From 3534b428629be185e096be99e3bd5fdfe32d5544 Mon Sep 17 00:00:00 2001 From: Laurent Bercot Date: Thu, 18 Sep 2014 18:55:44 +0000 Subject: initial commit with rc for skalibs-2.0.0.0 --- doc/crosscompile.html | 93 ++++ doc/djblegacy.html | 151 ++++++ doc/flags.html | 313 ++++++++++++ doc/index.html | 131 +++++ doc/libbiguint/index.html | 391 +++++++++++++++ doc/libdatastruct/index.html | 40 ++ doc/librandom/index.html | 113 +++++ doc/libskarnet.html | 100 ++++ doc/libstdcrypto/index.html | 110 +++++ doc/libstddjb/alloc.html | 98 ++++ doc/libstddjb/allreadwrite.html | 145 ++++++ doc/libstddjb/bitarray.html | 131 +++++ doc/libstddjb/djbtime.html | 201 ++++++++ doc/libstddjb/djbunix.html | 760 ++++++++++++++++++++++++++++++ doc/libstddjb/gccattributes.html | 48 ++ doc/libstddjb/genalloc.html | 46 ++ doc/libstddjb/genwrite.html | 98 ++++ doc/libstddjb/index.html | 125 +++++ doc/libstddjb/iopause.html | 196 ++++++++ doc/libstddjb/ip46.html | 172 +++++++ doc/libstddjb/lolstdio.html | 91 ++++ doc/libstddjb/safewrappers.html | 91 ++++ doc/libstddjb/selfpipe.html | 242 ++++++++++ doc/libstddjb/stralloc.html | 118 +++++ doc/libstddjb/tai.html | 462 ++++++++++++++++++ doc/libunixonacid/index.html | 58 +++ doc/libunixonacid/kolbak.html | 40 ++ doc/libunixonacid/skaclient.html | 34 ++ doc/libunixonacid/unix-timed.html | 34 ++ doc/libunixonacid/unix-transactional.html | 34 ++ doc/libunixonacid/unixmessage.html | 40 ++ doc/license.html | 82 ++++ doc/upgrade.html | 34 ++ 33 files changed, 4822 insertions(+) create mode 100644 doc/crosscompile.html create mode 100644 doc/djblegacy.html create mode 100644 doc/flags.html create mode 100644 doc/index.html create mode 100644 doc/libbiguint/index.html create mode 100644 doc/libdatastruct/index.html create mode 100644 doc/librandom/index.html create mode 100644 doc/libskarnet.html create mode 100644 doc/libstdcrypto/index.html create mode 100644 doc/libstddjb/alloc.html create mode 100644 doc/libstddjb/allreadwrite.html create mode 100644 doc/libstddjb/bitarray.html create mode 100644 doc/libstddjb/djbtime.html create mode 100644 doc/libstddjb/djbunix.html create mode 100644 doc/libstddjb/gccattributes.html create mode 100644 doc/libstddjb/genalloc.html create mode 100644 doc/libstddjb/genwrite.html create mode 100644 doc/libstddjb/index.html create mode 100644 doc/libstddjb/iopause.html create mode 100644 doc/libstddjb/ip46.html create mode 100644 doc/libstddjb/lolstdio.html create mode 100644 doc/libstddjb/safewrappers.html create mode 100644 doc/libstddjb/selfpipe.html create mode 100644 doc/libstddjb/stralloc.html create mode 100644 doc/libstddjb/tai.html create mode 100644 doc/libunixonacid/index.html create mode 100644 doc/libunixonacid/kolbak.html create mode 100644 doc/libunixonacid/skaclient.html create mode 100644 doc/libunixonacid/unix-timed.html create mode 100644 doc/libunixonacid/unix-transactional.html create mode 100644 doc/libunixonacid/unixmessage.html create mode 100644 doc/license.html create mode 100644 doc/upgrade.html (limited to 'doc') diff --git a/doc/crosscompile.html b/doc/crosscompile.html new file mode 100644 index 0000000..ba22b31 --- /dev/null +++ b/doc/crosscompile.html @@ -0,0 +1,93 @@ + + + + + How to cross-compile skalibs + + + + + + +

+skalibs
+Software
+skarnet.org +

+ +

How to cross-compile skalibs

+ +

+ There are three issues in the world of cross-compilation: +

+ + + +

Support for parallel builds

+ +

+ skalibs does not support out-of-tree builds at the moment, +but since the source code tree is small, it's not costly to duplicate it +to perform parallel builds. If the source code trees grows significantly +larger, out-of-tree build support will be considered. +

+ +

Support for build-time options

+ +

+ skalibs now uses a standard ./configure && make && make install +process, and cross-compiling build-time options can be given on the +./configure command line. +

+ +

Bypassing the build-time tests

+ +

+ This is the hardest part of cross-compilation, and very few build systems +get it right. (GNU autotools does not, which is one of the reasons why +skarnet.org packages do not use autotools.) +

+ +

+ Native build procedures usually perform build-time tests: they compile +executables and run them (on the build platform, which is the same as +the target platform) to check for features and system quirks. skalibs +does exactly that: the ./configure step performs tests on the +build platform and stores the system-dependent results in a directory +that it calls the sysdeps for this platform. +

+ +

+But in a +cross-compilation environment, build-time tests are invalid, since the build +platform and the target platform differ. + There is only one way to cross-compile portable code without resorting +to build-time autodetection: +you must provide by hand the sysdeps for your target +architecture, via the --with-sysdeps option to configure. +

+ +

+ The easiest way to get the correct sysdeps for a target achitecture is +to natively compile skalibs on that target, and steal the produced sysdeps +files. It can be easily done with a virtual machine, qemu for instance. +You could also (politely) ask for precompiled sysdeps on the +skaware mailing-list, if you cannot find them anywhere on the Internet. +

+ +

Credits

+ +

+Dan Kegel brought up the need for a +clean cross-compilation system. +

+ + + diff --git a/doc/djblegacy.html b/doc/djblegacy.html new file mode 100644 index 0000000..0cdd891 --- /dev/null +++ b/doc/djblegacy.html @@ -0,0 +1,151 @@ + + + + + skalibs: the djb legacy + + + + + + +

+skalibs
+Software
+skarnet.org +

+ +

The DJB legacy

+ +

Who is this DJB guy, and why is he so special anyway ?

+ +

+Dan J. Bernstein is a cryptologist and +a mathematician; he's also the author of a widely known and used MTA, +qmail, as well as a few +lesser known pieces of software. +

+ +

+ For some time he was quite active in some Unix software-related +Internet newsgroups and mailing-lists; he quickly became a +controversial figure of the Unix programming community, mostly +by being extremely vocal against well-known authors of +"mainstream" Unix software and by suggesting designs so removed +from traditional software design that a normal human reaction is +to first view him as a complete nut. +

+ +

+ I do not care for controversy. I am interested in the code. I was +a sysadmin at the time, and still learning to program in C beyond +what they teach you in school (i.e. not much). I had heard enough +horror stories with sendmail; so I gave a shot at qmail, trying to +understand its design principles and the way it was made. And then +I fell down the rabbit hole. +

+ +

+ Look, I don't care what you think of the guy, I don't know him +anyway, and this is totally beside the point. The only thing that +matters is that DJB's software is right in so +many ways. This software works. DJB's design +principles are sound and elegant; they are +sound foundations to build reliable, secure, and +low resource-consuming software. And the design, +when you get used to it, feels so unix-ish: it's Unix the way it +should have been from the start. +

+ +

+ Studying DJB's software was the best course in C/Unix programming +I ever had. Now I teach C/Unix; and I am really glad I +learned from the best. +

+ +

Building beyond DJB's works.

+ +

+ There's already +a lot you can do with +pristine DJB software and some brains. +

+ +

+ However, I mostly see DJB as a pioneer. He showed it was possible +to think Unix differently and build secure, reliable and efficient +software without investing millions of dollars into it; now it is +up to software architects and programmers to use the breakthrough +and build upon it. There's a real demand for quality Unix software +out there; it's time to supply. And +I am not the only +one thinking this way. +

+ +

+ So, skalibs. +

+ +

+ One of the "DJB philosophy" key points is to question the +interfaces. You have a task to do; you have existing interfaces. +What do you do? +

+ + + +

+ Interfaces should be questioned right down to the libc. You +cannot build strong software on flakey foundations. And from a system +and network programmer's point of view, one thing is clear: most +standard libc interfaces suck. There is no buffered asynchronous +I/O. There is no timed I/O. There is no heap management helper. Even +simple system calls are not +guaranteed to succeed! +

+ +

+ That is where skalibs comes from. skalibs results from questioning +the libc interfaces, and providing replacements or additions where +the existing interfaces do not make it easy to write reliable, secure +and efficient software. It is inspired by DJB's work. It is +not a shrine or anything of the kind. +

+ +

Conclusion

+ +

+ So, in short, DJB is not a guru, I'm not a mindless brainwashed fan, +and the "DJB advocates" are not a cult. We just think DJB brought +something to Unix and more generally to the software programming world; +we learned from him, we write software following +sound principles that he was one of the first to really apply, and we give +credit where credit is due. +

+ +

+ Use our software. You will never want to go back. +

+ + + diff --git a/doc/flags.html b/doc/flags.html new file mode 100644 index 0000000..2579d5a --- /dev/null +++ b/doc/flags.html @@ -0,0 +1,313 @@ + + + + + skalibs: configuration flags + + + + + + +

+skalibs
+Software
+skarnet.org +

+ +

skalibs configuration flags

+ +

+ The skalibs ./configure script comes with a few +uncommon options; this page explains what they are for. +

+ +

--enable-slashpackage[=sproot]

+ +

+ This flag tells configure that you want to install skalibs according to +the slashpackage convention. +If you enable it, and $v is the version of skalibs you're compiling, +make install will install the skalibs header files in +/package/prog/skalibs-$v/include, the static libraries in +/package/prog/skalibs-$v/library, the dynamic libraries in +/package/prog/skalibs-$v/library.so and the data files in +/package/prog/skalibs-$v/etc, all prefixed by sproot +it present. It will also add two more "make" targets: +

+ + + +

--enable-libc-replacements

+ +

+ If this option is given, then the low-level components +of libstddjb, such as byte_copy(), +will be built using independent, failsafe implementations; skalibs will +avoid relying on the libc when possible. +

+ +

+ If this option is not given, then native libc primitives such as +memmove() +will be used for the low-levels components of libstddjb. This is the default. +

+ +

+ This flag should be set if your libc has known bugs or you are uncertain +of it for some reason. Standard libcs on modern systems have been thoroughly +tested, so it's usually safe, and faster, to stick to the default. +

+ +

--enable-tai-clock

+ +

+ To understand what this flag is about - and the next three flags too - you +should start by reading +this +page about Unix time, +which David Madore wrote after +a long and fairly complete discussion we had on the subject. You can also +read what DJB says about Unix time. +Unfortunately, when he says "the POSIX rules are so outrageously dumb (...) +that no self-respecting engineer would obey them", DJB is wrong: a lot of +people follow the POSIX rules. Or maybe he's right... and there are very, +very few self-respecting engineers. +

+ +

+ Basically, when you configure a Unix system, there are essentially two +ways to deal with your system clock. +

+ +
    +
  1. You can set your system clock to TAI-10, which is the "right", but +uncommon, thing to do: +
      +
    • ↑ The main advantage of this setup is that it makes your system clock +linear. In other words, +gettimeofday() +becomes suitable for both timestamping (which needs absolute time) and timeout +computations (which need reliable interval measurements); if your clock is +accurate enough, it can function as both a wall clock and a stopwatch. +This simplifies keeping track of the current time considerably, +and makes event loop handling (with functions such as +iopause()) trivial.
      • +
    • ↑ skalibs uses TAI internally; setting your system clock to TAI-10 +saves a lot of conversions and makes time computations with skalibs more +efficient.
      • +
    • → In order to display GMT or local time properly, you have to +use the right/ timezones from Arthur David Olson's timezone +library. If your libc does not support them, see the +next flag.
      • +
    • ↓ This setup is arguably not SUSv4 conformant (a strict +interpretation of Single Unix requires the system clock to be set to UTC).
      • +
    • ↓ This setup is not compatible with +ntpd. ntpd's design +is flawed: it makes the mistake of setting the system clock itself - instead +of simply making the computed time available to other programs, one of which +could set the system clock - and it always sets it to UTC. (The +s6-networking +package provides alternate ways to synchronize your clock, though.)
      • +
    +
    2. +
  2. You can set your system clock to UTC, which is the common, strictly +POSIX setup: +
      +
    • ↑ This is strictly SUSv4-compliant. Most Unix machines all over +the world are set up like this.
      • +
    • ↑ This is compatible with ntpd.
      • +
    • → You should not use Olson's time library in that case.
      • +
    • ↓ skalibs time computations will take a bit more processing power.
      • +
    • ↓ Most importantly, you forsake all linearity - and even monotonicity +- on your system clock, which can now only be used as a wall clock, +not as a stopwatch. skalibs will try its best to do accurate time +computations, but there's no way gettimeofday() can be relied on +when a leap second is nearby; you have to use CLOCK_MONOTONIC as a stopwatch +for accurate interval measurement.
      • +
    +
    3. +
+ +

+ USe --enable-tai-clock if your system clock is set to TAI-10. +I generally recommend this setup +for computers you have full control on, on which you install and tweak +the software your way, like manually administered servers or embedded +boxes. If you do not run ntpd and do not mind breaking POSIX, it is the +sensible choice. +

+ +

+ Do not use this option if your system clock is set to UTC, i.e. if +you're in none of the above cases: you are a +POSIX freak, or your Unix distribution is running ntpd for you, or +other software is assuming you're on UTC. This is the default. +

+ + +

--enable-right-tz

+ +

+ This option instructs skalibs that you're using Olson's time +library, i.e. "right/" timezones. +

+ +

+ Normally, if you set --enable-tai-clock, you +should also set up your timezone to a "right/" one, and +set flag-tzisright. And if you don't use +--enable-tai-clock, you should also use a POSIX +timezone, and NOT use --enable-right-tz. Those two options +should always be used together. +

+ +

+ But some C libraries do not support the Olson time library's +timezone format, and just do not provide the "right/" timezones! For +instance, musl, +an alternative libc for Linux, only supports POSIX timezones. And you +might want to use such a libc, and still set up your clock to +TAI-10, for instance in embedded environments where accurate timekeeping +is important. In such cases, you'll set up a POSIX timezone, and use the +--enable-tai-clock option without the --enable-right-tz one. +

+ +

+ Be aware that setting your system clock to TAI-10 without having a +"right/" timezone will cause non-skalibs-using software to display +local time incorrectly; in such a setup, only skalibs-using software +will understand what is going on and do the proper computations to +display the correct local time. Keep your settings as consistent as +possible. +

+ +

+ By default, skalibs will consider you are using POSIX timezones (as well +as a UTC system clock). +

+ +

--enable-clock

+ +

+ The Open Group Base Specifications, issue 7, describes gettimeofday() +as obsolescent, and recommends the use of +clock_gettime() +with the CLOCK_REALTIME option instead. However: +

+ + + +

+ If --enable-clock is set, the tain_now() +and tain_setnow() functions for getting and setting time will be based on +the clock_gettime() and clock_settime() functions. +

+ +

+ Otherwise, the old-school gettimeofday() +and settimeofday() interfaces will be used. This is the default, +and it's usually safe. +

+ +

--enable-monotonic

+ +

+ Unless you have an accurate hardware system clock and you set it +on a linear time scale such as TAI-10 instead of UTC (see above), it is +generally a bad idea to trust the system clock for precise time interval +measurements. Single Unix recommends the use of clock_gettime() +with the CLOCK_MONOTONIC option to do such measurements: a stopwatch, not +a wall clock. However: +

+ + + +

+ If --enable-monotonic is set, then the absolute time given by the +tain_now() call will be computed with CLOCK_MONOTONIC. This +will ensure precise time arithmetic but may drift away from the system +clock. +

+ +

+ Otherwise, tain_now() will +return a time based on the system clock, and not use CLOCK_MONOTONIC. +This is the default. +

+ +

--disable-ipv6

+ +

+ If you set this option, then skalibs will be compiled without IPv6 support, +even if your target architecture supports it. This can significantly +reduce the size of your networking applications if they don't need IPv6 +support. +

+ +

+ If you don't set this option, then skalibs will include IPv6 support in the +relevant networking functions, if the target architecture supports it. +The safe option is to let this flag clear. +

+ +

--enable-force-devr

+ +

+ If this option is set, then the automatic sysdeps tests will assume the +target architecture has a working /dev/random and will skip +its autodetection. +

+ +

+ Otherwise, /dev/random will be autodetected +and tested; if entropy generation is low on the host, the compilation +process might hang for several minutes. It is safe to let this flag +clear; it should only be set to speed up the compilation process in a +known environment and for testing purposes. +

+ +

+ If skalibs is being cross-compiled, this flag obviously has no effect: +the presence of a working /dev/random is read from the user-provided +sysdeps. +

+ +

--enable-egd=path

+ +

+ If you set this option, then the librandom functions +will assume the presence of an EGD daemon listening on path, +and use it to get random data. +

+ +

+ By default, skalibs will not include EGD support. +

+ +

--with-default-path=path

+ +

+ The execvep() function uses +the value of the PATH environment variable as its executable search path. +Specifying this option to configure tells execvep() what executable +search path to use when PATH is undefined (which should not happen +often anyway). + The default is /usr/bin:/bin, which is usually safe. +

+ + + diff --git a/doc/index.html b/doc/index.html new file mode 100644 index 0000000..344ffff --- /dev/null +++ b/doc/index.html @@ -0,0 +1,131 @@ + + + + + skalibs: a C library for system programming + + + + + + +

+Software
+skarnet.org +

+ +

skalibs

+ +

What is it ?

+ +

+ skalibs is a package centralizing the free software / open source C development +files used for building all software at skarnet.org: it contains essentially +general-purpose libraries. +You will need to install skalibs if you plan to build skarnet.org software. + The point is that you won't have to download and compile big libraries, and care +about portability issues, +everytime you need to build a package: do it only once. +

+ +

+ skalibs can also be used as a sound basic start for C development. There +are a lot of general-purpose libraries out there; but if your main goal is +to produce small and secure C code with a focus on system +programming, skalibs might be for you. +

+ +
+ +

Installation

+ +

Requirements

+ + + +

Licensing

+ +

+ skalibs is free software. It is available under the +ISC license. +

+ +

Download

+ + + +

Compilation

+ + + +

Upgrade notes

+ + + +

Cross-compilation

+ +

+ Cross-compilation is tricky. skalibs provides system-agnostic interfaces, so all the +other skarnet.org packages cross-compile easily; but some effort needs to be made to +cross-compile skalibs, see this page. + +

+ +

Reference

+ +

Libraries

+ + + +
+ +

Similar work

+ +

+ Here are a few other libraries originating from the same place as +skalibs, i.e. people start to reuse and package, or rewrite, +code from Dan J. Bernstein, and then patch +after patch, addition after addition, the code evolves into a project +of its own: +

+ + + +

Related resources

+ + + + + diff --git a/doc/libbiguint/index.html b/doc/libbiguint/index.html new file mode 100644 index 0000000..30de27e --- /dev/null +++ b/doc/libbiguint/index.html @@ -0,0 +1,391 @@ + + + + + skalibs: the biguint library interface + + + + + + +

+libskarnet
+skalibs
+Software
+www.skarnet.org +

+ +

The biguint library interface

+ +

+biguint is set of simple primitives performing arithmetical +operations on (unsigned) integers of arbitrary length. It is nowhere +near as powerful or efficient as specialized, +assembly language-optimized libraries such as +GMP, but it has the advantages +of smallness and simplicity. +

+ +

Compiling

+ + + +

Programming

+ +

+ You should refer to the skalibs/biguint.h header for the exact function +prototypes. +

+ +

+Definitions

+ + + +

+Basic operations

+ +

Creating a biguint

+ +

+ Just declare uint32 x[n] ; - n being the length of the +biguint. You could also allocate x in the heap, possibly using a +uint32 genalloc. In the following, +a biguint is always referred to as a uint32 * with its +unsigned int length ; it must always be pre-allocated. +

+ +

+ If an operation fails because a biguint's length n is too small to +accommodate the result, the function will write the first (i.e. least significant) +n limbs of the result, truncating it, then return 0 with errno set to +EOVERFLOW. +

+ +

Setting it to zero

+ +
+uint32 *x ;
+unsigned int n ;
+
+ bu_zero(x, n) ;
+
+ +

+bu_zero() sets the first n limbs of x to zero. +

+ +

Copying a biguint

+ +
+uint32 const *x ;
+unsigned int xn ;
+uint32 *y ;
+unsigned int yn ;
+
+  bu_copy(y, yn, x, xn) ;
+
+ +

+bu_copy() copies x to y, setting higher limbs of y +to zero if needed. It then returns 1. If y is too small to contain x, +the function returns 0 EOVERFLOW. +

+ +

Calculating the order

+ +
+uint32 const *x ;
+unsigned int n ;
+unsigned int r ;
+
+  r = bu_len(x, n) ;
+
+ +

+bu_len() outputs the order of x of length n. +0 <= r <= n. +

+ +

Comparing two biguints

+ +
+uint32 const *a ;
+unsigned int an ;
+uint32 const *b ;
+unsigned int bn ;
+int r ;
+
+  r = bu_cmp(a, an, b, bn) ;
+
+ +

+bu_cmp() returns -1 if a < b, 1 if +a > b, and 0 if a = b. +

+ +

+I/O operations

+ +

Writing a biguint as an array of bytes

+ +
+char *s ;
+uint32 const *x ;
+unsigned int n ;
+
+  bu_pack(s, x, n) ;
+  bu_pack_big(s, x, n) ;
+
+ +

+bu_pack() writes 4*n bytes to s. The bytes +are a little-endian representation of x.
+bu_pack_big() is the same, with a big-endian representation. +

+ +

Reading a biguint from an array of bytes

+ +
+char const *s ;
+uint32 *x ;
+unsigned int n ;
+
+  bu_unpack(s, x, n) ;
+  bu_unpack_big(s, x, n) ;
+
+ +

+bu_unpack() reads 4*n little-endian bytes from s +and writes them into the corresponding biguint x.
+bu_unpack_big() is the same, but the bytes are interpreted as +big-endian. +

+ +

Formatting a biguint for readable output

+ +
+char *s ;
+uint32 const *x ;
+unsigned int n ;
+
+  bu_fmt(s, x, n) ;
+
+ +

+bu_fmt() writes x in s as a standard big-endian +hexadecimal number. x is considered of length n, so +8*n bytes will be written to s, even if it x +starts with zeros. bu_fmt returns the number of bytes written. +

+ +

Reading a biguint from readable format

+ +
+char const *s ;
+uint32 *x ;
+unsigned int xn ;
+unsigned int z ;
+unsigned int len ;
+
+  len = bu_scanlen(s, &z) ;
+  bu_scan(s, len, x, xn, z) ;
+
+ +

+ bu_scanlen() scans s for a biguint written as a hexadecimal +number and returns the number of +bytes read. The reading stops at the first byte encountered that is not +in the 0-9, A-F or a-f range. The z integer then contains the +number of bytes excluding leading zeros. +

+ +

+ If x has not been allocated yet, you can use xn = bitarray_div8(z) +(if you have included the bitarray.h header) +and allocate x with length xn. +

+ +

+bu_scan() then reads len bytes from s, assuming +there are z significant bytes (i.e. not leading zeros); it writes +the resulting biguint into x of length xn. It returns 1, +except if xn is too small, in which case it returns 0 EOVERFLOW. +

+ +

+Arithmetic operations

+ +

Addition

+ +
+uint32 const *a ;
+unsigned int an ;
+uint32 const *b ;
+unsigned int bn ;
+uint32 *c ;
+unsigned int cn ;
+unsigned char carrybefore ;
+unsigned char carryafter ;
+
+  bu_add(c, cn, a, an, b, bn) ;
+  bu_sub(c, cn, a, an, b, bn) ;
+
+ +

+bu_add() adds a and b, and puts the result +into c. It returns 1 unless it has to truncate it. +

+ +

+bu_sub() substracts b from a, and puts the +result into c. If the result should be negative, then it is +written as (2^32)^cn - c and the function returns 0 EOVERFLOW. +

+ +

Multiplication

+ +
+uint32 const *a ;
+unsigned int an ;
+uint32 const *b ;
+unsigned int bn ;
+uint32 *c ;
+unsigned int cn ;
+
+ bu_mul(c, cn, a, an, b, bn) ;
+
+ +

+bu_mul() computes c=a*b. +Make sure that cn ≥ bu_len(a, an) + bu_len(b, bn). +If it is not the case, the result will be truncated and bu_mul will return +0 EOVERFLOW. +

+ +

Division

+ +
+uint32 const *a ;
+unsigned int an ;
+uint32 const *b ;
+unsigned int bn ;
+uint32 *q ;
+unsigned int qn ;
+uint32 *r ;
+unsigned int rn ;
+
+ bu_div(a, an, b, bn, q, qn, r, rn) ;
+ bu_mod(r, rn, b, bn) ;
+
+ +

+bu_div() computes q, the quotient, and r, the +remainder, of a divided by b. If b is zero, it +returns 0 EDOM. If qn or rn is to small to store the +quotient or the remainder, it returns 0 EOVERFLOW. +bu_mod() computes only the remainder, and stores it in-place. +

+ +

GCD

+ +
+uint32 *r ;
+unsigned int rn ;
+uint32 const *a ;
+unsigned int an ;
+uint32 const *b ;
+unsigned int bn ;
+
+ bu_gcd(r, rn, a, an, b, bn) ;
+
+ +

+

+bu_gcd() computes the greatest common divisor between a +and b, and stores it into r. It returns 1 if all went well. +

+ +

+ Note that this function iterates on divisions, so it might use a non totally +negligible amount of CPU time. +

+ + +

Left-shifts and right-shifts

+ +
+uint32 *x ;
+unsigned int xn ;
+unsigned char carryafter ;
+unsigned char carrybefore ;
+
+ carryafter = bu_slbc(x, xn, carrybefore) ;
+ carryafter = bu_srbc(x, xn, carrybefore) ;
+
+ +

+bu_slbc() computes x <<= 1. +The least significant bit of x is then set to +carrybefore. bu_slbc() returns the +previous value of x's most significant bit.
+bu_srbc() computes x >>= 1. +The most significant bit of x is then set to +carrybefore. bu_slbc() returns the +previous value of x's least significant bit.
+bu_slb(x, n) and bu_srb(x, n) are macros for +respectively bu_slbc(x, n, 0) and bu_srbc(x, n, 0). +

+ +

Modular operations

+ +
+uint32 const *a ;
+unsigned int an ;
+uint32 const *b ;
+unsigned int bn ;
+uint32 *c ;
+unsigned int cn ;
+uint32 const *m ;
+unsigned int mn ;
+
+ bu_addmod(c, cn, a, an, b, bn, m, mn) ;
+ bu_submod(c, cn, a, an, b, bn, m, mn) ;
+ bu_mulmod(c, cn, a, an, b, bn, m, mn) ;
+ bu_divmod(c, cn, a, an, b, bn, m, mn) ;
+ bu_invmod(c, cn, m, mn) ;
+
+ +

+bu_addmod() computes c = (a+b) mod m.
+bu_submod() computes c = (a-b) mod m.
+bu_mulmod() computes c = (a*b) mod m.
+a and b must already be numbers modulo m.
+The functions return 1 if all went well. +

+ +

+bu_divmod() computes a divided by b modulo +m and stores it into c.
+bu_invmod() computes the inverse of c modulo m +and stores it into c.
+The divisor and m must be relatively prime, else +those functions return 0 EDOM.
+ The algorithm for modular division and inversion is due to +Sheueling +Chang Shantz. +

+ + + diff --git a/doc/libdatastruct/index.html b/doc/libdatastruct/index.html new file mode 100644 index 0000000..b32c7ce --- /dev/null +++ b/doc/libdatastruct/index.html @@ -0,0 +1,40 @@ + + + + + skalibs: the datastruct library interface + + + + + + +

+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The datastruct library interface

+ +

+libdatastruct implements generic data structures like chained +lists and AVL trees, in a memory-efficient and CPU-efficient way. +

+ +

Compiling

+ + + +

Programming

+ +

+FIXME: to be completed. +

+ + + diff --git a/doc/librandom/index.html b/doc/librandom/index.html new file mode 100644 index 0000000..b721594 --- /dev/null +++ b/doc/librandom/index.html @@ -0,0 +1,113 @@ + + + + + skalibs: the random library interface + + + + + +

+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The random library interface

+ +

+librandom is a small library designed to provide an +interface to some reasonable-quality pseudorandom number +generation. Some libcs have a bad +random() implementation; librandom is designed +to use system pseudorandom number generation when it's provided +via /dev/random and /dev/urandom, and to use +a good default PRNG otherwise. +

+ +

+ librandom also supports +EGD. If you have built +skalibs with --enable-egd, then the librandom +primitives will try and connect to an EGD service to get random bytes +if there is no kernel-based entropy generator such as /dev/random. +If the EGD connection fails, a SURF PRNG is used. +

+ +

Compiling

+ + + +

Programming

+ +

+ You should refer to the skalibs/random.h header for the exact +function prototypes. +

+ +

High quality, cryptographically strong random data

+ +
+  unsigned char c ;
+  unsigned int max ;
+  unsigned int n ;
+  unsigned int b ;
+  char data[at least b] ;
+  int r ;
+
+  goodrandom_init() ;
+  c = goodrandom_char() ;
+  n = goodrandom_int(max) ;
+  r = goodrandom_string(data, b) ;
+  goodrandom_finish() ;
+
+ +

+ goodrandom_init() becomes optional with skalibs-0.43. + It is recommended that you let the library perform cleanups after you +have used it, by calling goodrandom_finish(). +

+ + + +

+ If you have neither /dev/random nor EGD, a software PRNG is +used. This PRNG is based on the +SURF function, which +is unpredictable enough for most uses. +

+ +

Lower quality random data

+ +

+ It works basically the same, by replacing goodrandom_* with +badrandom_*. It uses /dev/urandom on systems that +support it; on systems that do not, but support EGD, non-blocking calls +to EGD are made ; if that is not enough, or EGD is not supported, +the SURF generator is used. +

+ +

+ The point of badrandom is to get random bytes instantly, +even at the expense of quality; whereas goodrandom always returns +high-quality random bytes, but may block if entropy is insufficient. In +practice, in spite of its name, badrandom will return quite +unpredictable pseudo-random data, so goodrandom should be used +only when paranoia is the rule and blocking is an option. +

+ + + diff --git a/doc/libskarnet.html b/doc/libskarnet.html new file mode 100644 index 0000000..9c29514 --- /dev/null +++ b/doc/libskarnet.html @@ -0,0 +1,100 @@ + + + + + skalibs: the skarnet library interface + + + + + + +

+skalibs
+Software
+skarnet.org +

+ +

The skarnet library interface

+ +

+ libskarnet is the library exported by skalibs; both a static +library and a shared library (if they are supported on your system) are +available. Every skarnet.org binary needs this library. +

+ +

Compiling

+ + + +

Linking

+ + + +

+ The skarnet library as a whole is big (833k for libskarnet.a and +257k for libskarnet.so.2.0.0.0 on x86_64), but the utmost care has been +given to separate functions so that linkers never pull in any more than they +need. Linking against the static version of libskarnet actually produces +very small executables, and if your libc is suited for that, since skalibs +only uses very basic libc interfaces, it is very possible to produce small +static binaries - in many cases, a static program written with skalibs APIs +will be an order of magnitude smaller than the equivalent program written +using libc's or other utility libraries' APIs. +

+ +

Programming

+ +

+ The skalibs source code is divided into several subdirectories, each +containing a logical unit of code implementing independent functions. +

+ + + + + diff --git a/doc/libstdcrypto/index.html b/doc/libstdcrypto/index.html new file mode 100644 index 0000000..bbc5f08 --- /dev/null +++ b/doc/libstdcrypto/index.html @@ -0,0 +1,110 @@ + + + + + skalibs: the stdcrypto library interface + + + + + + +

+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The stdcrypto library interface

+ +

+stdcrypto is a small collection of standard, +public-domain cryptographic primitives. Currently, the following +operations are provided: +

+ + + +

Compiling

+ + + +

Programming

+ +

+ You should refer to the skalibs/stdcrypto.h header and included headers +for the exact function prototypes. +

+ +

+RC4

+ +
+  RC4Schedule ctx ;
+  unsigned char const *key ;
+  unsigned int keylen ;
+  unsigned char const *in ;
+  unsigned char *out ;
+  unsigned int len ;
+
+  rc4_init(&ctx, key, keylen) ;
+  rc4(&ctx, in, out, len) ;
+
+ + + +

+MD5

+ +
+  MD5Schedule ctx ;
+  char const *message ;
+  unsigned int messagelen ;
+  char digest[16] ;
+
+  md5_init(&ctx) ;
+  md5_update(&ctx, message, messagelen) ;
+  md5_final(&ctx, digest) ;
+
+ + + +

+SHA1

+ +
+  SHA1Schedule ctx ;
+  char const *message ;
+  unsigned int messagelen ;
+  unsigned char digest[20] ;
+
+  sha1_init(&ctx) ;
+  sha1_update(&ctx, message, messagelen) ;
+  sha1_final(&ctx, digest) ;
+
+ + + + + diff --git a/doc/libstddjb/alloc.html b/doc/libstddjb/alloc.html new file mode 100644 index 0000000..e17fcc7 --- /dev/null +++ b/doc/libstddjb/alloc.html @@ -0,0 +1,98 @@ + + + + + skalibs: the alloc library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The alloc library interface

+ +

+ The following functions are declared in the skalibs/alloc.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ alloc is the skalibs heap memory manager. It's actually a +wrapper for the +malloc() +series of functions; it unifies a few system-dependent malloc +behaviours. It's also the API to implement and preload if for some +reason you need to plug in your own allocator: replacing alloc() +is much easier than replacing malloc() safely. +

+ +

+ As a general rule, you should not be using the alloc +interface directly. Allocating and freeing individual cells +in the heap is a recipe for heap fragmentation, as well as cell +tracking nightmares leading to memory leaks. You should use +the higher-level stralloc and +genalloc interfaces to handle dynamic +arrays of objects. +

+ +

+ C's lack of automatic management of heap memory is not a drawback: it's +a feature of the language. It allows for code that is one or two orders +of magnitude faster than the equivalent in a higher-level language, +and very low on resources consumption. However, it requires more attention +from the programmer. Good APIs can significantly reduce the difficulty of +keeping track of every heap-allocated cell, and every smart programmer +should favor them over basic interfaces like malloc(). +

+ +

+ alloc is used internally by skalibs to implement +stralloc, and nowhere else. +

+ +

Functions

+ +

+ char *alloc (unsigned int len)
+Allocates a block of len bytes in the heap and returns a pointer +to the start of the block (or NULL if it failed). Though the pointer type +is char *, the block of memory is correctly aligned for any type +of object. If len is 0, the function returns a pointer that +cannot be written to, but that is not null. Note that this is +different from the required C99 behaviour for malloc(). +

+ +

+ void alloc_free (void *p)
+Frees the block of heap memory pointed to by p. +

+ +

+ int alloc_realloc (char **p, unsigned int newlen)
+Redimension the block of heap memory pointed to by *p to +newlen bytes. The block may have to be moved, in which case +*p will be modified. Normally returns 1; if an error occurred, +returns 0 and sets errno, and neither *p nor its contents are +modified. +

+ +

+ int alloc_re (char **p, unsigned int oldlen, unsigned int newlen)
+Legacy interface for reallocation. It works like alloc_realloc, +except that the original block length must be provided as the oldlen +argument. +

+ + + diff --git a/doc/libstddjb/allreadwrite.html b/doc/libstddjb/allreadwrite.html new file mode 100644 index 0000000..847c4e3 --- /dev/null +++ b/doc/libstddjb/allreadwrite.html @@ -0,0 +1,145 @@ + + + + + skalibs: the allreadwrite library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The allreadwrite library interface

+ +

+ The following functions are declared in the skalibs/allreadwrite.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ allreadwrite is a set of IO function helpers. It's the +basis for safe reading and writing, either in blocking or in +non-blocking mode. The buffer interface +relies heavily on allreadwrite. +

+ +

+ Unless the IO you need is very simple, you generally should not +be using the allreadwrite functions directly; you should +use higher-level APIs such as bufalloc. +

+ +

Function types

+ +

+ typedef int iofunc_t (int fd, char *buf, unsigned int len)
+This is the simplified type of IO functions such as +read() +and +write(). +Unless your system's int is 64-bit, skalibs - which has been +optimized for small systems - does not support IO operations of more than +2 GB of data, for the sake of simplicity. In any case, it's always +possible to send data in several smaller chunks. +

+ +

+ typedef unsigned int alliofunc_t (int fd, char *buf, unsigned int len)
+This is the type of an IO operation that expects all of its +len bytes to be sent or received, and that will loop around a +lower-level IO function until either len bytes have been +transmitted or an error has occurred. The return value is the actual +number of transmitted bytes; if this value is lesser than len, +it means that an error has occurred and errno is set. +

+ +

Functions

+ +

+ int sanitize_read (int r)
+Reading functions such as read() and fd_read return +a positive number when they succeed, -1 when they fail, and 0 when they +read an EOF. No data available on the descriptor when reading in +non-blocking mode is treated as a failure: -1 EWOULDBLOCK. But sometimes +(namely, in asynchronous IO loops) it's preferrable to handle EOF as an +exception condition and EWOULDBLOCK as a normal condition. +sanitize_read(), when applied to the result of a basic reading +function, returns 0 if r is -1 and errno is EWOULDBLOCK (or +EAGAIN). If r is zero, it returns -1 EPIPE. Else it returns r. +

+ +

+ (No system reading function can ever set errno to EPIPE, and the +semantics are appropriate, so EPIPE is a good candidate to signal EOF +on reading.) +

+ +

+ unsigned int allreadwrite (iofunc_t *f, int fd, char *s, unsigned int len)
+*f must be a basic reading or writing function such as +fd_read or fd_write. allreadwrite() performs +*f on fd, s and len until len +bytes have been read or written, or until an error occurs. It returns the +total number of handled bytes, and sets errno if this number is not +len. allreadwrite may block if fd is in +blocking mode; if fd is in non-blocking mode, it might +set errno to EWOULDBLOCK or EAGAIN. +

+ +

+ int fd_read (int fd, char *s, unsigned int len)
+Safe wrapper around the +read() +function. +

+ +

+ int fd_write (int fd, char const *s, unsigned int len)
+Safe wrapper around the +write() +function. +

+ +

+ int fd_recv (int fd, char *s, unsigned int len, unsigned int flags)
+Safe wrapper around the +recv() +function. +

+ +

+ int fd_send (int fd, char const *s, unsigned int len, unsigned int flags)
+Safe wrapper around the +send() +function. +

+ +

+ unsigned int allread (int fd, char *s, unsigned int len)
+Equivalent to allreadwrite(&fd_read, fd, s, len) : attempts +to read len bytes from fd into s, looping around +fd_read() if necessary, until either len bytes are read or +an error occurs. EOF is reported as EPIPE. +

+ +

+ unsigned int allwrite (int fd, char const *s, unsigned int len)
+Equivalent to allreadwrite((iofunc_t *)&fd_write, fd, s, len) : +attempts to write len bytes from s to fd, looping +around fd_write() if necessary, until either len bytes are +written or an error occurs. +

+ + + diff --git a/doc/libstddjb/bitarray.html b/doc/libstddjb/bitarray.html new file mode 100644 index 0000000..1f88c3d --- /dev/null +++ b/doc/libstddjb/bitarray.html @@ -0,0 +1,131 @@ + + + + + skalibs: the bitarray library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The bitarray library interface

+ +

+ The following functions are declared in the skalibs/bitarray.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ bitarray is a set of primitives to operate efficiently on +large bitfields. +

+ +

+ A bitfield is represented by a pre-allocated block of +unsigned char; bitarray does not care if that +block has been BSS-, stack- or heap-allocated. Bitfields that +can grow in size should be stored in a +stralloc. +

+ +

+ Bits in a bitfield of length n are numbered from 0 to n-1. +

+ +

Functions

+ +

+ unsigned int bitarray_div8 (unsigned int n)
+Returns the minimum number of bytes needed to store a field of n bits. + +

+ +

+ void bitarray_clearsetn (unsigned char *s, unsigned int start, unsigned int len, int h)
+Sets (if h is nonzero) or clears (if h is zero) +len bits in field s, starting at bit start. +

+ +

+ void bitarray_clearn (unsigned char *s, unsigned int start, unsigned int len)
+Clears len bits in field s, starting at bit start. +

+ +

+ void bitarray_setn (unsigned char *s, unsigned int start, unsigned int len)
+Sets len bits in field s, starting at bit start. +

+ +

+ int bitarray_peek (unsigned char const *s, unsigned int n)
+Returns the value of the nth bit in field s. +

+ +

+ void bitarray_poke (unsigned char *s, unsigned int n, int h)
+Sets (if h is nonzero) or clears (if h is zero) +the nth bit in field s. +

+ +

+ void bitarray_clear (unsigned char *s, unsigned int n)
+Clears the nth bit in field s. +

+ +

+ void bitarray_set (unsigned char *s, unsigned int n)
+Sets the nth bit in field s. +

+ +

+ int bitarray_testandpoke (unsigned char *s, unsigned int n, int h)
+Sets (if h is nonzero) or clears (if h is zero) +the nth bit in field s, +and returns the previous value of that bit. +

+ +

+ int bitarray_testandclear (unsigned char *s, unsigned int n)
+Clear the nth bit in field s, +and returns the previous value of that bit. +

+ +

+ int bitarray_testandset (unsigned char *s, unsigned int n)
+Sets the nth bit in field s, +and returns the previous value of that bit. +

+ +

+ unsigned int bitarray_first (unsigned char const *s, unsigned int len, int h)
+Returns the number of the first set (if h is nonzero) or clear +(if h is zero) bit in s, len being +the total number of bits. If all bits in s are the negation of +h, then len is returned. +

+ +

+ unsigned int bitarray_firstclear (unsigned char const *s, unsigned int len)
+Returns the number of the first clear bit in s, len being +the total number of bits. If all bits in s are set, len is returned. +

+ +

+ unsigned int bitarray_firstset (unsigned char const *s, unsigned int len)
+Returns the number of the first set bit in s, len being +the total number of bits. If all bits in s are clear, len is returned. +

+ + + diff --git a/doc/libstddjb/djbtime.html b/doc/libstddjb/djbtime.html new file mode 100644 index 0000000..45876c8 --- /dev/null +++ b/doc/libstddjb/djbtime.html @@ -0,0 +1,201 @@ + + + + + skalibs: the djbtime library interface + + + + + + +

+libstddjb
+skalibs
+skalibs
+Software
+skarnet.org +

+ +

The djbtime library interface

+ +

+ The following functions are declared in the skalibs/djbtime.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ djbtime is a set of functions to convert +tai_t and tain_t structures, and +TAI time, from and to +other time formats and user-friendly representations. +

+ +

The /etc/leapsecs.dat file

+ +

+ User-friendly time is calculated from UTC. Internal time computations +should be performed on TAI time - because TAI flows linearly whereas +UTC does not. To convert between UTC and TAI time, you need a +leap second table. skalibs provides such a file in its +src/etc/leapsecs.dat subdirectory, which is copied +to /etc/leapsecs.dat at installation time (unless you specify +a --prefix or --datadir option to configure). +The /etc/leapsecs.dat file must remain accessible +on your system, else time conversions will not be computed +properly. +

+ +

Data structures

+ + + +

Functions

+ +

UTC

+ +

+ int utc_from_tai (uint64 *u, tai_t const *t)
+Converts the absolute TAI64 time in *t to an UTC time, stored in +*u as an unsigned 64-bit integer. *u is actually 2^62 +plus the number of seconds since the Epoch. +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs (for instance: the leap second table cannot be found). +

+ +

+ int tai_from_utc (tai_t *t, uint64 u)
+Converts the UTC time in u, stored +as an unsigned 64-bit integer (2^62 plus the number of seconds since +the Epoch), to a TAI64 time in *t. +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs (for instance: the leap second table cannot be found). +

+ +

NTP

+ +

+ int ntp_from_tain (uint64 *ntp, tain_t const *a)
+Converts the absolute TAI64N time in *a to a 64-bit NTP timestamp, +stored in *ntp. The higher 32 bits of *ntp represent a number +of seconds ; the lower 32 bits are the fractional part of the timestamp. +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs (for instance: the leap second table cannot be found, or +*a cannot be represented in the valid NTP range). +

+ +

+ int tain_from_ntp (tain_t *a, uint64 ntp)
+Converts the NTP timestamp in ntp to a TAI64N time in +*a. +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs (for instance: the leap second table cannot be found). +

+ +

Local time

+ +

+ The following functions convert time between an internal representation +and a broken-down struct tm. The +--enable-right-tz configure option is used in +determining how the conversion should proceed. If the --enable-tai-clock +and --enable-right-tz configure options have been both enabled +or both disabled, everything is naturally +converted as it should be. If only one of them has been enabled, +unholy magic happens here +to get the correct broken-down time despite the timezone definition being +wrong. +

+ +

+ int localtm_from_tai (struct tm *tm, tai_t const *t, int lo)
+Converts the TAI time in *t to broken-down GMT (if +lo is zero) or local (if lo is nonzero) time in +*tm. +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs (for instance: *t cannot be validly represented +in a struct tm). +

+ +

+ int localtm_from_utc (struct tm *tm, uint64 u, int lo)
+Converts the UTC time in u to broken-down GMT (if +lo is zero) or local (if lo is nonzero) time in +*tm. +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs (for instance: u cannot be validly represented +in a struct tm). +

+ +

+ int localtm_from_sysclock (struct tm *tm, uint64 u, int lo)
+Converts the time in u to broken-down GMT (if +lo is zero) or local (if lo is nonzero) time in +*tm. u will be interpreted as a TAI-10 value (with +--enable-tai-clock) or as a UTC value (without --enable-tai-clock). +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs (for instance: u cannot be validly represented +in a struct tm). +

+ +

+ int utc_from_localtm (uint64 *u, struct tm const *tm)
+Converts the broken-down local time in *tm to an UTC value +in *u. +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs. +

+ +

+ int tai_from_localtm (tai_t *t, struct tm const *tm)
+Converts the broken-down local time in *tm to a TAI value +in *t. +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs. +

+ +

+ int sysclock_from_localtm (uint64 *u, struct tm const *tm)
+Converts the broken-down local time in *tm to a value +in *u - either TAI-10 or UTC depending on your system clock. +The function returns 1 if it succeeds, or 0 (and sets errno) if an +error occurs. +

+ +

+ The following functions use the localtmn_t type to hold both +a broken-down time and a nanosecond count: +

+ +
typedef struct localtmn_s localtmn_t, *localtmn_t_ref ;
+struct localtmn_s
+{
+  struct tm tm ;
+  uint32 nano ;
+} ;
+
+ +

+ The prototypes are self-explaining: +

+ +

+ int localtmn_from_tain (localtmn_t_ref tmn, tain_t const *a, int lo) ;
+int tain_from_localtmn (tain_t *a, localtmn_t const *tmn) ;
+int localtmn_from_sysclock (localtmn_t_ref tmn, tain_t const *a, int lo) ;
+int sysclock_from_localtmn (tain_t *a, localtmn_t const *tmn) ;
+

+ + + diff --git a/doc/libstddjb/djbunix.html b/doc/libstddjb/djbunix.html new file mode 100644 index 0000000..0d6c89f --- /dev/null +++ b/doc/libstddjb/djbunix.html @@ -0,0 +1,760 @@ + + + + + skalibs: the djbunix library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The djbunix library interface

+ +

+ The following functions are declared in the skalibs/djbunix.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ djbunix is an alternative API to management of basic Unix +concepts: file descriptors, files, environment, and so on. It is a +rather chaotic mix of safe wrappers +around Unix system calls, better reimplementations of standard libc +functionalities, and higher-level manipulations of Unix concepts. +

+ +

+ Understanding djbunix is essential to understanding any piece +of code depending on skalibs. +

+ +

Functions

+ +

Basic fd operations

+ +

+ int coe (int fd)
+Sets the close-on-exec flag on fd. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int uncoe (int fd)
+Clears the close-on-exec flag on fd. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int ndelay_on (int fd)
+Sets the O_NONBLOCK flag on fd: sets it to non-blocking mode. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int ndelay_off (int fd)
+Clears the O_NONBLOCK flag on fd: sets it to blocking mode. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int pipenb (int *p)
+Like +pipe(), +but both ends of the created pipe are in non-blocking mode. +

+ +

+ int pipecoe (int *p)
+Like +pipe(), +but both ends of the created pipe are close-on-exec. +

+ +

+ int pipenbcoe (int *p)
+Like +pipe(), +but both ends of the created pipe are in non-blocking mode and close-on-exec. +

+ +

+ int fd_copy (int to, int from)
+Copies the open fd from to number to. to +must not refer to an already open fd. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int fd_copy2 (int to1, int from1, int to2, int from2)
+Copies the open fd from1 to number to2. Also copies +from2 to to2 at the same time. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int fd_move (int to, int from)
+Moves the open fd from to number to. to +must not refer to an already open fd, unless it's equal to from. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int fd_move2 (int to1, int from1, int to2, int from2)
+Moves the open fd from to number to. Also moves +from2 to to2 at the same time. This is useful for instance +when you want to swap two fds: fd_move2 will handle the situation +correctly. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int fd_close (int fd)
+Closes fd. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +This is a safe wrapper around +close(), +or rather as safe a wrapper as is possible to write: the close() +specification does not allow a 100% safe behaviour. So, in rare cases +it is possible for fd_close() to return 0 (instead of -1 EBADF) +when it is provided an argument that is not an open fd. This should not +be a problem, because giving wrong arguments to fd_close() is +always a static programming error. +

+ +

+ int fd_chmod (int fd, unsigned int mode)
+Safe wrapper around +fchmod(). +

+ +

+ int fd_chown (int fd, unsigned int uid, unsigned int gid)
+Safe wrapper around +fchown(). +This function requires root privileges. +

+ +

+ int fd_sync (int fd)
+Safe wrapper around +fsync(). +

+ +

+ int fd_chdir (int fd)
+Safe wrapper around +fchdir(). +

+ +

+ int fd_cat (int from, int to)
+Synchronously copies data from fd from to fd to, +until it encounters EOF or an error. Returns -1 (and sets errno) if +it fails; returns the number of transmitted bytes if it gets an EOF. +

+ +

+When the underlying OS allows it, zero-copy transmission is +performed. Currently, the following zero-copy implementations are +supported: +

+ + + +

+ unsigned int fd_catn (int from, int to, unsigned int n)
+Synchronously copies at most n bytes from fd from to fd to. +Returns the total number of transmitted bytes; sets errno if this number +is lesser than n. EOF is reported as EPIPE. See above for zero-copy +transmission; zero-copy transmission is not attempted for less than 64k of data. +

+ +

+ int fd_ensure_open (int fd, int w)
+If fd is not open, opens it to /dev/null, +for reading if w is zero, and for writing otherwise. +Returns 1 if it succeeds and 0 if it fails. +

+ +

+ int fd_sanitize (void)
+Ensures stdin and stdout are open. If one of those +file descriptors was closed, it now points to /dev/null. +Returns 1 if it succeeds and 0 if it fails. +

+ +

+ int lock_ex (int fd)
+Gets an exclusive advisory lock on fd. fd must point to +a regular file, open for writing. Blocks until the lock can be obtained. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int lock_exnb (int fd)
+Gets an exclusive advisory lock on fd. fd must point to +a regular file, open for writing. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. If the lock +is held and the function would block, it immediately returns with -1 EWOULDBLOCK. +

+ +

+ int lock_sh (int fd)
+Gets a shared advisory lock on fd. fd must point to +a regular file, open for reading. Blocks until the lock can be obtained. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int lock_shnb (int fd)
+Gets a shared advisory lock on fd. fd must point to +a regular file, open for reading. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. If the lock +is held and the function would block, it immediately returns with -1 EWOULDBLOCK. +

+ +

+ int lock_un (int fd)
+Releases a previously held lock on fd. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int open2 (char const *file, unsigned int flags)
+Safe wrapper around +open() +when it takes 2 arguments. +

+ +

+ int open3 (char const *file, unsigned int flags)
+Safe wrapper around +open() +when it takes 3 arguments. +

+ +

+ int open_read (char const *file)
+Opens file in read-only, non-blocking mode. +Returns a valid fd number if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int open_readb (char const *file)
+Opens file in read-only, blocking mode. +Returns a valid fd number if it succeeds, or -1 (and sets errno) if it fails. +This call does not block. The +open() +system call is actually performed with the O_NONBLOCK option, and blocking mode +is set afterwards; this behaviour allows for more transparent interactions +with FIFOs. +

+ +

+ int open_excl (char const *file)
+Opens file in write-only, non-blocking mode, with the +additional O_EXCL and O_CREAT flags. +Returns a valid fd number if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int open_append (char const *file)
+Opens file in write-only, non-blocking mode, with the +additional O_APPEND and O_CREAT flags. +Returns a valid fd number if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int open_trunc (char const *file)
+Opens file in write-only, non-blocking mode, with the +additional O_TRUNC and O_CREAT flags. +Returns a valid fd number if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int open_create (char const *file)
+Opens file in write-only, non-blocking mode, with the +additional O_CREAT flag. +Returns a valid fd number if it succeeds, or -1 (and sets errno) if it fails. +

+ +

+ int open_write (char const *file)
+Opens file in write-only, non-blocking mode. +Returns a valid fd number if it succeeds, or -1 (and sets errno) if it fails. +

+ +

Seek operations

+ +

+ long seek_cur (int fd)
+Returns the current file offset for descriptor fd. +

+ +

+ int seek_set (int fd, long pos)
+Sets the current file offset for fd to pos. +Returns 0 if it succeeds, or -1 (and sets errno) if it fails. +

+ +

Privilege management

+ +

+ int prot_readgroups (char const *name, gid_t *tab, unsigned int max)
+Reads the group database (normally /etc/group, but it can be +altered via NSS) to get the list of supplementary groups for user name. +Stores that list into the array pointed to by tab, which must be +preallocated. Stores at most max elements into tab. +Returns -1 and sets errno if it fails; else, returns the number of elements actually +stored into tab. +

+ +

+ int prot_grps (char const *name)
+Sets the kernel-maintained list of supplementary groups for the current process +to the list of supplementary groups for user name according to the +group database. This is a privileged operation. +Returns -1 and sets errno if it fails; returns 0 if it succeeds. +

+ +

+ int prot_gid (int gid)
+Alias to setgid. +

+ +

+ int prot_uid (int uid)
+Alias to setuid. +

+ +

Executable search and execution, and environment

+ +

+ void execvep (char const *file, char const *const *argv, char const *const *envp, char const *path)
+Executes into the executable file at file, with the command line +set to argv and the environment set to envp. +If file is not an absolute path, it is searched in the +path string, which must contain a colon-separated list of +search directories such as the contents of the PATH environment variable. +The function returns if it fails, and sets errno to the most relevant +error that happened. +

+ +

+ void pathexec_run (char const *file, char const *const *argv, char const *const *envp)
+Performs execvep(file, argv, envp, path), path being the +contents of the PATH environment variable. If PATH is not set, path +is set to the contents of the conf-compile/conf-defaultpath file in +the skalibs distribution. +The function returns if it fails, and sets errno appropriately. +

+ +

+ pathexec_run() is the standard skalibs API to perform an +exec call with a path search. It is recommended that you use +it instead of the Single Unix +execvp() or +execlp() +functions, because execvp and execlp default to execution of +the /bin/sh interpreter with file as an argument if they +cannot find a suitable executable file, and this is: +

+ +
    +
  1. a security risk,
    2. +
  2. probably not what you want.
    3. +
+ +

+ execvep() and pathexec_run() just fail with ENOENT +when they cannot find a file to exec into, which is the +sensible behaviour. +

+ +

+ void pathexec0_run (char const *const *argv, char const *const *envp)
+Performs pathexec_run(argv[0], argv, envp). If argv is empty, i.e. +argv[0] is null, the process exits 0 instead. Rationale: executing +the empty command line should amount to executing true, i.e. +simply exiting 0. +

+ +

+ void pathexec_r_name (char const *file, char const *const *argv, char const *const *envp, unsigned int envlen, char const *modifs, unsigned int modiflen)
+Alters envp (which does not have to be NULL-terminated, but the +number envlen of elements must be provided) with the modifier +string modifs of length modiflen, then performs +pathexec_run(file, argv, altered-envp). +

+ +

+ void pathexec_r (char const *const *argv, char const *const *envp, unsigned int envlen, char const *modifs, unsigned int modiflen)
+Same as pathexec_r_name, except that the file argument is read from argv[0]. +

+ +

+ int pathexec_env (char const *var, char const *value)
+Adds the "add variable var with value value" instruction +(if value is not null) or the "unset var" instruction +(if value is null) to a static hidden modifier string, used by the +following three functions. +Returns 1 if it succeeds and 0 (and sets errno) if it fails. +

+ +

+ void pathexec_fromenv (char const *const *argv, char const *const *envp, unsigned int envlen)
+Performs pathexec_r() with the given arguments and the hidden modifier +string. +

+ +

+ void pathexec (char const *const *argv)
+Executes into the argv command line, with the current environment +modified by the hidden modifier string. +

+ +

+ void pathexec0 (char const *const *argv)
+Executes into the argv command line, with the current environment +modified by the hidden modifier string. If this command line is empty, +exit 0 instead. +

+ +

+ The env library interface provides additional functions +to manipulate modifier strings and environments. +

+ +

Forking children

+ +

+ int doublefork ()
+Performs a double fork. Returns -1 if it fails (and +sets errno, EINTR meaning that the intermediate process +was killed by a signal), 0 if the current process is the grandchild, +and the grandchild's PID if the current process is the parent. +

+ +

+ pid_t child_spawn0 (char const *file, char const *const *argv, char const *const *envp)
+Forks and executes a child as with pathexec_run(file, argv, envp). +Returns 0 if it fails, and the pid of the child if it succeeds. +Implemented via posix_spawn() +on systems that support it. +

+ +

+ pid_t child_spawn1 (char const *file, char const *const *argv, char const *const *envp, int *fd, int w)
+Like child_spawn0(), except that a pipe is created between the child's +stdin (if w is 0) or stdout (if w is nonzero) and the parent. +The parent's end of the pipe will be stored in *fd. +

+ +

+ pid_t child_spawn (char const *file, char const *const *argv, char const *const *envp, int *fds, unsigned int nfds)
+More generic spawning function. fds must point to an array of at least nfds ints; +file descriptors reading from or writing to the child will be stored there. The function returns +0 on failure or the pid of the child on success. +

+ + +

Waiting for children

+ +

+ unsigned int wait_reap ()
+Instantly reaps all the pending zombies, without blocking, without a look at +the exit codes. +Returns the number of reaped zombies. +

+ +

+ int waitn (pid_t *pids, unsigned int n)
+Waits until all processes whose PIDs are stored in the +pids array, of size n, have died. +Returns 1 if it succeeds, and 0 (and sets errno) if it fails. The +pid array is not guaranteed to be unchanged. +

+ +

+ int waitn_reap (pid_t *pids, unsigned int n)
+Instantly reaps all zombies whose PIDs are stored in the +pids array, of size n. +Returns -1 (and sets errno) if it fails, and the number of reaped +zombies if it succeeds. The pid array is not guaranteed to +be unchanged. +

+ +

+ int wait_nohang (int *wstat)
+Instantly reaps one zombie, and stores the status information into +*wstat. +Returns the PID of the reaped zombie if it succeeds, 0 if there was +nothing to reap (and the current process still has children), -1 ECHILD +if there was nothing to reap (and the current process has no children), +or -1 (and sets errno) if it fails. +

+ +

+ int waitpid_nointr (pid_t pid, int *wstat, int flags)
+Safe wrapper around +waitpid(). +

+ +

+ int wait_pid_nohang (pid_t pid, int *wstat)
+Instantly reaps an undetermined number of zombies until it finds pid. +Stores the status information for dead pid into *wstat. +Returns pid if it succeeds, 0 if there was +nothing to reap (and the current process still has children), -1 ECHILD +if there was nothing to reap (and the current process has no children), +or -1 (and sets errno) if it fails. +

+ +

+ int wait_pids_nohang (pid_t const *pids, unsigned int len, int *wstat)
+Instantly reaps an undetermined number of zombies until it finds one whose +PID is in the pids array, of size len. +Stores the status information for that dead process into *wstat. +Returns the index of the found PID in pids, starting at 1. +Returns 0 if there was +nothing to reap (and the current process still has children), -1 ECHILD +if there was nothing to reap (and the current process has no children), +or -1 (and sets errno) if it fails. +

+ +

+ When asynchronously dealing with a child (resp. several children) and +getting a SIGCHLD - which should be handled via a +selfpipe - it is generally a good idea to +use the wait_pid_nohang() (resp. wait_pids_nohang()) +function over the basic Unix APIs. This allows a program to: +

+ + + +

Reading and writing whole files

+ +

+ int slurp (stralloc *sa, int fd)
+Slurps the contents of open descriptor fd into +the *sa stralloc. If you are +doing this, you should either have full control over the slurped +file, or run your process with suitable +limits +to the amount of heap memory it can get. +The function returns 1 if it succeeds, or 0 (and sets errno) if it fails. +

+ +

+ int openslurpclose (stralloc *sa, char const *file)
+Slurps the contents of file file into *sa. +Returns 1 if it succeeds, and 0 (and sets errno) if it fails. +

+ +

+ int openreadclose (char const *file, stralloc *sa, unsigned int dummy)
+Legacy interface for openslurpclose(sa, file). The dummy +argument is unused. Returns 0 if it succeeds, and -1 (and sets errno) if it fails. +

+ +

+ int openreadnclose (char const *file, char *s, unsigned int n)
+Reads at most n bytes from file file into preallocated +buffer s. Returns -1 (and sets errno) if it fails; else returns the +number of read bytes. If that number is not n, errno is set to EPIPE. +

+ +

+ int openreadfileclose (char const *file, stralloc *sa, unsigned int n)
+Reads at most n bytes from file file into the *sa +stralloc, which is grown (if needed) to just accommodate the file +size. Returns 1 if it succeeds and 0 (and sets errno) if it fails. +

+ +

+ int openwritenclose_unsafe_internal (char const *file, char const *s, unsigned int len, uint64 *dev, uint64 *ino, unsigned char dosync)
+Writes the n bytes stored at s into file file. +The previous contents of file are destroyed even if the function +fails. If dosync is nonzero, the new contents of file +are synced to disk before the function returns. If dev and ino +are not null, they're used to store the device and inode number of file. +The function returns 1 if it succeeds, or 0 (and sets errno) if it fails. +

+ +

+ int openwritenclose_unsafe (char const *file, char const *s, unsigned int len)
+int openwritenclose_unsafe_sync (char const *file, char const *s, unsigned int len)
+int openwritenclose_unsafe_devino (char const *file, char const *s, unsigned int len, uint64 *dev, uint64 *ino)
+int openwritenclose_unsafe_devino_sync (char const *file, char const *s, unsigned int len, uint64 *dev, uint64 *ino)
+Trivial shortcuts around openwritenclose_unsafe_internal(). The +reader can easily figure out what they do. +

+ +

+ int openwritenclose_suffix_internal (char const *file, char const *s, unsigned int len, uint64 *dev, uint64 *ino, unsigned char dosync, char const *suffix)
+Writes the n bytes stored at s into file file, +by first writing into filesuffix and atomically renaming +filesuffix to file. IOW, the old contents of file +are preserved if the operation fails, and are atomically replaced with the +new contents if the operation succeeds. +If dosync is nonzero, the new contents of filesuffix +are synced to disk before the atomic replace. If dev and ino +are not null, they're used to store the device and inode number of file. +The function returns 1 if it succeeds, or 0 (and sets errno) if it fails. +

+ +

+ int openwritenclose_suffix (char const *file, char const *s, unsigned int len, char const *suffix)
+int openwritenclose_suffix_sync (char const *file, char const *s, unsigned int len, char const *suffix)
+int openwritenclose_suffix_devino (char const *file, char const *s, unsigned int len, uint64 *dev, uint64 *ino, char const *suffix)
+int openwritenclose_suffix_devino_sync (char const *file, char const *s, unsigned int len, uint64 *dev, uint64 *ino, char const *suffix)
+Trivial shortcuts around openwritenclose_suffix_internal(). The +reader can easily figure out what they do. +

+ +

Filesystem deletion

+ +

+The following operations are not atomic, so if they fail, the +relevant subtree might end up partially deleted. +

+ +

+ int rm_rf (char const *path)
+Deletes the filesystem subtree at path. +Returns 0 if it succeeds or -1 (and sets errno) if it fails. +

+ +

+ int rm_rf_tmp (char const *path, stralloc *tmp)
+Deletes the filesystem subtree at path, using *tmp +as heap-allocated temporary space. +Returns 0 if it succeeds or -1 (and sets errno) if it fails. +

+ +

+ int rm_rf_in_tmp (stralloc *tmp, unsigned int n)
+Deletes a filesystem subtree, using *tmp +as heap-allocated temporary space. +Returns 0 if it succeeds or -1 (and sets errno) if it fails. +When the function is called, *tmp must contain the +null-terminated name of the subtree to delete at offset n. +

+ +

+ int rmstar (char const *dir)
+Deletes all the filesystem subtrees in directory dir. +Returns 0 if it succeeds or -1 (and sets errno) if it fails. +

+ +

+ int rmstar_tmp (char const *dir, stralloc *tmp)
+Deletes all the filesystem subtrees in directory dir, +using *tmp as heap-allocated temporary space. +Returns 0 if it succeeds or -1 (and sets errno) if it fails. +

+ +

Variable length wrappers around Single Unix calls

+ +

+ int sarealpath (stralloc *sa, char const *path)
+Resolves path into a symlink-free absolute path, appending +the result to the *sa +stralloc. +Returns 0 if it succeeds and -1 (and sets errno) if it fails. +

+ +

+ int sarealpath_tmp (stralloc *sa, char const *path, stralloc *tmp)
+Resolves path into a symlink-free absolute path, appending +the result to *sa. Uses *tmp as heap-allocated +temporary space. +Returns 0 if it succeeds and -1 (and sets errno) if it fails. +

+ +

+ int sabasename (stralloc *sa, char const *s, unsigned int len)
+Appends the basename of filename s (of length len) +to *sa. +Returns 1 if it succeeds and 0 (and sets errno) if it fails. +

+ +

+ int sadirname (stralloc *sa, char const *s, unsigned int len)
+Appends the dirname of filename s (of length len) +to *sa. +Returns 1 if it succeeds and 0 (and sets errno) if it fails. +

+ +

+ int sagetcwd (stralloc *sa)
+Appends the current working directory to *sa. +Returns 0 if it succeeds and -1 (and sets errno) if it fails. +

+ +

+ int sareadlink (stralloc *sa, char const *link)
+Appends the contents of symbolic link link to *sa. +Returns 0 if it succeeds and -1 (and sets errno) if it fails. +

+ +

+ int sagethostname (stralloc *sa)
+Appends the machine's hostname to *sa. +Returns 0 if it succeeds and -1 (and sets errno) if it fails. +

+ +

Temporization

+ +

+ void deepsleepuntil (tain_t const *deadline, tain_t *stamp)
+Sleeps until the absolute time represented by the +tain_t *deadline. *stamp +must contain the current time. When the function returns, *stamp +has been updated to reflect the new current time. +

+ +

+ void deepsleep (unsigned int n)
+Sleeps n seconds. Signals received during that time are handled, +but do not interrupt the sleep. +

+ +

+ void deepmillisleep (unsigned long n)
+Sleeps n milliseconds. Signals received during that time are handled, +but do not interrupt the sleep. +

+ + + diff --git a/doc/libstddjb/gccattributes.html b/doc/libstddjb/gccattributes.html new file mode 100644 index 0000000..6ee71ef --- /dev/null +++ b/doc/libstddjb/gccattributes.html @@ -0,0 +1,48 @@ + + + + + skalibs: the gccattributes header + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The skalibs/gccattributes.h header

+ +

+ skalibs/gccattributes.h is a set of wrappers around +gcc +attributes (duh). It defines macros that are always valid, and +that have no effect if the compiler is not gcc or the used version +of gcc does not support the wanted attribute. +

+ +

+ For instance: +

+ +
+ extern unsigned int str_len (char const *) gccattr_pure ;
+
+ +

+ defines the str_len function as pure if it is +supported. +

+ +

+ The source code is self-explanatory. +

+ + + diff --git a/doc/libstddjb/genalloc.html b/doc/libstddjb/genalloc.html new file mode 100644 index 0000000..b9ab934 --- /dev/null +++ b/doc/libstddjb/genalloc.html @@ -0,0 +1,46 @@ + + + + + skalibs: the genalloc library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The genalloc library interface

+ +

+ The following functions are declared in the skalibs/genalloc.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ genalloc is the skalibs way of handling dynamic arrays, i.e. +dynamically growing arrays of fixed-size objects. Any array that needs +to be stored in heap memory can be implemented via genalloc. +

+ +

+ Most genalloc functions are just macro calls around +stralloc functions. +

+ +

+ The genalloc.h header is actually very simple and the +prototypes there are self-explaining. +

+ + + diff --git a/doc/libstddjb/genwrite.html b/doc/libstddjb/genwrite.html new file mode 100644 index 0000000..33ff4d3 --- /dev/null +++ b/doc/libstddjb/genwrite.html @@ -0,0 +1,98 @@ + + + + + skalibs: the genwrite library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The genwrite library interface

+ +

+ The following functions are declared in the skalibs/genwrite.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ genwrite is syntactic sugar to help write functions that might +want to write either to memory or to a file descriptor. +

+ +

+ Writing to memory is achieved via appending to a +stralloc; writing to a file descriptor is achieved +via appending to a buffer or a +bufalloc. +

+ +

Usage

+ +

+ A genwrite_t structure contains a pointer to a function that writes +stuff to the target without flushing it +(which can be genwrite_put_stralloc, genwrite_put_buffer, +genwrite_put_bufalloc or any +compatible user-defined function) in .put, a pointer to a function +that flushes the target (which can be genwrite_flush_stralloc, +genwrite_flush_buffer, genwrite_flush_bufalloc or any +compatible user-defined function) in .flush, and a pointer to +the target writing structure in .target. +

+ +

+ Users should define a genwrite_t first, using the provided functions, +and give applications a pointer gp to this structure. To write len +characters at position s to the target, the application should then call +(*gp->put)(gp->target, s, len). When it is done writing, the +application should call (*gp->flush)(gp->target) to flush the +output. +

+ +

+ genwrite_stdout and genwrite_stderr are predefined; they +write to buffer_1 and buffer_2 respectively. +

+ +

Macros

+ +

+ GENWRITE_STRALLOC_INIT(sa)
+Declares a genwrite_t writing to the stralloc *sa. +

+ +

+ GENWRITE_BUFFER_INIT(b)
+Declares a genwrite_t writing to the buffer *b. Use +of such a buffer might interact badly with nonblocking I/O. +

+ +

+ GENWRITE_BUFALLOC_INIT(ba)
+Declares a genwrite_t writing to the bufalloc *ba. +

+ +

Note

+ +

+Object-oriented programming in C is inefficient and cumbersome. It is +usually possible to avoid it in Unix system programming, because Unix +primitives are often generic enough. Unfortunately, it is not the case +here: Unix does not provide an abstraction representing either a file +or a memory buffer. So an object-oriented approach is unavoidable. +

+ + + diff --git a/doc/libstddjb/index.html b/doc/libstddjb/index.html new file mode 100644 index 0000000..e0853cc --- /dev/null +++ b/doc/libstddjb/index.html @@ -0,0 +1,125 @@ + + + + + skalibs: the stddjb library interface + + + + + + +

+libskarnet
+skalibs
+Software
+www.skarnet.org +

+ +

The stddjb library interface

+ +

+ libstddjb is the base, and the most important part, of skalibs. +It is a set of general-purpose C functions wrapping some +system calls, hiding some Unix portability problems, providing some +basic low-level buffering functions and string handling, and generally +offering a nice API to Unix programming - in many ways nicer and safer +than the "standard" Unix APIs like stdio.h. +

+ +

+ It is mostly based on some excellent code written and placed into the +public domain by D. J. Bernstein. +

+ +

Compiling

+ + + +

Programming

+ + + +

+ The following headers are automatically generated at compile-time, when the +headers subsystem is made. The skalibs/stddjb.h file also +includes them. +

+ + + +

+ Additionally, stddjb.h also includes the following headers, which +are not associated with any code and are mostly self-explanatory: +

+ + + + + diff --git a/doc/libstddjb/iopause.html b/doc/libstddjb/iopause.html new file mode 100644 index 0000000..72eedde --- /dev/null +++ b/doc/libstddjb/iopause.html @@ -0,0 +1,196 @@ + + + + + skalibs: the iopause library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The iopause library interface

+ +

+ The following functions are declared in the skalibs/iopause.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ iopause is the skalibs API for event loop selection. It's a +wrapper around the system's +poll() +(if available) or +select() +(if poll() is unavailable) function. It +works around some system-dependent quirks; also it works with +absolute dates instead of timeouts. This is a good thing: +see below. +

+ +

+ iopause is a derived work from Dan J. Bernstein's +iopause library, but the +skalibs implementation is subtly different. +

+ +

Data structures

+ +

+ An iopause_fd structure is similar to a +struct pollfd +structure, and must be filled the same way. Usually, the user declares +an array of iopause_fd and fills it, one element per descriptor +to select on. If x is an iopause_fd: +

+ + + +

+ Unlike poll() or select(), which use a timeout +argument, the iopause() function uses a deadline argument, +i.e. an absolute time at which it must return 0 if no event has happened +so far, as well as a stamp argument, i.e. an absolute time meaning +now. Those arguments are stored in +tain_ts. Here is why: +

+ +

+ The event loop pattern is mostly used to multiplex several asynchronous +events that can happen independently, at the same time or not. Let's +say you have 3 events, x, y and z. Each of +those has a separate timeout: if x happens before x-timeout +milliseconds, you call the x-event-handler function, but +if x-timeout milliseconds elapse without x happening, +you call x-timeout-handler function. And similarly with y +and z. +

+ +

+ But the selection function returning does not mean x has happened +or that x has timed out. It might also mean that y has +happened, that y has timed out, that z has happened, that +z has timed out, or something else entirely. In the post-selection +part of the loop, the proper handler is called for the event or timeout +that has happened; then the loop is executed again, and in the +pre-selection part of the loop, the array describing the events is filled, +and the selection timeout is computed. +

+ +

+ How are you going to compute that global selection timeout? Easy: it's the +shortest of the three. But we just spent some amount of time waiting, so the +individual timeouts must be recomputed! This means: +

+ +

+ That is really cumbersome. A much simpler way of doing things is: +

+ + + +

+ Maintaining a global timestamp and using absolute times instead of relative +times really is the right way to work with event loops, and the iopause +interface reflects that. Of course, you need a reliable, bug-free time library +and a monotonic, constant system clock to handle absolute times correctly; +that is why iopause relies on the tai library. +

+ +

Functions

+ +

+ int iopause (iopause_fd *x, unsigned int len, tain_t const *deadline, tain_t const *stamp)
+Blocks until one of the events described in the x array, of length +len, happens, or until the absolute date *deadline is +reached. deadline may be null, in which case the function blocks +indefinitely until an event happens. If deadline is not null, then +stamp must not be null, and must contain an accurate estimation +of the current time. The function returns the number of events that have +happened, 0 for a timeout, or -1 (and sets errno) for an error. +

+ +

+ int iopause_stamp (iopause_fd *x, unsigned int len, tain_t const *deadline, tain_t *stamp)
+Like iopause(), but if stamp is not null, it is updated +right before the function returns. This helps the user always keep a +reasonably accurate estimation of the current time in stamp; +it is recommended to use this function instead of the lower-level +iopause(). +

+ +

Underlying implementations

+ +

+ iopause is an alias to either iopause_poll or +or iopause_select. By default, it is aliased to iopause_poll; to +alias it to iopause_select instead, configure skalibs with the +--enable-iopause-select option. +

+ +

+Both iopause_poll and iopause_select are implemented on top of the +ppoll() system call +if it is available; but if it is not, then iopause_poll defaults to +poll(), +which has a more comfortable API than +select(), +but a maximum precision of 1 millisecond which might not be enough for some applications; whereas +iopause_select defaults to select(), which incurs some CPU overhead for the +API conversion, but has a 1 microsecond precision. +

+ + + diff --git a/doc/libstddjb/ip46.html b/doc/libstddjb/ip46.html new file mode 100644 index 0000000..ee2f31c --- /dev/null +++ b/doc/libstddjb/ip46.html @@ -0,0 +1,172 @@ + + + + + skalibs: the ip46 library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The ip46 library interface

+ +

+ The following functions and structures are declared in the skalibs/ip46.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ ip46 is a set of macros and functions to support both IPv4 +and IPv6 network operations in an abstracted way. +

+ +

+ If skalibs has been built with the --disable-ipv6 +configure option, or it detects at build time than the target does not support IPv6, then +ip46 structures and functions will be directly aliased to their +IPv4 implementations with no overhead at all. +

+ +

Data structures

+ +

+ An ip46full_t is a structure that contains either an IPv4 or an IPv6 +address. + If a is an ip46full_t, then: +

+ +
    +
  • ip46_is6(&a) is 1 if a is +IPv6 and 0 otherwise.
    • +
  • a.ip points to 16 (if IPv6) or 4 (if IPv4) bytes containing +the address, in network byte order.
    • +
+ +

+ If skalibs has been build with IPv6 support, an ip46_t is +the same type as an ip46full_t. Otherwise, an ip46_t +is a structure that just contains an IPv4 address. +

+ +

Functions

+ +

+ int ip46_from_ip4 (ip46_t *a, char const *ip)
+Stores the IPv4 pointed to by ip into *a. Returns 1. +

+ +

+ int ip46_from_ip6 (ip46_t *a, char const *ip)
+Stores the IPv6 pointed to by ip into *a. Returns 1, +except if IPv6 is unavailable, in which case it returns 0 ENOSYS. +

+ +

+ unsigned int ip46_fmt (char *s, ip46_t const *a)
+Formats the address in *a into the string s, which +must be preallocated. Returns the number of bytes written. The address +will be accordingly formatted as IPv4 or IPv6. +

+ +

+ unsigned int ip46_scan (char const *s, ip46_t *a)
+Scans the string s for an IPv4 or IPv6 address. If it finds +one, writes it into *a and returns the number of bytes read. +If it cannot, returns 0. +

+ +

+ unsigned int ip46_scanlist (ip46_t *list, unsigned int max, char const *s, unsigned int *n)
+Scans the string s for a list of comma-, semicolon-, space-, tab- or +newline-separated IPv4 or IPv6 addresses, up to a maximum of max. It +stores them into the (preallocated) ip46_t array pointed to by list. +It returns the number of bytes read (0 if s does not contain a valid +IP list at all), and stores the number of found and scanned addresses into *n. +

+ +

+ int socket_connect46 (int fd, ip46_t *a, uint16 port)
+Connects the socket fd to address *a and port port. +Returns 0 in case of success, and -1 (and sets errno) in case of failure. +

+ +

+ int socket_bind46 (int fd, ip46_t *a, uint16 port)
+Binds the socket fd to address *a and port port. +Returns 0 in case of success, and -1 (and sets errno) in case of failure. +

+ +

+ int socket_bind46_reuse (int fd, ip46_t *a, uint16 port)
+Same as the previous function, with the SO_REUSEADDR option. +

+ +

+ int socket_deadlineconnstamp46 (int fd, ip46_t const *a, uint16 port, tain_t const *deadline, tain_t *stamp)
+Attempts to synchronously connect the socket fd to address aa +and port port. Returns 1 if it succeeds and 0 (and sets errno) +if it fails. stamp must contain an accurate enough +timestamp, and is updated when the function returns. If the connection is +still pending by deadline, then the attempt stops and the function +returns 0 ETIMEDOUT. +

+ +

+ int socket_recv46 (int fd, char *s, unsigned int len, ip46_t *a, uint16 *port)
+Reads a datagram from socket fd. The message is stored into buffer s +of max length len, and stores the sender information into address *a +and port *port. Returns the length of the read datagram, or -1 if it fails. +

+ +

+ int socket_send46 (int fd, char const *s, unsigned int len, ip46_t const *a, uint16 port)
+Writes a datagram to socket fd. The message is read from buffer s +of length len, and the recipient information is address *a +and port port. Returns the number of written bytes, or -1 if it fails. +

+ +

+ int socket_local46 (int fd, ip46_t *a, uint16 *port)
+Gets the local information about bound socket fd: the local IP +address is stored into *a and the local port into *port. +Returns 0 in case of success, and -1 (and sets errno) in case of failure. +

+ +

+ int socket_remote46 (int fd, ip46_t *a, uint16 *port)
+Gets the peer information about connected socket fd: the remote IP +address is stored into *a and the remote port into *port. +Returns 0 in case of success, and -1 (and sets errno) in case of failure. +

+ +

+ int socket_recvnb46 (int fd, char *s, unsigned int len, ip46_t *a, uint16 *port, +tain_t const *deadline, tain_t *stamp)
+Like socket_recv46, except that the function blocks until a datagram +is received. *stamp must be an accurate enough approximation of the +current time, and is updated when the function returns. If no datagram has +arrived by absolute date *deadline, the function returns -1 ETIMEOUT. +

+ +

+ int socket_sendnb46 (int fd, char const *s, unsigned int len, ip46_t const *a, uint16 port, +tain_t const *deadline, tain_t *stamp)
+Like socket_send46, except that the function blocks until a datagram +has been effectively sent. *stamp must be an accurate enough approximation of the +current time, and is updated when the function returns. If the message still has +not been sent by absolute date *deadline, the function returns -1 ETIMEOUT. +

+ + + diff --git a/doc/libstddjb/lolstdio.html b/doc/libstddjb/lolstdio.html new file mode 100644 index 0000000..e059f24 --- /dev/null +++ b/doc/libstddjb/lolstdio.html @@ -0,0 +1,91 @@ + + + + + skalibs: the lolstdio library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The lolstdio library interface

+ +

+ The following functions are declared in the skalibs/lolstdio.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ lolstdio is a set of convenience functions providing +printf-style +formatting but interacting with buffers or +bufallocs instead of stdio FILEs. +

+ +

+ Like any printf-style functions, the lolstdio functions are rather +complex and inefficient, and not recommended for general use; they are +provided as a quick and dirty way to debug or test things. Programmers +are advised to use the type-specific formatting +functions instead in production-quality code. +

+ +

+ Be aware that functions writing into buffers interact badly with +non-blocking fds (and asynchronism in general) - just as you cannot +use FILEs with non-blocking output. Functions writing into bufallocs, +however, are fine, because bufallocs are much more suited to asynchronous +writing than fixed-size buffers or FILEs are. +

+ +

+ The current lolstdio implementation relies on the libc's +vsnprintf +function. +

+ +

Functions

+ +

+ int vbprintf (buffer *b, char const *format, va_list args)
+Like vfprintf +except that the result is written to the buffer b. +

+ +

+ int bprintf (buffer *b, char const *format, ...)
+Like fprintf +except that the result is written to the buffer b. +

+ +

+ int lolprintf (char const *format, ...)
+Like printf +except that the result is written to the buffer buffer_1. +

+ +

+ int vbaprintf (bufalloc *ba, char const *format, va_list args)
+Like vfprintf +except that the result is written to the bufalloc ba. +

+ +

+ int baprintf (bufalloc *ba, char const *format, ...)
+Like fprintf +except that the result is written to the bufalloc ba. +

+ + + diff --git a/doc/libstddjb/safewrappers.html b/doc/libstddjb/safewrappers.html new file mode 100644 index 0000000..6d889d6 --- /dev/null +++ b/doc/libstddjb/safewrappers.html @@ -0,0 +1,91 @@ + + + + + skalibs: safe wrappers + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

Safe wrappers

+ +

+ Lots of functions in libstddjb, declared for instance in +allreadwrite.h or +djbunix.h, are just "safe wrappers" +around corresponding system functions. For instance, +fd_read() is a safe wrapper around the system read() +function. +

+ +

The problem

+ +

+ Quite a lot of system calls are defined by +The +Open Group Base Specifications as interruptible: when the process is in +the middle of such a system call and receives a signal that it does not +ignore, the system call immediately returns -1 EINTR (after the signal +handler, if any, has been executed). +

+ +

+ This means that the intended execution of the process is at the mercy +of a stray signal. If a signal happens at the wrong time, a system call +fails when it could have succeeded. This is not acceptable. +

+ +

The solution

+ +

+ So, in order to be perfectly reliable, when a program makes an interruptible +system call, it must check whether the return value is -1 EINTR, +and restart the system call if it is the case. This is annoying to write; +so, libstddjb provides small wrappers around interruptible system +calls, so that programmers can just call those safe wrappers and +never bother with this again. +

+ +

+ The performance loss from having a wrapper layer is totally negligible +compared to the cost of using a system call in the first place. +

+ +

But isn't it what the SA_RESTART flag is meant to address?

+ +

+ Yes, it is. Unfortunately, SA_RESTART only protects interruptible +system calls from signals you actually have control over, and set a +handler for with +sigaction(). +This is not enough. You cannot decide that every signal sent +to your process should have SA_RESTART behaviour; and the Single Unix +specification says nothing about signals you do not control. For instance, +you cannot trap SIGSTOP; SIGSTOP does not kill your process, which +should resume flawlessly at the next SIGCONT; and according to the +specification, it is valid for SIGSTOP and SIGCONT to not +have SA_RESTART behaviour. So if you get a SIGSTOP while performing +an interruptible system call, that system call may return -1 EINTR, +this is not an OS bug, and there's nothing you can do about it with +sigaction(). +

+ +

+ SA_RESTART is only a partial solution: in other words, it doesn't work. +Until the Single Unix specification explicitly states that untrapped +non-lethal signals MUST have SA_RESTART behaviour by default, you +need safe wrappers to protect interruptible system calls. +

+ + + diff --git a/doc/libstddjb/selfpipe.html b/doc/libstddjb/selfpipe.html new file mode 100644 index 0000000..7eff430 --- /dev/null +++ b/doc/libstddjb/selfpipe.html @@ -0,0 +1,242 @@ + + + + + skalibs: the selfpipe library interface + + + + + + +

+libstddjb
+skalibs
+skalibs
+Software
+skarnet.org +

+ +

The selfpipe library interface

+ +

+ The selfpipe functions are declared in the +skalibs/selfpipe.h header and implemented in the libskarnet.a +or libskarnet.so library. +

+ +

What does it do ?

+ +

+Signal handlers suck. +

+ +

+They do. I don't care how experienced you are with C/Unix programming, +they do. You can be Ken Thompson, if you use signal handlers as a +regular part of your C programming model, you are going to +screw up, and write buggy code. +

+ +

+ Unix is tricky enough with interruptions. Most of libstddjb's wrappers +are there to protect system calls from EINTR. (And no, the SA_RESTART +option in sigaction() isn't protection +enough.) But signal handlers are +more than just pesky interruptions: they can totally change the +execution flow. They mess up the logic of linear and structured code, +they introduce non-determinism; you always have to think "and what +if I get interrupted here and the flow goes into a handler...". This +is annoying. +

+ +

+ Moreover, signal handler code is very limited in what it can +do. It can't use any non-reentrant function! If you call a non-reentrant +function, and by chance you were precisely in that non-reentrant function +code when you got interrupted by a signal... you lose. That means, no +malloc(). No bufferized IO. No globals. The list goes on and on.
+ If you're going to catch signals, you'll want to handle them outside +the signal handler. You actually want to spend the least possible +time inside a signal handler - just enough to notify your main +execution flow that there's a signal to take care of. +

+ +

+ And, of course, signal handlers don't mix with event loops, which is +a classic source of headaches for programmers and led to the birth of +abominations such as + +pselect. So much for the "everything is a file" concept that Unix was +built on. +

+ +

+ A signal should be an event like any other. +There should be a unified interface - receiving a signal should make some +fd readable or something. +

+ +

+ And that's exactly what the +self-pipe trick, invented +by DJB, does. +

+ +

+ As long as you're in some kind of event loop, the self-pipe trick allows +you to forget about signal handlers... forever. It works this way: +

+ +
    +
  1. Create a pipe p. Make both ends close-on-exec and nonblocking.
    2. +
  2. Write a tiny signal handler ("top half") for all the signals you want to +catch. This +signal handler should just write one byte into p[1], and do nothing +more; ideally, the written byte identifies the signal.
    3. +
  3. In your event loop, add p[0] to the list of fds you're watching +for readability.
    4. +
+ +

+ When you get a signal, a byte will be written to the self-pipe, and your +execution flow will resume. When you next go through the event loop, +p[0] will be readable; you'll then be able to read a byte from +it, identify the signal, and handle it - in your unrestricted main +environment (the "bottom half" of the handler). +

+ +

+ The selfpipe library does it all for you - you don't even have to write +the top half yourself. You can forget their existence and recover +some peace of mind. Of course, you still need to protect your +system calls against EINTR: the self-pipe trick doesn't prevent signals +from happening. +

+ +

How do I use it ?

+ +

Starting

+ +
+int fd = selfpipe_init() ;
+
+ +

+selfpipe_init() sets up a selfpipe. You must use that +function first.
+If fd is -1, then an error occurred. Else fd is a +non-blocking descriptor that can be used in your event loop. It will +be selected for readability when you've caught a signal. +

+ +

Trapping/untrapping signals

+ +
+int r = selfpipe_trap(SIGTERM) ;
+
+ +

+selfpipe_trap() catches a signal and sends it to the selfpipe. +Uncaught signals won't trigger the selfpipe. r is 0 if +the operation succeeded, and -1 if it failed. If it succeeded, you +can forget about the trapped signal entirely.
+In our example, if r is 0, then a SIGTERM will instantly +trigger readability on fd. +

+ +
+int r = selfpipe_untrap(SIGTERM) ;
+
+ +

+Conversely, selfpipe_untrap() uncatches a signal; the selfpipe +will not manage it anymore. r is 0 if the operation succeeded +and -1 if it failed. +

+ +
+int r ;
+sigset_t set ;
+sigemptyset(&set) ;
+sigaddset(&set, SIGTERM) ;
+sigaddset(&set, SIGHUP) ;
+r = selfpipe_trapset(&set) ;
+
+ +

+selfpipe_trap() and selfpipe_untrap() handle signals one +by one. Alternatively (and often preferrably), you can use +selfpipe_trapset() to directly handle signal sets. When you call +selfpipe_trapset(), signals that are present in set will +be caught by the selfpipe, and signals that are absent from set +will be uncaught. r is 0 if the operation succeeded and -1 if it +failed. +

+ +

Handling events 

+ +
+int c = selfpipe_read() ;
+
+ +

+ Call selfpipe_read() when your fd is readable. +That's where you write your real signal handler: in the +body of your event loop, in a "normal" context.
+c is -1 if an error occurred - in which case chances are +it's a serious one and your system has become very unstable. +c is 0 if there are no more pending signals. If c +is positive, it is the number of the signal that was caught. +

+ +

Finishing

+ +
+selfpipe_finish() ;
+
+ +

+ Call selfpipe_finish() when you're done using the selfpipe. +Signal handlers will be restored to their previous value. +

+ +

Any limitations ?

+ +

+ Some, as always. +

+ +
    +
  • The selfpipe library uses a global pipe; +so, it's not safe for multithreading. I'm not sure how multithreaded +programs handle signals; I personally don't like multithreading and +never use it, so I'm not knowledgeable about it. Anyway, if your +program is multithreaded, chances are you don't have an asynchronous +event loop, so the self-pipe trick has less benefits for you.
    • +
  • In rare cases, the self-pipe can theoretically be filled, if some +application sends more than PIPE_BUF signals before you have time to +selfpipe_read(). On most Unix systems, PIPE_BUF is 4096, +so it's a very acceptable margin. Unless your code is waiting where +it should not be, only malicious applications will fill the self-pipe +- and malicious applications could just send you a SIGKILL and be done +with you, so this is not a concern. Protect yourself from malicious +applications with clever use of uids.
    • +
+ +

Hey, Linux has signalfd() for this !

+ +

+ Yes, the Linux team loves to gratuitously add new system calls to do +things that could already be done before without much effort. This +adds API complexity, which is not a sign of good engineering. +

+ +

+ However, now that signalfd() exists, it is indeed marginally more +efficient than a pipe, and it saves one fd: so the selfpipe library +is implemented via signalfd() when this call is available. +

+ + + diff --git a/doc/libstddjb/stralloc.html b/doc/libstddjb/stralloc.html new file mode 100644 index 0000000..a5a1c7e --- /dev/null +++ b/doc/libstddjb/stralloc.html @@ -0,0 +1,118 @@ + + + + + skalibs: the stralloc library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The stralloc library interface

+ +

+ The following functions are declared in the skalibs/stralloc.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ stralloc is the preferred skalibs way of storing objects into +heap memory. It focuses on strings of char, which is the generic +way to handle any object. For easy structure manipulation, the +genalloc +series of functions can be used; those functions are mostly macros wrapping +calls to their stralloc counterparts. +

+ +

+ A stralloc is a structure containing the following fields: +

+ +
    +
  • s: a pointer to a zone of heap memory. The stralloc +functions internally manipulate those via the +alloc series of functions. It is recommended +to never modify that field manually.
    • +
  • len: the used length of the +allocated zone. It is the only field that the user can modify +directly.
    • +
  • a: the number of allocated bytes. The user should never +have to access that field, because the memory allocation management +should be entirely transparent. len cannot exceed a: +if an operation needs a bigger len, it will automatically +reallocate as needed.
    • +
+ +

+ The benefits of using stralloc are as follows: +

+ +
    +
  • Memory allocation is performed on-demand and automatically, with +a suitable size. Heuristics are used to accommodate constantly growing +strings while minimizing the amount of needed reallocations.
    • +
  • If every heap-allocated object is represented by a stralloc, then +it is very easy to identify what pointer is in the heap. When you stop +using a pointer p, should you free it ? Sometimes it's not +easy to find out. When you stop using a stralloc sa, you +know you must call stralloc_free(&sa). Store +your strong references as strallocs and weak references as simple +pointers, and never free simple pointers. This policy allows me to +boast that no skarnet.org software has ever leaked memory.
    • +
  • Repeated for emphasis: + the golden rule for programming with strallocs is +every pointer to the heap must be owned by a stralloc. +Every pointer you handle yourself either does not point to the heap, +or is weak. That sometimes implies unusual programming practices, such +as having storage separated from structure, but it's hands down the +easiest way to keep control of your heap in complex situations.
    • +
  • The indirection layer makes weak references immune to +reallocation troubles. The s field may change when a +reallocation happens, but the stralloc structure's address never +changes.
    • +
+ +

+ A stralloc can be declared anywhere: static/global data, stack or heap. (Of course, +as a general rule, you should favor the stack whenever possible.) +A stralloc should be initialized to STRALLOC_ZERO before its first use. +

+ +

Functions

+ +

+ int stralloc_catb (stralloc *sa, char const *s, unsigned int len)
+Appends the len bytes pointed to by s to the end of the +memory zone handled by *sa, automatically allocating more memory +if needed. Returns 1 if it succeeds, and 0 if it fails. +

+ +

+ void stralloc_free (stralloc *sa)
+Frees *sa, i.e. calls alloc_free +on sa→s then zeroes the structure. *sa is +then reusable. However, it is not good practice to call this function +if you're going to reuse *sa soon: it takes time and causes +memory fragmentation. Just setting sa→len to 0 allows +you to instantly reuse the allocated block of memory. +

+ +

+ The above are the most important and fundamental functions of +skalibs/stralloc.h. Other functions can be found in this header and +their prototypes are self-explaining. +

+ + + diff --git a/doc/libstddjb/tai.html b/doc/libstddjb/tai.html new file mode 100644 index 0000000..3524c05 --- /dev/null +++ b/doc/libstddjb/tai.html @@ -0,0 +1,462 @@ + + + + + skalibs: the tai library interface + + + + + + +

+libstddjb
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The tai library interface

+ +

+ The following functions are declared in the skalibs/tai.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ tai is a set of data structures and primitives to represent +and perform operations on time. +

+ +

+ The point of tai is to handle time without ever having to +deal with annoyances such as Y2K, Y2038, NTP limits, non-linear +clocks, and the like. By using the tai interface, you ensure +your program will behave properly no matter what. +

+ +

What is the problem ?

+ +

+ The standard APIs for time management under Unix are broken in more +or less subtle ways. The most obvious thing is that they do not pass +year 2038. A less obvious problem is that they do not handle leap +seconds correctly. Here are a few references you should read to +understand what is going on: +

+ + + +

+ The meat and potatoes of all this is that programmers cannot simply rely on +standard Unix APIs such as +gettimeofday() +(which, by the way, is marked as obsolescent, but it's not going to disappear tomorrow) +to measure time intervals or even to give precise absolute time, and in +any case those APIs will become obsolete in 2038. +

+ +

So what does tai do ?

+ +

+ tai implements - among other things - the +TAI64 and TAI64N +formats, which are used in all of skalibs. This gives a programmer access +to precise linear absolute time, which is suitable for both +timestamping (wallclock usage) and time interval measurements +(stopwatch usage). Additionally, TAI64 passes Y2038 (it can +represent dates exceeding the estimated lifespan of the universe). +

+ +

+ tai has been inspired by Dan J. Bernstein's +libtai library, but does not +borrow any code from it. +

+ +

Data structures

+ +

+ A tai_t structure holds an absolute date with a one-second +precision. A tain_t structure holds an absolute date with a +maximum of one-nanosecond precision, as permitted by the underlying system +call. If flag-usert is clear, the system +clock will be read via +gettimeofday() +system call, which has a one-microsecond precision; if it is set, the +system clock will be read via the +clock_gettime() +system call, which has a one-nanosecond precision. In either case, a current +tain_t will be unable to be more precise than the underlying +implementation. +

+ +

+ A tai_t, as well as a tain_t, can also +hold a (possibly negative) relative time, i.e. a difference of absolute +dates. It is up to the programmer to make sure that a relative time is +never interpreted as an absolute TAI64 date, and vice-versa. +

+ +

The leap second table

+ +

+ skalibs provides a src/etc/leapsecs.dat file, +which is copied to /etc/leapsecs.dat at installation time +(or wherever you specified with the --prefix or --datadir +options to configure). +Make sure this file is always present and readable. +This file contains the leap second table, which is needed for +conversions between TAI and UTC. If you call a function that needs such +a conversion (for instance, you call tain_sysclock() and your +system clock is set to UTC) and the file cannot be read, the function +call will fail. +

+ +

+ The leap second table is read once in every process that needs it +(the first time a TAI ↔ UTC conversion is made) and then is +stored in memory. If the leapsecs.dat file changes, long-lived +processes will need to be restarted to take the change into account. +

+ +

Functions

+ +

Wallclock operations

+ +

+ int tai_now (tai_t *t)
+Writes the current time as a TAI value to *t, with a +1-second precision. The current time is based on the system clock. +Make sure skalibs has been compiled with or without the +--enable-tai-clock configure option according +to your system clock synchronization method: skalibs supports a +system clock set to TAI-10 or to UTC. +The function returns 1 if it succeeds, or 0 (and sets errno) if +it fails. +

+ +

+ int sysclock_get (tain_t *a)
+Reads the current value of the system clock into *a, with +a 1-nanosecond (resp. 1-microsecond ) precision if skalibs has been +configured with (resp. without) the +--enable-clock option. +Returns 1 if it succeeds or 0 (and sets errno) if it +fails. Note that despite being a tain_t, *a +does not contain a TAI value - it only contains +an internal, Y2038-safe representation of the value of the system +clock, which should be either TAI-10 or UTC. You should not use +this function directly unless you know exactly what you are doing. +

+ +

+ int sysclock_set (tain_t const *a)
+Sets the system clock to *a, provided *a has +the correct internal representation. You should not use this +function directly unless you know exactly what you are doing. +

+ +

+ int tain_sysclock (tain_t *a)
+Reads the current time into *a, as a TAI64N value, +with a 1-nanosecond (resp. 1-microsecond) precision if skalibs +has been configured with (resp. without) the +--enable-clock +option. Returns 1 if it succeeds or 0 (and sets errno) if it +fails. + Here a contains a valid TAI stamp, no matter what the +system clock is set to: arithmetic operations can be performed +on it. +

+ +

+ int tain_setnow (tain_t const *a)
+Sets the current time to *a, with a 1-nanosecond +(resp. 1-microsecond) precision if skalibs has been configured +with (resp. without) the +--enable-clock +option. Returns 1 if it succeeds or 0 (and sets errno) if it +fails. a must contain a valid TAI stamp; proper +operations will be automatically run to convert that stamp into +the right format for the system clock. +

+ +

Stopwatch operations

+ +

+ The following 3 operations are only defined if your system +provides the +clock_gettime() +primitive with the CLOCK_MONOTONIC option. +

+ +

+ int tain_clockmon_init (tain_t *offset)
+Initializes a stopwatch in *offset. The actual value of +*offset is meaningless to the user; offset's only +use is to be given as a second parameter to tain_clockmon(). +The function returns 1 if it succeeds or 0 (and sets errno) if it fails. +

+ +

+ What tain_clockmon_init() does is synchronize the "stopwatch +clock" (CLOCK_MONOTONIC) to the system clock. Right after +tain_clockmon_init() has been called, the absolute times given +by tain_clockmon() and tain_sysclock() are similar. Then, +depending on the accuracy of the system clock, a drift may appear; calling +tain_clockmon_init() again resets that drift to zero. +

+ +

+ int tain_clockmon (tain_t *a, tain_t const *offset)
+ Gives the absolute time, as a TAI64N value, in *a. This +absolute time is computed as a linear increase (as measured with +CLOCK_MONOTONIC) since the last time tain_clockmon_init() +was called with parameter offset. tain_clockmon() +guarantees precise time interval measurements; however, the time it +gives can slightly differ from the result of tain_sysclock(). +The function returns 1 if it succeeds or 0 (and sets errno) if it fails. +

+ +

All-purpose time reading

+ +

+ int tain_init (void)
+If skalibs has been configured with the +--enable-monotonic option: this +function initializes a process-global stopwatch, that future +tain_now invocations will depend on. +Without the --enable-monotonic option: this +function does nothing. +The function returns 1 if it succeeds or 0 (and sets errno) if it fails. +

+ +

+ int tain_now (tain_t *a)
+Writes the current time, as a TAI value, to *a. This is the +function you should use to read time by default. It returns 1 if it succeeds or +0 (and sets errno) if it fails. +

+ +

+ If skalibs has been configured with the +--enable-monotonic option: +tain_now() is computed as a linear increase from the last time +tain_init() was called. (If tain_init() has never +been called before, the first invocation of tain_now() +automatically calls tain_init().) + Without the --enable-monotonic option: +tain_now() is the same as tain_sysclock(). +

+ +

+ If the above is unclear to you: just use tain_now() +everytime you need to read time, and you will always get a reasonable +approximation of the current time, in a format suited for arithmetic +computations. +

+ +

Converting to/from libc representations

+ +

+ int tai_from_timeval (tai_t *t, struct timeval const *tv)
+int tai_from_timespec (tai_t *t, struct timespec const *ts)
+int tai_relative_from_timeval (tai_t *t, struct timeval const *tv)
+int tai_relative_from_timespec (tai_t *t, struct timespec const *ts)
+Those functions convert an absolute (resp. relative) time in a +struct timeval (resp. struct timespec) to an absolute (resp. relative) +time in a tai_t, with a 1-second precision. They return 1. +

+ +

+ int timeval_from_tai (struct timeval *tv, tai_t const *t)
+int timespec_from_tai (struct timespec *ts, tai_t const *t)
+int timeval_from_tai_relative (struct timeval *tv, tai_t const *t)
+int timespec_from_tai_relative (struct timespec *ts, tai_t const *t)
+Those functions do the opposite conversion. They normally return 1; +however, struct timeval and struct timespec cannot +represent an absolute date before the Epoch, or a negative relative time; +if *t cannot be converted, 0 EINVAL is returned. +

+ +

+ int tain_from_timeval (tain_t *a, struct timeval const *tv)
+int tain_from_timespec (tain_t *a, struct timespec const *ts)
+int tain_relative_from_timeval (tain_t *a, struct timeval const *tv)
+int tain_relative_from_timespec (tain_t *a, struct timespec const *ts)
+int timeval_from_tain (struct timeval *tv, tain_t const *a)
+int timespec_from_tain (struct timespec *ts, tain_t const *a)
+int timeval_from_tain_relative (struct timeval *tv, tain_t const *a)
+int timespec_from_tain_relative (struct timespec *ts, tain_t const *a)
+Same conversion operations, but with a tain_t. The 1-microsecond +(for struct timeval) or 1-nanosecond (for struct timespec) +precision is preserved. +

+ +

+ void tain_uint (tain_t *a, unsigned int c)
+Stores a relative time of c seconds into a. +

+ +

+ int tain_from_millisecs (tain_t *a, int ms)
+This function makes a tain_t representing a relative +time of ms milliseconds. ms must be non-negative. +The function returns 1, unless ms is negative, in which case +it returns 0 EINVAL. +

+ +

+ int tain_to_millisecs (tain_t const *a)
+If *a contains a non-negative relative time that fits into +a 31-bit integer number of milliseconds, the function returns that +number. Else it returns -1 EINVAL. +

+ +

Time computations

+ +

+ void tai_add (tai_t *t, tai_t const *t1, tai_t const *t2)
+Stores *t1 + *t2 into t. Of course, *t1 +and *t2 must not both represent absolute times. +

+ +

+ void tai_sub (tai_t *t, tai_t const *t1, tai_t const *t2)
+Stores *t1 - *t2 into t. Of course, *t1 +and *t2 must be of the same type (relative or absolute), and +*t will always be relative. +

+ +

+ void tain_add (tain_t *a, tain_t const *a1, tain_t const *a2)
+void tain_sub (tain_t *a, tain_t const *a1, tain_t const *a2)
+Same thing with tain_t. +

+ +

+ void tain_addsec (tain_t *a, tain_t const *a1, int c)
+Adds c seconds to *a1 and stores the result into a. +c may be negative. +

+ +

+ void tain_half (tain_t *a, tain_t const *b)
+Stores *b/2 into a. *b must be relative. +

+ +

Comparing dates or durations

+ +

+ int tai_less (tai_t const *t1, tai_t const *t2)
+int tain_less (tain_t const *t1, tain_t const *t2)
+Those functions return nonzero iff *t1 is lesser than *t2. +*t1 and *t2 must be both relative, or both absolute. +

+ +

Packing and unpacking

+ +

+ void tai_pack (char *s, tai_t const *t)
+Marshals *t into the buffer s points to, which +must be preallocated with at least TAI_PACK (8) characters. Afterwards, +the buffer contains the +external TAI64 format +representation of *t. +

+ +

+ void tai_unpack (char const *s, tai_t *t)
+Unmarshals the +external TAI64 format +label pointed to by s (at least TAI_PACK characters) and stores +the result into t. +

+ +

+ void tain_pack (char *s, tain_t const *a)
+void tain_unpack (char const *s, tain_t *a)
+void tain_pack (char *s, tain_t const *a)
+void tain_unpack (char const *s, tain_t *a)
+Same thing with +external TAI64N format, +using TAIN_PACK (12) characters. +

+ +

Formatting and scanning

+ +

+ unsigned int tain_fmt (char *s, tain_t const *a)
+Writes into s an ASCII representation of *a in external +TAI64N format. s must point to a preallocated buffer of at least +TAIN_PACK*2 (24) characters. The function returns the number of bytes that +have been written to s (24). +

+ +

+ unsigned int tain_scan (char const *s, tain_t *a)
+Reads 24 characters from s; if those characters are a valid ASCII +representation of the external TAI64N format of some time value, this value +is stored into a, and 24 is returned. Else 0 is returned. +

+ +

Timestamping

+ +

+ A TAI64N timestamp is a string of 25 characters: a single '@' +character followed by the ASCII representation of the TAI64N external +format of an absolute date. +

+ +

+ unsigned int timestamp_fmt (char *s, tain_t const *a)
+Writes a TAI64N timestamp representing the absolute date *a +into the 25 characters pointed to by s. Returns 25. +

+ +

+ unsigned int timestamp_scan (char const *s, tain_t *a)
+Reads 25 characters at s. If those characters are a valid TAI64N +timestamp, stores the absolute date in a and returns 25. Else, +returns 0. +

+ +

+ int timestamp (char *s)
+Writes the current time (read from the system clock) as a TAI64N timestamp +into s. Returns 1 if it succeeds or 0 (and sets errno) if it fails. +

+ +

+ TAI64N timestamps are an efficient, robust, and easy-to-use way of +timestampping log lines. They're easy to recognize in automatic data +parsers. Log files where every line starts with a TAI64N timestamp can +be merged and alphanumerically sorted: the resulting file will be +chronologically sorted. +

+ +

+ The s6 package +provides tools to convert TAI64N timestamps into human-readable +dates. Please do not embed human-readable dates in your log files, +thus making parsing tools unnecessarily hard to write; +use TAI64N timestamps instead, design tools that can parse them, +and translate them to human-readable form at human analysis time. +

+ + + diff --git a/doc/libunixonacid/index.html b/doc/libunixonacid/index.html new file mode 100644 index 0000000..81b024f --- /dev/null +++ b/doc/libunixonacid/index.html @@ -0,0 +1,58 @@ + + + + + skalibs: the unixonacid library interface + + + + + +

+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The unixonacid library interface

+ +

+libunixonacid provides higher-level interfaces to Unix +concepts such as the filesystem - for instance, it provides a way to +access several files atomically, be it for reading or for writing - or +interprocess communication. +

+ + +

Compiling

+ +
    +
  • Use #include <skalibs/unixonacid.h>
    • +
+ +

Programming

+ +

+ The skalibs/unixonacid.h header is actually a concatenation of other +headers, every one of each declaring related structures, macros and +functions. +

+ + + + + diff --git a/doc/libunixonacid/kolbak.html b/doc/libunixonacid/kolbak.html new file mode 100644 index 0000000..a7480e8 --- /dev/null +++ b/doc/libunixonacid/kolbak.html @@ -0,0 +1,40 @@ + + + + + skalibs: the kolbak library interface + + + + + + +

+libunixonacid
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The kolbak library interface

+ +

+ The following functions are declared in the skalibs/kolbak.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ kolbak implements simple queuing of callback functions to use when +sending a message to a peer and expecting an answer, which will be handled +by the callback function. The queue is a circular buffer. +

+ +

+ FIXME: To be completed. +

+ + + diff --git a/doc/libunixonacid/skaclient.html b/doc/libunixonacid/skaclient.html new file mode 100644 index 0000000..7deb38b --- /dev/null +++ b/doc/libunixonacid/skaclient.html @@ -0,0 +1,34 @@ + + + + + skalibs: the skaclient library interface + + + + + + +

+libunixonacid
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The skaclient library interface

+ +

+ The following functions are declared in the skalibs/skaclient.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+FIXME: to be completed. +

+ + + diff --git a/doc/libunixonacid/unix-timed.html b/doc/libunixonacid/unix-timed.html new file mode 100644 index 0000000..5261475 --- /dev/null +++ b/doc/libunixonacid/unix-timed.html @@ -0,0 +1,34 @@ + + + + + skalibs: the unix-timed library interface + + + + + + +

+libunixonacid
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The unix-timed library interface

+ +

+ The following functions are declared in the skalibs/unix-timed.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+FIXME: to be completed. +

+ + + diff --git a/doc/libunixonacid/unix-transactional.html b/doc/libunixonacid/unix-transactional.html new file mode 100644 index 0000000..036d8cd --- /dev/null +++ b/doc/libunixonacid/unix-transactional.html @@ -0,0 +1,34 @@ + + + + + skalibs: the unix-transactional library interface + + + + + + +

+libunixonacid
+libskarnet
+skalibs
+Software
+www.skarnet.org +

+ +

The unix-transactional library interface

+ +

+ The following functions are declared in the skalibs/unix-transactional.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+FIXME: to be completed. +

+ + + diff --git a/doc/libunixonacid/unixmessage.html b/doc/libunixonacid/unixmessage.html new file mode 100644 index 0000000..2a6c46d --- /dev/null +++ b/doc/libunixonacid/unixmessage.html @@ -0,0 +1,40 @@ + + + + + skalibs: the unixmessage library interface + + + + + + +

+libunixonacid
+libskarnet
+skalibs
+Software
+skarnet.org +

+ +

The unixmessage library interface

+ +

+ The following functions are declared in the skalibs/unixmessage.h header, +and implemented in the libskarnet.a or libskarnet.so library. +

+ +

General information

+ +

+ unixmessage implements message transmission over Unix domain sockets. +Messages are made of standard untyped data (strings), but can also include file +descriptors using fd-passing. +

+ +

+ FIXME: To be completed. +

+ + + diff --git a/doc/license.html b/doc/license.html new file mode 100644 index 0000000..21865c0 --- /dev/null +++ b/doc/license.html @@ -0,0 +1,82 @@ + + + + + skalibs: license + + + + + + +

+skalibs
+Software
+skarnet.org +

+ +

skalibs license

+ +

Source code

+ +

+The skalibs source code from skalibs is released under the +ISC license, +the text of which can be found in the COPYING file enclosed in the +package. +

+ +

+ The license has been chosen for its simplicity - it's short +and to the point - and above all its permissivity. There is nothing +legitimate you should not be able to do with the skalibs code. If +the license somehow stands in the way, please let me know. +

+ +

Documentation

+ +

+ However, the skalibs documentation, which you are currently +reading, and which is available in the doc/ subdirectory of +the official skalibs tarball, is not provided under +the same license. +

+ +

+ The license for the skalibs documentation is more restrictive than +the license for the source code. Namely: +

+ +
    +
  • The exact same disclaimer applies.
    • +
  • You have the right to download, use and modify the documentation.
    • +
  • You have the right to distribute unmodified copies of the documentation.
    • +
  • You have the right to distribute modified copies of the documentation, +provided that: +
      +
    • The contents and links (not necessarily the appearance) of this file, +license.html, are left unmodified;
      • +
    • On your documentation's main page, +you explicitly state that the documentation has been modified, and you +set up a +link to the original +skalibs site, explicitly stating that the original software and +documentation can be found there;
      • +
    • The parts of the documentation that you have modified are +unarguably and undeniably distinguishable from the unmodified parts. +You can, for instance, use a different text color, different background +color, or different text font.
      • +
    +
    • +
+ +

+ I am aware that the previous restrictions sound completely +ridiculous while the official skalibs documentation is incomplete. +As of 2.0.0.0, I'm not going to enforce those restrictions, but if you're +going to provide documentation for skalibs, don't keep it to yourself, +please send it to me instead. :-) +

+ + + diff --git a/doc/upgrade.html b/doc/upgrade.html new file mode 100644 index 0000000..70c14b9 --- /dev/null +++ b/doc/upgrade.html @@ -0,0 +1,34 @@ + + + + + skalibs: how to upgrade + + + + + + +

+skalibs
+Software
+skarnet.org +

+ +

Upgrade incompatibilities: notes for developers

+ +

From previous versions to 2.0.0.0

+ +
    +
  • No compatibility is ensured. Most APIs haven't changed, but no guarantee +is offered.
    • +
  • The most important change is probably the disparition of struct tai +and struct taia, replaced with the tai_t and tain_t +types. Attosecond precision has been removed - processor speed is almost capped now, +and it looks like nanosecond precision will be enough for the foreseeable future.
    • +
  • Programs should now link with -lskarnet followed by the appropriate +sysdeps links.
    • +
+ + + -- cgit v1.3.1