From vegard.stikbakke at gmail.com Fri Apr 3 06:10:28 2020 From: vegard.stikbakke at gmail.com (Vegard Stikbakke) Date: Fri, 3 Apr 2020 08:10:28 +0200 Subject: Hi friends! I'm Vegard Message-ID: I'm 26 years old, from Norway, and I work as a backend developer. I'm "originally" a statistician, so I don't have as much theoretical computer science grounding as I'd like. I found this list through Joe Nelson after reading (not all of!) the concurrency primer blog post he recently posted, and figured I'd like to join. Joe said "It's kind of a systems programming crowd," which sounds fun! I use Python at work, and that's also what I gravitate towards using if I do any side projects - probably because it's the only language I know! I've recently started getting interested in files and storage and such. I'm currently trying to learn more about database systems. I have a blog with a few posts, e.g. this one https://www.vegardstikbakke.com/how-do-pipes-work-sigpipe/, which I posted on Hacker News, and learned a lot from the comments there. Looking forward to perusing this list! Best, Vegard From vegard.stikbakke at gmail.com Fri Apr 3 06:35:53 2020 From: vegard.stikbakke at gmail.com (Vegard Stikbakke) Date: Fri, 3 Apr 2020 08:35:53 +0200 Subject: Hi friends! I'm Vegard In-Reply-To: References: Message-ID: I'm also interested in the history of computing. I read The Dream Machine by M. Mitchell Waldrop and UNIX: A History and a Memoir by Brian Kernighan last year - both are highly recommended! On Fri, Apr 3, 2020 at 8:10 AM Vegard Stikbakke wrote: > > I'm 26 years old, from Norway, and I work as a backend developer. I'm > "originally" a statistician, so I don't have as much theoretical > computer science grounding as I'd like. > > I found this list through Joe Nelson after reading (not all of!) the > concurrency primer blog post he recently posted, and figured I'd like > to join. > Joe said "It's kind of a systems programming crowd," which sounds fun! > > I use Python at work, and that's also what I gravitate towards using > if I do any side projects - probably because it's the only language I > know! > I've recently started getting interested in files and storage and such. > I'm currently trying to learn more about database systems. > > I have a blog with a few posts, e.g. this one > https://www.vegardstikbakke.com/how-do-pipes-work-sigpipe/, which I > posted on Hacker News, and learned a lot from the comments there. > > Looking forward to perusing this list! > > Best, Vegard From mxu at uribe.cc Fri Apr 3 12:02:14 2020 From: mxu at uribe.cc (Mauricio Uribe) Date: Fri, 3 Apr 2020 08:02:14 -0400 Subject: Hi friends! I'm Vegard In-Reply-To: References: Message-ID: <778d31d3-081b-1c79-192f-cb914aaf8879@uribe.cc> On 4/3/20 2:35 AM, Vegard Stikbakke wrote: > I'm also interested in the history of computing. I read The Dream > Machine by M. Mitchell Waldrop and UNIX: A History and a Memoir by > Brian Kernighan last year - both are highly recommended! > > On Fri, Apr 3, 2020 at 8:10 AM Vegard Stikbakke > wrote: >> >> I'm 26 years old, from Norway, and I work as a backend developer. I'm >> "originally" a statistician, so I don't have as much theoretical >> computer science grounding as I'd like. >> >> I found this list through Joe Nelson after reading (not all of!) the >> concurrency primer blog post he recently posted, and figured I'd like >> to join. >> Joe said "It's kind of a systems programming crowd," which sounds fun! >> >> I use Python at work, and that's also what I gravitate towards using >> if I do any side projects - probably because it's the only language I >> know! >> I've recently started getting interested in files and storage and such. >> I'm currently trying to learn more about database systems. >> >> I have a blog with a few posts, e.g. this one >> https://www.vegardstikbakke.com/how-do-pipes-work-sigpipe/, which I >> posted on Hacker News, and learned a lot from the comments there. >> >> Looking forward to perusing this list! >> >> Best, Vegard Hi Vegard, and welcome! I'm not a systems programmer, but often simply lurk on the discussions. And already, I've learned quite a bit from the smart folks around here! They're all smart and good people. Oh, and also, I'm using python myself for side projects too. For crafting rapid prototypes I've never been as productive as I am now; i just love it as a language! Once again, welcome! Best regards, Mauricio ("mxu") Uribe mxu at uribe.cc From dklann at grunch.org Fri Apr 3 14:13:40 2020 From: dklann at grunch.org (David Klann) Date: Fri, 3 Apr 2020 09:13:40 -0500 Subject: Hi friends! I'm Vegard In-Reply-To: <778d31d3-081b-1c79-192f-cb914aaf8879@uribe.cc> References: <778d31d3-081b-1c79-192f-cb914aaf8879@uribe.cc> Message-ID: <8909def9-f1b6-47f2-00fc-8794a3488495@grunch.org> Greetings Vegard, welcome to the list! On 4/3/20 7:02 AM, Mauricio Uribe wrote: > On 4/3/20 2:35 AM, Vegard Stikbakke wrote: >> I'm also interested in the history of computing. I read The Dream >> Machine by M. Mitchell Waldrop and UNIX: A History and a Memoir by >> Brian Kernighan last year - both are highly recommended! >> Not general computing history, but Unix-specific: there's also The Unix Heritage Society (TUHS) led by Warren Toomey: https://www.tuhs.org/ and mailing list (and archive) https://minnie.tuhs.org/mailman/listinfo/tuhs I am quite impressed with Mr Toomey's work, and the fact that so many people who were there 30 and 50 years ago are still around and actively participating in shaping computing and the community today. I'm a Perl and shell hacker from way back and have tried to selectively learn new languages (including Python and Go) as time allows. >> On Fri, Apr 3, 2020 at 8:10 AM Vegard Stikbakke >> wrote: >>> >>> I'm 26 years old, from Norway, and I work as a backend developer. I'm >>> "originally" a statistician, so I don't have as much theoretical >>> computer science grounding as I'd like. >>> Best, ~David Klann From joe at begriffs.com Sat Apr 4 18:01:51 2020 From: joe at begriffs.com (Joe Nelson) Date: Sat, 4 Apr 2020 13:01:51 -0500 Subject: What are the "serious" libraries? Message-ID: <20200404180151.fiftnvub272lmv3x@begriffs.com> I've been looking for major libraries that do the heavy lifting behind the scenes for most software, and that power features in other languages (like how Haskell relies on libgmp to do the actual numerical work). I came up with the list below. Anybody have more suggestions for serious and portable libraries? For instance, I'm looking for a good cross platform GUI library. Also is there a good container library for C that provides generic data structures? LibreSSL https://www.libressl.org/ Symmetric ciphers Public key cryptography and key agreement Certificates Authentication codes and hash functions Input, output, and data encoding ICU http://site.icu-project.org/home Code Page Conversion Collation Formatting numbers, dates, times and currency amounts Time calculations with calendars and time zones Unicode: character properties, normalization, case folding, other fundamental operations Regular expressions Bidirectional writing Text Boundaries: Locate the positions of words, sentences, paragraphs within a range of text, or identify locations that would be suitable for line wrapping when displaying the text SQLite https://sqlite.org/index.html an in-process library that implements a self-contained, zero-configuration, transactional SQL database engine uses a high-level, stable, reliable, cross-platform, widely-deployed, extensible, performant, accessible, concurrent file format GMP https://gmplib.org/ arbitrary precision arithmetic, operating on signed integers, rational numbers, and floating-point numbers carefully designed to be as fast as possible by using fullwords as the basic arithmetic type, by using fast algorithms, with highly optimised assembly code for the most common inner loops for a lot of CPUs X/Open Curses https://publications.opengroup.org/c094 (ncurses is an implementation and extension https://invisible-island.net/ncurses/) terminal capability detection "raw" mode keyboard input cursor motion line drawing highlighting, underlining {insert,delete} {line,character} status line area clear windows color libcurl https://curl.haxx.se/libcurl/ multi-protocol file transfer lex / yacc (flex and bison are widely available implementations) https://pubs.opengroup.org/onlinepubs/9699919799/utilities/lex.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/yacc.html generate lexers and parsers libarchive https://www.libarchive.org/ multi-format archive and compression (arguably it uses more fundamental libraries internally like zlib) From june at causal.agency Sat Apr 4 18:12:43 2020 From: june at causal.agency (June Bug) Date: Sat, 4 Apr 2020 14:12:43 -0400 Subject: What are the "serious" libraries? In-Reply-To: <20200404180151.fiftnvub272lmv3x@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: <2F5EBFA1-A09C-445F-857A-0C4DE427CD84@causal.agency> > On Apr 4, 2020, at 14:01, Joe Nelson wrote: > > I've been looking for major libraries that do the heavy lifting behind > the scenes for most software, and that power features in other languages > (like how Haskell relies on libgmp to do the actual numerical work). > > I came up with the list below. Anybody have more suggestions for serious > and portable libraries? For instance, I'm looking for a good cross > platform GUI library. Also is there a good container library for C that > provides generic data structures? Off the top of my head I?d say SDL2 belongs on this list. Really good cross-platform way to get a window open and put some graphics up, used in loads of games. From fpereiro at gmail.com Sat Apr 4 19:41:57 2020 From: fpereiro at gmail.com (Fede Pereiro) Date: Sat, 4 Apr 2020 21:41:57 +0200 Subject: What are the "serious" libraries? In-Reply-To: <20200404180151.fiftnvub272lmv3x@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: Hi Joe & friends! While not a library per se (it's a server), Redis provides well implemented data structures written in C. I believe all its primitives (like its dynamic string library) are modular and can be repurposed as libraries, at least to some extent. Cheers! ??, 4 ???. 2020 ?. ? 20:02, Joe Nelson : > > I've been looking for major libraries that do the heavy lifting behind > the scenes for most software, and that power features in other languages > (like how Haskell relies on libgmp to do the actual numerical work). > > I came up with the list below. Anybody have more suggestions for serious > and portable libraries? For instance, I'm looking for a good cross > platform GUI library. Also is there a good container library for C that > provides generic data structures? > > LibreSSL https://www.libressl.org/ > Symmetric ciphers > Public key cryptography and key agreement > Certificates > Authentication codes and hash functions > Input, output, and data encoding > > ICU http://site.icu-project.org/home > Code Page Conversion > Collation > Formatting numbers, dates, times and currency amounts > Time calculations with calendars and time zones > Unicode: character properties, normalization, case folding, other fundamental operations > Regular expressions > Bidirectional writing > Text Boundaries: Locate the positions of words, sentences, > paragraphs within a range of text, or identify locations that would > be suitable for line wrapping when displaying the text > > SQLite https://sqlite.org/index.html > an in-process library that implements a self-contained, > zero-configuration, transactional SQL database engine > uses a high-level, stable, reliable, cross-platform, > widely-deployed, extensible, performant, accessible, > concurrent file format > > GMP https://gmplib.org/ > arbitrary precision arithmetic, operating on signed integers, > rational numbers, and floating-point numbers > carefully designed to be as fast as possible by using fullwords as > the basic arithmetic type, by using fast algorithms, with highly > optimised assembly code for the most common inner loops for a lot > of CPUs > > X/Open Curses https://publications.opengroup.org/c094 > (ncurses is an implementation and extension https://invisible-island.net/ncurses/) > terminal capability detection > "raw" mode keyboard input > cursor motion > line drawing > highlighting, underlining > {insert,delete} {line,character} > status line > area clear > windows > color > > libcurl https://curl.haxx.se/libcurl/ > multi-protocol file transfer > > lex / yacc (flex and bison are widely available implementations) > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/lex.html > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/yacc.html > generate lexers and parsers > > libarchive https://www.libarchive.org/ > multi-format archive and compression > (arguably it uses more fundamental libraries internally like zlib) From dklann at grunch.org Sun Apr 5 00:32:55 2020 From: dklann at grunch.org (David Klann) Date: Sat, 4 Apr 2020 19:32:55 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200404180151.fiftnvub272lmv3x@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: <4ebabd35-96b2-01dd-6ecc-f87ff171827a@grunch.org> All good suggestions Joe! On 4/4/20 1:01 PM, Joe Nelson wrote: > I've been looking for major libraries that do the heavy lifting behind > the scenes for most software, and that power features in other languages > (like how Haskell relies on libgmp to do the actual numerical work). > > I came up with the list below. Anybody have more suggestions for serious > and portable libraries? For instance, I'm looking for a good cross > platform GUI library. Also is there a good container library for C that > provides generic data structures? > > LibreSSL https://www.libressl.org/ > ... > ICU http://site.icu-project.org/home > ... > SQLite https://sqlite.org/index.html > ... > GMP https://gmplib.org/ > ... > X/Open Curses https://publications.opengroup.org/c094 There's also the new-ish NotCurses: https://github.com/dankamongmen/notcurses I haven't yet used it, but it looks to be a modern interpretation of the long-time standard. > > libcurl https://curl.haxx.se/libcurl/ > ... > lex / yacc (flex and bison are widely available implementations) > ... > libarchive https://www.libarchive.org/ > ... There's also the Linux (and *BSD) kernel(s): for uni-kernel embedded applications, the kernel is a must-use "library" of functionality. Warning: I'm not a *BSD user, so the BSD links may not be of much use... Linux: https://kernel.org/ NetBSD: http://www.netbsd.org/docs/kernel/ OpenBSD: https://man.openbsd.org/release FreeBSD: https://www.freebsd.org/doc/handbook/makeworld.html ~David From bavier at posteo.net Sun Apr 5 03:05:17 2020 From: bavier at posteo.net (Eric Bavier) Date: Sat, 04 Apr 2020 22:05:17 -0500 Subject: What are the "serious" =?UTF-8?Q?libraries=3F?= In-Reply-To: <20200404180151.fiftnvub272lmv3x@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: On 04.04.2020 13:01, Joe Nelson wrote: > I've been looking for major libraries that do the heavy lifting behind > the scenes for most software, and that power features in other > languages > (like how Haskell relies on libgmp to do the actual numerical work). Without taking the time to actually put a nice list together myself, I'll point to the American Fuzzy Lop webpage which has a nice "bug-case". Most of the software listed there (many of which are libraries) was fuzzed precisely because it's a high-impact bit of software: https://lcamtuf.coredump.cx/afl/ -- `~Eric From salo at saloits.com Sun Apr 5 05:42:46 2020 From: salo at saloits.com (Timothy J. Salo) Date: Sun, 5 Apr 2020 00:42:46 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200404180151.fiftnvub272lmv3x@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: On 4/4/2020 1:01 PM, Joe Nelson wrote: > I've been looking for major libraries that do the heavy lifting behind > the scenes for most software, and that power features in other languages > (like how Haskell relies on libgmp to do the actual numerical work). I'm not sure what a "serious" library is, but it seems to me that pretty much any such list ought to include glibc. From the GNU C Library pages: The GNU C Library provides many of the low-level components used directly by programs written in the C or C++ languages. Many programming languages use the GNU C Library indirectly including C#, Java, Perl, Python, and Ruby (interpreters, VMs, or compiled code for these langauges use glibc directly). -tjs From joe at begriffs.com Sun Apr 5 17:11:13 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 5 Apr 2020 12:11:13 -0500 Subject: What are the "serious" libraries? In-Reply-To: References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: <20200405171113.bke4o6zlut2deorv@begriffs.com> Fede Pereiro wrote: > While not a library per se (it's a server), Redis provides well > implemented data structures written in C. Good point, I see a bunch of things in https://github.com/antirez/redis/tree/unstable/src Kind of wish they had released that part as a shared library that other projects could use too. Not to say you couldn't just copy the sources into your own project, but it would be nice to be able to install the library with an OS package manager. From joe at begriffs.com Sun Apr 5 17:19:54 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 5 Apr 2020 12:19:54 -0500 Subject: What are the "serious" libraries? In-Reply-To: <4ebabd35-96b2-01dd-6ecc-f87ff171827a@grunch.org> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <4ebabd35-96b2-01dd-6ecc-f87ff171827a@grunch.org> Message-ID: <20200405171954.2ynuiayhmyp4jgjo@begriffs.com> > > X/Open Curses https://publications.opengroup.org/c094 David Klann wrote: > There's also the new-ish NotCurses: > https://github.com/dankamongmen/notcurses I haven't yet used it, but it > looks to be a modern interpretation of the long-time standard. While, according to their readme, "notcurses is not an implementation of X/Open (aka XSI) Curses, nor part of SUS4-2018," it still looks interesting. I like that notcurses is * thread-safe The curses standard says that implementations don't have to be thread-safe, and in fact ncurses is not. I'd have to put locks around ncurses calls, or nominate a certain thread to do UI and take requests from the others. * supports Unicode ncurses supports wide characters (wchar_t), which can't *portably* handle Unicode. https://begriffs.com/posts/2019-01-19-inside-c-standard-lib.html#stdlib.h-wchar_t Thanks for sharing the library. > There's also the Linux (and *BSD) kernel(s): for uni-kernel embedded > applications, the kernel is a must-use "library" of functionality. > Warning: I'm not a *BSD user, so the BSD links may not be of much use... Can you tell me more about uni-kernel embedded applications? I don't know anything about that. Perhaps some examples? From june at causal.agency Sun Apr 5 17:32:36 2020 From: june at causal.agency (June Bug) Date: Sun, 5 Apr 2020 13:32:36 -0400 Subject: What are the "serious" libraries? In-Reply-To: <20200405171954.2ynuiayhmyp4jgjo@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <4ebabd35-96b2-01dd-6ecc-f87ff171827a@grunch.org> <20200405171954.2ynuiayhmyp4jgjo@begriffs.com> Message-ID: > On Apr 5, 2020, at 13:19, Joe Nelson wrote: > > * supports Unicode > ncurses supports wide characters (wchar_t), which can't *portably* > handle Unicode. > https://begriffs.com/posts/2019-01-19-inside-c-standard-lib.html#stdlib.h-wchar_t Noticed something in that post: "Even Mac OS and iOS use 16 bit wchar_t for whatever reason.? That doesn?t seem to be true: printf("%zu\n", sizeof(wchar_t)); prints 4 on macOS. However, __STDC_ISO_10646__ doesn?t seem to be defined. From joe at begriffs.com Sun Apr 5 17:33:41 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 5 Apr 2020 12:33:41 -0500 Subject: What are the "serious" libraries? In-Reply-To: References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: <20200405173341.rpcsa2cnrwru7yti@begriffs.com> Martin VanWinkle wrote: > * https://www.boost.org Boost is C++ only, correct? I don't know this for a fact, but I'm assuming that it's harder for other languages to call C++ libraries than to call C libraries because of complexities in the object files. Maybe someone who knows about that can chime in. > * Readline Oh for sure, it's used by many programs. Looks like it handles these tasks: * move the text cursor (vi and emacs modes) * search the command history * control a kill ring * and use tab completion It's straight up GPL though, not LGPL. There are more permissive knockoffs like: * libedit http://thrysoee.dk/editline/ requires ncurses * linenoise https://github.com/antirez/linenoise restricted to VT100 terminals/emulators From joe at begriffs.com Sun Apr 5 17:49:52 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 5 Apr 2020 12:49:52 -0500 Subject: What are the "serious" libraries? In-Reply-To: References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: <20200405174952.wowkxhmmp26mh27n@begriffs.com> Timothy J. Salo wrote: > I'm not sure what a "serious" library is What I mean by serious is this combination: * Required painstaking work (e.g. ncurses cross-terminal abstraction) or specialist knowledge (e.g. crypto algorithms) * Available on many platforms and callable from many languages Zooming out a little about my motivation, I've been wondering what the best use of my time would be for learning tech topics. What topics are going to allow me to do cool new things, and also not become quickly obsolete? I thought that identifying and learning serious libraries would be a good contender. > but it seems to me that pretty much any such list ought to include > glibc. Good point. Looks like glibc provides an implementation of both the C standard library, and POSIX.1-2008. Wikipedia identifies some other implementations as well: BSD libc, dietlibc, ?Clibc, Newlibc, klibc, musl, Bionic. Looks like BSD libc also has some extensions to the standard. https://en.wikipedia.org/wiki/C_standard_library#BSD_libc Fragmentation pains me, I wish the implementations would stick to the spec and then offer the extensions as another library. That way you wouldn't have to buy into the whole platform. Or maybe you can use BSD libc even on Mac and Linux? Is that possible? From samuel.stuewe at gmail.com Sun Apr 5 17:51:54 2020 From: samuel.stuewe at gmail.com (Sam Stuewe) Date: Sun, 5 Apr 2020 12:51:54 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200405174952.wowkxhmmp26mh27n@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <20200405174952.wowkxhmmp26mh27n@begriffs.com> Message-ID: Musl and sqlite are widely considered some of the highest-quality codebases in C out there. Can't recommend either highly enough. All the best, -Sam On 4/5/20, Joe Nelson wrote: > Timothy J. Salo wrote: >> I'm not sure what a "serious" library is > > What I mean by serious is this combination: > > * Required painstaking work (e.g. ncurses cross-terminal abstraction) > or specialist knowledge (e.g. crypto algorithms) > * Available on many platforms and callable from many languages > > Zooming out a little about my motivation, I've been wondering what the > best use of my time would be for learning tech topics. What topics are > going to allow me to do cool new things, and also not become quickly > obsolete? I thought that identifying and learning serious libraries > would be a good contender. > >> but it seems to me that pretty much any such list ought to include >> glibc. > > Good point. Looks like glibc provides an implementation of both the C > standard library, and POSIX.1-2008. Wikipedia identifies some other > implementations as well: BSD libc, dietlibc, ?Clibc, Newlibc, klibc, > musl, Bionic. > > Looks like BSD libc also has some extensions to the standard. > https://en.wikipedia.org/wiki/C_standard_library#BSD_libc > Fragmentation pains me, I wish the implementations would stick to the > spec and then offer the extensions as another library. That way you > wouldn't have to buy into the whole platform. Or maybe you can use BSD > libc even on Mac and Linux? Is that possible? > From june at causal.agency Sun Apr 5 17:57:36 2020 From: june at causal.agency (June Bug) Date: Sun, 5 Apr 2020 13:57:36 -0400 Subject: What are the "serious" libraries? In-Reply-To: <20200405174952.wowkxhmmp26mh27n@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <20200405174952.wowkxhmmp26mh27n@begriffs.com> Message-ID: > On Apr 5, 2020, at 13:49, Joe Nelson wrote: > > Looks like BSD libc also has some extensions to the standard. > https://en.wikipedia.org/wiki/C_standard_library#BSD_libc > Fragmentation pains me, I wish the implementations would stick to the > spec and then offer the extensions as another library. That way you > wouldn't have to buy into the whole platform. Or maybe you can use BSD > libc even on Mac and Linux? Is that possible? For other platforms there is which implements functions present in *BSD libc?s but missing elsewhere. The libc on macOS is FreeBSD?s libc, thankfully. From joe at begriffs.com Mon Apr 6 02:19:22 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 5 Apr 2020 21:19:22 -0500 Subject: What are the "serious" libraries? In-Reply-To: References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: <20200406021922.zjpr3kpo5aq5i5q7@begriffs.com> Eric Bavier wrote: > Most of the software listed there (many of which are libraries) was > fuzzed precisely because it's a high-impact bit of software: Great observation. Looks like there are quite a few libraries for individual image and sound formats. Then there are these two which seem pretty serious: ffmpeg (seems tasteful https://ffmpeg.org/developer.html#C-language-features) libavcodec encoding/decoding library libavfilter graph-based frame editing library libavformat I/O and muxing/demuxing library libavdevice special devices muxing/demuxing library libavutil common utility library libswresample audio resampling, format conversion and mixing libpostproc post processing library libswscale color conversion and scaling library libsndfile http://www.mega-nerd.com/libsndfile/ Read/write/convert many formats Examine and seek through "frames" (aka samples for channels) of audio Both file and stream I/O C99, tested on tons of systems From joe at begriffs.com Mon Apr 6 03:36:08 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 5 Apr 2020 22:36:08 -0500 Subject: What are the "serious" libraries? In-Reply-To: References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <4ebabd35-96b2-01dd-6ecc-f87ff171827a@grunch.org> <20200405171954.2ynuiayhmyp4jgjo@begriffs.com> Message-ID: <20200406033608.hxkxqklufsoeodmr@begriffs.com> > > https://begriffs.com/posts/2019-01-19-inside-c-standard-lib.html#stdlib.h-wchar_t June Bug wrote: > Noticed something in that post: "Even Mac OS and iOS use 16 bit > wchar_t for whatever reason.? That doesn?t seem to be true: > > printf("%zu\n", sizeof(wchar_t)); > > prints 4 on macOS. However, __STDC_ISO_10646__ doesn?t seem to be > defined. Hm, thanks for fact checking, I confirmed it's four bytes on my mac as well. The header /usr/include/i386/_types.h has a comment about the choice of wchar_t representation: /* * The rune type below is declared to be an ``int'' instead of the more * natural ``unsigned long'' or ``long''. Two things are happening * here. It is not unsigned so that EOF (-1) can be naturally assigned * to it and used. Also, it looks like 10646 will be a 31 bit standard. * This means that if your ints cannot hold 32 bits, you will be in * trouble. The reason an int was chosen over a long is that the is*() * and to*() routines take ints (says ANSI C), but they use * __darwin_ct_rune_t instead of int. By changing it here, you lose a * bit of ANSI conformance, but your programs will still work. * * NOTE: rune_t is not covered by ANSI nor other standards, and should * not be instantiated outside of lib/libc/locale. Use wchar_t. * wchar_t and rune_t must be the same type. Also wint_t must be no * narrower than wchar_t, and should also be able to hold all members of * the largest character set plus one extra value (WEOF). wint_t must be * at least 16 bits. */ So this brings us to the next question, why isn't __STDC_ISO_10646__ defined when wchar_t is wide enough to hold UTF-32 code units? The C99 spec mentions briefly: __STDC_ISO_10646__ An integer constant of the form yyyymmL (for example, 199712L). If this symbol is defined, then every character in the Unicode required set, when stored in an object of type wchar_t, has the same value as the short identifier of that character. The Unicode required set consists of all the characters that are defined by ISO/IEC 10646, along with all amendments and technical corrigenda, as of the specified year and month. I'm not familiar with the "short identifier" lingo, and I don't believe the Unicode standard uses that term. So I downloaded the ISO 10646 spec, and they say: 6.5 Short identifiers for code points (UIDs) This International Standard defines short identifiers for each code point, including code points that are reserved (unassigned). A short identifier for any code point is distinct from a short identifier for any other code point. If a character is allocated at a code point, a short identifier for that code point can be used to refer to the character allocated at that code point. [?] EXAMPLE The short identifier for LATIN SMALL LETTER LONG S may be notated in any of the following forms: 017F +017F U017F U+017F It's basically just the codepoint, written with a certain syntax. So all I can conclude is that if __STDC_ISO_10646__ is not defined, then there is a Unicode character which is stored in wchar_t using a different numerical value than its codepoint. Why would that be, does anyone have insight into this? From joe at begriffs.com Mon Apr 6 03:52:13 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 5 Apr 2020 22:52:13 -0500 Subject: What are the "serious" libraries? In-Reply-To: <2F5EBFA1-A09C-445F-857A-0C4DE427CD84@causal.agency> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <2F5EBFA1-A09C-445F-857A-0C4DE427CD84@causal.agency> Message-ID: <20200406035213.6rt34totcfkbv6xn@begriffs.com> > > I'm looking for a good cross platform GUI library. June Bug wrote: > Off the top of my head I?d say SDL2 belongs > on this list. Really good cross-platform way to get a window open > and put some graphics up, used in loads of games. Sounds great for raw graphical access. Can you also recommend a GUI library that has widgets like buttons and text fields for more desktop-y apps? From dklann at grunch.org Mon Apr 6 13:18:59 2020 From: dklann at grunch.org (David Klann) Date: Mon, 6 Apr 2020 08:18:59 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200406035213.6rt34totcfkbv6xn@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <2F5EBFA1-A09C-445F-857A-0C4DE427CD84@causal.agency> <20200406035213.6rt34totcfkbv6xn@begriffs.com> Message-ID: <95269718-86c3-deb3-5523-266a31196ae8@grunch.org> Hi Joe & June & Friends, On 4/5/20 10:52 PM, Joe Nelson wrote: >>> I'm looking for a good cross platform GUI library. > > June Bug wrote: >> Off the top of my head I?d say SDL2 belongs >> on this list. Really good cross-platform way to get a window open >> and put some graphics up, used in loads of games. > > Sounds great for raw graphical access. > > Can you also recommend a GUI library that has widgets like buttons and > text fields for more desktop-y apps? > The obvious candidates here are: - Qt (sometimes pronounced "cute"): https://www.qt.io/developers - GTK: https://www.gtk.org/docs/ - WxWidgets: https://www.wxwidgets.org/ These are probably the most mature of the bunch, and I suspect there are many others. I just started (re)learning GTK after looking into all three of the above for a small desktop GUI app I wrote (in Python). From what I can tell, all three of these have quite large footprints (maybe necessarily so, due to the nature of GUI applications *and* cross-platform functionality). I wonder if there is a "smaller" or more appropriate library for small-ish desktop apps... ~David From june at causal.agency Mon Apr 6 13:37:00 2020 From: june at causal.agency (June Bug) Date: Mon, 6 Apr 2020 09:37:00 -0400 Subject: What are the "serious" libraries? In-Reply-To: <20200406035213.6rt34totcfkbv6xn@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <2F5EBFA1-A09C-445F-857A-0C4DE427CD84@causal.agency> <20200406035213.6rt34totcfkbv6xn@begriffs.com> Message-ID: <9EA34480-468F-4DDE-A4C3-F79041D14604@causal.agency> > On Apr 5, 2020, at 23:52, Joe Nelson wrote: > >>> I'm looking for a good cross platform GUI library. > > June Bug wrote: >> Off the top of my head I?d say SDL2 belongs >> on this list. Really good cross-platform way to get a window open >> and put some graphics up, used in loads of games. > > Sounds great for raw graphical access. > > Can you also recommend a GUI library that has widgets like buttons and > text fields for more desktop-y apps? I haven?t had a chance to use any yet, but I?m very interested in ?immediate mode GUI? libraries: - seems to be the big one; - is a much smaller one I recently found. The gist of these as I understand them is that unlike ?retained mode GUI? APIs, rather than creating and storing a widget hierarchy, you just call functions inline with the rest of your code (which is what makes it appealing for game debugging UI), and then when you go to render your frame you ask the library what to draw based on the calls you made. From dfeldman.mn at gmail.com Mon Apr 6 16:49:39 2020 From: dfeldman.mn at gmail.com (Daniel Feldman) Date: Mon, 6 Apr 2020 11:49:39 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200404180151.fiftnvub272lmv3x@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: An interesting one that comes to mind is BLAS! It's at the core of pretty much everything that does matrix math. It predates open source (1979) and is the main reason Fortran compilers are still maintained. And hardly anyone knows about it! http://www.netlib.org/blas/ Daniel From andrew at apastron.io Mon Apr 6 18:33:11 2020 From: andrew at apastron.io (Andrew Fischer) Date: Mon, 06 Apr 2020 13:33:11 -0500 Subject: What are the "serious" libraries? In-Reply-To: Message-ID: And don't forget about lapack! http://www.netlib.org/lapack/ That and blas are incredibly powerful, widely used, and rarely mentioned (unless you work in the linalg space). They are amazing. On Mon Apr 6, 2020 at 11:49 AM Daniel Feldman wrote: > An interesting one that comes to mind is BLAS! It's at the core of > pretty much everything that does matrix math. It predates open source > (1979) and is the main reason Fortran compilers are still maintained. > And hardly anyone knows about it! http://www.netlib.org/blas/ > Daniel From bavier at posteo.net Mon Apr 6 22:26:13 2020 From: bavier at posteo.net (Eric Bavier) Date: Mon, 06 Apr 2020 17:26:13 -0500 Subject: What are the "serious" =?UTF-8?Q?libraries=3F?= In-Reply-To: References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: <6e373eeceb125c0c35901c12959ab008@posteo.net> On 06.04.2020 11:49, Daniel Feldman wrote: > An interesting one that comes to mind is BLAS! It's at the core of > pretty much everything that does matrix math. It predates open source > (1979) and is the main reason Fortran compilers are still maintained. Most serious implementations of BLAS are not implemented in Fortran anymore. With respect to this conversation it might be worth noting that BLAS is a standard with many different implementations. Fortran compilers still exist because there are very large, very important codes with huge budgets that are implemented in Fortran. -- `~Eric From joe at begriffs.com Mon Apr 6 23:28:24 2020 From: joe at begriffs.com (Joe Nelson) Date: Mon, 6 Apr 2020 18:28:24 -0500 Subject: What are the "serious" libraries? In-Reply-To: <6e373eeceb125c0c35901c12959ab008@posteo.net> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <6e373eeceb125c0c35901c12959ab008@posteo.net> Message-ID: <20200406232824.hmvf34lfvlzqaq5s@begriffs.com> Eric Bavier wrote: > With respect to this conversation it might be worth noting that BLAS > is a standard with many different implementations. While I was reading more about this, I found an interesting story of a single guy -- kind of an outsider to high performance computing -- whose implementation of BLAS outperformed a whole team and their automated code generator called Atlas. https://www.nytimes.com/2005/11/28/technology/writing-the-fastest-code-by-hand-for-fun-a-human-computer-keeps.html The NY Times missed the opportunity for the headline, "Goto considered helpful" though. ;) From joe at begriffs.com Mon Apr 6 23:43:15 2020 From: joe at begriffs.com (Joe Nelson) Date: Mon, 6 Apr 2020 18:43:15 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200404180151.fiftnvub272lmv3x@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: <20200406234315.j6l52xaog2gfrbuj@begriffs.com> There are also those libraries which provide strings that keep track of their length with a number rather than relying on \0 termination. I've always shied away from this kind of thing since I don't use standard library functions that could overflow, but I did find this comparison interesting: http://bstring.sourceforge.net/features.html It identifies libsrt as a runner up to bstring. What's interesting about libsrt is that it includes more than just strings -- namely vector, bit set, set, map, hash set, and hash map. https://github.com/faragon/libsrt It's fairly new, but looks thorough and tested. From joe at begriffs.com Tue Apr 7 03:50:53 2020 From: joe at begriffs.com (Joe Nelson) Date: Mon, 6 Apr 2020 22:50:53 -0500 Subject: What are the "serious" libraries? In-Reply-To: <95269718-86c3-deb3-5523-266a31196ae8@grunch.org> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <2F5EBFA1-A09C-445F-857A-0C4DE427CD84@causal.agency> <20200406035213.6rt34totcfkbv6xn@begriffs.com> <95269718-86c3-deb3-5523-266a31196ae8@grunch.org> Message-ID: <20200407035053.rbprjopflh2axtny@begriffs.com> > > Can you also recommend a GUI library that has widgets like buttons > > and text fields for more desktop-y apps? David Klann wrote: > The obvious candidates here are: > > - Qt (sometimes pronounced "cute"): https://www.qt.io/developers > - GTK: https://www.gtk.org/docs/ > - WxWidgets: https://www.wxwidgets.org/ I spent some time tonight comparing these and some others. There are plenty of libraries for C++ it seems, and the only mainstream one for C is GTK+. That thing is a bit of a beast, and reportedly its API isn't very stable across releases. One thought I had was to combine a C application with a scripting language like Tcl or Lua to handle the GUI, using Tk or a Lua wrapper for one of the C++ frameworks. However there's another option, a truly oldskool library called Motif. It might just fit the bill, if you don't mind a grey rectangular interface that looks like it's from 1985 (I don't). The Definitive Guides To The X Window System, Vol 6, Motif Programming Manual says this by way of introduction: So why motif? Because it remains what it has long been: the common native windowing toolkit for all the UNIX platforms, fully supported by all the major operating system vendors. It is still the only truly industrial strength toolkit capable of supporting large scale and long term projects. Everything else is tainted: it isn't ready or fully functionally complete, or the functional specification changes in a non-backwards-compatible manner per release, or there are performance issues. Perhaps it doesn't truly port across UNIX systems, or it isn't fully ICCCM compliant with software written in any other toolkit on the desktop, or there are political battles as various groups try to control the specification for their own purposes. Indeed it may matter very much whose version of the toolkit you have managed to acquire, or if the toolkit is open sourced so you have no idea who is going to stick their untrusted fingers into it at any time. So many problems with each choice you make. With motif, you know where you are: its stable, its robust, its professionally supported, and it all works. I was a little worried about Mac support, but I just got the hello world example from that book to work. Mac has an X11 port called XQuartz. I also installed the openmotif package from macports, and used the following Makefile settings (couldn't find a pkg-config definition, but it may be on my system somewhere). CFLAGS = -std=c99 -Wall -Wextra -pedantic -I/opt/local/include LDFLAGS = -L/opt/local/lib -lXt -lXm The hello world source file is attached inline to this message so you can try it yourself. :) -------------- next part -------------- #include #include void button_pushed(Widget widget, XtPointer client_data, XtPointer call_data) { (void)widget; (void)client_data; (void)call_data; printf("Hello Yourself!\n"); } int main(int argc, char *argv[]) { Widget toplevel, button; XtAppContext app; XmString label; Arg args[2]; XtSetLanguageProc (NULL, NULL, NULL); toplevel = XtVaOpenApplication( &app, "Hello", NULL, 0, &argc, argv, NULL,sessionShellWidgetClass, NULL); label = XmStringCreateLocalized("Push here to say hello"); XtSetArg(args[0], XmNlabelString, label); button = XmCreatePushButton (toplevel, "pushme", args, 1); XmStringFree (label); XtAddCallback (button, XmNactivateCallback, button_pushed, NULL); XtManageChild (button); XtRealizeWidget (toplevel); XtAppMainLoop (app); return 0; } From joe at begriffs.com Fri Apr 10 04:23:49 2020 From: joe at begriffs.com (Joe Nelson) Date: Thu, 9 Apr 2020 23:23:49 -0500 Subject: What are the "serious" libraries? In-Reply-To: References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <20200405174952.wowkxhmmp26mh27n@begriffs.com> Message-ID: <20200410042349.zsv4dltr3k2rxcmz@begriffs.com> > > Or maybe you can use BSD libc even on Mac and Linux? Is that > > possible? June Bug wrote: > For other platforms there is > which implements functions present in *BSD libc?s but missing > elsewhere. The libc on macOS is FreeBSD?s libc, thankfully. Very interesting. I spent some time looking into this library and it's nice and light. Ultimately the problem I'm running into is how to set up a portable makefile which falls back to BSD functionality when present, or links with libbsd when needed? The project man page https://helpmanual.io/man7/libbsd/ talks about an overlay mode, and that's what I want, but the makefile gymnastics are puzzling. The libbsd author linked to some other libraries of interest. One is called c-algorithms, and has a nice baseline of generic containers. Haven't tried it yet to see how easy it is to use: https://fragglet.github.io/c-algorithms/ I also noticed that there are some generic algorithms provided by POSIX in search.h -- https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/search.h.html They've got a tree, queue, hash table, and list. However the hash table didn't look so great, since hcreate() allocates a hidden global. Wouldn't work with threads. From joe at begriffs.com Sat Apr 11 02:56:24 2020 From: joe at begriffs.com (Joe Nelson) Date: Fri, 10 Apr 2020 21:56:24 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200404180151.fiftnvub272lmv3x@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> Message-ID: <20200411025624.u2yooy3lyrxccybz@begriffs.com> I'm working on the template for a portable Makefile which searches for prerequisite libraries to link to the main program. Here's what I've devised (using a made-up example that needs the ncurses and gpgme libraries): ---------------------------------------- .POSIX: SHELL=`command -v sh` DEPS = 'ncurses >= 6.0' 'gpgme >= 1.13' CFLAGS = -std=c99 -pedantic -Wall -Wextra myapp : .lib.deps myapp.c $(CC) $(CFLAGS) `pkg-config --cflags $(DEPS)` \ $(LDFLAGS) `pkg-config --libs $(DEPS)` \ -o $@ myapp.c .lib.deps : @pkg-config --exists --print-errors --short-errors $(DEPS) && \ pkg-config --modversion $(DEPS) > $@ ---------------------------------------- The idea is you put the library names in DEPS, along with version constraints. Then there's a .lib.deps target which will only be created if the libraries are present on the system in satisfactory versions. Any target which requires the libraries can add .lib.deps as a dependency in its make rule. In the build command, we add pkg-config's output along with standard CFLAGS and LDFLAGS. What's nice about this is it fails right away with a clear message when libraries don't exist, rather than getting a confusing linker error later on. If I change the ncurses version constraint to >=6.2, here's how it fails: $ make Requested 'ncurses >= 6.2' but version of ncurses is 6.1.20180127 You may find new versions of ncurses at https://invisible-island.net/ncurses make: *** [.lib.deps] Error 1 Another nice thing is how the pkg-config utility provides me with the appropriate flags for my platform. On mac I'm getting these LDFLAGS: -L/opt/local/lib -lncurses -lgpgme -lassuan -lgpg-error Other targets can continue to use the standard CFLAGS / suffix rules / whatever. Open question: how to specify lower AND upper version bounds for each package? Is the following our best option? DEPS = 'ncurses >= 6.0' 'ncurses < 7' 'gpgme >= 1.13' 'gpgme < 2' This also raises the problem that if the project was previously built, and afterward dependency libraries were upgraded too far, our .lib.deps is already cached and make won't raise an error on the next build. From june at causal.agency Sat Apr 11 15:14:24 2020 From: june at causal.agency (June Bug) Date: Sat, 11 Apr 2020 11:14:24 -0400 Subject: What are the "serious" libraries? In-Reply-To: <20200411025624.u2yooy3lyrxccybz@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <20200411025624.u2yooy3lyrxccybz@begriffs.com> Message-ID: <69B66B57-0CB3-4B1E-8365-169C1388E4BF@causal.agency> > On Apr 10, 2020, at 22:56, Joe Nelson wrote: > > I'm working on the template for a portable Makefile which searches for > prerequisite libraries to link to the main program. Here's what I've > devised (using a made-up example that needs the ncurses and gpgme > libraries): Why do this all inside the Makefile? I think it would be easier to read as a simple configure script, and running ./configure before make is what?s expected. I write mine to output config.mk and `include config.mk` in my Makefile. Some advantages: the flags output by pkg-config are ?cached? in the file, and can be manually adjusted in that file if needed. It?s also a lot easier if you need to set PKG_CONFIG_PATH (for a library installed in a non-standard location), you only need to set it for ./configure and after that make will keep using the right flags. From joe at begriffs.com Sat Apr 11 17:49:07 2020 From: joe at begriffs.com (Joe Nelson) Date: Sat, 11 Apr 2020 12:49:07 -0500 Subject: What are the "serious" libraries? In-Reply-To: <69B66B57-0CB3-4B1E-8365-169C1388E4BF@causal.agency> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <20200411025624.u2yooy3lyrxccybz@begriffs.com> <69B66B57-0CB3-4B1E-8365-169C1388E4BF@causal.agency> Message-ID: <20200411174907.34gizsusgrl7duar@begriffs.com> June Bug wrote: > > Why do this all inside the Makefile? I think it would be easier to > read as a simple configure script, and running ./configure before make > is what?s expected. I write mine to output config.mk and `include > config.mk` in my Makefile. That's a much better idea, thank you! I see an example of what you mean in one of your projects: https://git.causal.agency/litterbox/tree/configure One question: in your Makefile why do you prefix include with a dash? -include config.mk Doesn't that ignore errors in the execution of config.mk? I think I'd like to be aware of any errors. This includes config.mk not existing, in which case I'd like the main Makefile to complain. From drewbenson at netjack.com Sat Apr 11 23:54:05 2020 From: drewbenson at netjack.com (Andrew Benson) Date: Sat, 11 Apr 2020 18:54:05 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200411174907.34gizsusgrl7duar@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <20200411025624.u2yooy3lyrxccybz@begriffs.com> <69B66B57-0CB3-4B1E-8365-169C1388E4BF@causal.agency> <20200411174907.34gizsusgrl7duar@begriffs.com> Message-ID: <2E0465C5-2ECC-4FC8-99AC-670FBEB66A34@netjack.com> How can that configure script work? It?s specifying Bourne shell at the top and then using non-bourne-shell syntax. > On Apr 11, 2020, at 12:49 PM, Joe Nelson wrote: > > June Bug wrote: >> >> Why do this all inside the Makefile? I think it would be easier to >> read as a simple configure script, and running ./configure before make >> is what?s expected. I write mine to output config.mk and `include >> config.mk` in my Makefile. > > That's a much better idea, thank you! I see an example of what you mean > in one of your projects: > https://git.causal.agency/litterbox/tree/configure > > One question: in your Makefile why do you prefix include with a dash? > > -include config.mk > > Doesn't that ignore errors in the execution of config.mk? I think I'd > like to be aware of any errors. This includes config.mk not existing, in > which case I'd like the main Makefile to complain. From drewbenson at netjack.com Sun Apr 12 00:05:37 2020 From: drewbenson at netjack.com (Andrew Benson) Date: Sat, 11 Apr 2020 19:05:37 -0500 Subject: What are the "serious" libraries? In-Reply-To: <2E0465C5-2ECC-4FC8-99AC-670FBEB66A34@netjack.com> References: <2E0465C5-2ECC-4FC8-99AC-670FBEB66A34@netjack.com> Message-ID: Like the +=, which is a bash addition. Sent from my iPhone On Apr 11, 2020, at 6:54 PM, Andrew Benson wrote: ?How can that configure script work? It?s specifying Bourne shell at the top and then using non-bourne-shell syntax. >> On Apr 11, 2020, at 12:49 PM, Joe Nelson wrote: >> >> June Bug wrote: >> >> Why do this all inside the Makefile? I think it would be easier to >> read as a simple configure script, and running ./configure before make >> is what?s expected. I write mine to output config.mk and `include >> config.mk` in my Makefile. > > That's a much better idea, thank you! I see an example of what you mean > in one of your projects: > https://git.causal.agency/litterbox/tree/configure > > One question: in your Makefile why do you prefix include with a dash? > > -include config.mk > > Doesn't that ignore errors in the execution of config.mk? I think I'd > like to be aware of any errors. This includes config.mk not existing, in > which case I'd like the main Makefile to complain. From june at causal.agency Sun Apr 12 00:18:21 2020 From: june at causal.agency (June Bug) Date: Sat, 11 Apr 2020 20:18:21 -0400 Subject: What are the "serious" libraries? In-Reply-To: References: <2E0465C5-2ECC-4FC8-99AC-670FBEB66A34@netjack.com> Message-ID: <48C74981-6176-432E-A123-5829E7F22949@causal.agency> > On Apr 11, 2020, at 20:05, Andrew Benson wrote: > > Like the +=, which is a bash addition. Those are inside the here-documents of makefile it?s generating. From june at causal.agency Sun Apr 12 00:22:52 2020 From: june at causal.agency (June Bug) Date: Sat, 11 Apr 2020 20:22:52 -0400 Subject: What are the "serious" libraries? In-Reply-To: <20200411174907.34gizsusgrl7duar@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <20200411025624.u2yooy3lyrxccybz@begriffs.com> <69B66B57-0CB3-4B1E-8365-169C1388E4BF@causal.agency> <20200411174907.34gizsusgrl7duar@begriffs.com> Message-ID: > On Apr 11, 2020, at 13:49, Joe Nelson wrote: > > One question: in your Makefile why do you prefix include with a dash? > > -include config.mk > > Doesn't that ignore errors in the execution of config.mk? I think I'd > like to be aware of any errors. This includes config.mk not existing, in > which case I'd like the main Makefile to complain. As far as I know that only ignores it if it?s missing, and I think is just left over from before I had configure scripts and config.mk was optional. It should indeed be just `include` now. From drewbenson at netjack.com Sun Apr 12 04:49:19 2020 From: drewbenson at netjack.com (Andrew Benson) Date: Sat, 11 Apr 2020 23:49:19 -0500 Subject: What are the "serious" libraries? In-Reply-To: <48C74981-6176-432E-A123-5829E7F22949@causal.agency> References: <48C74981-6176-432E-A123-5829E7F22949@causal.agency> Message-ID: <26D90867-F389-4923-8196-7D106D071940@netjack.com> Oh lol I see cat < wrote: ? > On Apr 11, 2020, at 20:05, Andrew Benson wrote: > > Like the +=, which is a bash addition. Those are inside the here-documents of makefile it?s generating. From dave at 19a6.net Sun Apr 12 15:32:53 2020 From: dave at 19a6.net (Dave Bucklin) Date: Sun, 12 Apr 2020 10:32:53 -0500 Subject: WhyTheName Message-ID: <20200412153253.GA10241@19a6.tech> Lots of a-ha moments in here, for me. https://wiki.debian.org/WhyTheName > Giving cryptic names to software is a well-established UNIX tradition, > and the explanations are often missing from the documentation, either > because the developers imagine it's obvious (usually wrongly) or because > they think nobody cares (and here they're usually right, or it would > turn up as FAQ material). From drewbenson at netjack.com Sun Apr 12 16:34:59 2020 From: drewbenson at netjack.com (Andrew Benson) Date: Sun, 12 Apr 2020 11:34:59 -0500 Subject: WhyTheName In-Reply-To: <20200412153253.GA10241@19a6.tech> References: <20200412153253.GA10241@19a6.tech> Message-ID: Fun link - thanks! > On Apr 12, 2020, at 10:32 AM, Dave Bucklin wrote: > > Lots of a-ha moments in here, for me. > > https://wiki.debian.org/WhyTheName > >> Giving cryptic names to software is a well-established UNIX tradition, >> and the explanations are often missing from the documentation, either >> because the developers imagine it's obvious (usually wrongly) or because >> they think nobody cares (and here they're usually right, or it would >> turn up as FAQ material). From joe at begriffs.com Sun Apr 12 18:11:38 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 12 Apr 2020 13:11:38 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200406033608.hxkxqklufsoeodmr@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <4ebabd35-96b2-01dd-6ecc-f87ff171827a@grunch.org> <20200405171954.2ynuiayhmyp4jgjo@begriffs.com> <20200406033608.hxkxqklufsoeodmr@begriffs.com> Message-ID: <20200412181138.p7o6ytuutfukt4xd@begriffs.com> I wrote: > So all I can conclude is that if __STDC_ISO_10646__ is not defined, > then there is a Unicode character which is stored in wchar_t using a > different numerical value than its codepoint. Why would that be, does > anyone have insight into this? Trying to get to the bottom of the mystery. Thought perhaps Mac stores UTF-16 rather than UTF-32 in wchar_t, but the following program disproves that hypothesis. Really wondering why Mac doesn't conform to ISO 10646. -------------- next part -------------- #include #include #include int main(void) { /* U+1F41A, outside the BMP */ char *seashell_utf8 = "\xf0\x9f\x90\x9a"; wchar_t *seashell_wide = malloc(5 * sizeof(wchar_t)); if (!setlocale(LC_CTYPE, "en_US.UTF-8")) { fputs("Cannot set locale\n", stderr); return EXIT_FAILURE; } if ((size_t)(-1) == mbstowcs(seashell_wide, seashell_utf8, sizeof(seashell_wide)-1)) { fputs("Invalid multibyte character\n", stderr); return EXIT_FAILURE; } while (*seashell_wide) printf("%x ", *seashell_wide++); puts(""); return EXIT_SUCCESS; } From joe at begriffs.com Sun Apr 12 22:43:59 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 12 Apr 2020 17:43:59 -0500 Subject: What are the "serious" libraries? In-Reply-To: <20200412181138.p7o6ytuutfukt4xd@begriffs.com> References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <4ebabd35-96b2-01dd-6ecc-f87ff171827a@grunch.org> <20200405171954.2ynuiayhmyp4jgjo@begriffs.com> <20200406033608.hxkxqklufsoeodmr@begriffs.com> <20200412181138.p7o6ytuutfukt4xd@begriffs.com> Message-ID: <20200412224359.xsbdsgpiijbuhxf7@begriffs.com> > mbstowcs(seashell_wide, seashell_utf8, sizeof(seashell_wide)-1)) Oops, that whole sizeof part is wrong, but not wrong enough to affect the particular example. From Kurtis at riseup.net Mon Apr 13 03:58:00 2020 From: Kurtis at riseup.net (Kurtis Hanna) Date: Mon, 13 Apr 2020 03:58:00 +0000 Subject: WhyTheName In-Reply-To: <20200412153253.GA10241@19a6.tech> References: <20200412153253.GA10241@19a6.tech> Message-ID: <7b54708a-13b4-84de-c90e-d6f65dd04283@riseup.net> Thanks Dave! I particularly liked this one... > chromium - web browsers on Linux spent a decade going through a cycle of slick new slimline web browsers gradually getting buried in creeping features until they were as weighed down with chrome grills and ornamental fins as a fifties US car, at which point everyone would switch to some new minimalist alternative. Meanwhile these GUI widgets came to be referred to as "chrome", which explains why Google would choose to advertise their browser as if it was manufactured entirely out of deadweight bling... From joe at begriffs.com Sun Apr 19 22:11:29 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 19 Apr 2020 17:11:29 -0500 Subject: LDLIBS in portable makefiles Message-ID: <20200419221129.bpppejhwuni6udzc@begriffs.com> Hi Chris, I notice that your makefile article [1] uses the macro name LDLIBS for -l options like -lm. That's reminiscent of GNU make's .c suffix rule [2]: LINK.c = $(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH) .c: $(LINK.c) $^ $(LOADLIBES) $(LDLIBS) -o $@ Do you know why they separate the -L flags in LDFLAGS vs -l flags in LDLIBS? I realize that GCC's linker has trouble if the -l flags precede the source file(s), but my question is why don't they put all the flags (both -L and -l) together into LDFLAGS and put that macro at the end? For instance, using this suffix rule: .c: $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) POSIX make [3] doesn't recognize LDLIBS in its default rules. And its .c suffix rule puts the LDFLAGS before the source name: .c: $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $< The clang compiler is fine with the -l flags going before the source file actually, which makes it work with the POSIX defaults. Do you know the background behind GNU's divergence? 1: https://nullprogram.com/blog/2017/08/20/ 2: https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html 3: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html#tag_20_76_13_09 From joe at begriffs.com Mon Apr 20 02:11:29 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 19 Apr 2020 21:11:29 -0500 Subject: What are the "serious" libraries? In-Reply-To: References: <20200404180151.fiftnvub272lmv3x@begriffs.com> <4ebabd35-96b2-01dd-6ecc-f87ff171827a@grunch.org> <20200405171954.2ynuiayhmyp4jgjo@begriffs.com> Message-ID: <20200420021129.7fglpaptbufwvnmg@begriffs.com> June Bug wrote: > printf("%zu\n", sizeof(wchar_t)); > > prints 4 on macOS. However, __STDC_ISO_10646__ doesn?t seem to be > defined. I did more testing and an pretty sure __STDC_ISO_10646__ should be defined on macOS with clang. Don't know whether the bug is in clang or libc, but I filed a bug on clang with steps to reproduce my test: https://bugs.llvm.org/show_bug.cgi?id=45613 From wellons at nullprogram.com Mon Apr 20 13:57:40 2020 From: wellons at nullprogram.com (Christopher Wellons) Date: Mon, 20 Apr 2020 09:57:40 -0400 Subject: LDLIBS in portable makefiles In-Reply-To: <20200419221129.bpppejhwuni6udzc@begriffs.com> References: <20200419221129.bpppejhwuni6udzc@begriffs.com> Message-ID: <20200420135740.fyky7dkb64mgyxe7@nullprogram.com> > why don't they put all the flags (both -L and -l) together into > LDFLAGS and put that macro at the end? That's a really good question, and I don't know the reason behind it. As you pointed out, as long as the -L option precedes the corresponding -l option, I believe it shouldn't matter that -L comes later, even for gcc. I suspect it's a historical accident. Dynamic linking came relatively late, and along with it linker sensitivity to argument order (at least when invoked via some compilers). By convention, options go before file arguments so the POSIX make documentation put LDFLAGS first. However, with dynamic linking we needed some arguments passed after the source file names. Rather than override the established inference rules to put LDFLAGS last, introduce a new variable, LDLIBS, to hold just the arguments that must go last. Perhaps whoever decided to create LDLIBS thought moving LDFLAGS was too disruptive, and that introducing LDLIBS was a softer fix. Unfortunately this history predates GNU Make's use of source control, as LDLIBS first appears in a January 1992 commit titled "Initial revision". However, it turns out this isn't a GNU-ism! Digging around the Unix History Repository, I see LDLIBS makes its first appearance in April 1986 in BSD 4.3 (commit 7887d35c750). It's in its modern form: at the end of the command with a separate LDFLAGS. This predates GNU Make, which was announced in February 1988 and released in June 1988. So GNU Make was just following conventions established by BSD. It may just be rationalization, but I actually prefer that these are separate since they configure two different sorts of knobs. It's common for the default LDFLAGS to be empty, and LDLIBS populated with the necessary libraries. I can override LDFLAGS independently of LDLIBS, meaning I don't need to worry about preserving anything about LDLIBS: make LDFLAGS='-pie -Wl,-rpath=$HOME/.local/lib' CFLAGS='-fpie' I do want to override LDFLAGS on occasion, but I rarely want or need to override LDLIBS. Makefiles are also a little tidier with them separated. From joe at begriffs.com Sun Apr 26 17:11:30 2020 From: joe at begriffs.com (Joe Nelson) Date: Sun, 26 Apr 2020 12:11:30 -0500 Subject: Finished pthreads article In-Reply-To: <20200328013834.ia7n36zuczohbrxe@begriffs.com> References: <20200325014438.oqi3dibnhart3x2h@begriffs.com> <20200328013834.ia7n36zuczohbrxe@begriffs.com> Message-ID: <20200426171130.fmt7maglw6ugwfvh@begriffs.com> Joe Nelson wrote: > Nicholas Drozd wrote: > > - I was unable to build life.c on Linux. > > -- undefined reference to `pthread_barrier_wait' > > -- undefined reference to `pthread_barrier_init' > > -- undefined reference to `pthread_barrier_destroy' > > Similar compilation errors on Mac. > > Interesting, I wonder if this is an instance of gcc being a special > snowflake. Or maybe my Makefile isn't as portable as it should be. > > Doing a quick search, I see people say that gcc uses the -pthread > option rather than the linker directive -lpthread. Update: I was previously going down some pretty complicated paths trying to see why -lpthread wasn't working with gcc. Turns out that I just needed to put -lpthread after the source file inputs to gcc, not before them. It's the LDFLAGS/LDLIBS distinction that Chris Wellons and I were talking about in another thread. This makefile should work for the examples in my article: .POSIX: CFLAGS = -std=c99 -pedantic -D_POSIX_C_SOURCE=200809L -Wall -Wextra LDLIBS = -lpthread .c: $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $< $(LDLIBS) I also updated the article. Thanks again for pointing out the build failure. Oh and the gcc-specific `-pthread` option for CFLAGS isn't needed anymore. Its purpose was to add -lpthread and the glibc symbol -D_REENTRANT, but the latter is deprecated https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html#index-_005fREENTRANT From nicholasdrozd at gmail.com Tue Apr 28 14:56:47 2020 From: nicholasdrozd at gmail.com (Nicholas Drozd) Date: Tue, 28 Apr 2020 09:56:47 -0500 Subject: Finished pthreads article In-Reply-To: <20200426171130.fmt7maglw6ugwfvh@begriffs.com> References: <20200325014438.oqi3dibnhart3x2h@begriffs.com> <20200328013834.ia7n36zuczohbrxe@begriffs.com> <20200426171130.fmt7maglw6ugwfvh@begriffs.com> Message-ID: > This makefile should work for the examples in my article: > > .POSIX: > CFLAGS = -std=c99 -pedantic -D_POSIX_C_SOURCE=200809L -Wall -Wextra > LDLIBS = -lpthread > > .c: > $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $< $(LDLIBS) > > I also updated the article. Thanks again for pointing out the build failure. I can confirm that this works on Linux, and on Mac the #error message is reported for life.c.