From nicholasdrozd at gmail.com Wed Jul 1 17:58:50 2020 From: nicholasdrozd at gmail.com (Nicholas Drozd) Date: Wed, 1 Jul 2020 12:58:50 -0500 Subject: A Makefile for Getting Dressed In-Reply-To: <20200625160336.GI1930@begriffs.com> References: <20200625160336.GI1930@begriffs.com> Message-ID: JB: Thanks, I corrected the comment about the `all` target. JN: `tsort` syntax is simpler and more compact than `make` syntax, but it doesn't look very ergonomic and I don't think I would want to write directly in it. It looks more like a compile target. For example, the target `jacket : vest cufflinks` could be compiled down to `jacket vest` and `jacket cufflinks`. For all I know, an implementation of `make` could actually do this, then call to `tsort` to flatten the dependency graph. (Would that be POSIX-compliant?) `tsort` also lacks abstraction facilities. You pointed out that since a suit just is jacket and pants, it doesn't need to show up in the final output. That's true, but it can still be handy to refer to a suit. In project management, we might refer to an "epic" that is nothing other than a collection of "stories". The "epic" is complete exactly when all of its "stories" are, but that doesn't mean the "epic" abstraction is not useful. Different "epics" might depend on each other, and it would be unwieldy to have to to masses of "stories" directly. From 24dch42 at gmx.com Sun Jul 5 21:52:41 2020 From: 24dch42 at gmx.com (harv) Date: Sun, 5 Jul 2020 16:52:41 -0500 Subject: advice BHCS Message-ID: <1MMXQF-1kAYfH088t-00JZDJ@mail.gmx.com> Hello all, Since I have a server I installed OpenBSD on a while back and have done nothing with since I figured it was high time I did something with it so intend to implement the OpenBSD+http+C+sqlite stack on it. As I have no experience with web servers or databases what I am looking for are reference links to get me up to speed and tips/suggestions. In other words I'm not looking for hand holding just am confused by all the conflicting/contradictory information found on the net so far. The goals of this project are- learn a modicum of server administration same for database administration force myself to learn C as it has been on my todo list much too long probably learn some html as I'm going to need a front end sooner or later. and to make it interesting, have server draw data from zfs storage pool on freeBSD box. Only real question I have right now is is my partitioning scheme adequate or should I have www on its own slice and should I have a separate slice for the database(s)? # size offset fstype [fsize bsize cpg] a: 1.0G 64 4.2BSD 2048 16384 12958 # / b: 4.2G 2097216 swap # none c: 298.1G 0 unused d: 4.0G 10878016 4.2BSD 2048 16384 12958 # /tmp e: 11.9G 19266624 4.2BSD 2048 16384 12958 # /var f: 2.0G 44168256 4.2BSD 2048 16384 12958 # /usr g: 1.0G 48362560 4.2BSD 2048 16384 12958 #/usr/X11R6 h: 20.0G 50459712 4.2BSD 2048 16384 12958 #/usr/local i: 2.0G 92402752 4.2BSD 2048 16384 12958 #/usr/src j: 6.0G 96597056 4.2BSD 2048 16384 12958 #/usr/obj k: 246.0G 109179968 4.2BSD 4096 32768 26062 # /home I believe I took the default scheme on install as I didn't know what I was going to do with the box at the time. Thanks in advance and apologies in advance for late replies as I work a 10hr night shift with a 130 mile round trip commute 5-6 nights a week so keyboard time is limited to weekends. harv From hello at robertdherb.com Mon Jul 6 02:37:24 2020 From: hello at robertdherb.com (Robbie D) Date: Sun, 5 Jul 2020 21:37:24 -0500 Subject: advice BHCS In-Reply-To: <1MMXQF-1kAYfH088t-00JZDJ@mail.gmx.com> References: <1MMXQF-1kAYfH088t-00JZDJ@mail.gmx.com> Message-ID: <6fce867b-aa9c-00c0-7c85-b77e979b693a@robertdherb.com> Hey harv, You might find your disk being mostly taken over by home to be a bit limiting, but it's not the end of the world, especially while you're starting out, 12GB? on /var will be fine. Is this a VM? If so, it's not a problem to add another drive and just mount it to /var/www if you get in a pickle. That's probably easier than shuffling the slices around. But if it comes down to it, well, you said you want to learn more about server administration, and blowing a server away and starting over is a big part of running servers. If you haven't, I would start by reading the httpd manpage, and all the man pages mentioned in SEE ALSO. OpenBSD has some really great documentation! If you have no prior knowledge of HTML, you might find the Mozilla Developer Network a good place to stop: https://developer.mozilla.org/en-US/docs/Web/HTML I'm not familiar with how good their tutorials are, but I use it as a reference all the time, especially when I have to, bleh, write javascript. The sqlite3 man page doesn't look like it has a lot of info, but this website is mentioned, so that's probably where I would go looking for info on that: http://www.sqlite.org/cli.html. SQL is a weird language with its own quirks and and a strange heritage. I would recommend if you're not super familiar with it to maybe get a web frontend like webmin or phpmyadmin, to help visualize what your tables are doing. And I guess my final piece of advice is... If your goal is just to "learn bchs", well, that's going to make progress slow, because there's so much to know, and it's hard to tell where you should start. It's easier to learn if your goal is something like, "Make a weird web form to water my plants" or something specific. Good luck on your journey. rdh PS: Oh, you should also look into cutting that commute down. Yikes! From j3s at c3f.net Tue Jul 7 00:14:33 2020 From: j3s at c3f.net (j3s at c3f.net) Date: Tue, 07 Jul 2020 00:14:33 +0000 Subject: advice BHCS In-Reply-To: <1MMXQF-1kAYfH088t-00JZDJ@mail.gmx.com> References: <1MMXQF-1kAYfH088t-00JZDJ@mail.gmx.com> Message-ID: <846322f17b992e5858de1da32b7a9ebc@c3f.net> July 5, 2020 5:00 PM, "harv" <24dch42 at gmx.com> wrote: > and to make it interesting, have server draw data from zfs storage pool > on freeBSD box. My first piece of advice is to take things one step at a time. This sounds like a very exciting project, but I'd caution against doing too much - bringing in ZFS sounds like too much, since you'll already be learning a lot. I vote in favor of bringing in ZFS later, if you want, but not at the beginning. > Only real question I have right now is is my partitioning scheme > adequate or should I have www on its own slice and should I have a > separate slice for the database(s)? I don't speak for everyone - but nowadays it's _very_ common to have a single large root partition instead of separate partitions at all - or, if there is more than one partition, there are normally two - one for root + everything else, and one for a huge data directory (say, /var/lib/postgresql). That said, if you're set on making partitions for everything, I recommend giving your database the lions share of your storage space, since the www directory is unlikely to take up much space at all. From j3s at c3f.net Tue Jul 7 00:16:19 2020 From: j3s at c3f.net (j3s) Date: Tue, 07 Jul 2020 00:16:19 +0000 Subject: advice BHCS In-Reply-To: <6fce867b-aa9c-00c0-7c85-b77e979b693a@robertdherb.com> Message-ID: July 5, 2020 5:00 PM, "harv" <24dch42 at gmx.com> wrote: > and to make it interesting, have server draw data from zfs storage > pool > on freeBSD box. My first piece of advice is to take things one step at a time. This sounds like a very exciting project, but I'd caution against doing too much - bringing in ZFS sounds like too much, since you'll already be learning a lot. I vote in favor of bringing in ZFS later, if you want, but not at the beginning. > Only real question I have right now is is my partitioning scheme > adequate or should I have www on its own slice and should I have a > separate slice for the database(s)? I don't speak for everyone - but nowadays it's _very_ common to have a single large root partition instead of separate partitions at all - or, if there is more than one partition, there are normally two - one for root + everything else, and one for a huge data directory (say, /var/lib/postgresql). That said, if you're set on making partitions for everything, I recommend giving your database the lions share of your storage space, since the www directory is unlikely to take up much space at all. Also, some other notes: If you are interested in database administration, I recommend learning something like PostgreSQL or MySQL instead of SQLite, since with SQLite there's really not much to be administered. As far as server administration goes, be sure to learn about OpenSSH and how to properly secure access to your system - it's fundamental and priority #1 on a new system. Be sure to use key-based auth, not password auth. Also, have fun! I'm well versed in this stuff so feel free to ask questions. j3s From 24dch42 at gmx.com Tue Jul 7 10:31:46 2020 From: 24dch42 at gmx.com (harv) Date: Tue, 7 Jul 2020 05:31:46 -0500 Subject: advice BHCS In-Reply-To: <846322f17b992e5858de1da32b7a9ebc@c3f.net> References: <1MMXQF-1kAYfH088t-00JZDJ@mail.gmx.com> <846322f17b992e5858de1da32b7a9ebc@c3f.net> Message-ID: <1N8XPt-1kx97d1wXJ-014VKE@mail.gmx.com> j3s at c3f.net opined: > July 5, 2020 5:00 PM, "harv" <24dch42 at gmx.com> wrote: > > > and to make it interesting, have server draw data from zfs storage pool > > on freeBSD box. > > My first piece of advice is to take things one step at a time. This sounds > like a very exciting project, but I'd caution against doing too much - bringing > in ZFS sounds like too much, since you'll already be learning a lot. I vote in favor > of bringing in ZFS later, if you want, but not at the beginning. > Not really bringing in ZFS as the freeBSD box in question is my main system and has been running with ZFS for several years now. > > Only real question I have right now is is my partitioning scheme > > adequate or should I have www on its own slice and should I have a > > separate slice for the database(s)? > > I don't speak for everyone - but nowadays it's _very_ common to have a single large > root partition instead of separate partitions at all - or, if there is more than one > partition, there are normally two - one for root + everything else, and one for a huge > data directory (say, /var/lib/postgresql). > > That said, if you're set on making partitions for everything, I recommend giving your > database the lions share of your storage space, since the www directory is unlikely to > take up much space at all. Thanks for the advice. Much appreciated. From joe at begriffs.com Sat Jul 11 05:48:26 2020 From: joe at begriffs.com (joe at begriffs.com) Date: Sat, 11 Jul 2020 00:48:26 -0500 Subject: Introducing my project, "mimedown" Message-ID: <20200711054827.60A6730600A9@mailuser.nyi.internal> This is a test ============== Hi friends, This is an example email written in the CommonMark [0] markdown standard, and turned into a multipart MIME message with begriffs/mimedown [1] . Hopefully the end result is beautiful in both web-based clients like Gmail, and in textual command-line email clients like Mutt. The message is converted into two parts: * Tasteful HTML which graphical clients can display. * Plaintext alternative wrapped at 72 characters but using format=flowed [2] so that smaller screens like phones will understand how to reflow the text and quotes. Not only that, but code blocks are split into inline attachments. Text clients like Mutt will display the code along with the message contents, and both graphical and text clients will allow the recipient to save the attachments to disk. That makes it easier to use them than copying and pasting into a text editor. Code blocks are also not text wrapped at all but preserved verbatim. Why bother? ----------- There's a school of thought [3] that says HTML mail is evil. This can be true if people are using HTML to embed tracking images, or aren't providing a graceful alternative for console users. However I could flip the argument around and say that plaintext emails are not properly supporting graphical clients! Why not support true text emphasis, lists, and headers? My mimedown program uses a tasteful subset of HTML. I'm trying to enable the best of both worlds automatically. How can you help? ================= This software is in beta. 1. Send me screenshots of how this looks in your graphical client. The more providers the better. I know that people on this list use many different providers. I'm also curious to see how it renders on mobile. 2. Jump into the github issues [4] (yeah yeah, I'm using github rather than an email based workflow...) 3. Report bugs. 4. Star the project on github. ;) By the way, here's a quote: > Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do > eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim > ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut > aliquip ex ea commodo consequat. Internal structure ================== The markdown for this message turned into this MIME structure: -------------- next part -------------- [multipa/alternativ, 7bit, 6.8K] ??> [multipa/mixed, 7bit, 3.4K] ? ??> [text/plain, 7bit, utf-8, 2.3K] ? ??>mime-structure.txt [text/plain, 7bit, utf-8, 0.5K] ? ??>references.uri [text/uri-list, 7bit, utf-8, 0.3K] ??> [text/html, 7bit, utf-8, 3.3K] From joe at begriffs.com Sat Jul 11 05:53:48 2020 From: joe at begriffs.com (Joe Nelson) Date: Sat, 11 Jul 2020 00:53:48 -0500 Subject: Introducing my project, "mimedown" In-Reply-To: <20200711054827.60A6730600A9@mailuser.nyi.internal> References: <20200711054827.60A6730600A9@mailuser.nyi.internal> Message-ID: <20200711055348.GB77799@begriffs.com> > 3. Report bugs. Well, found one problem already... my mailing list software stripped parts of the message out, like the HTML alternative and the url references attachment. I'll try to adjust the content filter and try sending the message again. From joe at begriffs.com Sat Jul 11 05:56:21 2020 From: joe at begriffs.com (joe at begriffs.com) Date: Sat, 11 Jul 2020 00:56:21 -0500 Subject: Mimedown message, take two Message-ID: <20200711055622.345C63280060@mailuser.nyi.internal> This is a test ============== Hi friends, This is an example email written in the CommonMark [0] markdown standard, and turned into a multipart MIME message with begriffs/mimedown [1] . Hopefully the end result is beautiful in both web-based clients like Gmail, and in textual command-line email clients like Mutt. The message is converted into two parts: * Tasteful HTML which graphical clients can display. * Plaintext alternative wrapped at 72 characters but using format=flowed [2] so that smaller screens like phones will understand how to reflow the text and quotes. Not only that, but code blocks are split into inline attachments. Text clients like Mutt will display the code along with the message contents, and both graphical and text clients will allow the recipient to save the attachments to disk. That makes it easier to use them than copying and pasting into a text editor. Code blocks are also not text wrapped at all but preserved verbatim. Why bother? ----------- There's a school of thought [3] that says HTML mail is evil. This can be true if people are using HTML to embed tracking images, or aren't providing a graceful alternative for console users. However I could flip the argument around and say that plaintext emails are not properly supporting graphical clients! Why not support true text emphasis, lists, and headers? My mimedown program uses a tasteful subset of HTML. I'm trying to enable the best of both worlds automatically. How can you help? ================= This software is in beta. 1. Send me screenshots of how this looks in your graphical client. The more providers the better. I know that people on this list use many different providers. I'm also curious to see how it renders on mobile. 2. Jump into the github issues [4] (yeah yeah, I'm using github rather than an email based workflow...) 3. Report bugs. 4. Star the project on github. ;) By the way, here's a quote: > Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do > eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim > ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut > aliquip ex ea commodo consequat. Internal structure ================== The markdown for this message turned into this MIME structure: -------------- next part -------------- [multipa/alternativ, 7bit, 6.8K] ??> [multipa/mixed, 7bit, 3.4K] ? ??> [text/plain, 7bit, utf-8, 2.3K] ? ??>mime-structure.txt [text/plain, 7bit, utf-8, 0.5K] ? ??>references.uri [text/uri-list, 7bit, utf-8, 0.3K] ??> [text/html, 7bit, utf-8, 3.3K] -------------- next part -------------- A non-text attachment was scrubbed... Name: references.uri Type: text/uri-list Size: 262 bytes Desc: not available URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicholasdrozd at gmail.com Sat Jul 11 17:33:44 2020 From: nicholasdrozd at gmail.com (Nicholas Drozd) Date: Sat, 11 Jul 2020 12:33:44 -0500 Subject: Mimedown message, take two In-Reply-To: <20200711055622.345C63280060@mailuser.nyi.internal> References: <20200711055622.345C63280060@mailuser.nyi.internal> Message-ID: Attached are two screenshots, one for Gmail on Linux desktop and one for the HTML attachment displayed in Emacs. Why doesn't the title "Foo" show up in Gmail? A similar project: https://github.com/org-mime/org-mime Related notes: https://kitchingroup.cheme.cmu.edu/blog/2016/10/29/Sending-html-emails-from-org-mode-with-org-mime/ -------------- next part -------------- A non-text attachment was scrubbed... Name: mimedown-gmail-desktop.png Type: image/png Size: 101634 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mimedown-emacs-eww.png Type: image/png Size: 244182 bytes Desc: not available URL: From dave at 19a6.net Sat Jul 11 18:52:32 2020 From: dave at 19a6.net (Dave Bucklin) Date: Sat, 11 Jul 2020 13:52:32 -0500 Subject: Mimedown message, take two In-Reply-To: References: <20200711055622.345C63280060@mailuser.nyi.internal> Message-ID: Here's a screenshot of K-Mail on my Samsung G7. It looks good! -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot_20200711-135046.png Type: image/png Size: 473117 bytes Desc: not available URL: From 42dch42 at gmail.com Sat Jul 11 19:17:56 2020 From: 42dch42 at gmail.com (harv) Date: Sat, 11 Jul 2020 14:17:56 -0500 Subject: Mimedown message, take two In-Reply-To: <20200711055622.345C63280060@mailuser.nyi.internal> References: <20200711055622.345C63280060@mailuser.nyi.internal> Message-ID: <5f0a1065.1c69fb81.3579d.46a5@mx.google.com> > This software is in beta. > ? Send me screenshots of how this looks in your graphical client. The more providers the better. I know that people on this list use many different providers. I'm also curious to see how it renders on mobile. > ? Jump into the github issues (yeah yeah, I'm using github rather than an email based workflow...) > ? Report bugs. > ? Star the project on github. ;) > Here is what Claws-mail shows me. HTML rendered by Dillo. -------------- next part -------------- A non-text attachment was scrubbed... Name: html.png Type: image/png Size: 128019 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mime-structure.png Type: image/png Size: 19889 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: plain_text.png Type: image/png Size: 78431 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: rfc822.png Type: image/png Size: 96993 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: uri-list.png Type: image/png Size: 23594 bytes Desc: not available URL: From joe at begriffs.com Sat Jul 11 21:50:00 2020 From: joe at begriffs.com (Joe Nelson) Date: Sat, 11 Jul 2020 16:50:00 -0500 Subject: Mimedown message, take two In-Reply-To: <5f0a1065.1c69fb81.3579d.46a5@mx.google.com> References: <20200711055622.345C63280060@mailuser.nyi.internal> <5f0a1065.1c69fb81.3579d.46a5@mx.google.com> Message-ID: <20200711215000.GG77799@begriffs.com> harv wrote: > Here is what Claws-mail shows me. HTML rendered by Dillo. Thanks for those screenshots, they reveal three issues: * The plaintext view isn't using a monospaced font, so the "====" under a header is too wide, and spaces to indent overhung list items are too narrow. I wonder if there's a hint I can include so clients will prefer monospace? * The quoted lorem ipsum doesn't really look like quoted text, it's just indented. Is that how quoted replies look in Claws, or am I indicating the quote wrong? * The placeholder title "Foo" shows through in one of your pictures, so I'll need to think of a more reasonable title. (The title tag is required in well-formed HTML.) From joe at begriffs.com Sat Jul 11 22:01:53 2020 From: joe at begriffs.com (Joe Nelson) Date: Sat, 11 Jul 2020 17:01:53 -0500 Subject: Mimedown message, take two In-Reply-To: References: <20200711055622.345C63280060@mailuser.nyi.internal> Message-ID: <20200711220153.GH77799@begriffs.com> Nicholas Drozd wrote: > Attached are two screenshots, one for Gmail on Linux desktop and one > for the HTML attachment displayed in Emacs. Interesting to see what clients do with the HTML, since I don't override the style at all. The h2 is bold, and the h3 is italicized. Your text view on the left looks like it's not respecting format=flowed, because that would have eaten the list item indentation. (There's an open issue for me to use hard linebreaks on list items to preserve the indentation in format=flowed.) > Why doesn't the title "Foo" show up in Gmail? I can see why clients might take either approach to the title, showing it or hiding it. Typically it would be for a web page, but emails already have a subject, so the title seems a little redundant. Nonetheless HTML requires it, so I need to think of a better value than "Foo." :) > A similar project: https://github.com/org-mime/org-mime > Related notes: https://kitchingroup.cheme.cmu.edu/blog/2016/10/29/Sending-html-emails-from-org-mode-with-org-mime/ Looks like that project turns markdown to HTML, but doesn't include a text alternative? Or does it send the markdown itself along as a text part? From joe at begriffs.com Sat Jul 11 22:04:44 2020 From: joe at begriffs.com (Joe Nelson) Date: Sat, 11 Jul 2020 17:04:44 -0500 Subject: Mimedown message, take two In-Reply-To: References: <20200711055622.345C63280060@mailuser.nyi.internal> Message-ID: <20200711220444.GI77799@begriffs.com> Dave Bucklin wrote: > Here's a screenshot of K-Mail on my Samsung G7. It looks good! That does look sharp! :) However I notice it shows the blockquote as merely indented rather than what I would consider a more "email quote" kind of look, such as a blue stripe on the side. I wonder whether I need different markup or some CSS to achieve a better HTML quote style. From 42dch42 at gmail.com Sat Jul 11 23:50:21 2020 From: 42dch42 at gmail.com (harv) Date: Sat, 11 Jul 2020 18:50:21 -0500 Subject: Mimedown message, take two In-Reply-To: <20200711215000.GG77799@begriffs.com> References: <20200711055622.345C63280060@mailuser.nyi.internal> <5f0a1065.1c69fb81.3579d.46a5@mx.google.com> <20200711215000.GG77799@begriffs.com> Message-ID: <5f0a503e.1c69fb81.bee1d.3587@mx.google.com> > * The quoted lorem ipsum doesn't really look like quoted text, it's just > indented. Is that how quoted replies look in Claws, or am I indicating > the quote wrong? Claws-mail uses Dillo browser to render html messages. So probably related to Dillo more than Claws. I haven't gotten an html message with quoted replies in so long I don't remember how it handles the quotes. From jesse at jesselucas.com Sat Jul 11 22:21:35 2020 From: jesse at jesselucas.com (jesse at jesselucas.com) Date: Sat, 11 Jul 2020 22:21:35 +0000 Subject: Mimedown message, take two In-Reply-To: <20200711055622.345C63280060@mailuser.nyi.internal> References: <20200711055622.345C63280060@mailuser.nyi.internal> Message-ID: This is great! ?? I?ve attached screenshots for ProtonMail mobile clients on iOS. On Sat, Jul 11, 2020 at 1:56 AM, wrote: > This is a test > > Hi friends, > > This is an example email written in the [CommonMark](https://commonmark.org) markdown standard, and turned into a multipart MIME message with [begriffs/mimedown](https://github.com/begriffs/mimedown). Hopefully the end result is beautiful in both web-based clients like Gmail, and in textual command-line email clients like Mutt. > > The message is converted into two parts: > > - Tasteful HTML which graphical clients can display. > - Plaintext alternative wrapped at 72 characters but using [format=flowed](https://joeclark.org/ffaq.html) so that smaller screens like phones will understand how to reflow the text and quotes. > > Not only that, but code blocks are split into inline attachments. Text clients like Mutt will display the code along with the message contents, and both graphical and text clients will allow the recipient to save the attachments to disk. That makes it easier to use them than copying and pasting into a text editor. Code blocks are also not text wrapped at all but preserved verbatim. > > Why bother? > > There's a [school of thought](https://useplaintext.email) that says HTML mail is evil. This can be true if people are using HTML to embed tracking images, or aren't providing a graceful alternative for console users. However I could flip the argument around and say that plaintext emails are not properly supporting graphical clients! Why not support true text emphasis, lists, and headers? My mimedown program uses a tasteful subset of HTML. > > I'm trying to enable the best of both worlds automatically. > > How can you help? > > This software is in beta. > > - Send me screenshots of how this looks in your graphical client. The more providers the better. I know that people on this list use many different providers. I'm also curious to see how it renders on mobile. > - Jump into the [github issues](https://github.com/begriffs/mimedown/issues) (yeah yeah, I'm using github rather than an email based workflow...) > - Report bugs. > - Star the project on github. ;) > > By the way, here's a quote: > >> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. > > Internal structure > > The markdown for this message turned into this MIME structure: > > [multipa/alternativ, 7bit, 6.8K] > ??> [multipa/mixed, 7bit, 3.4K] > ? ??> [text/plain, 7bit, utf-8, 2.3K] > ? ??>mime-structure.txt [text/plain, 7bit, utf-8, 0.5K] > ? ??>references.uri [text/uri-list, 7bit, utf-8, 0.3K] > ??> [text/html, 7bit, utf-8, 3.3K] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 871620FF-A574-4986-A556-220B51A03537.jpg Type: image/jpeg Size: 459270 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 517D8B0D-E98B-420D-A8EC-DC021D5F274A.jpg Type: image/jpeg Size: 456007 bytes Desc: not available URL: From bavier at posteo.net Sun Jul 12 03:04:06 2020 From: bavier at posteo.net (Eric Bavier) Date: Sat, 11 Jul 2020 22:04:06 -0500 Subject: Mimedown message, take two In-Reply-To: <20200711055622.345C63280060@mailuser.nyi.internal> References: <20200711055622.345C63280060@mailuser.nyi.internal> Message-ID: On Sat, 2020-07-11 at 00:56 -0500, joe at begriffs.com wrote: > 1. Send me screenshots of how this looks in your graphical client. > The > more providers the better. I know that people on this list use > many > different providers. I'm also curious to see how it renders on > mobile. Hi Joe, This is what I see in Evolution version 3.34.2. The configured "HTML Mode" is "Show HTML if present", but it still wanted to show the plaintext first. Cool project, good luck! `~Eric -------------- next part -------------- A non-text attachment was scrubbed... Name: mimedown-2.png Type: image/png Size: 172312 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mimedown-1.png Type: image/png Size: 173905 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mimedown-0.png Type: image/png Size: 164390 bytes Desc: not available URL: From joe at begriffs.com Sun Jul 12 04:50:40 2020 From: joe at begriffs.com (Joe Nelson) Date: Sat, 11 Jul 2020 23:50:40 -0500 Subject: Mimedown message, take two In-Reply-To: <5f0a503e.1c69fb81.bee1d.3587@mx.google.com> References: <20200711055622.345C63280060@mailuser.nyi.internal> <5f0a1065.1c69fb81.3579d.46a5@mx.google.com> <20200711215000.GG77799@begriffs.com> <5f0a503e.1c69fb81.bee1d.3587@mx.google.com> Message-ID: <20200712045040.GK77799@begriffs.com> harv wrote: > I haven't gotten an html message with quoted replies in so long I don't > remember how it handles the quotes. (Tried replying with my fancy system, but must have gotten the headers wrong, so here's a lower tech reply.) True, most HTML users don't do inline replies. Usually the old messages are just appended to the bottom of the reply. It's kind of crude. I tried using Mac Mail and Fastmail to send myself "rich text" inline replies, and this is the resulting markup:

This is a test

Hi friends,

Well hello self.

Some pretty nasty markup, a bunch of useless divs, and a nonstandard "type" attribute of blockquote. Another approach would be to use the "cite" attribute of blockquote, and in fact this [w3c submission](https://www.w3.org/TR/1998/NOTE-HTMLThreading-0105) about quotes in HTML replies recommends setting the attribute to the Message-ID of the quoted message. Fastmail creates slightly better markup:
On Wed, Jul 8, 2020, at 11:19 PM, joe at begriffs.com wrote:

This is a test

Hi friends,


Well hi yourself!
Notice the empty title... guess that's how they dealt with the title requirement we talked about earlier. Similar to Mac Mail, Fastmail throws in some superfluous br tags. They also do the type="cite" thing. Searching for that pattern brought up an old, rather heated, [Thunderbird bug report](https://bugzilla.mozilla.org/show_bug.cgi?id=183219). The final comments from a few months ago suggest that HTML should standardize the type="cite" based on years of existing practice. Perhaps my tool should use type="cite" and also cite="mid:message-id", supposing it knows the Message-ID of the mail it is replying to. Could have a command line option for reply mode, where it reads in an mbox file of the message we're replying to, to fill the blockquote attributes, as well as the In-Reply-To and References headers. Might be a low priority issue though. A simple blockquote might be enough for most people and mail clients. From j3s at c3f.net Mon Jul 13 05:21:55 2020 From: j3s at c3f.net (j3s at c3f.net) Date: Mon, 13 Jul 2020 05:21:55 +0000 Subject: Mimedown message, take two In-Reply-To: <20200711055622.345C63280060@mailuser.nyi.internal> References: <20200711055622.345C63280060@mailuser.nyi.internal> Message-ID: Looks amazing in rainloop! -------------- next part -------------- An HTML attachment was scrubbed... URL: From joe at begriffs.com Fri Jul 24 04:06:31 2020 From: joe at begriffs.com (Joe Nelson) Date: Thu, 23 Jul 2020 23:06:31 -0500 Subject: The ideal Message-ID format Message-ID: <20200724040120.made-you-look@begriffs.com> Ideal Message-ID format ======================= Here's my latest obsessive and inconsequential investigation, hope you enjoy. :) I want to add the ability for mimedown [0] to generate a Message-ID for the messages it creates. (Without one, a mail transfer agent will append one during the first hop.) Since I'm trying to make impeccable emails with mimedown, I'm wondering if there's an ideal Message-ID format. For background, Message-ID is an SMTP header which must be unique for every email. Email replies are linked to previous messages by their ids, which is how clients build and display trees for threads. Criteria -------- RFC 2822 ?3.6.4 [1] defines the grammar of Message-IDs, roughly like with certain characters allowed. Furthermore the RFC advises: > Though other algorithms will work, it is RECOMMENDED that the right > hand side contain some domain identifier (either of the host itself > or otherwise) such that the generator of the message identifier can > guarantee the uniqueness of the left hand side within the scope of > that domain. I think these are the qualities we want: 1. Unique! (RFC: "The message identifier itself MUST be a globally unique identifier for a message.") 2. Short. Presuming we can keep the ids unique, using fewer bytes is better 3. Not revealing unnecessary information, like process ID, internal hostname, or process owner 4. Not triggering spam filters. Jes ran into this with initial code in his Cyberia mailing list, getting a MSGID_RANDY penalty score from SpamAssassin. Here are regexes to avoid [2] . Sadly, they include a rule that ids shouldn't be too short! A note about revealing unnecessary information. Typically mailers combine sources of uniqueness between dots, like Meetup.com does with its JavaMail engine below. Here are two ids I got from meetup, and notice which one was running as root. ;) * <207885924.1338514418286.JavaMail.nobody at james3.pvt.meetup.com> * <1000751181.1336532452057.JavaMail.root at jobs0.meetup.com> Patterns I've seen ------------------ The simplest ids I've seen come from GovDelivery, a Twin Cities company. They're short, like <16840138.6818 at public.govdelivery.com>. The first number is monotonically increasing, so it must be related to time or some kind of counter for all messages coming out of their domain. I don't know what the second number is, but it looks like process id. Theo de Raadt, the OpenBSD maintainer, also uses very short ones like <11736.1595428218 at cvs.openbsd.org>. Google uses big ones, which makes sense given the volume of mail from their domain. The typical pattern is like this: * * * However it looks like they also accept whatever desktop mail clients send and just change the domain name, because I've seen some like this: * <1251158382.15456 at gmail.com> * <21E55C16-9966-454C-B622-F82A57D52195 at gmail.com> * The xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx pattern is typical of a 128-bit UUID [3] . I've seen this style on our friends list from various mailers: Thunderbird, Apple Mail, Riseup web, Fastmail web, Protonmail web. The mutt algorithm ------------------ Mutt and NeoMutt have distinctive ids. Here's the relevant Mutt code [4] : -------------- next part -------------- A non-text attachment was scrubbed... Name: mutt_msgid.c Type: text/x-c Size: 502 bytes Desc: not available URL: -------------- next part -------------- It's a timestamp down to the second, a counter going from A-Z in a loop, and the process id. This will be unique unless you send more than twenty-six messages in a second from the same process, or unless there are two machines with the same process id sending at the same time for the same domain. Both of which sound like safe assumptions. There is some information leakage, because you can piece together how many messages someone is sending over time if they keep a long-running Mutt process, unless they send so many that it loops the counter between when you're able to sample their messages. The process id range probably fingerprints their machine a little too. Also sending the timestamp in decimal wastes bytes. The internet draft standard, recommendations for generating Message IDs [5] , recommends encoding such numbers in base-36 (to pack the values into fewer characters, presumably). The draft standard is closer to how NeoMutt does it [6] : -------------- next part -------------- A non-text attachment was scrubbed... Name: neomutt_msgid.c Type: text/x-c Size: 522 bytes Desc: not available URL: -------------- next part -------------- The randomness comes from the Linuxism getrandom(2) [7] or reading from /dev/urandom if getrandom is not available, so probably rather portable. I considered whether random() from POSIX would suffice, but the question would be how to seed it. Typically people seed with the current time, as exemplified in MSC30-C [8] , like srandom(ts.tv_nsec ^ ts.tv_sec) but then the pseudo-random value doesn't add anything beyond the timestamp if you think about it, except the greater precision of tv_nsec. The NeoMutt approach leaks less information than Mutt. All emails are timestamped anyway, so it adds no information to include the time in the message id. What's the verdict then? ======================== Maybe the NeoMutt approach is basically what we want, except to base-36 the timestamp and the random value to make them shorter. Maybe it's safe to decrease the timestamp precision (ala GovDelivery?) to an hour or day to shorten it, leaning on the random value to keep us safe. Finally the UUID approach is certainly safe. However it's overkill because it's designed to be globally unique where all we need is uniqueness per domain. Any opinions? Parting evil thought -------------------- Given how hard mailers work to create unique ids, I wonder what would happen if I sent messages with intentionally duplicate ids. Would it break this list, or break your mail client? Are systems built on the assumption of uniqueness? Should I try it? -------------- next part -------------- A non-text attachment was scrubbed... Name: references.uri Type: text/uri-list Size: 903 bytes Desc: not available URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From salo at saloits.com Fri Jul 24 08:17:14 2020 From: salo at saloits.com (Timothy J. Salo) Date: Fri, 24 Jul 2020 03:17:14 -0500 Subject: The ideal Message-ID format In-Reply-To: <20200724040120.made-you-look@begriffs.com> References: <20200724040120.made-you-look@begriffs.com> Message-ID: <34c31b53-84d1-d6c2-8bc3-7d63369b02ce@saloits.com> On 7/23/2020 11:06 PM, Joe Nelson wrote: > Parting evil thought > > Given how hard mailers work to create unique ids, I wonder what would > happen if I sent messages with intentionally duplicate ids. Would it > break this list, or break your mail client? Are systems built on the > assumption of uniqueness? Should I try it? Well, of course, googling [email duplicate message ids] returns some interesting discussions. There is a view that an MTA must accept email with duplicate message IDs. However, it appears that some software (MTA or MUA) doesn't agree with this view. Of course, I would expect the recipient's MUA to screw up threading. -tjs From joe at begriffs.com Tue Jul 28 04:24:14 2020 From: joe at begriffs.com (Joe Nelson) Date: Mon, 27 Jul 2020 23:24:14 -0500 Subject: The ideal Message-ID format In-Reply-To: <20200724040120.made-you-look@begriffs.com> Message-ID: The format ---------- I thought of another constraint for the Message-ID: it should consist of URL-safe characters. Many web archives expose individual messages by putting the msg id in the url, like this postgres patch [0] . Ultimately the format I chose was where rnd is at least 64 bits ( uint_least64_t ). The numbers are encoded in base 70, because there are 70 characters that are both "atext tokens" and URL safe. (You can see an example in this message's headers themselves.) OS function detection --------------------- One challenge was being able to generate random numbers without a seed, and doing it portably without requiring an extra library dependency like libsodium. BSD has arc4random_buf() , and Linux has getrandom() , but which variants of BSD, and which versions of Linux? How can my program know when to call the right one? Matthew Martin (phy1729) uses an approach where he builds a little static library, libcompat [1] , in a subdirectory of his projects. This library contains symbols that shim missing OS functions. I tried this, but it was heavyweight, used the GNUism #include_next , and the ar program didn't like adding an empty object file to my library on macOS. Kind of a hack. My next approach was to set feature macros for myself saying which functions are available. I was thinking of heuristics to determine my environment, and proposed them on the #openbsd channel. The channel told me my approach was too unreliable, and suggested an article by EWONTFIX [2] that talked about mistakes people make when checking for the availability of functions. I came up with a trick that you all might like. I attempt to compile little test programs that check whether the functions exist and whether they have the expected signature. Sounds like it could be a pain, right? Well it's actually pretty clean. I include this in my configure script: -------------- next part -------------- A non-text attachment was scrubbed... Name: configure-example.sh Type: application/x-sh Size: 667 bytes Desc: not available URL: -------------- next part -------------- Then my portable function to get random data checks the flags. -------------- next part -------------- A non-text attachment was scrubbed... Name: get_random_bytes.c Type: text/x-c Size: 257 bytes Desc: not available URL: -------------- next part -------------- Maybe it's a technique as old as time...it's worked well for me so far. -------------- next part -------------- A non-text attachment was scrubbed... Name: references.uri Type: text/uri-list Size: 210 bytes Desc: not available URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: