Unicode study

Joe Nelson joe at begriffs.com
Sat Feb 16 21:46:24 UTC 2019


> I wanted to make sure this topic does not fall off the front of our minds.

Don't worry, I still remember. Working on finishing up that ftp project,
and then I'll get started on the unicode book. Also waiting for a number
of people who said they were interested in doing the Unix book...most of
whom are flaking.

> I have started to read on this

Are you reading Unicode Demystified? Please keep notes about things that
surprised you or anything extra you want to add that the book left out.

> I wrote a very simple percent encoder, which I will share. Nothing fancy
> there to get the work done,

Nice, yeah send a link.

> I will be writing a base64 encoder, which is more interesting.

A few years ago I used base64 encoding/decoding as an example to study
property based testing. The tests would generate random binary data
and check that the byte length of the message expanded by a certain
percent after encoding, that the result was padded with the correct
number of equal signs, and that encoding-decoding would undo each other
properly. (RFC4648 guarantees the roundtrip will work only on "canonical
encodings," for example decoding "1yx=" and re-encoding it produces
"1yw=". Found this out through a fuzz testing failure actually.)

Here's the relevant section in my article about it:
https://begriffs.com/posts/2017-01-14-design-use-quickcheck.html#test-case-distribution-and-shrinking

> but I will welcome tricks to make it obscure and cool-looking.

As you can see from the above, I've got you covered for making things
obscure and weird. :-P


More information about the Friends mailing list