From 1df3566997535ab563b5fa76724a8c6c55b96483 Mon Sep 17 00:00:00 2001 From: Paul Brinkmeier Date: Mon, 10 Aug 2026 11:19:52 +0200 Subject: [PATCH] UTF-63: Snd version --- blog/utf63/index.html | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/blog/utf63/index.html b/blog/utf63/index.html index 341b5dd..5bce5c6 100644 --- a/blog/utf63/index.html +++ b/blog/utf63/index.html @@ -15,11 +15,10 @@ By

- UTF-8 is almost everywhere these days. - The web runs on it, most common file formats use it, macOS, Linux and increasingly even Windows systems use it as the default option. + UTF-8 is almost everywhere these days. + The Internet runs on it, most common file formats use it, macOS, Linux and increasingly even Windows systems use it as the default option. It makes sense: It's quite compact for mostly-ASCII text and therefore most keywords and symbols of text-based file formats and protocols.

-

What if I told you that in this age of memory bandwidth-constrained workloads you are using a wasteful encoding for text storage? Most text these days is 99% ASCII with a few emoji sprinkled in (source: Made it up). @@ -27,7 +26,7 @@ Even if we ignore the costly multibyte emoji encodings, using UTF-8, 1 out of every 8 bits of memory is wasted on the encoding, because 7-bit ASCII characters are encoded using a whole byte, i.e. we have an encoding overhead of roughly 15%.

- For example, consider the UTF-8 encoding of the string β€œπŸ™ˆπŸ™‰πŸ™Š encoding” (see here for codepoints): + For example, consider the UTF-8 encoding of the string β€œπŸ™ˆπŸ™‰πŸ™Š encoding” (see here for code points):

11110000100111111001100110001000  πŸ™ˆ
 11110000100111111001100110001001  πŸ™‰
@@ -43,9 +42,9 @@
       And even for good old reliable ASCII, useless bits are sprinkled here and there.
       There's gotta be a better way!
     

-

UTF-63

+

A Better Way

- Enter: An entirely new way to encode Unicode! + Enter: UTF-63, an entirely new way to encode Unicode!

10000111110110010010000000111110  πŸ™ˆπŸ™‰
 11001001001000011111011001001010  πŸ™Š
@@ -58,8 +57,8 @@
     

Decoding is dead simple: We consider each set of 8 bytes a Big Endian 64-bit code unit. - If the most significant bit is 0, the other 63 bits contain nine Basic Latin characters (i.e. ASCII characters) at 7 bits each. - If it's 1, they contain any three Unicode code points at 21 bits each (the maximum codepoint is U+10FFFF). + If the most significant bit is 0, the other 63 bits contain nine Basic Latin code points (ASCII characters) at 7 bits each. + If it's 1, the other 63 bits contain any three Unicode code points at 21 bits each (the maximum code point is U+10FFFF). When any of the encoded code points is U+0000 NULL, the string ends there (yes, we're bringing null-terminated strings back. So. many. applications. don't allow NULL anyways).

Usage

@@ -69,6 +68,9 @@ It wouldn't surprise me if VLC supported UTF-63 subtitles or something though. You can do your part by spreading the word among friends, family, flatmates, coworkers and anybody in between.

+

+ If you enjoyed my Unicode ramblings you can read on over at Channable. +