Skipping the Photo Booth Line at Google’s WebAI Summit: Reverse-Engineering the 4-Color Espruino ePaper Badge (and Building an On-Device WebGPU Studio for It)
How a 45-minute portrait queue turned into a live USB-C hardware hack, a bit-exact 96,000-byte 2bpp BWYR codec, surgical .bootcde firmware patching, and an open-source WebAI Badge Studio running LiteRT-LM and MediaPipe over WebGPU.
When Jason Mayes invites you to Google’s WebAI Summit, you already know the demos are going to run locally in the browser without a cloud GPU in sight. What I did not expect when walking into the registration hall was to be handed one of the coolest pieces of conference hardware I have ever worn: a custom Espruino 4-color ePaper smart badge (EPAPER_BADGE), built in collaboration with Espruino creator Gordon Williams.
Instead of a paper lanyard card, every attendee received a bare-PCB hardware badge powered by an ESP32-C3 running Espruino JavaScript firmware (2v29.490), a crisp 480×800 4-color e-ink panel (Black, White, Yellow, Red), ten addressable WS2812B NeoPixel LEDs around the perimeter, two tactile buttons (BTN1 and BTN2), and a Bluetooth Low Energy mesh beacon that lets badges exchange contact cards and increment a live connection counter at the bottom corner of the screen.
There was just one bottleneck. To get your face and name onto the ePaper screen, you had to stand in the official photo booth line. And because every single engineer in the building wanted a custom e-ink portrait, that line stretched across the hall.
The 15-Minute Hallway REPL Hack
I looked at the 45-minute photo line, looked down at the exposed USB-C connector on the bottom of the PCB (J1 USB-C), and pulled my laptop out of my backpack. Because the badge runs Espruino, plugging it in over USB-C exposes a live interactive JavaScript REPL at 115200 baud. Twenty minutes later, I had dumped the flash filesystem, reverse-engineered the 96,000-byte 2bpp framebuffer layout, dithered a stage selfie with Jason, embedded a pixel-snapped LinkedIn QR code, and patched the firmware to render my connection counter in Roman numerals (XIV).
When I posted photos of that hacked badge on LinkedIn, messages immediately started rolling in from other summit attendees: “How did you get the QR code to scan without e-ink dither noise?”, “How do I change the second screen when I press BTN1?”, and “Can you share the script so I don’t overwrite my badge’s calibration or connection log?”
A raw Node script is fine for a quick hallway mod, but this hardware deserved a proper tool. So I packaged everything we learned about the board into the open-source WebAI Badge Studio: a zero-install, browser-native hardware lab, 4-ink compositor, .bootcde binary patcher, Wi-Fi provisioner, and Web Serial / Web Bluetooth flasher. And because this is the WebAI Summit, the studio itself runs local vision and language models in the browser over WebGPU using LiteRT-LM and MediaPipe.
Figure 1: End-to-end tour of WebAI Badge Studio: 8× 2-bit pixel dither loupe, MediaPipe WebAI portrait cutout, WebGPU LiteRT-LM theme synthesis, 1-bit vCard QR generator, BTN1 second-screen templates, wifi.json provisioning, .bootcde Roman counter patching, and BLE attendee constellation backup.
1. Under the Hood of the Espruino EPAPER_BADGE
Before writing a single pixel to flash, I needed to understand how the badge stores and renders images. Connecting to the badge over Web Serial and running require("Storage").list() revealed the entire on-chip SPI flash layout:
| Flash File | Size | Role on the Badge |
|---|---|---|
| .bootcde | 8,971 B | Primary boot firmware: defines showBadge(), BLE scanning, button handlers, and dynamic counter overlay. |
| badge.epaper | 96,000 B | Primary screen raw 2bpp BWYR framebuffer (800×480 rotated -90°). |
| programme.epaper | 96,000 B | Secondary screen framebuffer toggled when pressing physical button BTN1. |
| id.json | ~360 B | Attendee identity payload broadcast over BLE during badge-to-badge handshakes. |
| wifi.json | ~95 B | Primary and fallback Wi-Fi SSID/password credentials used for cloud sync. |
| scans.txt & connections.json | Variable | Newline-delimited MAC log (counted at boot for the corner badge counter) and resolved attendee profiles. |
The ePaper display measures 480×800 pixels when held upright in portrait orientation, for a total of 384,000 pixels. At 2 bits per pixel (4 pixels packed MSB-first into each byte), a full-screen image is always exact to the byte: 96,000 bytes. However, the physical controller scans in landscape (800×480) with a -90° rotation transform. Writing an upright image directly to badge.epaper without rotating coordinates produces scrambled diagonal stripes.
Once you know the coordinate mapping (rx = 799 - y, ry = x) and the 2-bit ink table, encoding and decoding between a browser HTML5 ImageData buffer and a physical .epaper file takes twenty lines of typed-array math (`src/epaper/codec.js`):
export function encodeRGBAToEpaper(rgbaData) {
const out = new Uint8Array(96000);
for (let y = 0; y < 800; y++) {
const rx = 799 - y;
for (let x = 0; x < 480; x++) {
const idx = (y * 480 + x) * 4;
const c = nearestPaletteIndex(rgbaData[idx], rgbaData[idx + 1], rgbaData[idx + 2]);
const ry = x;
const p = ry * 800 + rx;
const byteIdx = p >> 2;
const shift = (3 - (p & 3)) * 2;
out[byteIdx] |= (c & 0x03) << shift;
}
}
return out;
}2. Taming 4-Color Skin Tones and Zero-Noise 1-Bit QR Codes
A 4-color Black/White/Yellow/Red display has no green or blue primaries and no mid-gray ink. If you run standard Euclidean RGB distance against [#000000, #FFFFFF, #FFCC33, #DB3727], two bad things happen immediately:
- Light gray backgrounds turn yellow: Any neutral shadow around
RGB(210, 210, 210)is mathematically closer in raw RGB space to Amber Gold (#FFCC33) than to pure White or Black, polluting neutral areas with yellow speckles. - QR codes become unscannable: Error-diffusion dithering bleeds yellow and red error pixels into the white quiet zone and module edges of a QR code, causing phone cameras to fail under conference lighting.
In src/epaper/codec.js and src/epaper/dither.js, we solved the portrait problem in two stages. First, we pre-blend image pixels toward a warm black-white-amber-red gamut curve and apply a 3×3 unsharp mask before error diffusion. Second, inside nearestPaletteIndex(r, g, b), we compute a weighted redmean perceptual distance with a neutral-chroma penalty: when a pixel’s chroma (max(r,g,b) - min(r,g,b)) is below 28, yellow and red inks receive an extra distance penalty so neutral grays dither strictly between Black and White paper ink.
For the QR code, we wrote a zero-dependency ISO/IEC 18004 QR matrix generator (src/epaper/qrcode.js) supporting Galois Field GF(256) Reed-Solomon error correction across Versions 1 through 10. Crucially, the composer renders the portrait and background through Atkinson dithering first, and only afterward stamps the QR matrix onto the canvas using integer module widths (boxSize) and pure #000000 / #FFFFFF pixels. Every QR module aligns to exact hardware pixel boundaries with zero dither noise, supporting LinkedIn URLs, multi-field BEGIN:VCARD contact cards, and WIFI:S:... network strings.
3. Surgical .bootcde Patching: Why Roman Numerals Clipped at 48 Pixels
One of my favorite details on the badge is the dynamic connection counter in the bottom-left corner. Rather than baking the number into badge.epaper, the badge’s startup script (.bootcde) counts the lines in scans.txt and composites the number and a tiny person silhouette over the bottom-left strip of the image buffer on every refresh.
When I first patched showBadge() at the summit to format my 14 connections as Roman numerals (XIV), I noticed a subtle bug in the stock firmware:
// Stock .bootcde strip renderer (48px wide):
return Badge.showImageFileRendering("badge.epaper", 48, function(g) {
let img = atob("ICOBAAABgAAAD/AAAB/4..."); // 144-byte 1-bit transparent icon
g.setRotation(1);
g.setColor(1).drawImage(img, 24 - 16, 759);
g.setFont("22").setFontAlign(0, 0).drawString(connectionCount, 24, 745);
});
Notice the second argument to Badge.showImageFileRendering("badge.epaper", 48, ...). To save RAM on the ESP32-C3, the firmware only allocates a 48-pixel-wide overlay strip at the left edge of the portrait display (x = 0..47), centered at x = 24. A 2-digit Arabic number fits inside 48 pixels, but strings like XVIII, hexadecimal 0x1F, or binary 1110 overflow past x = 48 and get sliced in half!
Furthermore, the atob("ICOB...") string inside showBadge() is a 144-byte Espruino 1-bit bitmap header ([32, 35, 129, 0, ...], where byte 129 = 0x80 | 1 sets the transparency bit). If you accidentally alter a single byte of that base64 string, the person icon turns into a solid white rectangle.
Our binary patcher in src/epaper/bootcde-patcher.js locates the exact showBadge = function(options) { ... }; block inside the 8,971-byte .bootcde file, preserves the 144-byte transparent icon byte-for-byte, widens the overlay strip from 48 to 64 pixels, and dynamically shifts the text anchor Math.max(24, (g.stringWidth(r) >> 1) + 4) so Roman, Hexadecimal, and Binary counters never clip.
4. Defeating ESP32-C3 Serial FIFO Drops & CRC32-Verified Flashing
Flashing 96,000 bytes into Espruino SPI flash from Chrome requires sending base64 chunks into require("Storage").write("badge.epaper", atob("..."), offset, totalSize). During early testing, writing a 1,500-character REPL command in one writable.write() call over Web Serial corrupted the image roughly one out of every three flashes.
The culprit is the ESP32-C3’s hardware USB-Serial/JTAG controller. When Espruino pauses interrupts for a few milliseconds to erase a 4 KB SPI flash sector, incoming USB serial packets can overrun the small hardware FIFO if the host browser keeps blasting bytes at full USB speed. In src/transport/espruino-client.js, we slice every outgoing REPL command into 64-byte packets with a 3 ms inter-packet pause, and follow every file upload with an on-device IEEE 802.3 checksum verification:
// Verify on-chip flash contents against local IEEE 802.3 CRC32 before refreshing ePaper
const expectedCrc = crc32(uint8Bytes);
const remoteCrc = await client.evalExpr(
`E.CRC32(require("Storage").read(${JSON.stringify(filename)}))`
);
if (Number(remoteCrc) !== expectedCrc) {
throw new Error(`CRC32 mismatch on ${filename}: expected ${expectedCrc}, got ${remoteCrc}`);
}
We also added a dedicated Wi-Fi Provisioning (wifi.json) card in Tab 3 (`src/transport/wifi-provisioning.js`). After leaving the conference venue, your badge still tries to connect to the summit Wi-Fi whenever it syncs. Now you can read or write primary and backup Wi-Fi credentials directly from the browser over USB-C or Web Bluetooth, verified end-to-end with E.CRC32.
5. Meta-WebAI: Running LiteRT-LM and MediaPipe Over WebGPU Inside the Badge Studio
It felt only right that a studio built for Google’s WebAI Summit should itself be a showcase of modern client-side WebAI. Every AI feature in the studio runs 100% locally inside your browser tab with zero server calls and zero API keys:
- WebGPU Portrait Cutout & Die-Cut Sticker Outline (
src/ai/vision.js): Using@mediapipe/tasks-visionand the float16selfie_segmenter.tflitemodel delegated to the GPU, the studio isolates people from cluttered backgrounds so the geometric 4-ink backdrop shines behind them, optionally tracing a morphological die-cut sticker outline around the foreground mask. - In-Browser LLM Art Director (
src/ai/llm-designer.js): Users can load lightweight instruction-tuned models directly in the browser over WebGPU via@litert-lm/core(.litertlm) or@mediapipe/tasks-genai(.task), including Qwen3-0.6B-INT4 (329 MB), SmolLM-135M-Instruct (167 MB), or Gemma-4-E2B-Web, cached persistently in the browser’sCacheStorage(or loaded from a local.litertlm/.taskfile on disk). - Telemetry-Grounded 4-Ink Themes (
src/epaper/ai-layout.js): Before prompting the model, the studio measures your portrait’s mean luminance, RMS contrast, and warm-channel ratio. The model combines your natural-language prompt with those optical metrics to emit a validatedBadgeDesignSpecJSON that configures five 4-ink architectural themes (cyber-pcb,sunburst-pop,bauhaus-grid,retro-terminal,editorial-minimal), five procedural 4-ink vector backdrops (circuit,sunburst,halftone,bauhaus,matrix-grid), dithering parameters, NeoPixel LED accents, and matchingprogramme.epapersecond-screen cards.
Got a WebAI Summit Badge on Your Desk? Plug It In.
Grab a data-capable USB-C cable (or turn on Bluetooth), open the studio in Chrome or Edge, and start hacking. Your first click should be Full Backup (.ZIP) on Tab 4 to save a bit-exact snapshot of your stock flash files before flashing custom screens or Roman numeral firmware.
👉 Live WebAI Badge Studio: https://rabimba.github.io/webai-badge/
💻 GitHub Repository & Reverse-Engineered Specs: https://github.com/rabimba/webai-badge
🔗 Original Summit Live-Hack Post: LinkedIn Post from Google WebAI Summit
Comments
Post a Comment