RK's Rambling

Racecraft · Prologue , The Origin Story It Started With a Wine List and a Question About Racing How a happy-hour conversation in the Bay Area turned into a trustable AI race coach , and then into a second version that runs entirely on a phone, on the NPU. This is the prologue to a five-part series. Two years ago(1st November, 2024) I was in the Bay Area for a GDE Summit. If you've never been: it's a couple of days of talks among Google Developer Experts, the kind of people who get unreasonably excited about a new on-device runtime, and then , mercifully , a happy hour where everyone stops performing and just eats. We ended up at a restaurant(Puesto Santa Clara), a long table of GDEs, and I was doing the most important engineering of the evening: trying to decide which wine to order. Across the table was Ajeet Mirwani . I don't even remember how the wine talk turned into racing talk , these things drift , but the moment the word "racing" ...

"If you measure the wrong thing, you optimize for the wrong thing. And when dealing with frontier, multimodal models, the wrong thing might just break your entire system without throwing a single error." We are remarkably comfortable evaluating the AI models we’ve already built. We have our standard suites, our automated leaderboards, and our hard-coded unit tests. We feel in control. But let me drop a truth bomb that’s been brewing across frontier labs: we are profoundly, staggeringly bad at evaluating the models we are about to build . Most benchmarks, safety evaluations, and red-teaming protocols operate on a comforting but incredibly lazy assumption. They treat the next iteration of an LLM or a large multimodal model like a linear upgrade,like turning a dial from 8 to 10. But if you’ve spent any time hacking away at deep learning architectures or building agentic frameworks, you know that complex networks don’t scale smoothly. They undergo massive, silent phase transi...

Gemma 4 Good Hackathon · Impact Track · Health & Sciences It started with two envelopes. One contained a single sheet of paper, a radiologist's report for my friend. It was a wall of text that might as well have been written in another language. Words like " parenchymal volume ," " hyperintensities ," and " susceptibility artifact " stared back at us, creating more anxiety than they resolved. The other was a flimsy paper sleeve containing a CD-ROM. This, we were told, held the actual images from her MRI scan. The ground truth. And we couldn't even look at it. Our laptops, like most these days, don't have disc drives. For a moment, this crucial, deeply personal piece of her health information was a coaster. I felt that familiar, hot-wired frustration every engineer knows: the feeling of being locked out by a dumb problem. The powerlessness was infuriating. So, I did what any slightly obsessive software engineer would do...

When I set out to implement Ring Attention for long-context models , I hit a wall that none of the papers prepared me for. My performance was capping out at half of what it should be, no matter how many accelerators I threw at the problem. It turns out, there's a hidden performance trap in the standard recipe for causal attention . Here’s the story of that bug, how to prove it exists without burning a single TPU hour, and how a simple trick called " zigzag sharding " fixed it completely. This post is a walkthrough of ring-flash-jax , a small JAX project that explores this problem and implements the fix. We'll add the four things you actually need before Ring Attention is usable for training a real-world causal language model . TL;DR ring-flash-jax is a JAX implementation of ring attention with four key additions to the standard pattern: Causal masking — required for any decoder language model. Zigzag (striped) sharding — fixes the critical load...

The Curse of Superposition: Why LLMs are Black Boxes Large Language Models like Google’s Gemma are incredibly powerful, but they suffer from a phenomenon known as Superposition . Neural networks naturally want to represent more concepts than they have mathematical dimensions. To accomplish this, they pack multiple unrelated concepts into the same neurons—a property called polysemanticity . When Gemma processes the word "Paris," it doesn't activate a neat, dedicated "City" neuron. Instead, it fires a dense, entangled vector of floating-point numbers in a 2,304-dimensional space that simultaneously represents "France," "capital," "tourism," and "linguistics." For researchers trying to build safer, steerable AI, this is a massive problem. How do we debug an AI if its internal thoughts are entangled in a dense manifold? The state-of-the-art solution is Mechanistic Interpretability via Sparse Autoencoders (SAEs...