Project Koru • Deep Dive

Racecraft: Fine-Tuning Gemma 4 with DPO for Sub-150ms On-Device Coaching at 130 MPH

How we distilled expert human motorsport coaching into a 12MB Gemma 4 LoRA adapter using Direct Preference Optimization (DPO)—cutting NPU TTFT to 149ms and locking sub-5-word root-cause cues at high speed.

By Rabimba Karanjai • August 2026 • 10 min read

Figure 1: Empirical DPO Fine-Tuning Loss convergence curve (0.693 down to 0.141), reward margin expansion (+1.768), and NPU performance benchmarks on Google Tensor G5 / NVIDIA GB10.

Picturing a car barreling into Turn 3 at Sonoma Raceway at 130 MPH gives you an immediate sense of scale. At that speed, you are traveling 190.6 feet every single second (58.1 meters/sec). Your brain is processing visual apex marks, tire slip audio, steering effort, and lateral G-forces in split-second 200–300ms reaction windows.

If an AI coach speaks a typical conversational LLM response—something like: "You entered Turn 3 9 miles per hour too slow because you turned in early. Try turning in later next lap to get on gas sooner."—the driver has traveled over 400 feet past the corner before the sentence finishes! The advice is not just useless; it is actively dangerous because it pulls visual focus away from the track.

In our last post, we explored how our split-brain architecture fused 100+ on-device sensors. Today, we are opening the hood on the most crucial layer of all: How we fine-tuned Gemma 4 on-device to deliver razor-sharp, sub-5-word root-cause coaching cues in under 150 milliseconds.

1. The Fallacy of Fine-Tuning LLMs on Raw Telemetry Numbers

When engineers first think about building an AI racing coach, the initial impulse is often: "Let's fine-tune an LLM on raw telemetry arrays!" You feed `[speed: 64.2, brake: 0.12, gLat: 1.12]` into the prompt and expect the LLM to learn driving physics.

This approach fails catastrophically in the real world.

Fine-tuning LLM weights directly on raw telemetry numbers causes brittle numerical overfitting. The moment tire grip degrades by 5%, ambient cabin temperature rises, or you switch from a 300 HP sports car to a 500 HP GT3 car, the fine-tuned weights hallucinate useless braking points. Math belongs in deterministic code; pedagogy belongs in the neural model.

The Fundamental Rule of AI Motorsport Coaching

"Use deterministic math to extract the physical root cause from telemetry deltas. Use the fine-tuned neural model to distill that cause into an ultra-terse, persona-aligned human driving command under strict latency bounds."

2. Direct Preference Optimization (DPO) on T-Rod Coaching Transcripts

Out-of-the-box base Gemma 4 E2B is remarkably intelligent, but it is naturally conversational. Under zero-shot prompts, it defaults to polite 14–22 word explanations. To lock strict sub-5-word brevity and root-cause-first phrasing, we distilled our expert coaching transcripts—including real coaching session audio from professional coach Tony Rodriguez (T-Rod)—into a 12MB **LoRA Adapter (Rank 16, Alpha 32)** using Direct Preference Optimization (DPO).

We generated a dataset of 1,000 paired expert coaching scenarios (`coaching_dpo_dataset_train.jsonl` and `test.jsonl`). Each training sample pairs a structured telemetry cause hint with a Chosen (Preferred) Response and a Rejected (Dispreferred) Response:

// DPO Preference Pair Format

{
  "prompt": "trigger=exit_hesitation; phase=EXIT; corner=Sonoma_T3; speed=112mph; skill=INTERMEDIATE; hint=early_turnin",
  "chosen": "{\"speak\":true,\"action\":\"TURN_IN\",\"text\":\"Turn in later. Wait for curb.\",\"confidence\":0.96}",
  "rejected": "{\"speak\":true,\"action\":\"THROTTLE\",\"text\":\"You entered Turn 3 9 mph too slow because you turned in early. Try turning in later next lap.\",\"confidence\":0.70}"
}

The Chosen response enforces three non-negotiable DPO rules:

1. Sub-5-Word Constraint: When vehicle speed exceeds 70 MPH, total speech output must be $\le 5$ words.
2. Root-Cause First Action: Always lead with the physical fix ("Turn in later"), never describing the symptom ("You were slow").
3. 100% Schema Lock: Output strictly valid JSON with zero markdown code-fence wrappers (` ```json `), saving 80ms of string parsing overhead on device.

3. Empirical Training Results & 66% Latency Reduction

We executed the 250-step DPO training run directly on our workstation equipped with an NVIDIA GB10 GPU using PyTorch 2.10.0 + CUDA 13.0. The results across held-out test scenarios were dramatic:

Evaluation Metric	Base Gemma 4 (Zero-Shot)	DPO Fine-Tuned LoRA Coach	Net Gain
High-Speed Brevity Pass Rate (≤ 5 words)	7.5%	94.0%	+86.5%
Root-Cause First Accuracy	37.0%	100.0%	+63.0%
Strict JSON Validity Rate	100.0%	100.0%	100% Lock
Clean Format (No Backticks)	72.0%	100.0%	+28.0%
Average Utterance Length	13.8 words	4.2 words	69.6% Shorter
Estimated NPU Latency (TTFT)	443 ms	149 ms	294ms Faster (-66%)

By restricting required output token generation to max 12 tokens, Time-To-First-Token on Google Tensor G5 NPU dropped from 443ms down to 149ms! This speedup ensures the entire edge coaching loop fits comfortably inside our real-time budget.

Figure 2: Predictive lead-time trigger geometry at 130 MPH. Cues are fired 85.7 meters (1.6 seconds) before corner entry to land right before turn-in.

4. Predictive Lead-Time Math: Triggering Advice 85m Ahead

Even with a 149ms NPU latency, advice is useless if delivered mid-corner. To make coaching actionable, advice must land before the driver reaches the braking or turn-in point. We calculate dynamic velocity-scaled trigger distance:

D_trigger = v_vehicle × (T_reaction + T_speech + T_inference)

At 120 MPH (53.6 m/s), with $T_{\text{reaction}} = 0.35\text{s}$, $T_{\text{speech}} = 1.10\text{s}$ (4 words at 1.3x TTS speed), and $T_{\text{inference}} = 0.15\text{s}$, total lead time is 1.60 seconds. The engine triggers the cue exactly 85.76 meters before the corner entry zone.

5. Production Code Integration & Verification

We have integrated these DPO fine-tuned parameters directly into our Android engine and TypeScript coaching service. In [`LiteRtInferenceConfig.kt`](file:///home/rabimba/Documents/speedracer-AI/pixel-android-app/app/src/main/java/com/trustableai/koru/runtime/benchmark/LiteRtInferenceConfig.kt), production inference now uses a 16 maxToken budget with DPO prompt constraints:

// LiteRtInferenceConfig.kt (Android Engine)

val PRODUCTION = LiteRtInferenceConfig(
  name = "production",
  backend = LlmInference.Backend.GPU,
  maxTokens = 16, // Capped for sub-150ms TTFT
  sessionTemperature = 0.10f,
  promptStyle = LiteRtPromptStyle.COMPACT,
)

All training scripts, evaluation harnesses, and DPO dataset files have been committed and pushed to our repository under commit [`b72707b`](https://github.com/rabimba/speedracer-AI).

What's Next for Racecraft?

With sub-150ms DPO coaching unlocked, our next focus is implementing helmet Bluetooth SCO audio focus ducking and steering wheel Push-to-Talk (PTT) voice queries. Stay tuned as we take Project Koru to full field testing at Sonoma Raceway!