Blog
Notes from the lab.
Longer write-ups of things I've built and measured. Mostly robots that learn, and what happens when they don't.
- What work gloves do to hand tracking Human video is becoming robot training data, and in factories and warehouses the people in that video wear gloves. I measured what that does to 3D hand-pose estimation, and which capture choices actually fix it.
- The policy that ignored the instruction I fine-tuned a 500M vision-language model into a robot policy. It scored 0%, and the reason turned out to be a property of the data, not a bug in the model.
Physical-AI notes
Study notes, behind a very small door.
Interactive notes I keep while learning the robot-learning stack: quantization, NPUs, foundation models, deployment, reconstruction and touch. Rough around the edges, and full of widgets.
- Optimization Fewer bits, same robot Number formats, calibration, GPTQ, AWQ and SmoothQuant, rooflines, fusion, CUDA graphs and TensorRT, each with a widget to poke at.
- Edge NPUs Qualcomm AI stack field guide QAIRT, QNN, the Hexagon NPU and its siblings, op support, picking a backend, and getting from a checkpoint to a tuned context binary.
- Overview Physical AI systems map The whole pipeline in fifteen stages, from sensors and data collection to training, edge inference, fleets and safety.
- Models Foundation models, read like code ViT, CLIP, DINOv2, LLaVA, OpenVLA and π0.5: clickable architectures, the math, tensor shapes step by step, and the lines of code that matter.
- Deployment GR00T N1.6 → ONNX → TensorRT Splitting a VLA into six graphs, building engines on the target, binding tensors, and proving the actions didn't move.
- Reconstruction Real2sim: TSDF scans vs Gaussian splats Two ways to turn the real world into a sim asset: robot-guided RGB-D fusion, and a 3D Gaussian splatting pipeline.
- Touch Self-supervised tactile encoders Learning one representation from very different touch sensors with per-sensor stems, a shared transformer and the DINOv2 recipe.