Hey everyone, welcome to my first blog post!
I’ve been planning to start writing for a while now, mostly as a way to document what I’m learning and, honestly, to force myself to think more clearly about it. So what better way to kick things off than by talking about the thing I’ve been completely obsessed with lately: robotics and Physical AI.
Let me back up a little. Over the past few months, I’ve become genuinely convinced that robotics and Physical AI are going to define the next era of technology. We’ve spent the last decade or so getting AI to understand and generate language, images, and video, but all of that intelligence has mostly stayed trapped inside a screen. The next frontier is getting that intelligence out into the physical world, into machines that can see a cluttered kitchen counter, understand what needs to happen, and actually do something about it. Autonomous vehicles, robots that can help around the house, robots that can work alongside humans in warehouses and factories, this is the direction things are heading, and I want to be part of building it, not just watching it happen.
That curiosity is what led me down the Vision-Language-Action (VLA) rabbit hole, and more specifically, to a model called Pi0.5.
So What Is Pi0.5?
Pi0.5 is an open source VLA model built on a flow matching policy, developed by the folks at Physical Intelligence. In simple terms, it’s a model that can look at a camera feed, read a natural language instruction like “put the mug in the sink” and directly output the motor commands needed to do it. So cool isn’t it?
But the thing that makes it stand out is how it handles generalization: it’s designed to work in messy, real-world environments it has never seen before, not just the tidy lab setups it was trained on, and it can transfer what it’s learned to entirely new robot embodiments with just a bit of fine-tuning on task specific data.
There’s no shortage of VLA models out there competing in this space, OpenVLA and Wall-OSS being two well-known examples, and each has its own strengths. But Pi0.5 is the one I ended up gravitating toward, mostly for practical reasons. Being open source means I can actually get under the hood: the checkpoints and weights are accessible and easy to fine tune, and I’m not stuck waiting on someone else’s API. On top of that, the documentation is genuinely some of the best I’ve come across in this space, clear, well-structured, and easy to follow even when the underlying concepts get technical. That’s not something I can say about every VLA project I’ve poked around in, where I’ve often found myself digging through sparse READMEs or piecing things together from GitHub issues just to get a basic setup running.
Where I’m At Right Now
Right now, I’m knee-deep in the unglamorous but necessary part of this project: building the software to collect training data. I’m using an industrial-grade manipulator from Elite Robotics CS66, and I’ve spent the last stretch of time teleoperating it using Meta Quest 3, essentially controlling it by hand to generate the kind of demonstration data that VLA models like Pi0.5 need to learn from.
The data collection pipeline is almost done. What’s left is testing it end to end and then starting to record actual sample data, just to make sure the quality holds up before I commit to collecting the full dataset. It’s the kind of work that doesn’t produce flashy results day to day, but it’s the foundation everything else depends on. Garbage data in, garbage policy out, so I’d rather get this part right than rush it. I won’t pretend this has been fast. Robotics projects like this move slowly, and there’s a real amount of patience required, debugging hardware quirks, tweaking the teleoperation setup, testing and re-testing before anything even resembles real progress. But slow doesn’t mean stuck. Every week I’m closer to having a working pipeline, and that counts for something.
What’s Next
Once the software is tested and validated, the real fun begins: actual data collection. I’m planning to start simple, pick and place tasks, but with enough variety baked in to make the dataset useful. That means switching up scenarios, changing the environment, and introducing a range of different props and objects rather than repeating the same setup over and over. Diversity in the data is what should eventually let the model generalize instead of just memorizing one narrow task.
I’ll keep documenting this as I go, partly for anyone else curious about this space, and partly to keep myself honest about the progress. If you’re working on anything similar or have thoughts on VLA models, teleoperation setups, or data collection strategies, I’d love to hear from you. You can contact me here
Thank you for reading!
More soon.