Google Research just dropped something that actually made me stop scrolling: Vibe Coding XR. It’s a prototyping workflow that hooks Gemini Canvas into the open-source XR Blocks framework, letting you turn a plain English prompt into a working Android XR app in under a minute. No game engine degree required.
The demo video shows someone typing “create a beautiful dandelion” into a browser on a Galaxy XR headset. Within 60 seconds, Gemini spits out a fully interactive 3D dandelion that responds to pinch gestures. The petals scatter when you touch them. That’s not a mockup—that’s real physics-aware code generated on the fly.
What’s actually new here
LLMs generating code isn’t new. We’ve all seen someone prompt ChatGPT into producing a tic-tac-toe game or a half-baked React component. What’s different here is the target: extended reality. XR prototyping has traditionally meant wrestling with Unity, Unreal, or custom perception pipelines. You’d spend days wiring up hand tracking, spatial anchors, and physics before you even saw anything in a headset.
Vibe Coding XR skips all that. The system combines Gemini’s long-context reasoning with curated XR Blocks code templates and specialized system prompts. It handles spatial logic automatically—scene layout, object physics, hand interactions, depth sensing. You describe what you want, and it builds the experience. The team claims it works on both desktop Chrome (with a simulated reality environment) and actual Android XR headsets like the Galaxy XR.
The workflow feels right
The process is straightforward: open the XR Blocks Gem in Chrome, type or speak your prompt, and Gemini designs the XR scene using multi-step planning. On a headset, you pinch the “Enter XR” button to launch the experience immediately. If something’s off, you tweak the prompt and regenerate. The share button creates a public link for others to test.
What I appreciate is that they didn’t force everything into the headset. The desktop simulator lets you iterate without strapping on hardware every five seconds. Advanced features like depth sensing and hand physics still need the real device, but basic layout and interaction testing works on a laptop. That’s practical.
The technical bits
The underlying framework, XR Blocks, is open-source on GitHub. It’s a web-based XR framework that handles the heavy lifting: scene graphs, physics engines, hand tracking integration, and rendering. Gemini acts as the code generator, using few-shot examples from XR Blocks to produce correct, runnable WebXR applications.
The team is presenting this at ACM CHI 2026, which makes sense—it’s a human-computer interaction conference, and this is fundamentally about lowering the barrier to entry for spatial computing. They’ve put up a live demo and a technical report if you want the nitty-gritty.
Where it falls short
Let’s be honest: vibe coding has limits. Complex multi-step interactions, custom animations, or anything requiring nuanced sensor fusion will probably still trip up the LLM. The generated apps likely work best for single-object scenes or simple spatial UIs. If you’re building a full multiplayer XR experience, you’re not getting there with a single prompt.
Also, the dependency on Chrome on Android XR means you’re tied to Google’s ecosystem and WebXR’s current capabilities. Native XR features like foveated rendering or advanced passthrough AR might not be accessible through this pipeline. It’s a prototyping tool, not a production framework.
Why this matters
Despite the caveats, this is the kind of tool that could actually get more people building for XR. The learning curve for spatial development is absurdly steep right now. Anything that cuts that down to a 60-second prompt-response loop is worth paying attention to. I’ve been burned by “no-code” XR tools before—they usually trade simplicity for capability. Vibe Coding XR seems to trade capability for speed, which is a better tradeoff for early-stage prototyping.
The team’s blog post mentions educational experiences and UI testing as use cases. I’d add quick concept validation for indie developers and hackathon projects. If you can sketch an idea in 60 seconds and test it in a headset, you’ve saved yourself a day of building something nobody wants.
You can try the live demo now, or dig into the XR Blocks source on GitHub. The video demo is worth watching—seeing a dandelion materialize from a text prompt and respond to your hand is still impressive, even in 2026.
Comments (0)
Login Log in to comment.
Be the first to comment!