Google’s new TPUs: one for thinking, one for doing

Google just dropped its eighth generation of TPUs, and for the first time they’re splitting the workload across two distinct chips. Not a single do-everything accelerator, but two specialized ones designed for what they’re calling the “agentic era.”

If you’ve been following the AI hardware race, this is a notable shift. For years, TPUs have been general-purpose matrix engines, good for training and inference alike. The new lineup breaks that mold.

One chip, codenamed “Thinker,” is optimized for reasoning, planning, and chain-of-thought processing. The other, “Doer,” is tuned for high-throughput execution — the kind of work you need when an agent is calling APIs, retrieving data, or generating responses in real time.

This makes sense if you’ve spent any time building agentic systems. The bottleneck isn’t just raw compute anymore — it’s the mismatch between planning latency and execution throughput. A model that spends half its time reasoning and half acting needs different hardware profiles for each phase. Google’s bet is that splitting them yields better overall efficiency.

I’m curious about the software story here. TPUs have always been tied tightly to TensorFlow and JAX, and this two-chip architecture will demand some careful orchestration. If Google’s compiler stack can automatically route planning ops to Thinker and execution ops to Doer, that’s a win. If developers have to manually annotate their models, adoption might be slower.

Performance numbers weren’t detailed in the announcement, but Google claims significant improvements in tokens-per-second for agentic workloads compared to the previous generation. Given that agent loops are where inference costs really balloon, any efficiency gain there is welcome.

The bigger picture is clear: the industry is moving beyond simple text generation and into autonomous agents that plan, act, and iterate. Hardware vendors are responding. NVIDIA’s Grace Hopper and AMD’s MI300 are also targeting agentic patterns, but Google’s approach of dedicated silicon is more aggressive.

Whether this pays off depends on how quickly agentic workloads become mainstream. Right now, most production AI is still single-turn inference or simple RAG. But if the predictions are right and agents explode in the next 18 months, Google will have hardware that’s purpose-built for the job.

For now, I’m cautiously optimistic. The specialization makes sense on paper, and Google has the scale to iterate fast. But I’ll reserve judgment until I see real benchmarks — and until the software tooling catches up to the hardware ambition.