Benchmarking Pocket-Scale Inference

Let's be honest: for years, "mobile AI" meant shipping your keystrokes off to a server farm and praying for low latency. This kind of benchmarking from Artificial Analysis is a welcome corrective. When you're measuring tokens per second on a phone, you're not just flexing on a spec sheet — you're asking a genuinely revolutionary question: can a model live entirely on a device you actually own, without phoning home? That's the territory where open source models like Llama and Qwen start to matter, because you can't run a proprietary cloud model on your iPhone without someone else's blessing. The broader story here is sovereignty. On-device inference isn't just a performance metric; it's a political and licensing battleground. If the best model you can run locally is a 7B parameter open weights model, then the license that governs those weights — and the hardware that runs them — becomes the new chokepoint. Apple, Qualcomm, and MediaTek are now in the business of making their silicon a *platform* for open models, and that's a massive shift from the closed, API-driven world of "AI." This is where the open source community wins or loses the next decade. I want to see more granular, reproducible benchmarks like this — power draw, memory bandwidth, quantization levels — because the hardware stack is the quiet enabler of the software freedom movement. A phone that runs a solid open model at 20 tokens per second is a device that can't be spied on, can't be rate-limited, and can't be shut down by a cloud provider. That's a big deal. The article's approach of testing the entire inference stack, not just the model, is exactly the kind of rigor we need to hold vendors' marketing claims accountable. So here's the pragmatic takeaway: if you care about open source AI, you should care about mobile silicon as much as you care about the model weights. Read the benchmarks, and let's push for standards. Because in the pocket-scale inference war, the open source community's best ally is a well-measured watt and a permissive license. Check the full data at the source: https://artificialanalysis.ai/hardware-inference-stack/mobile-phones.