July 14, 2026, (Inside AI) — A new measurement study has quantified the actual electricity cost of running local large language models (LLMs) on a single NVIDIA RTX 3090, revealing surprising efficiency gaps that defy conventional assumptions about model size and cost.
The research, conducted by an independent developer using an open-source monitoring tool called HomeLab Monitor, benchmarked eight quantized models served by Ollama on a machine running openSUSE. The findings, published on Towards Data Science, show that five models came in cheaper than a hosted cloud API, but three did not — and the most expensive was not the largest.
“I measured the actual GPU electricity for eight local models on one RTX 3090 — and the cheapest wasn't the smallest, nor the priciest the biggest,” the author noted.
The core metric is euros per million output tokens, calculated by sampling GPU power draw every 10 seconds via nvidia-smi and integrating consumption over each run’s exact duration. Electricity costs used real Bulgarian day/night tariffs (converted at the fixed ECB peg of 1 BGN ≈ €0.5113). This granular approach moves beyond theoretical thermal design power estimates to actual measured energy use.
The benchmark tested three Gemma models — gemma3:1b, gemma4:26b, and gemma3:27b — with a dedicated 240-second sustained generation loop. Five additional models — GLM-4.5-Air (106B), DeepSeek-R1-Distill (32.8B), Seed-OSS (36B), Devstral (24B), and Qwen3-Coder (30.5B) — were evaluated using real multi-step coding-agent tasks. All models used Q4_K_M quantized GGUF weights for like-for-like comparison.
The results upend the folk wisdom that local inference is essentially free after hardware purchase. At €0.046 per million tokens, gemma3:1b was cheapest, while DeepSeek-R1-Distill topped the list at €1.526 — more than triple the cost of some larger models. For reference, hosted Flash-class APIs like Gemini 2.5 Flash or GPT-4o-mini run about €0.55 per million tokens, with the cheapest tier (Gemini 3.1 Flash-Lite) around €0.40.
The key driver is effective throughput, not raw generation speed. DeepSeek-R1-Distill, a reasoning-distilled model, spent significant time deliberating between generations, yielding an effective delivery rate of just 3.7 tokens per second — the lowest of all models tested. In contrast, gemma3:1b delivered 136 tokens per second at 154 watts.
“Size was never the mechanism. Effective speed was — and effective speed is not the number these models are usually benchmarked at,” the author explained.
The study underscores that per-token energy cost is a function of watts divided by throughput, but the right throughput number must account for real-world usage patterns including tool calls and reasoning gaps. This challenges the common practice of comparing models solely on parameter count or raw generation benchmarks.
Notably, the analysis only covers marginal GPU electricity — not CPU, DRAM, idle power, hardware amortization, or cooling. The author cautioned that the real cost of local LLM deployment is dominated by the upfront GPU purchase, which only pays off under high, steady utilization. The measured figures represent a floor, not a total cost of ownership.
The methodology is fully reproducible using the open-source HomeLab Monitor, which tracks GPU energy and prices runs in any configured currency. The tool integrates with Ollama and provides a programmatic API for cost analysis.
The findings arrive amid growing enterprise interest in on-premise AI to reduce API bills and address data privacy. However, they highlight that not all local models are cost-effective, and that reasoning-heavy architectures can incur hidden energy penalties. Industry analysts have long noted that inference efficiency depends on hardware-software co-design, and this data provides a concrete, measured example.
For practitioners, the takeaway is clear: choose the smallest, fastest model that meets quality requirements, and measure actual effective throughput under realistic workloads. As the author put it, “the variable that matters is tokens delivered per wall-clock second, full stop.”
