November 11, 2026, (Inside AI) — Amazon Web Services has introduced usage-based metering for bidirectional streaming inference on Amazon SageMaker AI, a technical fix that lets model providers bill customers for actual consumption during live audio sessions rather than flat hourly rates. The capability, announced today, launches with Deepgram, the real-time speech AI company, as its first partner. Deepgram will use the system to bring per-second Speech-to-Text and Text-to-Speech billing to AWS Marketplace.
The change addresses a structural gap in cloud billing for streaming AI. Under the previous AWS Marketplace metering model for SageMaker AI, model providers reported a unit count in an HTTP response header. That worked for request-response APIs where the billable quantity was known at the moment of response. Bidirectional streaming, however, keeps a session open for anywhere from 30 seconds to 30 minutes. Usage accumulates continuously, and the final total is unknown until the session ends. A single response header cannot report periodic usage mid-stream.
The new system solves this by opening a second WebSocket connection alongside the inference stream. When a streaming session starts, SageMaker AI establishes two connections to the model server: a data connection carrying audio and transcriptions, and a metadata connection dedicated to metering data. Model servers opt in by returning a specific header on the WebSocket upgrade response. SageMaker AI then opens the companion connection at a designated path, with both connections carrying the same request ID so the model server can correlate metering records with the active session.
Model servers send metering data as JSON text frames on the metadata connection. AWS recommends sending records incrementally as usage accrues rather than waiting until session end, which reduces unreported usage if a session terminates unexpectedly. The metadata stream has a rate quota of one message per second per connection and a maximum message size of 512 bytes. The ClientToken field supports idempotency, and duplicate tokens on the same connection are silently dropped.
Read: ElevenLabs Expands Nvidia Partnership to Power Voice Agents with Nemotron 3.5 and Vera Rubin
The design includes a configurable failure mode. By default, if the metadata connection fails to establish or drops mid-session, inference continues. Providers can set a header to true to stop inference if metering becomes unavailable, trading inference availability for metering continuity. The documentation notes this choice does not guarantee that malformed or dropped records are billed.
For customers, the shift means the AWS Marketplace software charge aligns with actual audio processed rather than endpoint uptime. If a product charges per second of audio, 12 minutes of metered audio equals 720 billable seconds. An open connection lasting 12 minutes does not by itself define the billable quantity. The provider's counting rules determine metered usage. SageMaker AI endpoint infrastructure charges remain separate and continue to apply while the endpoint is provisioned.
The launch builds on a series of integrations between Deepgram and AWS. Earlier work introduced bidirectional streaming for real-time inference on SageMaker AI, followed by Deepgram's addition of AWS Identity and Access Management temporary delegation for secure authentication and expanded observability metrics. The usage-based metering capability completes a billing layer that was missing from the streaming stack.
For model providers, the integration requires three steps. First, the model server opts in by including headers in the WebSocket upgrade response. Second, it accepts the metadata WebSocket connection and uses the request ID to correlate it with the data connection. Third, it sends metering messages as JSON objects in WebSocket text frames, with each frame containing exactly one record. AWS provides an onboarding agent skill with starter templates that advertise metadata-stream support and build validated ConsumedUnits frames, though the company advises reviewing and production-hardening the code before submission.
Validation guidance is specific. AWS recommends testing with a Limited listing and an allowlisted buyer, running a known-quantity test such as sending 12 minutes of audio and checking that records total 720 units. Providers should reconcile model-server records with seller usage reports and buyer charges, allowing time for reports to populate. They should also test normal completion, interrupted sessions, metadata-stream loss, and malformed messages, comparing reported and billed quantities for each case before release.
The billing change arrives as real-time voice AI moves from experimentation to production in contact centers, voice agents, and captioning systems. Hourly software pricing can charge customers during quiet periods when no audio flows. Usage-based metering lets providers align charges with consumption, a model that better fits the economics of streaming workloads where demand fluctuates second by second.
Read: Singapore Airlines Reports 2x Chatbot Satisfaction With OpenAI Models
For customers deploying Deepgram through AWS Marketplace, the process requires no client-side configuration. They subscribe to a Deepgram streaming model, deploy a SageMaker AI real-time endpoint using the model package, and connect through the bidirectional streaming API. Deepgram's model server calculates metering automatically, and AWS Marketplace processes it. The software charge flows into the consolidated AWS bill, and buyers can negotiate private offers with per-unit rates or fixed-price contracts.
The capability also carries implications for how AI infrastructure gets priced more broadly. As more model providers offer streaming inference for voice, video, and other continuous data, the ability to meter consumption at fine granularity becomes a competitive requirement. AWS has now built that meter into its marketplace plumbing. Whether competitors follow with equivalent systems will shape how enterprises compare costs across clouds for real-time AI workloads.
Shamil has a low tolerance for AI that only works in controlled demos. He covers agentic systems and generative AI with the skepticism of someone who has seen too many “revolutionary” tools quietly disappear six months after launch.
