Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
There is no permanently cheapest cloud for AI inference. In an April 2026 comparison of on-demand NVIDIA T4 instances, Google Cloud was lowest, followed by AWS and then Azure. That result applies to one GPU, one pricing mode and one date—not to every model, region or workload. Managed APIs charge by tokens, while self-hosted inference charges for GPUs and the machines around them, so the cheapest choice depends on your model, token mix, utilization, latency target, data-residency requirement and tolerance for interruptions.
The short answer: Google Cloud leads one T4 comparison, but that is not a universal verdict
TCOIQ’s 2026 cross-provider table lists these on-demand accelerator prices for a comparable NVIDIA T4 configuration:
| Provider | GPU and pricing basis | Price per GPU-hour | What the figure does not include |
|---|---|---|---|
| Google Cloud | NVIDIA T4, on demand | $0.35 | Machine-type cost, storage, networking, monitoring, orchestration and idle capacity |
| AWS | NVIDIA T4, on demand | $0.53 | Host instance, storage, data transfer, operations and idle capacity |
| Azure | NVIDIA T4, on demand | $0.56 | Host instance, storage, networking, observability, orchestration and idle capacity |
On that narrowly matched comparison, Google Cloud is about 34 percent below AWS and about 38 percent below Azure for the accelerator line item. It does not prove that Google Cloud will deliver the lowest cost per generated token. A different GPU, region, machine shape, model, quantization level or utilization rate can reverse the order. The table also compares a self-hosted compute component, not a managed large-language-model API.
For a cost decision, treat the T4 figures as a starting point. First decide whether you need a managed endpoint or control of your own serving stack; then compare equivalent capacity and performance.
#1 Best Overall
- Video/Sound Cards
- Passive Cooling
Two different products are often called inference
Cloud bills become confusing when a token API and a rented GPU are put in the same spreadsheet. They solve different operational problems and use different meters.
Managed, pay-per-token inference
A managed service runs the model, scales the endpoint and exposes an API. You generally pay for input and output tokens, sometimes for cached tokens, batch processing, provisioned capacity or a fine-tuned deployment. You avoid paying for a GPU while there are no requests, which is valuable for development, low volume and unpredictable traffic. The trade-off is less control over model weights, serving software, accelerator placement and sometimes data location.
Self-hosted inference on a GPU instance
You select a machine, install or deploy the model server and pay for the accelerator-hour plus the host machine and supporting services. This can be economical at sustained utilization or when you need a custom model, a particular quantization or a regional deployment unavailable through an API. You also assume capacity planning, patching, autoscaling, queue management, failover and security work.
Recommended Free Tools
Do not compare a provider’s price per million API tokens with a GPU-hour quote without converting both to the same unit, such as cost per one million generated tokens at a specified latency and utilization.
How the three providers bill managed inference
The exact same-model winner cannot be established from the available public catalogs because model names, regions, input and output meters, discounts and deployment options differ. Prices and availability change frequently, so record the model SKU, region and date whenever you quote a result.
AWS Bedrock
Amazon Bedrock publishes model-specific rates per 1 million input tokens and per 1 million output tokens. Some models also have provisioned model-unit charges. Your calculation therefore depends on the model selected and the ratio of prompt tokens to completion tokens. A workload with long retrieved context can cost very differently from one with short prompts and long answers, even when request counts are identical.
Azure Foundry
Microsoft Foundry documents token billing for model inputs and outputs and hourly hosting for some fine-tuned deployments. Its deployment choices matter:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Rank #2
- Original premium quality
- Item weight: 0.55 kg
- Size: Full-Height/Full-Length (FH/FL)
- Global Standard: pay per token for standard online requests.
- Global Batch: asynchronous, delay-tolerant processing; Microsoft Learn documents a 50 percent discount for eligible models and workloads.
- Provisioned throughput: purchase predictable capacity when demand is steady enough to justify a commitment.
Global Batch is not a blanket half-price setting for every model. Confirm eligibility, throughput behavior, regional processing terms and the current rate card before moving production traffic.
Google Cloud managed endpoints
Google Cloud’s managed model offerings, including Vertex AI endpoints, use model-specific pricing and availability rather than a single universal token rate. Check the selected model’s input and output meters, context or caching charges, batch options, regional restrictions and any provisioned-capacity requirement. A lower GPU rate on Compute Engine does not automatically make a managed endpoint cheaper, because the API may use a different accelerator fleet and include managed-serving overhead.
Self-hosted GPU prices and commitment options
Google Cloud’s official GPU pricing page, accessed October 18, 2026, lists the following NVIDIA T4 accelerator rates. The GPU charge is added to the cost of the machine type, as Google states: each GPU adds to the instance cost in addition to the machine-type cost.
| Google Cloud T4 option | GPU price | Commitment or availability condition |
|---|---|---|
| On demand | $0.35 per GPU-hour | Pay as used; region and machine availability still apply |
| One-year commitment | $0.22 per GPU-hour | Requires a one-year commitment; verify the associated commitment and machine charges |
| Three-year commitment | $0.16 per GPU-hour | Requires a three-year commitment; verify whether your forecasted utilization can sustain it |
| Spot | Dynamic | Google reports typical discounts of 60–91 percent for most machine types and GPUs, with interruption and price variability |
The commitment figures are accelerator charges, not an all-in server price. A commitment can reduce the unit rate while increasing financial exposure if your traffic falls, a model becomes more efficient or a different region is required.
GPU type also changes the comparison. TCOIQ lists a Google Cloud L4 at $0.72 per hour in its 2026 on-demand table. A faster or more memory-rich GPU may process more tokens per second and still produce a lower cost per token than a cheaper T4. Always benchmark throughput and latency with your actual model and precision.
nOps reported an August 2026 spot-to-on-demand range of $1.95–$12.29 per GPU-hour across H100, A100 and V100 instances on the three major providers. That range covers different GPU generations and is not a direct T4 comparison. It illustrates why a Spot quote must include the GPU type, date, region and interruption policy.
A fair comparison method
Use the following process before declaring a provider cheapest.
Rank #3
- NVIDIA Tesla T4 brings GPU Boost technology to boost performance of any application. Includes Error-Correcting-Codes (ECC) for protecting data reliability.
- PCI Express 5.0 host interface ensures dependable data transfer for maximum efficiency
- GDDR6 memory technology effectively enables data to be moved at various points in a CPU clock cycle to allow maximum productivity
- Plug-in Card form factor allows hassle-free and easy usage with increased efficiency
- Comes in 11.5" height for maximum productivity and easy carrying
- Freeze the model and precision. Specify the exact model version, quantization or numerical precision, context length and serving engine. Comparing a full-precision model on one provider with a quantized model on another changes quality and throughput.
- Define the request mix. Record input tokens, output tokens, requests per second, daily or monthly volume, batch percentage and cache-hit assumptions. Keep prompt and completion tokens separate because providers meter them separately.
- Set the service target. Write down time-to-first-token, tokens-per-second, p95 or p99 latency, availability target and maximum queue time. A cheap instance that misses the target is not a cheaper solution.
- Fix geography and residency. Choose the region or regions, state where data may be processed, and note cross-region replication or failover requirements. A globally processed deployment may not satisfy a residency boundary.
- Calculate managed API cost. Multiply input and output token volumes by the selected provider’s current rates. Add caching, batch, provisioned-capacity, fine-tuned-hosting and other applicable meters. Apply only discounts whose eligibility and term you have verified.
- Calculate self-hosted cost. Add GPU charges, host VM charges, boot and persistent storage, snapshots, inter-zone or internet networking, load balancing, logs, metrics, tracing, orchestration, support and security services.
- Model utilization, not just capacity. Calculate average and peak GPU utilization separately. Include warm-up time, autoscaler lag, failover replicas and capacity held idle for bursts.
- Show three pricing scenarios. Report on-demand, commitment or provisioned capacity, and Spot separately. State the commitment duration, Spot interruption assumption and the date of every rate.
- Convert to a common unit. Express the result as monthly cost, cost per request and cost per million input or output tokens at the same quality and latency. Include the operational labor needed to run the service.
Worked T4 example: why utilization changes the answer
Suppose a team keeps one T4 accelerator allocated for 730 hours, the approximate number of hours in a 30-day billing month. Using only the accelerator line item from the TCOIQ table, the arithmetic is:
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11| Provider | Rate used | 730-hour accelerator subtotal |
|---|---|---|
| Google Cloud | $0.35/hour | $255.50 |
| AWS | $0.53/hour | $386.90 |
| Azure | $0.56/hour | $408.80 |
These subtotals deliberately exclude the machine, disk, network and operations. If the same accelerator is active for only 20 percent of the month and the service can scale to zero, a pay-per-token API or an interruptible instance may have a better total cost. If traffic is continuous and the model saturates the GPU, a commitment or provisioned-throughput arrangement may amortize fixed capacity more effectively.
The example also shows why a lower hourly quote is not enough. If one provider’s machine delivers materially higher tokens per second, its cost per generated token can be lower even with a higher GPU-hour rate. Measure with production-shaped prompts rather than relying on accelerator names.
When each pricing model tends to win
Low-volume or bursty traffic
Managed pay-per-token inference usually fits workloads with long idle periods because you pay for requests rather than a continuously allocated GPU. Spot instances can be an alternative for asynchronous work if your queue can retry interrupted jobs and your data can be checkpointed. For interactive traffic, interruption risk may create unacceptable latency spikes.
Steady, high-throughput traffic
Reserved capacity, one- or three-year commitments, or provisioned throughput can lower unit cost when utilization is predictable. Before committing, calculate the break-even utilization: divide the committed monthly charge by the on-demand cost you would otherwise incur, then include minimums, scaling limits and cancellation risk.
Free tools Windows power users keep installed
One-click scans. No signup required.
Batch and asynchronous jobs
Azure Global Batch documents a 50 percent discount for eligible large asynchronous jobs. Its data-processing description allows any Azure region, while the deployment is billed per token. Check equivalent batch programs and current discounts on AWS and Google Cloud at the time you schedule the job; do not assume the same percentage or service limits.
Custom models and strict locality
Self-hosting may be preferable when you need proprietary weights, custom adapters, a specific tokenizer, an unusual context window or a region that managed catalogs do not support. The decision can be rational even when the headline hourly rate is higher because residency, control and predictable latency have business value.
Rank #4
Interruption-tolerant experimentation
Spot capacity can reduce compute cost substantially, but it requires retry logic, checkpointing, capacity diversification and a fallback plan. Treat the discount as variable rather than guaranteed. Maintain an on-demand path for deadlines or interactive requests.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Costs that headline GPU rates hide
Include these items in the same worksheet as the accelerator price:
- Host machine: CPU, memory, local SSD, operating-system disk and any minimum VM charge.
- Storage: model weights, container images, checkpoints, snapshots and replicated backups.
- Networking: ingress and egress, inter-zone traffic, private endpoints, gateways and cross-region replication.
- Serving layer: container orchestration, autoscaling, load balancing, API gateways, queues and deployment rollouts.
- Observability: logs, metrics, traces, request payload retention, dashboards and alerting.
- Reliability: standby replicas, health checks, failover regions, spare capacity and disaster recovery tests.
- Security and compliance: key management, private networking, policy enforcement, vulnerability scanning and audit retention.
- Operations: engineering time for upgrades, model releases, capacity planning, incident response and performance tuning.
- Idle and warm capacity: GPUs held ready for latency targets, autoscaler buffers and traffic that never arrives.
For managed APIs, replace most infrastructure lines with the provider’s token, batch, caching, provisioned-throughput and fine-tuned-hosting meters, but still budget for application networking, logging, retries and engineering work.
Region, latency and data residency can outweigh price
A rate card is meaningful only with its region and processing terms. A model available in one location may be absent in another, and a global deployment may process data outside the country your policy permits. Cross-region failover can add replication and network charges. Locating inference near users can reduce latency and egress even when the regional GPU rate is higher.
Write the residency boundary into the comparison: permitted countries or regions, whether prompts may leave that boundary, where logs are stored and how backups are handled. Then verify that the selected managed deployment or GPU stock actually satisfies it. If not, remove that option before comparing price.
Should you buy a GPU instead of renting one?
An NVIDIA T4 can be used for a self-hosted inference system, but a purchase is not automatically cheaper. Compare the fully loaded ownership cost with cloud rental:
- Purchase price, shipping, taxes and expected replacement cycle.
- Power draw, electricity rate, cooling and rack or office space.
- Host server, storage, networking, spare parts and warranty.
- Utilization across the year, including nights, weekends and model migrations.
- Administrator time, security updates, monitoring and physical access controls.
- Opportunity cost when a newer model needs more memory or a different accelerator.
Divide capital cost plus operating cost over the number of productive GPU-hours, then compare that figure with the cloud’s all-in on-demand, commitment and Spot scenarios. Ownership can win for a heavily utilized, stable workload, while rental preserves flexibility for changing models and demand.
A practical decision framework
- Start with a managed API when traffic is small, unpredictable or primarily an experiment. It minimizes idle capacity and shortens time to production.
- Move to batch pricing when work is asynchronous and can tolerate queueing. Confirm the model’s eligibility and regional processing terms.
- Evaluate provisioned or committed capacity after several weeks of measured demand show a stable baseline. Use the observed utilization, not a hopeful forecast.
- Consider self-hosting when custom weights, specialized serving, residency or sustained throughput justify the operational burden.
- Use Spot selectively for retryable jobs, evaluation runs and offline generation. Keep critical interactive paths on capacity you can depend on.
- Recalculate after every model or region change. A new context window, tokenizer, precision, accelerator or residency requirement invalidates the old comparison.
How to keep the result current
Cloud pricing and model catalogs are volatile. Store a dated record containing provider, service, model or GPU SKU, region, precision, token assumptions, request volume, latency target, discount type, commitment term and source page. Recheck quotas, availability, Spot behavior, batch eligibility and provisioned-capacity terms immediately before a production decision. A spreadsheet or FinOps calculator should show both peak and average utilization and expose every assumption rather than presenting one timeless “cheapest” number.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
