DeepInfra Emerges from Stealth With $8M to Make Running AI Inferences More Affordable

iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more

DeepInfra emerged from stealth on November 9, 2023, with an $8 million seed round led by A.Capital and Felicis to make open-source AI inference cheaper. Founded by former IMO Messenger engineers, the company initially hosted models such as Llama 2 and Code Llama; by 2026, it says it had become a production-scale inference cloud.

The launch focused on a part of AI infrastructure that was receiving less attention than model training: serving trained models to real users. DeepInfra’s argument was that better systems engineering, shared model hosting, and higher GPU utilization could reduce the cost of each generated token without requiring every developer to operate a GPU fleet.

#	Preview	Product	Price
1		Plugable Thunderbolt 5 AI eGPU Enclosure & Dock: 80Gbps, TAA Compliant	$629.99	Buy on Amazon
2		VIKINYEE Thunderbolt 3/4 USB4 to PCIe Expansion Chassis for Laptop and Mini PC, External Enclosure...	$319.00	Buy on Amazon
3		OwlTree PCIe 5.0 x4 128Gbps eGPU Dock, for 50 Series Graphics Cards, M.2 NVME to PCIe x16 Riser...	$99.99	Buy on Amazon

Key takeaways

• DeepInfra emerged from stealth on November 9, 2023, after raising an $8 million seed round led by A.Capital and Felicis.
• The company’s original product hosted open-source models, including Meta’s Llama 2 and Code Llama, through an inference API.
• VentureBeat reported a historical launch price of $1 per million input or output tokens, compared with $10 for GPT-4 Turbo and $11.02 for Claude 2 at the time; those figures are not current prices or an independent apples-to-apples benchmark.
• DeepInfra’s current documentation describes an OpenAI-compatible cloud covering language, vision, embeddings, image, video, speech, private deployments, and GPU rental.
• DeepInfra announced a $107 million Series B on May 4, 2026, and said it processed nearly five trillion tokens per week across more than 190 open-source models.
• Low token pricing alone does not prove lower total cost; buyers also need to test latency, concurrency, model quality, reliability, licensing, privacy, and migration effort.

What did DeepInfra announce in November 2023?

DeepInfra announced its emergence from stealth on November 9, 2023, alongside an $8 million seed financing led by A.Capital and Felicis. VentureBeat reported that Georges Harik and SVA also participated. The founding team came from IMO Messenger, giving the startup experience operating large-scale distributed systems rather than only building individual machine-learning models.

The initial focus was hosted inference for open-source models. Launch coverage named Meta’s Llama 2 and Code Llama, along with variants, tuned models, and other open-source systems. Instead of asking each developer to acquire GPUs, install a serving stack, load model weights, and manage capacity, DeepInfra offered access through a hosted API.

#1 Best Overall

Plugable Thunderbolt 5 AI eGPU Enclosure & Dock: 80Gbps, TAA Compliant

• Build Your Own AI Enclosure: The Plugable TBT5-AI is an 80Gbps high-performance Thunderbolt 5 eGPU enclosure featuring an 850W ATX 3.1 PSU and PCIe x16 slot with 4 lanes PCIe 4.0 to host your own GPU for offline AI models. (GPU not provided).
• Intelligence You Own: Resolve the innovation vs. privacy deadlock by running models like Llama 3 with an air gap. This secure system supports Ollama, LM Studio, Foundry Local, NVIDIA NIM, and llama.cpp, ensuring your sensitive prompts, data, and results never leave your perimeter. No cloud risks or subscription fees.
• Modular Performance Scales With Your Workflow: More than an external GPU enclosure, the TBT5-AI includes features like 96W host charging, 2.5Gbps Ethernet, downstream Thunderbolt 5 port, and 10Gbps USB-A and USB-C ports. The 850W PSU (80+ Gold) provides a dedicated 600W to your GPU, leveraging 80Gbps Thunderbolt 5 speeds for double the bandwidth of Thunderbolt 4.
• Works With: Thunderbolt 5, 4, and USB4 systems. USB4 must support eGPU: Designed for Windows 11, it connects via a single Thunderbolt 5 cable (included). Supports GPUs up to 346mm x 170mm x 77mm, and 3.5-slots wide, and 600W, fitting most high-end cards like NVIDIA, AMD. Check GPU dimensions before purchase. Not compatible with macOS, Linux, ChromeOS, or Thunderbolt 3.
• Lifetime Support: This TAA-compliant AI enclosure has been designed with reliability at its core and was built to meet the deployment demands of IT departments and the ease of use necessary for home offices. Includes lifetime support from our North American team of connectivity experts.

$629.99

Buy on Amazon

The November 2023 VentureBeat report on DeepInfra’s launch is the primary source for the round, founding background, initial model examples, and the company’s original cost argument.

What is AI inference, and why does it matter?

AI inference is the act of running a trained model to produce an answer or prediction. Training changes a model’s parameters; inference uses those parameters to answer a prompt, classify a document, generate an image, transcribe audio, create an embedding, or perform another task.

Inference becomes an infrastructure problem when an application moves beyond a single demonstration. A production service may need to handle many simultaneous users, large models, long context windows, lengthy generated responses, repeated requests, or chains of calls made by an AI agent. Every request must be processed quickly enough for the user while the provider keeps enough capacity available for traffic spikes.

The central challenge is not simply buying more GPUs. A provider must place many concurrent requests on hardware with limited GPU memory and memory bandwidth, load model weights efficiently, schedule work, manage queues, and keep tail latency under control. A GPU that is technically available but poorly utilized can still produce an unattractive cost per token.

DeepInfra’s CEO described the launch problem in terms of fitting concurrent users and model executions onto the same hardware. Each generated token requires computation and memory movement, so the economics depend on how efficiently the service handles requests—not merely on the theoretical performance of the accelerator.

How did DeepInfra propose to lower inference costs?

DeepInfra’s original proposal combined hosted infrastructure with systems optimization. The company aimed to operate server fleets as an inference service, share popular models across customers, and use the founding team’s distributed-systems experience to improve utilization.

• Shared model hosting: A provider can serve a popular model for many customers instead of requiring every customer to maintain a separate deployment.
• More efficient scheduling: Concurrent requests can be organized so that expensive hardware spends less time idle.
• Lower operational burden: Customers avoid purchasing GPUs, loading model weights, upgrading drivers, and building their own production serving layer.
• Model choice: Developers can select among open models, variants, and tuned versions rather than committing to one proprietary API.
• Distributed-systems experience: The founders’ background at IMO Messenger was presented as relevant to operating services across distributed infrastructure.

The available 2023 launch coverage explains the concurrency, computation, and memory-bandwidth problem, but it does not provide an independent benchmark, audited cost model, utilization rate, or detailed scheduler design. Claims about specific techniques such as continuous batching, quantization, speculative decoding, or kernel-level optimization should therefore not be attributed to the launch unless separately documented.

What was the original DeepInfra price comparison?

The historical comparison was deliberately attention-grabbing. The November 2023 launch coverage cited DeepInfra at $1 per million input or output tokens, compared with $10 per million tokens for OpenAI GPT-4 Turbo and $11.02 per million tokens for Anthropic Claude 2.

According to the November 2023 VentureBeat coverage, these were the prices used to illustrate DeepInfra’s cheaper-inference thesis. The figures should not be treated as current rates or as an independent, normalized benchmark. The compared models, input and output pricing structures, context limits, latency, throughput, reliability, support, and enterprise controls may not have been equivalent.

A lower price per token can also fail to produce a lower application bill. A less capable model may need longer prompts, more output, additional retries, or more agent steps. Higher latency can create business costs, and rate limits or failures can require extra engineering. The meaningful comparison is cost per successful task at the required quality and latency—not a headline token price in isolation.

Why were open-source models central to DeepInfra’s strategy?

Open-source and open-weight models can give developers more control over deployment, customization, fine-tuning, and model selection. They can also reduce dependence on a single proprietary provider and create opportunities to choose a smaller or specialized model for a particular workload.

DeepInfra’s 2023 thesis was that strong open models would lead to more derivatives, tuned versions, and application-specific systems. That ecosystem would create demand for a service that made models easy to discover and inexpensive to run.

However, “open-source model” is not a complete legal or technical description. Model weights, source code, training data, commercial-use rights, redistribution rules, acceptable-use restrictions, and attribution obligations can vary. A buyer should review the license for the exact model version being served rather than assuming that every model in a catalog has identical rights.

What does DeepInfra offer now?

DeepInfra’s current documentation describes an inference cloud rather than only the small hosted API announced in 2023. The platform provides an OpenAI-compatible interface and a catalog spanning multiple types of machine-learning workloads.

The current DeepInfra documentation describes these product areas. The documentation’s “100-plus LLMs” description and the company’s later “190-plus open-source models” announcement are different statements made in different contexts; they should not be collapsed into one supposedly exact catalog count.

Rank #2

VIKINYEE Thunderbolt 3/4 USB4 to PCIe Expansion Chassis for Laptop and Mini PC, External Enclosure with Dual PCIe Slot Compatible with Decklink Capture Card, for Windows/MacOS/Linux (VK-K2L)

• Expand Your Connectivity: Instantly add two external PCIe 3.0 x16 slots to any Thunderbolt 3, Thunderbolt 4, or USB4 enabled laptop and desktop. This expansion chassis is fully compatible with Mac, Windows, and Linux systems, ideal for pro hardware integration.
• 40Gbps Thunderbolt Bandwidth: Get max 40Gbps speed for high-performance cards. Ideal for video capture, 10GbE network adapters, NVMe SSDs, and pro audio interfaces. This PCIe enclosure delivers full bandwidth without bottlenecks. Includes a DisplayPort 1.4 output.
• 15W Laptop Charging & Stable Power: A robust 60W internal PSU ensures stable card operation. Each Thunderbolt port provides 15W PD for peripherals.
• Silent, All-Aluminum Enclosure: Engineered for durability and superior heat dissipation. The rugged aluminum body and passive cooling design operate silently, making it an ideal choice for noise-sensitive environments like recording studios or quiet offices.
• Plug-and-Play & Card Compatibility: Driverless setup for the chassis (PCIe cards may need drivers). Connect and go. Fits half-length, full-height PCIe cards up to 205x145x25mm (LxHxW). Please verify your card's dimensions before purchase.

$319.00

Buy on Amazon

How does the OpenAI-compatible DeepInfra API work?

For a simple chat-completion integration, the current OpenAI-compatible base URL is https://api.deepinfra.com/v1/openai. A developer creates an account, generates an API key in the dashboard, stores the key securely, and changes the SDK base URL.

1. Create a DeepInfra account.
2. Generate an API key from the dashboard.
3. Store the key as an environment variable rather than placing it directly in source code:

export DEEPINFRA_TOKEN="your_token_here"

A basic command-line request is:

curl "https://api.deepinfra.com/v1/openai/chat/completions" 
  -H "Content-Type: application/json" 
  -H "Authorization: Bearer $DEEPINFRA_TOKEN" 
  -d '{
    "model": "deepseek-ai/DeepSeek-V3",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'

The Python client can use the same general approach:

from openai import OpenAI

client = OpenAI(
    api_key="$DEEPINFRA_TOKEN",
    base_url="https://api.deepinfra.com/v1/openai",
)

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3",
    messages=[
        {"role": "user", "content": "Hello!"}
    ],
)

print(response.choices[0].message.content)

The official DeepInfra quick-start documentation says that existing OpenAI Python and Node.js SDK integrations can be adapted by changing the base URL and API key.

OpenAI compatibility is an API migration aid, not a promise of complete behavioral or feature parity. Structured outputs, tool calls, streaming events, audio, assistants, batch requests, embeddings, moderation, provider-specific parameters, and error formats may behave differently. Test the exact SDK features used by an application before treating a base-URL change as production migration.

How much does DeepInfra cost in 2026?

DeepInfra’s pricing is model-specific. The pricing page says language models may be billed by input and output tokens, while many other models are billed according to inference execution time. The page also says there are no long-term contracts or upfront costs.

The following examples were visible on DeepInfra’s pricing page on August 18, 2026. Prices are listed per million tokens and can change; they are not a universal DeepInfra rate.

Check the current DeepInfra pricing page before making a purchasing decision. The 2023 $1-per-million-token launch figure and the 2026 model-specific rates describe different points in time and should never be presented as one continuous price.

For image, video, speech, and other non-language workloads, execution-time pricing can make the calculation different from a token-only estimate. Buyers should check minimum charges, GPU idle time, dedicated-deployment costs, batch pricing, and the cost of retries or failed requests.

What changed from the $8 million seed round to 2026?

The company’s reported scale changed substantially after the 2023 launch. DeepInfra’s current company materials say it was founded in 2022. On May 4, 2026, DeepInfra announced a $107 million Series B and described a broader infrastructure business.

In the May 4, 2026 Series B announcement, DeepInfra said it processed nearly five trillion tokens per week, supported more than 190 open-source models, operated GPU infrastructure across eight U.S. data centers, and was expanding internationally.

Those scale figures are company-reported. The available dossier does not provide independent measurements of token volume, GPU utilization, latency, uptime, or profitability. The defensible conclusion is that DeepInfra’s stated business evolved from an early low-cost API thesis into a larger inference-cloud and infrastructure platform—not that every performance or cost claim has been independently established.

What does DeepInfra claim about privacy and enterprise controls?

At launch, CEO Nikola Borisov told VentureBeat that prompts were not stored or used. DeepInfra’s current materials advertise zero data retention, SOC 2 certification, ISO 27001 certification, secure U.S.-based data centers, dedicated deployments, and autoscaling.

These are company statements and should be checked against the scope of the applicable policies, reports, and contract terms. “Zero retention” should not be shortened to “DeepInfra stores no data.” A buyer must distinguish inference prompts and outputs from account, billing, security, abuse-prevention, support, and operational metadata.

Review whether retention rules apply identically to public APIs, private deployments, batch jobs, logs, support interactions, subprocessors, and customer-configured services. The DeepInfra data-privacy documentation is the relevant starting point, but regulated deployments should also require a contractual and security review.

DeepInfra’s current materials list support for A100, H100, H200, B200, and B300 deployments. Hardware availability, geographic placement, pricing, and compliance scope can vary by product and date, so those claims should be validated for the intended deployment rather than assumed from a general product page.

Rank #3

Sale

OwlTree PCIe 5.0 x4 128Gbps eGPU Dock, for 50 Series Graphics Cards, M.2 NVME to PCIe x16 Riser Cable 50cm, Supports Standard ATX Power Supply & External GPU for Mini PC NUC Laptop

• Package Include: NVME M.2 Female to PCIe x16 Enclosure Dock 50cm/19.7inch (Note: The GPU and Power Supply are not included)
• Compatibility (NVMe Protocol Only):✅ Supported: Laptops, ITX/STX motherboards, NUCs | M.2 interface with NVMe Protocol only. ❌ Not supported: M.2 SATA, M.2 WiFi, M.2 WWAN, USB interfaces
• Compatible Graphics Cards and Power Supply: Supports NVIDIA RTX 50 Series, AMD RX 90xx Series, new-generation PCIe 5.0 graphics cards (backward compatible) with transfer speed up to PCIe 5.0 x4 (128Gbps), as well as full-size high-performance graphics cards, with no concerns over length restrictions Compatible Power Supplies: standard ATX power supplies ONLY (500W or higher), featuring dual screw mounting (top & bottom) for enhanced stability.
• NVME M.2 to PCIe Cable: Ultra-thin & flexible design (0.8mm thickness) with premium aesthetics, eliminating port damage risks from rigid traditional cables. Flat cable architecture with full-coverage shielding and advanced EMI materials to minimize interference and performance degradation. The cable is highly flexible and can be bent freely without affecting functionality, but avoid pulling or tugging it excessively.
• Advantage of the dock: Our enclosue detachable design on both ends for improved portability and easy storage. PCB board with 10μ gold-plated contacts ensure superior conductivity and reduce oxidation/rust-related resistance that may cause system crashes or BSOD. Multi-status LED indicators provide clear visual feedback for real-time device monitoring.

$99.99

Buy on Amazon

When should a company use shared inference or a private deployment?

Shared inference is generally the simpler starting point for public models and variable experimentation. Private deployment is more appropriate when a team needs custom weights, private endpoints, stronger isolation, or more control over capacity and model lifecycle.

DeepInfra’s private-model documentation presents private deployments for fine-tuned and customer-owned models, with private endpoints and autoscaling described as part of the offering. The practical question is whether the additional isolation and control justify dedicated GPU and operational costs for the workload.

Who is DeepInfra a good fit for?

DeepInfra is most compelling for developers and companies that want hosted access to open models through a familiar API without operating GPUs themselves. High-volume applications may benefit from model-specific token prices, while teams with custom weights can evaluate private deployments.

• Open-model developers: Teams can experiment with multiple language, vision, embedding, speech, and generative-media models.
• Cost-sensitive applications: Applications can compare smaller or specialized models against expensive proprietary alternatives.
• OpenAI-SDK users: Simple chat-completion integrations may require only a base-URL and credential change before deeper testing.
• Enterprise model owners: Teams with fine-tuned or customer-owned weights can investigate private serving.
• Infrastructure teams: GPU instances and clusters may suit organizations that need more control than a hosted endpoint.

DeepInfra may be a poor fit for applications that require a particular proprietary model’s exact behavior, a hyperscaler-only procurement path, untested low-latency performance at high concurrency, or compliance guarantees that have not been confirmed contractually.

What alternatives should buyers compare?

The right alternative depends on whether the buyer wants a turnkey API, a broad model marketplace, managed private serving, hyperscaler governance, or raw GPU control.

Current competitor pricing and plan terms were not verified for this article. Compare each provider using the same model version, region, concurrency, quality target, and production requirements rather than relying on advertised starting prices.

How should you evaluate DeepInfra before production?

A provider test should measure the complete cost and behavior of the intended application, not just one successful API call.

1. Select representative work: Use at least three real prompt types, including short and long contexts, normal and worst-case outputs, and any tool or structured-output calls the application needs.
2. Pin the model: Record the exact model name, version, precision or quantization where disclosed, context limit, and license.
3. Measure performance: Record time to first token, total latency, tokens per second, tail latency, streaming behavior, and error rates.
4. Test realistic concurrency: Repeat the workload at expected average and peak concurrency. A single-request benchmark says little about queueing or capacity behavior.
5. Calculate successful-task cost: Include input tokens, output tokens, cached tokens where available, retries, failures, routing, monitoring, and any dedicated GPU charges.
6. Check reliability: Review rate limits, 429 and 5xx behavior, queue handling, failover options, status communication, and any service-level commitment.
7. Review privacy and compliance: Confirm retention, training use, encryption, access controls, SSO, audit logs, data residency, subprocessors, and the scope of SOC 2 or ISO 27001 claims.
8. Test migration risk: Exercise tool calls, structured responses, streaming, embeddings, batch jobs, error handling, and model replacement before committing to compatibility.
9. Confirm licensing: Review commercial-use and redistribution terms for every model used in the product.

Does the original “cheaper inference” thesis still hold?

The original thesis still describes a real buyer concern: inference economics depend on serving efficiency, not just on the cost of acquiring a model. DeepInfra’s later funding and company-reported usage indicate that the market for production AI inference expanded beyond the narrow 2023 launch product.

But “cheaper inference” is not a universal verdict. DeepInfra’s current prices vary sharply by model and workload, and a low per-token rate may be outweighed by lower quality, higher latency, more retries, capacity constraints, private-GPU costs, or the engineering required to manage model changes.

The strongest conclusion is narrower and more useful: DeepInfra’s $8 million seed round backed an early bet that open-model serving would become a major infrastructure market. By 2026, the company said that bet had grown into a vertically integrated inference cloud serving production-scale and agentic workloads. Whether DeepInfra is the economical choice for a particular buyer remains a workload-specific question that requires testing and contract review.

Frequently Asked Questions

When did DeepInfra emerge from stealth?

DeepInfra emerged from stealth on November 9, 2023, announcing an $8 million seed round led by A.Capital and Felicis. The company was founded by former IMO Messenger engineers and initially focused on hosting open-source models such as Llama 2 and Code Llama.

What was DeepInfra’s original launch price?

VentureBeat reported a historical DeepInfra price of $1 per million input or output tokens in November 2023. That figure is not a current quote and was not an independent apples-to-apples comparison with GPT-4 Turbo or Claude 2.

Is DeepInfra OpenAI-compatible?

Yes. DeepInfra documents an OpenAI-compatible API with the base URL https://api.deepinfra.com/v1/openai. Simple chat-completion applications can often adapt an existing OpenAI SDK integration by changing the base URL and API key, but feature compatibility should be tested.

Does DeepInfra retain prompts and outputs?

DeepInfra advertises zero data retention for inference data, but buyers should review the current privacy documentation and contract terms. Zero retention of prompts and outputs does not necessarily mean that account, billing, security, abuse-prevention, support, or operational metadata is never retained.

What did DeepInfra announce in 2026?

DeepInfra announced a $107 million Series B on May 4, 2026. The company said it processed nearly five trillion tokens per week, supported more than 190 open-source models, and operated GPU infrastructure across eight U.S. data centers; those figures are company-reported.

The Bottom Line

DeepInfra began as a November 2023 bet that open-source model inference could become cheaper through shared hosting and better systems utilization. The company’s 2026 expansion suggests that the inference market did grow, but the best provider is determined by successful-task cost, latency, reliability, model rights, privacy scope, and operational control—not by the lowest token price alone.

Quick Recap

Bestseller No. 1

Plugable Thunderbolt 5 AI eGPU Enclosure & Dock: 80Gbps, TAA Compliant

$629.99

Bestseller No. 2

VIKINYEE Thunderbolt 3/4 USB4 to PCIe Expansion Chassis for Laptop and Mini PC, External Enclosure with Dual PCIe Slot Compatible with Decklink Capture Card, for Windows/MacOS/Linux (VK-K2L)

$319.00

SaleBestseller No. 3

OwlTree PCIe 5.0 x4 128Gbps eGPU Dock, for 50 Series Graphics Cards, M.2 NVME to PCIe x16 Riser Cable 50cm, Supports Standard ATX Power Supply & External GPU for Mini PC NUC Laptop

$99.99

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.