The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
LiteLLM lets a homelab expose OpenAI-compatible API endpoints for hosted services, local Ollama models, and other providers through one proxy. Your applications keep one base URL while LiteLLM handles provider-specific requests, retries, access control, spend records, and ordered fallback between deployments.
For a single machine, start with a configuration-file container on port 4000. Add Docker Compose and PostgreSQL when you need the Admin UI, virtual keys, persisted model settings, spend logs, or budgets. Add Redis only when multiple LiteLLM instances must share routing and rate-limit state. In LiteLLM terms, “waterfall” is best understood as ordered fallback: try the preferred deployment, then retry or send the request to another deployment after an error or outage.
What is LiteLLM, and why is it useful in a homelab?
LiteLLM is an open-source, MIT-licensed AI gateway that presents one OpenAI-format interface in front of many model providers. The documentation describes a self-hosted proxy for more than 100 providers, while the product site currently advertises 140-plus providers and 1,892 models (figures accessed October 26, 2026). Provider catalogs and those counts change, so treat them as current vendor-maintained figures rather than a permanent specification.
The practical benefit is application stability. A script can send requests to your LiteLLM address using a normal OpenAI client; changing from a hosted model to an Ollama model, or moving to a second provider during an outage, happens in the gateway configuration rather than in every application.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
- One endpoint: applications use one base URL and authentication pattern.
- Provider abstraction: LiteLLM translates the OpenAI-style request to the selected provider.
- Resilience: retries and fallbacks can move a request to another deployment when the first one fails.
- Administration: the self-hosted proxy can provide virtual keys, spend tracking, an Admin UI, load balancing, guardrails, and logging integrations.
- Privacy options: a local provider such as Ollama can remain inside the homelab, while hosted providers remain available through the same API.
Self-hosting is free under the MIT license, but upstream model providers still charge for their own APIs, and you remain responsible for the compute, storage, backups, and security of your gateway.
Which LiteLLM deployment should you choose?
Choose the smallest architecture that supplies the control you actually need. A file-based container is simplest; the PostgreSQL-backed Compose stack is the useful middle ground for most serious homelabs.
| Deployment | What it provides | What it does not provide | Best fit |
|---|---|---|---|
| Single container with YAML | One OpenAI-compatible endpoint, provider routing, retries, and ordered fallbacks | LiteLLM virtual keys and global budget enforcement without a database | Personal projects, a few trusted applications, and a single host |
| Docker Compose plus PostgreSQL | Admin UI at /ui, persisted models and keys, spend logs, virtual keys, and budget-related management |
Shared routing state across replicas unless Redis is added | A homelab serving several users or applications |
| Helm or Terraform production pattern | Replicated stateless services, load balancing, managed data stores, and secret-manager integration | The simplicity of a one-box installation | Multiple nodes, high availability, or an existing Kubernetes and cloud-operations practice |
Do not add PostgreSQL merely because a model provider needs it; LiteLLM’s database is for gateway identity, configuration, spend, and management features. Add Redis when more than one LiteLLM instance must coordinate rate limits, router state, or shared caching.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWhat do I need before installing LiteLLM?
Prepare the network address, provider credentials, and a decision about persistence before starting.
- A Linux host, NAS, or other machine that can run Docker, with port 4000 available on the local network.
- API credentials for each hosted provider you intend to use. Keep them in an environment file or secret store, never in a public repository.
- An Ollama host and reachable API address if you want local models. Containerized LiteLLM must be able to resolve that address; on some Docker setups, the Ollama service is on the host rather than inside the container.
- A model configuration file for the container-only path, or PostgreSQL credentials for the Admin UI path.
- A plan for backups. Preserve the YAML file, environment secrets, and PostgreSQL data together so a replacement host can reproduce the gateway.
The gateway itself does not remove upstream account requirements. Each provider still controls model availability, quotas, authentication, and billing.
How do I run a basic LiteLLM proxy in Docker?
The minimal path mounts a YAML file, passes provider credentials as environment variables, and publishes port 4000. This keeps the footprint small and avoids introducing a database before you need one.
- Create the configuration file. Save a file named
litellm_config.yamlin a directory reserved for the gateway. Define one or more logical model names and map each to a provider-specific model. - Store secrets separately. Put variables such as
OPENAI_API_KEYor an equivalent provider credential in a protected.envfile. Restrict its permissions and exclude it from source control. - Start the container. A typical Docker invocation mounts the configuration at
/app/config.yaml, loads the environment file, and starts the proxy on port 4000:
docker run -d
--name litellm
--restart unless-stopped
-p 4000:4000
--env-file .env
-v "$PWD/litellm_config.yaml:/app/config.yaml:ro"
ghcr.io/berriai/litellm:main-latest
--config /app/config.yamlPin a tested release tag instead of tracking a moving tag for a long-lived installation. The exact image tag and command-line options can change between releases; check the version’s current startup instructions before upgrading.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
- Check the logs. Run
docker logs -f litellmand look for a successful server start and a listener on port 4000. Authentication or YAML errors normally appear before the listener is ready. - Point a client at the proxy. Use the proxy URL as the OpenAI client’s base URL and send the logical model name from your configuration. The application no longer needs provider-specific endpoint logic.
Keep port 4000 on the LAN or behind a VPN while you validate the setup. Exposing the port directly to the internet turns the gateway’s credentials and upstream spending into an internet-facing security problem.
How should the model configuration map providers to one endpoint?
Give applications stable logical names and keep provider-specific details in litellm_config.yaml. A minimal pattern looks like this:
model_list:
- model_name: fast-chat
litellm_params:
model: openai/gpt-4o-mini
api_key: os.environ/OPENAI_API_KEY
- model_name: local-chat
litellm_params:
model: ollama/llama3
api_base: http://ollama:11434
The names and provider model identifiers in this example are illustrative. Replace them with models available in your accounts and with the network address that your LiteLLM container can reach. If Ollama runs directly on the Docker host, http://ollama:11434 may need to be replaced with a host gateway address or the host’s LAN address.
Applications can then use the usual OpenAI client shape, changing only the base URL and key used to reach LiteLLM:
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsfrom openai import OpenAI
client = OpenAI(
base_url="http://litellm-host:4000/v1",
api_key="your-litellm-key"
)
response = client.chat.completions.create(
model="fast-chat",
messages=[{"role": "user", "content": "Hello"}]
)For the no-database container, treat the gateway credential as a trusted internal secret. Virtual-key management and global budget controls require a connected database, so the simple path is not a substitute for the Admin UI when several people or applications need separate quotas.
How do I configure waterfall-style fallback?
LiteLLM does not need a separate product mode called “waterfall” for this pattern. Define multiple deployments behind one logical model name and configure retries and fallbacks so the preferred deployment is attempted first and another deployment is selected after an eligible failure.
Build an ordered deployment group
Use the same model_name for equivalent deployments, such as a hosted model followed by a second provider or a local model. Keep the preferred deployment first in your operational plan, then identify which failures should trigger a retry and which should move to the next deployment. Authentication failures, exhausted quotas, timeouts, and provider outages often deserve different handling from a malformed request that will fail everywhere.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
Set retries and fallback targets
LiteLLM Router supplies retry and fallback behavior. Configure the retry count conservatively so a slow provider does not multiply latency, then list fallback deployments for the logical model. Validate the exact configuration keys against the LiteLLM release you install because Router settings evolve; the concepts are a retry count, fallback mapping, and a routing strategy.
Test the failure path
- Send a normal request and confirm it reaches the preferred deployment.
- Temporarily make that deployment unreachable or use a test credential with a controlled failure.
- Confirm the proxy retries only as configured, then sends the request to the next deployment.
- Inspect logs to verify which deployment answered and whether the response remains compatible with your application.
- Restore the preferred provider and repeat the test, because a fallback that never returns to the primary can hide a permanent configuration error.
Fallback is not semantic equivalence. A local model may have a smaller context window, different tool support, or different output quality. Applications that depend on a specific context length or structured-output feature should detect those differences rather than silently accepting any response.
Which Router strategy should a homelab use?
LiteLLM Router supports several selection strategies. For a single instance, simple-shuffle is the documented production default because it distributes requests without the extra coordination overhead of usage-based decisions.
| Strategy | How it selects a deployment | Trade-off |
|---|---|---|
| Simple shuffle | Shuffles eligible deployments before selection | Low overhead and a practical default; it does not optimize for current latency or cost |
| Weighted pick | Uses configured weights to send more traffic to selected deployments | Useful for gradual migrations or different capacities; weights require deliberate tuning |
| Rate-limit aware | Considers remaining provider limits | Can avoid exhausted deployments, but depends on accurate limits and state |
| Latency based | Prefers deployments with better observed response times | Can improve responsiveness while adding measurement and state overhead |
| Least busy | Chooses the deployment with the fewest active requests | Helpful for uneven workloads; coordination is more complex than simple shuffle |
| Lowest cost | Prefers the deployment configured as least expensive | Useful for budget-sensitive traffic, but cost data and quality differences must be maintained |
| Custom | Applies your own selection function | Most flexible and most operationally expensive to test and maintain |
Router can also apply deployment requests-per-minute and tokens-per-minute limits before a call. Those pre-call checks filter deployments that would exceed their configured limits, reducing avoidable provider errors.
When do I need PostgreSQL?
Use PostgreSQL when LiteLLM must remember management data beyond a process restart. The Docker quickstart’s Compose stack runs LiteLLM with PostgreSQL and exposes the Admin UI at /ui.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Download the current Docker Compose quickstart supplied by the LiteLLM project and review its image tags before starting it.
- Generate a strong
LITELLM_MASTER_KEYand a separateLITELLM_SALT_KEYin the environment used by the stack. - Start the Compose services and wait for both LiteLLM and PostgreSQL to report healthy.
- Open the proxy’s
/uipath, add provider models, create virtual keys, and send a Playground request. - Back up PostgreSQL and the environment secrets together.
The database-backed workflow stores models, keys, and spend logs. It is also the practical path for budget-related controls. Without a database, the official quickstart states that global budgets are not enforced and virtual keys fail because there is nowhere to persist them.
What do the master key and salt key protect?
LITELLM_MASTER_KEY is the gateway’s root credential. Anyone who obtains it has full administrative access, so do not commit it to a repository, place it in a chat message, or reuse it for ordinary application traffic. Rotate it after a suspected leak and update every client or automation that used it.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
LITELLM_SALT_KEY encrypts provider API keys stored by the UI. Preserve it as carefully as the database backup: changing the salt makes existing stored credentials unreadable until they are entered again. A lost salt therefore creates a credential-reentry outage even if PostgreSQL itself is intact.
When should I add Redis or multiple LiteLLM instances?
A single container does not need Redis. Add Redis when you run replicas and need shared rate limiting, Router state, or caching across those instances; otherwise each process has only its local view.
The documented production architecture separates an HTTPS load balancer, stateless LiteLLM services, PostgreSQL for keys, teams, users, spend, and configuration, Redis for rate limiting and router state, and a secret manager for gateway and provider credentials. Helm deployment is documented for EKS, GKE, and AKS, while Terraform modules cover AWS and GCP.
For a homelab growth path, keep the stages explicit:
- One container and YAML for trusted, low-volume use.
- Compose plus PostgreSQL when identity, virtual keys, spend records, or budgets matter.
- Redis plus replicas when a single process is no longer an availability target you can accept.
- Helm or Terraform only when you already operate Kubernetes or managed cloud infrastructure well enough to justify the additional failure modes.
What commonly breaks, and how can I fix it?
The proxy starts but a model request fails
Check the provider prefix and model identifier in the YAML, then verify that the environment variable is present inside the container rather than only in your shell. Review container logs for authentication, DNS, TLS, or provider quota errors.
Ollama cannot be reached
Test the Ollama URL from the LiteLLM container’s network namespace. A service name works only when both containers share a Docker network; a host-local address such as localhost points back to the LiteLLM container, not the Docker host.
Virtual keys do not work
Confirm that LiteLLM is connected to PostgreSQL and that the database is reachable with the configured credentials. The no-database mode cannot persist virtual keys.
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
Budgets appear to be ignored
Global budget enforcement requires the database-backed deployment. If you deliberately remain on the simple container path, enforce spending limits at the upstream provider as well.
Stored provider keys stopped decrypting
Check whether LITELLM_SALT_KEY changed. Restore the original salt where possible; otherwise re-enter the provider credentials through the UI after confirming that the database and master key are correct.
Replicas make inconsistent routing decisions
Verify that all instances use the same Router configuration and that Redis is reachable by every replica. Without shared state, rate limits and routing observations can diverge between processes.
Free tools Windows power users keep installed
One-click scans. No signup required.
How much does a homelab LiteLLM setup cost to operate?
The gateway software is free to self-host under the MIT license. Your actual cost comes from electricity or hosted compute, storage for PostgreSQL backups, and the model providers’ own usage charges. A local Ollama deployment can avoid per-request provider fees but still requires suitable hardware and power; a hosted fallback shifts cost to the provider whenever it is used.
Keep provider pricing and model quality in the routing decision. A lowest-cost route may increase latency or reduce output quality, while a high-capability fallback may be inappropriate for routine traffic. Record which deployment answered each request so a sudden provider bill or quality change can be traced to a routing change.
What is the practical homelab recommendation?
Start with one container and a read-only YAML mount if you only need a stable OpenAI-compatible endpoint. Move to Docker Compose with PostgreSQL as soon as multiple users, virtual keys, spend logs, or budget controls become requirements. Treat waterfall routing as explicitly ordered fallback, test each failure branch, protect both LiteLLM keys, and introduce Redis only when replicas must share state.
Frequently Asked Questions
Can I expose LiteLLM safely outside my home network?
Use a VPN or an authenticated HTTPS reverse proxy rather than forwarding port 4000 directly. Terminate TLS at the edge, restrict administrative paths, apply client-specific keys, and monitor access logs; the gateway credential can expose every configured provider.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Will a request sent to an Ollama deployment leave my LAN?
Not if the selected deployment points to an Ollama service running on your own network and no fallback is triggered. If that local deployment fails and the route permits a hosted fallback, the request can then be sent to the external provider, so document that behavior for sensitive workloads.
How do I move a LiteLLM installation to another server?
For the container-only mode, copy the tested YAML and the protected environment file, then recreate the container. For the Admin UI mode, restore PostgreSQL and the original salt key before starting LiteLLM; restoring the database without the salt leaves encrypted provider credentials unusable.
Does LiteLLM replace my model provider accounts or invoices?
No. It is an orchestration and gateway layer. Each hosted provider still authenticates your account, applies its quotas and terms, and bills usage; LiteLLM’s spend records are an operational view, not a replacement for provider invoices.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Recommended Free Tools
