Complete Ollama Tutorial (2026): LLMs via CLI, Cloud, and Python

iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more

This Complete Ollama Tutorial (2026): LLMs via CLI, Cloud, and Python shows how to install Ollama on macOS, Windows, or Linux, run open models locally, authenticate for cloud models, call the local or cloud API, and use the official Python client for chat, streaming, embeddings, structured outputs, vision, and tools.

Model names, cloud tags, package versions, hardware support, context defaults, pricing, and limits change quickly. The commands below use documented examples, so verify those details against Ollama’s current documentation before publication or production deployment.

#	Preview	Product	Price
1		AI Engineering: Building Applications with Foundation Models	$45.98	Buy on Amazon
2		Test Yourself on Sebastian Raschka's Build a Large Language Model (From Scratch): 300+ practice...		Buy on Amazon
3		The Developer's Playbook for Large Language Model Security: Building Secure AI Applications	$51.30	Buy on Amazon
4		Small Language Models for Mobile Devices: A Guide to On-Device AI, Model Optimization, and Edge...	$24.99	Buy on Amazon
5		Advancing Software Engineering Through AI, Federated Learning, and Large Language Models	$332.68	Buy on Amazon

Key takeaways

• Ollama runs on macOS, Windows, and Linux, and the basic local workflow is to install Ollama, open a terminal, download or select a model, and run it.
• The local Ollama API normally starts at http://localhost:11434/api, while direct cloud API access uses https://ollama.com/api.
• The official Ollama Python library supports Python 3.8 and newer and exposes chat, generation, streaming, asynchronous clients, embeddings, model management, and cloud hosts.
• Ollama Cloud requires account authentication, and direct programmatic cloud requests require an API key sent as an Authorization: Bearer header.
• According to Ollama’s 2026 context-length documentation, the documented default context is 4K below 24 GiB of VRAM, 32K at 24–48 GiB, and 256K at 48 GiB or more; these are configuration defaults, not performance benchmarks.
• Tool calling, structured outputs, embeddings, and vision are supported capabilities, but the selected model must support the capability you want to use.

What is Ollama?

Ollama is a local-first runtime and API layer for downloading, running, and interacting with open models. Ollama is not itself an LLM: the model, model tag, quantization, context window, and supported capabilities determine what the runtime can do.

“Ollama runs on macOS, Windows, and Linux.” — Ollama, official Quickstart documentation

Popular on Amazon

Sale

AI Engineering: Building Applications with Foundation Models

$45.98

Buy on Amazon

Ollama provides a command-line interface for interactive use, a local HTTP API for applications, a cloud API, and an official Python client that mirrors the API. A local installation can keep prompts on the user’s machine when the selected workflow genuinely runs locally. A cloud model instead sends processing to Ollama’s cloud service, so cloud authentication and cloud data-handling terms matter.

The official API documentation describes interfaces for generation and chat, embeddings, model listing, running-model inspection, model creation, copying, pulling, pushing, deletion, and version retrieval. The API is not strictly versioned; Ollama says the API is expected to remain stable and backward compatible, but production applications should still pin and test the behavior they depend on rather than treating undocumented behavior as a permanent contract. Read the official Ollama API introduction and documentation index when implementing against a particular release.

How do you install Ollama and get a first response?

Install the version for macOS, Windows, or Linux by following the current official Ollama Quickstart. The installer and platform details can change, so use the current instructions rather than copying an old operating-system-specific command from an unrelated guide.

1. Install Ollama using the current instructions for your operating system.
2. Open a terminal and run ollama. The official quickstart uses this command to open Ollama’s interactive menu.
3. Download a model explicitly when you want to separate downloading from execution:

ollama pull gemma4

Start a local chat with the documented example:

ollama run gemma4

After the model starts, enter a prompt in the terminal. gemma4 is an example tag from the documentation, not a permanent recommendation. Confirm the current model name, tag, size, context window, and capability labels in Ollama’s model library immediately before publication or deployment because model identifiers change.

A quick installation check is:

• The ollama command is available in a new terminal session.
• ollama ls can list the model you pulled.
• ollama run gemma4, or the current model tag you selected, accepts a prompt and returns a response.
• Your application can reach the local API at http://localhost:11434/api if you intend to use HTTP or Python.

Which Ollama CLI commands matter?

The Ollama CLI is more than a chat launcher: the CLI can download, inspect, run, stop, remove, serve, authenticate, create, and integrate models. The following compact reference is based on the official Ollama CLI reference.

Replace MODEL with the exact current model identifier. Do not assume that a cloud suffix, model size, or capability tag remains unchanged between publications.

Should you run a model locally or use Ollama Cloud?

Choose local execution when local data routing, offline control, and hardware ownership matter; choose Ollama Cloud when you want Ollama to offload inference and you accept account authentication, network dependence, and the current cloud service conditions. Neither option is universally better.

The official Ollama Cloud documentation shows the same familiar workflow with a cloud model tag:

ollama signin
ollama run gpt-oss:120b-cloud

gpt-oss:120b-cloud is a documentation example. Verify the current cloud model names, tags, account requirements, rate limits, pricing, and data-handling terms before using this command in a published tutorial or production system.

How do you authenticate direct cloud API requests?

Direct requests to https://ollama.com/api require an API key and an Authorization: Bearer header according to Ollama’s official API authentication documentation. Store the key in a secret manager or environment variable; do not commit the key to source control, paste it into a public issue, or hard-code it in a client-side application.

Also popular on Amazon

Test Yourself on Sebastian Raschka's Build a Large Language Model (From Scratch): 300+ practice problems to cement your learning

Buy on Amazon

export OLLAMA_API_KEY=your_api_key

The command above uses POSIX-style shell syntax. The important security principle is independent of operating system: keep the secret outside the source file and inject it at runtime.

What is the Ollama API URL, and how do you call it?

The default Ollama API URL is http://localhost:11434/api for a local server. Ollama’s cloud API URL is https://ollama.com/api. The local base URL is documented in the Ollama API introduction; cloud authentication is documented separately.

A minimal local generation request uses the /api/generate endpoint:

curl http://localhost:11434/api/generate -d '{
  "model": "gemma4",
  "prompt": "Explain vector databases in two sentences."
}'

Use the chat interface when your application needs a sequence of role-based messages, and use generation when a single prompt is the natural input. The API also documents operations for embeddings, model listing, running-model inspection, model creation, copying, pulling, pushing, deletion, and version retrieval.

The equivalent cloud pattern requires the cloud URL and bearer authentication:

curl https://ollama.com/api/generate 
  -H "Authorization: Bearer $OLLAMA_API_KEY" 
  -H "Content-Type: application/json" 
  -d '{
    "model": "CURRENT_CLOUD_MODEL",
    "prompt": "Explain vector databases in two sentences."
  }'

CURRENT_CLOUD_MODEL is deliberately a placeholder. Use a currently supported cloud model identifier rather than assuming that the example cloud tag in an older article still exists.

How do you use Ollama with Python?

Install the official Ollama Python library with pip. The package metadata states that the client supports Python 3.8 and newer:

python -m pip install ollama

The Python client is designed around Ollama’s REST API and exposes chat, generation, listing, model inspection, creation, copying, deletion, pulling, pushing, embeddings, and running-model inspection. See the official Ollama Python library and its package metadata for the current installation requirements and signatures.

With the local Ollama server running, a basic chat call looks like this:

from ollama import chat

response = chat(
    model='gemma4',
    messages=[
        {'role': 'user', 'content': 'Explain recursion simply.'},
    ],
)

print(response.message.content)

The Python example uses the same illustrative gemma4 tag as the CLI example. Replace it with a model that is installed locally and currently supports the task you need.

Also popular on Amazon

Sale

The Developer's Playbook for Large Language Model Security: Building Secure AI Applications

$51.30

Buy on Amazon

How do you stream Ollama responses in Python?

Streaming lets an interactive application render partial output instead of waiting for the complete response. Streaming changes when text becomes available; it does not by itself prove that the model will produce tokens faster overall.

from ollama import chat

stream = chat(
    model='gemma4',
    messages=[{'role': 'user', 'content': 'Write a short poem about terminals.'}],
    stream=True,
)

for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

How do you use asynchronous Python with Ollama?

Use AsyncClient when the surrounding application is asynchronous and you want to await Ollama calls. Asynchronous code helps an application coordinate I/O; do not describe it as a guaranteed model-performance improvement without testing a defined workload.

import asyncio
from ollama import AsyncClient

async def main():
    client = AsyncClient()
    response = await client.chat(
        model='gemma4',
        messages=[{'role': 'user', 'content': 'Give me three concise test cases.'}],
    )
    print(response.message.content)

asyncio.run(main())

The official Python repository also documents asynchronous streaming for applications that need both non-blocking control flow and incremental output.

How do you call Ollama Cloud from Python?

Configure the Python client with https://ollama.com when you want the client to talk directly to Ollama Cloud. A normal Client() points to the local Ollama server; a client configured with the cloud host sends requests to the cloud API.

import os
from ollama import Client

client = Client(
    host='https://ollama.com',
    headers={
        'Authorization': 'Bearer ' + os.environ['OLLAMA_API_KEY'],
    },
)

response = client.chat(
    model='gpt-oss:120b',
    messages=[{'role': 'user', 'content': 'Explain quantum computing.'}],
)

print(response.message.content)

The cloud model identifier in this example is subject to change. Confirm current cloud access conditions and model identifiers in Ollama’s cloud documentation before deploying the example.

How do Modelfiles customize an Ollama model?

A Modelfile defines a customized Ollama model configuration. It can select a base model, set runtime parameters, add a system prompt, define a template, apply an adapter, include a license, and add message history; it does not automatically retrain the underlying model weights.

“A Modelfile is the blueprint to create and share customized models using Ollama.” — Ollama, official Modelfile Reference

×

Free tools Windows power users keep installed

One-click scans. No signup required.

Fix Windows issues →Update all drivers →Clean & speed up →

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The official Modelfile Reference requires a FROM instruction. A minimal tutor configuration is:

FROM gemma4
SYSTEM """You are a concise technical tutor."""
PARAMETER temperature 0.2

Save that text as Modelfile, then create and run the customized model:

ollama create tutor -f Modelfile
ollama run tutor

A system prompt changes the model’s instructions, and a temperature parameter changes generation behavior. Neither change turns a general model into a domain-expert model or guarantees factual accuracy. Use retrieval, evaluation, fine-tuning, or a different base model when the application needs capabilities that a runtime configuration cannot provide.

Also popular on Amazon

Small Language Models for Mobile Devices: A Guide to On-Device AI, Model Optimization, and Edge Computing for Android and iOS

$24.99

Buy on Amazon

How much VRAM does Ollama need?

Ollama does not have one meaningful VRAM requirement for every model and workload. Model size, quantization, prompt length, context length, concurrency, operating system, and GPU backend all affect memory use. The following figures are Ollama’s documented context defaults, not a universal minimum VRAM table or an independent speed benchmark.

According to Ollama’s context-length documentation (2026), the documented default context lengths are:

Ollama also documents at least 64K tokens as the minimum recommended context for tasks such as web search, agents, and coding tools. That recommendation can exceed the documented default for systems with less than 48 GiB of VRAM, which is why context-heavy workloads may require more memory or a different deployment choice.

“Context length is the maximum number of tokens that the model has access to in memory.” — Ollama, official Context length documentation

Which GPU backends does Ollama support?

Ollama documents NVIDIA, AMD, Apple Metal, and Vulkan paths in its hardware-support documentation. NVIDIA support begins at documented compute capability 5.0 with driver requirements. AMD support uses ROCm on documented platforms, Apple GPU acceleration uses Metal, and Vulkan provides additional support on documented Windows and Linux configurations.

Can you use Ollama without a GPU? Do not treat a dedicated GPU as a universal installation prerequisite, but do not promise a particular CPU-only speed or model capacity either: the supplied official hardware material provides no universal CPU minimum or CPU performance benchmark. For CPU-only or memory-constrained use, test the exact model, context length, response time, and concurrency that your application requires.

If you are comparing local LLM hardware, define the model size, quantization, context target, operating system, concurrency, and budget first. A generic “AI computer” recommendation is not technically meaningful without those constraints.

How do tool calling, JSON, embeddings, and vision work?

These capabilities turn Ollama from a text chat runtime into an application component, but capability support belongs to the selected model rather than to every model automatically.

How does Ollama tool calling work?

Tool calling allows a model to request a function and incorporate the function’s returned result into a later response.

“Ollama supports tool calling (also known as function calling) which allows a model to invoke tools and incorporate their results into its replies.” — Ollama, official Tool calling documentation

Also popular on Amazon

Sale

Advancing Software Engineering Through AI, Federated Learning, and Large Language Models

$332.68

Buy on Amazon

The following simplified Python flow follows the documented single-tool pattern:

from ollama import chat

def add(a: int, b: int) -> int:
    return a + b

messages = [{'role': 'user', 'content': 'What is 12 + 30?'}]

response = chat(model='qwen3', messages=messages, tools=[add], think=True)
messages.append(response.message)

if response.message.tool_calls:
    for call in response.message.tool_calls:
        result = add(**call.function.arguments)
        messages.append({
            'role': 'tool',
            'tool_name': call.function.name,
            'content': str(result),
        })

    final = chat(model='qwen3', messages=messages, tools=[add], think=True)
    print(final.message.content)

The arithmetic function is safe because it has no external side effect. An application—not the model—must validate tool names, argument types, permissions, authorization, returned data, and side effects. Shell commands, file writes, database changes, payments, and account actions should use explicit allowlists, least-privilege credentials, confirmation steps, and sandboxing. Ollama’s official documentation covers single-tool calls, parallel tool calls, multi-turn agent loops, and streaming tool calls, but no model is guaranteed to call tools reliably.

How do you make Ollama return JSON?

Use Ollama’s structured-output capability when downstream code needs machine-readable JSON. Supply a schema through the supported structured-output interface, then parse the response and validate it against the schema before using any field. A prompt that merely says “return JSON” is weaker than a schema-enforced contract and should not replace validation.

The exact schema and client argument should follow the current Ollama Structured Outputs documentation. Schema validation is still an application responsibility: valid JSON can contain the wrong values, missing business fields, unsafe instructions, or data that fails domain rules.

How do you use Ollama for RAG?

Use Ollama for RAG by embedding your source documents, storing the vectors, retrieving the most relevant chunks for a question, and passing those chunks to a chat model as grounded context.

1. Collect and clean the documents your application is allowed to use.
2. Split documents into retrieval-sized chunks while retaining titles, source identifiers, and other metadata.
3. Use an embedding model through Ollama’s embedding operation to turn each chunk into a vector.
4. Store the vectors and metadata in a vector store selected for your application.
5. Embed the user’s question with the same embedding model, then retrieve the closest relevant chunks.
6. Place the retrieved text and source metadata in the chat prompt, with instructions to distinguish supplied evidence from unsupported guesses.
7. Evaluate retrieval quality and answer faithfulness on your own corpus before increasing model size or context length.

Embeddings are not the same as chat completion. Choose an embedding model appropriate for the language and content of the target corpus, and confirm its current model requirements. Ollama documents the embedding operation in its official embeddings capability guide.

How does Ollama vision input work?

Vision input works only with a model that supports images. Do not assume that a text model or every Ollama model is multimodal; check the current model page for the required image input format, model requirements, and supported behavior before building an image workflow. Ollama’s vision documentation describes the capability.

Which Ollama workflow should you choose?

The right Ollama workflow depends on whether you are experimenting, building an application, grounding answers in documents, or allowing a model to take actions.

How do small and large models differ?

Smaller models generally reduce memory pressure and can be easier to run locally, while larger models may offer different task quality or tool behavior at higher hardware or cloud cost. The dossier provides no universal quality ranking, latency benchmark, or best model, so choose by measuring the exact tasks, context, language, latency, and concurrency that matter to your application.

What should you verify before using Ollama in production?

Production readiness is less about making one successful chat request and more about controlling changing models, memory, credentials, outputs, and actions.

• Pin the model identifier you tested. Record the model tag, configuration, context target, and any Modelfile rather than silently accepting a replacement.
• Test the real workload. Measure response quality, latency, memory use, concurrency, long prompts, tool calls, and retrieval on representative data.
• Separate local and cloud data paths. Make the selected host explicit and document which prompts may leave the local machine.
• Protect credentials. Keep cloud API keys in environment-level secrets or a secret manager, and rotate exposed keys.
• Validate structured responses. Parse JSON, enforce the schema, and apply business validation before storing or executing data.
• Constrain tools. Allow only approved functions and arguments, require authorization for side effects, and sandbox high-risk operations.
• Plan for memory. Treat model size, quantization, context length, and concurrency as capacity variables rather than relying on a single VRAM number.
• Recheck volatile documentation. Confirm the current Ollama release, CLI syntax, model names and tags, cloud limits and pricing, Python package version and signatures, GPU and driver support, and context defaults immediately before publication or deployment.

Ollama troubleshooting checklist

Ollama is most useful when you treat the runtime, model, hardware, host, and application controls as separate decisions. Start with the CLI for exploration, move to the local API or Python client for repeatable software, use cloud execution when its authentication and data-routing trade-offs fit, and add RAG, structured outputs, vision, or tools only after confirming support for the specific model you selected.

The Bottom Line

Bottom line: Ollama gives you one familiar workflow across local models, cloud models, the REST API, and Python, but the model tag and workload determine the result. Verify current model, cloud, package, hardware, and context documentation before treating any example as a production specification.

Quick Recap

SaleBestseller No. 1

AI Engineering: Building Applications with Foundation Models

$45.98

Bestseller No. 2

Test Yourself on Sebastian Raschka's Build a Large Language Model (From Scratch): 300+ practice problems to cement your learning

SaleBestseller No. 3

The Developer's Playbook for Large Language Model Security: Building Secure AI Applications

$51.30

Bestseller No. 4

Small Language Models for Mobile Devices: A Guide to On-Device AI, Model Optimization, and Edge Computing for Android and iOS

$24.99

SaleBestseller No. 5

Advancing Software Engineering Through AI, Federated Learning, and Large Language Models

$332.68