iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Edge AI means running at least part of an AI model’s inference on the device that creates data or on a nearby edge node, rather than sending every input to a distant cloud service. A camera, machine sensor, vehicle computer, gateway or local server can therefore make a decision close to where an event occurs. The cloud may still train models, retain history, coordinate a fleet or handle analysis that does not fit the endpoint. In standards terms, edge AI is a distributed device–edge–cloud arrangement, not a requirement to move all AI out of the cloud.
What does edge AI mean for connected devices?
In a connected-device system, sensors continuously produce data: images, audio, vibration, temperature, location or machine telemetry. Traditional designs send much of that data to a central cloud, where a model performs inference and returns a result. Edge AI moves some inference closer to the source. “Edge” can mean the endpoint itself, a gateway serving a room or factory, or a nearby network node in a carrier or enterprise facility.
The distinction is about where a function runs, not about a particular model architecture. A small classifier may run inside a microcontroller; a larger vision model may run on an on-site gateway; a cloud service may still perform training and long-term analytics. The International Organization for Standardization describes edge computing for IoT as covering data management, coordination, processing, network functions, heterogeneous computing, security and hardware/software optimization, so edge computing is broader than putting a neural network inside a sensor. See ISO/IEC TR 30164:2020.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsThe June 2026 ITU-T reference model places AIoT functions across device, edge and cloud layers. It defines the concept this way: Artificial intelligence of things (AIoT) is the combination of artificial intelligence (AI) technologies with the Internet of things (IoT) infrastructure to enable more efficient IoT operations, improve human-machine interactions and enhance data management and analytics, among other capabilities.
The full model and requirements are in ITU-T Y.4618 (06/2026).
#1 Best Overall
- HIGH PERFORMANCE: Features 26 TOPS (Trillion Operations Per Second) AI acceleration capability through the Hailo AI Accelerator for advanced machine learning applications
- COMPATIBILITY: Specifically designed for the Raspberry Pi 5, connecting via PCIe interface for optimal data transfer and processing speeds
- COMPACT DESIGN: Measures 65mm x 56.5mm, offering a space-efficient solution while maintaining full functionality as an AI acceleration add-on board
- TEMPERATURE RANGE: Operates reliably in temperatures from 0°C to +50°C (32°F to 122°F), ensuring stable performance in various environments
- SEAMLESS INTEGRATION: Functions as a HAT (Hardware Attached on Top) add-on board, providing plug-and-play compatibility with Raspberry Pi ecosystem
How the device, edge and cloud layers cooperate
The layers are functional roles. A real deployment can omit a layer, combine layers in one box or move a function as requirements change.
| Layer | Typical work | Why it is useful | Typical limitation |
|---|---|---|---|
| Device | Sensor access, filtering, feature extraction, lightweight inference and immediate control | Fast local response and operation when connectivity is poor or unavailable | Limited compute, memory, energy, storage and physical security |
| Edge node | Contextual inference, aggregation, coordination, model deployment and device management for several endpoints | More capacity than an endpoint while remaining close to the data source | Requires local networking, power, administration and protection |
| Cloud | Large-scale storage, model training, orchestration, fleet-wide analytics and lifecycle management | Elastic compute, centralized visibility and broad historical context | Network delay, transfer cost and dependence on an available connection |
A factory, for example, might classify a dangerous machine vibration on the sensor or a nearby gateway, send only an event and summary to the cloud, and use cloud infrastructure to retrain a model from data collected across many sites. That is still edge AI even though the cloud remains essential.
What an edge-AI inference path looks like
- Capture: A sensor or connected device records a signal such as an image, sound sample or vibration window.
- Prepare: Software converts the signal into the format expected by the model. Resizing an image, filtering noise or calculating a short feature vector can reduce the amount of data that must be processed or transmitted.
- Infer: A model executes on the endpoint or a nearby node and produces a prediction, score, classification or control recommendation.
- Act locally: The device can trigger an alarm, stop a machine, adjust a setting or queue a message without waiting for a round trip to the cloud.
- Share selectively: The system may send an event, a compact feature set, selected media or raw samples for later analysis. Policies should specify what is retained, for how long and who can access it.
- Manage centrally: Edge or cloud services can distribute model versions, monitor health, collect feedback and retrain models. Those management functions do not have to run on the endpoint.
This split lets designers put each function where its timing, data and resource requirements fit best.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Why put inference near the data?
Lower transfer volume
Filtering and classifying locally can mean transmitting an alert or summary instead of continuous raw video, audio or telemetry. That can reduce bandwidth demand and the amount of data stored centrally. The saving depends on the workload and on how often the system still needs to upload raw samples for audit, debugging or retraining.
More responsive control
A local decision avoids the variable delay of a network round trip and a remote queue. This is valuable for alarms and control loops where a delayed result is less useful. Local execution is not automatically faster: a poorly sized model, slow storage or an overloaded gateway can still add delay. Measure the complete path, including sensor capture and actuation.
Rank #2
- This kit includes an AI HAT+, a metal case and an active cooler. It's compatible with Raspberry Pi 5.
- The Raspberry Pi AI HAT+ features a built-in neural network accelerator, turning your Raspberry Pi 5 into a high-performance, accessible, and power-efficient AI machine.The 13 TOPS variant capably runs neural networks for applications including object detection, semantic and instance segmentation, pose estimation, and more.
- The AI HAT+ communicates using Raspberry Pi 5’s PCIe Gen 3 interface. When the host Raspberry Pi 5 is running an up-to-date Raspberry Pi OS image, it automatically detects the on-board Hailo accelerator and makes the NPU available for AI computing tasks. The built-in rpicam-apps camera applications in Raspberry Pi OS natively support the AI module, automatically using the NPU to run compatible post-processing tasks.
- Conforms to Raspberry Pi HAT+ specification; Supplied with 16mm stacking header, spacers, and screws to enable fitting on Raspberry Pi 5 with Raspberry Pi Active Cooler in place.
- The metal case can protect the Raspberry Pi 5 board from damage, dust and scratches. It can access most ports, including usb-c power jack, micro HDMI ports, usb ports, Ethernet jack, sd card slot, power button and GPIO port.
Operation through connectivity loss
An endpoint can continue a defined set of functions while a wide-area link is down or degraded. It may buffer records and synchronize later. The offline behavior must be designed explicitly; cloud-dependent identity checks, model retrieval or policy updates can still fail during an outage.
Data minimization and local context
Keeping selected inputs on site can support a data-minimization policy and allow decisions to use local context. It does not guarantee privacy. A device can be compromised, local logs can expose sensitive information, and any data sent for monitoring or retraining remains subject to access and retention controls.
What the cloud still does well
Cloud infrastructure generally offers more compute and storage for global training, long-term history, cross-site analysis and centralized orchestration. A nearby edge node can coordinate many devices or run a model too large for each endpoint. A hybrid design often keeps urgent inference local while using edge or cloud resources for updates, monitoring and heavier analysis.
Device, edge or cloud: how to choose
There is no universal winner. Evaluate the same workload against the following questions before selecting a placement.
| Decision axis | Device placement | Nearby edge placement | Cloud placement |
|---|---|---|---|
| Response time | Best fit when a very short, predictable path matters | Can provide low latency for multiple endpoints on one site | Delay varies with the connection and service path |
| Connectivity loss | Can continue local functions if dependencies are packaged locally | Can serve a site during wide-area outages if local networking and power remain | Usually unavailable when the required connection is lost |
| Data sensitivity | Raw data can remain at the source, but the endpoint must be secured | Data stays within a controlled local or regional boundary when configured that way | Central access simplifies analysis but increases transfer and governance obligations |
| Bandwidth | Can transmit events or summaries instead of raw streams | Aggregates traffic from several devices before selected uploads | May require continuous or large uploads for inference and storage |
| Model and resource size | Must fit endpoint compute, memory, storage and energy budgets | Supports larger workloads shared by a site | Provides the largest pool of elastic compute and storage |
| Scale and coordination | Each device needs provisioning, monitoring and update handling | Can coordinate a group of endpoints and enforce site policies | Centralizes fleet-wide orchestration and cross-site analytics |
| Security work | Protect hardware, firmware, model files, interfaces and local credentials | Protect the node, its network interfaces and every device connection | Protect accounts, APIs, data pipelines and model-serving infrastructure |
| Lifecycle | Updates and diagnostics must work on constrained or intermittent links | Can stage models and updates locally for a fleet | Convenient for retraining, version control and long-term observability |
A small, time-critical sensor task may suit device inference. A site-wide video or equipment workload may fit an edge node. A fleet-wide training job generally needs cloud-scale resources. These are architecture patterns, not guarantees of a particular latency, cost or accuracy.
Rank #3
- Hailo-10H AI accelerator delivering 40 TOPS (INT4) inferencing performance.
- Performance for computer vision models comparable to the Raspbery Pi AI HAT+ (26 TOPS).
- Runs generative AI models efficiently using 8GB on-board RAM.
- Fully integrated into Raspbery Pi’s camera software stack.
- Conforms to Raspbery Pi HAT+ specification.
Constraints that make device-level edge AI difficult
Compute and memory
The model, runtime and intermediate tensors must fit the processor and memory available while meeting the required response time. Quantization, a smaller architecture or a hardware accelerator can help, but each change must be checked for accuracy and supported operators. A model that runs in a development environment may not run on the target device without conversion or a compatible runtime.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Energy and thermal limits
Inference consumes energy and can increase heat. Battery-powered products may need event-triggered sampling, duty cycling or a smaller model. Always-on vision or audio workloads can change battery life and enclosure requirements even when network traffic falls.
Communications and intermittent networks
Devices still need a plan for enrollment, time synchronization, telemetry, model updates and recovery. Queueing and retry policies should prevent a reconnecting fleet from overwhelming a gateway or cloud endpoint. Decide which functions are mandatory offline and which can wait.
Data distribution and model behavior
Data collected at one location may not match the data used for training elsewhere. Lighting, temperature, users, machinery and sensor calibration can shift the input distribution. NIST lists non-identical or non-independent data distributions, communication constraints, privacy requirements, resource constraints and security vulnerabilities among edge-AI challenges. Its overview is available at NIST Edge AI (updated August 12, 2026).
Model lifecycle
Plan version identifiers, compatibility checks, staged rollout, rollback and a way to verify that a device is running the intended model. Keep enough diagnostics to detect drift without collecting more raw data than the policy allows. Retraining can remain centralized even when inference is local.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Rank #4
- The Raspbery Pi AI HAT+ is an add-on board with a built-in Hailo AI accelerator designed for RPi 5. It provides an accessible, cost-effective, and power-efficient way to integrate high-performance AI. It's suited to everything from entry-level applications to more complex neural processing, with the ability to process multiple concurrent models and AI tasks. Explore applications including process control, security, home automation, and robotics.
- This AI HAT+ is available in 13 TOPS variants, built around the Hailo-8L neural network inference accelerators. The 13 TOPS variant capably runs neural networks for applications including object detection, semantic and instance segmentation, pose estimation, and more.
- The AI HAT+ communicates using Raspbery Pi 5's PCIe Gen 3 interface. It automatically detects the onboard Hailo accelerator and makes the NPU available for AI computing tasks. The built-in rpicam-apps camera applications in Raspbery Pi OS natively support the AI module, automatically using the NPU to run compatible post-processing tasks.
- Hailo-8L accelerator offering 13 TOPS inferencing performance respectively. Fully integrated into Raspbery Pi's camera software stack. Conforms to Raspbery Pi HAT+ specification.
- Comes with 16mm stacking header, spacers, and screws to enable fitting on Raspbery Pi 5 with Raspbery Pi Active Cooler in place.
Security is more than keeping data on the device
Local inference changes the attack surface; it does not remove it. An attacker who gains access to a device may read cached inputs, extract model files, alter predictions or use the device as a route into the local network. Edge nodes add management interfaces and connections among many endpoints.
Security controls should cover secure application deployment, access control, interface security, runtime protection and verification that devices connecting to an edge network are legitimate. Those areas are addressed in ITU-T X.1648 (04/2025). In practice, establish device identity and least-privilege permissions, protect communications in transit, verify software and model updates, restrict administrative interfaces, log security-relevant events and define a recovery path for a compromised endpoint.
Can AI run on an IoT device without sending data to the cloud?
Yes, if the device has a supported model, runtime and sufficient compute, memory and power. The inference input and result can remain local. “Without the cloud” should not be interpreted as “without any network”: provisioning, model updates, fleet coordination, monitoring, policy distribution or later training may still use an edge service or cloud. A robust design states exactly which functions continue offline and which require connectivity.
Hands-on example: Raspberry Pi AI HAT+ for Raspberry Pi 5
The Raspberry Pi AI HAT+ is an add-on board for Raspberry Pi 5 that includes a Hailo neural processing unit. Raspberry Pi’s documentation states: AI HATs are add-on boards for Raspberry Pi 5 that come with a built-in AI accelerator chip: the Hailo neural processing unit (NPU).
The official product page lists 13 TOPS and 26 TOPS variants and names applications including process control, home automation, research, object detection, image segmentation and pose estimation. TOPS is a manufacturer specification for the accelerator, not an independent benchmark of a complete application. See Raspberry Pi AI HAT+ and the AI HAT documentation.
| Item | What is established | What you must verify |
|---|---|---|
| Host | Designed for Raspberry Pi 5 | Board revision, power, cooling and physical installation requirements |
| Accelerator options | 13 TOPS and 26 TOPS variants are listed by Raspberry Pi | Whether the chosen model and workload need one variant or the other |
| Software | Supported workloads use the accelerator with compatible software and models | Current operating-system support, runtime, model format and operator compatibility |
| Role | It accelerates inference; it is not a standalone computer | The Pi 5, storage, power, cooling, sensors and application software required for the prototype |
A practical prototype sequence
- Define the task and acceptance limits: input type, response time, offline behavior, accuracy and energy budget.
- Use a Raspberry Pi 5 and select the AI HAT+ variant after checking the current compatibility lists in the official documentation.
- Install the supported software stack and accelerator runtime specified by Raspberry Pi, then confirm that the board is detected.
- Choose a model listed as supported, convert or compile it only using the documented toolchain, and test representative sensor data rather than a few ideal samples.
- Measure end-to-end behavior, including capture, preprocessing, inference, actuation, temperature and power. Do not treat the TOPS label as an application performance result.
- Secure the prototype before connecting it to a wider network: change default credentials, restrict management access, verify updates and decide what data leaves the device.
Availability, supported models and software compatibility can change, so check the manufacturer’s current pages before purchasing or designing around the board.
A decision checklist for an edge-AI project
- What decision must be made, and what is the maximum acceptable delay?
- Which inputs are sensitive, and what data-minimization and retention rules apply?
- How much raw data can the network carry, and what can be summarized locally?
- Must the function continue during loss of wide-area connectivity?
- What compute, memory, storage, power and thermal budget is available at the endpoint?
- How many devices need coordinated deployment, monitoring and rollback?
- How will identities, interfaces, runtime behavior and software/model updates be protected?
- Where will training, retraining, long-term storage and fleet analytics run?
- What evidence will show that the model remains accurate as local conditions change?
Answering these questions usually leads to a split architecture: keep only the latency-sensitive or data-reduction step local, place shared contextual work at an edge node, and reserve cloud resources for coordination, training and history. Edge AI is therefore a placement strategy for connected-device intelligence, not a blanket replacement for cloud computing.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
