← Back to blog

Local AI Camera System on the Orange Pi 5: Frigate Without the Cloud, Without Monthly Fees

27 July 2026 · 15 min read

homelabself-hosted

A practical guide built on real, long-running production use — not a weekend experiment.

Why this project exists

This system wasn't built as a tech toy or an AI experiment. It came out of a practical need — keeping an eye on what's happening around the house, without depending on the cloud, without monthly fees, and with full control over my own data.

The first attempts ran on an ordinary PC with no hardware AI acceleration. It worked, but in practice that meant high, constant CPU load, performance dips under heavier activity, noise, and unpredictable behavior. AI detection running purely on CPU can work fine in a test environment, but it's an inefficient approach for long-term production use.

The real difference came with the Orange Pi 5 and its RK3588 chip, which has an integrated NPU (Neural Processing Unit). Moving AI inference off the CPU brought stable inference times, significantly lower CPU load, lower temperatures, and predictable behavior — exactly what a system meant to run continuously needs.

The Orange Pi 5 wasn't picked for marketing reasons, but for the RK3588 chip, its integrated NPU, available hardware video acceleration, vendor kernel support, and a reasonable price. In real production, the combination of Orange Pi 5 + Frigate + the rknn detector has proven very effective — not a compromise, but a practical foundation for a local AI system.

The goal of this article isn't to show "everything you can configure." It's more of a guide to building a system that runs stably, has performance headroom, doesn't need daily intervention, and can be restored from a backup with no fuss. AI is the tool. Stability is the goal.

Reference system

• Orange Pi 5, 8 GB RAM, 128 GB SD card (OS + Frigate)
• 5 IP cameras (4× Hikvision, 1× a different brand) — 4× 2560×1440, 1× 1920×1080, 8 FPS
• Frigate in Docker, the rknn detector (RK3588 NPU)
• average inference time ~23 ms, CPU load 22–31%
• stable 24/7 operation

Maximum load with 8 cameras hasn't been tested, since other services also run on the device. Based on current resource usage, though, it's very likely the system could handle 7–8 cameras, as long as significantly more other processes aren't also running. The goal isn't to push the system to its performance limit — the goal is to keep headroom.

Rough costs: Orange Pi 5 8GB (€120-150), 128GB SD card (€15-25), optionally an NVMe drive (€40-80) or an external HDD/SSD (€50-120). The whole system can be built for roughly €150 to €250. Plenty of people also already have suitable drives lying around from older devices (an SSD from a laptop, an NVMe from an old PC) — no need to buy everything new.

Hardware: a sensible baseline instead of extremes

Hardware is the foundation of the whole system. Undersized, and the system will be unstable. Needlessly oversized, and it'll just be expensive and inefficient.

The deciding factor wasn't a high CPU clock speed, but the presence of an NPU. The RK3588 packs an 8-core CPU, a VPU (hardware video decoding), a GPU, and an NPU. For Frigate, the NPU is what matters — without it, AI detection runs on the CPU, performance fluctuates, and the system behaves unpredictably. With the NPU, inference is offloaded from the CPU, inference times stay stable, and the CPU keeps headroom for everything else.

8 GB of RAM is plenty for 5–8 cameras, as long as the system runs headless, on a minimal OS, with Frigate isolated in Docker and no unnecessary services. More RAM only starts to matter with a significantly higher camera count or a lot more other services running on the same device.

Power supply

A short note to close out the hardware section: don't ignore the quality of your power adapter. I'd recommend a proper 5.1V/5A (27W) USB-C PD supply (the same type as the official Raspberry Pi 27W adapter), not a cheap universal charger. Why I'm stressing this so much, and exactly what happened to me because of an underpowered adapter, I cover in a separate article.

RTSP and differences between camera brands

Every camera brand uses a different RTSP path — this is one of the most common sources of problems during setup.

# Hikvision
rtsp://USER:PASSWORD@CAMERA_IP:554/Streaming/Channels/101

# Dahua and compatible
rtsp://USER:PASSWORD@CAMERA_IP:554/cam/realmonitor?channel=1&subtype=0

# a simpler format (various brands)
rtsp://CAMERA_IP/ch0_0.h264

Hikvision and Dahua behave stably and predictably in practice. Reolink often has different RTSP behavior and can be more troublesome. With less standard brands, expect more testing.

Before configuring anything in Frigate, always test the stream in VLC first. If a stream doesn't work in VLC, it won't work in Frigate either — and it's the fastest way to diagnose the problem.

Why the main stream, 1440p, and 8 FPS

Many guides recommend using the sub-stream (lower quality, lower load) for AI detection. This system deliberately uses the main stream instead — thanks to the NPU and hardware acceleration, there's no reason to sacrifice image quality for performance, and the payoff is higher-quality snapshots, more accurate detection, and a better base for future extensions (e.g. license plate recognition).

1440p provides enough detail for both AI detection and good-quality snapshots. 8 FPS is fully sufficient for detecting people and vehicles — a higher FPS increases data throughput and load without dramatically improving detection.

The recommended codec is H.264 — more stable hardware acceleration, better compatibility, fewer problems. H.265 works, but can be more sensitive to configuration and slightly increase load.

Operating system: Armbian, vendor kernel

The system runs on Armbian (Debian) with the vendor kernel — not mainline. A mainline kernel is fine for development and experimentation, but on the RK3588 it can have incomplete NPU support and limited hardware video acceleration. The vendor kernel includes RK3588-specific patches, has better NPU support, and is more stable for production use. Stability takes priority over running the newest kernel version.

The Armbian minimal image is recommended — no graphical interface, no unnecessary services, lower RAM usage. This system isn't meant to be worked on directly at a monitor — it's meant to run in the background.

After the first login (SSH, which will prompt for a password change and creating a regular user), a normal update follows:

sudo apt update
sudo apt upgrade -y

The most common post-install mistakes: switching to the mainline kernel for no reason, installing a desktop environment "just in case," using nightly builds. If the system works, leave it alone — changes made without a real reason are a frequent source of instability. The same goes for temperature: it isn't a target metric, it's a consequence of load. Fix the configuration first, worry about cooling only after.

Docker and deploying Frigate

Frigate runs as a Docker container — not as a "trendy add-on," but as a tool that substantially improves stability and maintainability. A manual install directly onto the system would mean wrestling with dependencies, library conflicts, and awkward updates. In Docker, if something goes wrong, the container can simply be restarted or the config restored, without disturbing the rest of the system.

sudo apt update
sudo apt install -y docker.io docker-compose-plugin
sudo usermod -aG docker $USER
# log out and back in over SSH

Project structure:

opi-frigate/
├── docker-compose.yml
└── config/
    └── config.yml

docker-compose.yml (a production example for the RK3588)

services:
  frigate:
    container_name: frigate
    image: ghcr.io/blakeblackshear/frigate:stable-rk
    privileged: true
    restart: unless-stopped
    shm_size: "2gb"
    ports:
      - "5000:5000"
    environment:
      TZ: Europe/Bratislava
    security_opt:
      - apparmor:unconfined
      - systempaths=unconfined
    devices:
      - /dev/dri:/dev/dri
      - /dev/dma_heap:/dev/dma_heap
      - /dev/rga:/dev/rga
      - /dev/mpp_service:/dev/mpp_service
    volumes:
      - ./config:/config
      - /media/frigate:/media/frigate
      - /sys/:/sys/:ro
    tmpfs:
      - /tmp/cache:size=1000000000

The devices section is the key part — it maps the devices needed for hardware video acceleration and NPU inference. Without them, inference can end up falling back to the CPU with much lower performance. Port 5000 is left as standard local access — it's not recommended to expose it directly to the internet (see the remote access section below).

docker compose up -d
docker compose ps      # check status — should say "running"
docker logs -f frigate # logs are the first place to look when something's wrong

config.yml — a minimal working configuration (1 camera)

The most common early mistake is adding all 5 cameras at once and trying to sort out zones, masks, and notifications before the system even works at all. The right order is: 1 camera → verify the NPU and stable detection → only then expand.

mqtt:
  enabled: false

detectors:
  rknn:
    type: rknn
    num_cores: 3

model:
  path: frigate-fp16-yolov9-t
  model_type: yolo-generic
  width: 320
  height: 320
  input_tensor: nhwc
  input_pixel_format: bgr

ffmpeg:
  hwaccel_args: preset-rk-h264

objects:
  track:
    - person
    - car

cameras:
  cam1:
    ffmpeg:
      inputs:
        - path: rtsp://USER:PASSWORD@CAMERA_IP:554/Streaming/Channels/101
          roles: [detect]
    detect:
      width: 2560
      height: 1440
      fps: 8

The detectors: rknn section makes sure inference doesn't run on the CPU — if it's misconfigured, inference time will be high and the CPU overloaded. Just as important is the hwaccel_args: preset-rk-h264 line — without it, video decoding falls back to the CPU instead of using hardware acceleration.

Once one camera is confirmed working stably with the NPU active, more can be added — one at a time, watching the logs and inference time after each change. YAML indentation is critical here; a single wrong space can keep the container from starting at all.

Zones and masks: what actually makes Frigate usable

This is, in my view, the single most important part of the whole setup — and also the one most guides dismiss in a single sentence.

Once Frigate is running stably and detection works, you'll quickly notice one thing: not everything that moves matters. Without properly configured zones and masks, the system turns noisy, gets flooded with events, and becomes practically unusable. AI detection on its own isn't enough — the system needs context.

A zone defines where in the frame detection actually matters — it answers the question "if something moves here, do I want to know about it?" Without zones, every car on the road is an event, every person on the sidewalk is a detection, and the system generates a pile of pointless notifications. Well-configured zones significantly cut down the number of events and make the remaining ones relevant. Zones are easiest to draw directly in the Frigate UI (pick a camera → zone editor → draw the area → save). Rule of thumb: one zone, one meaning (e.g. "gate," not "gate + road + yard"), and a simple shape beats an extremely precise one.

Masks, on the other hand, define areas where detection shouldn't run at all — typically trees moving in the wind, busy roads outside your area of interest, or reflective surfaces. They cut down false alarms and save performance. The most common mistake is masking large areas "just in case" — the better approach is to let the system run without masks first, watch where false detections come from, and mask only the actual problem spots.

In real-world use, the biggest improvement doesn't come from more performance or a better model. It comes from zones.

How to tell the system is healthy

With SBC setups, a lot of people tend to watch every graph, every CPU percentage, and every temperature change. In practice, it's more useful to know what to watch, what to ignore, and when it's actually worth stepping in. Stability isn't about hitting maximum numbers — it's about predictability.

The essentials are all in the Frigate UI:

• Inference time — the most important metric. The reference system holds around ~23 ms, with a normal range of 20–40 ms. Sustained values above 80–100 ms, or a gradual increase over time, mean a problem — most often an inactive NPU, a config error, or too many cameras/too high an FPS.
• CPU load — with a correct configuration, the CPU doesn't handle AI inference at all, it just decodes video (with hardware acceleration) and manages the system. The reference system holds 22–31%. Brief spikes during container startup or bursts of activity are normal; a problem starts when it's sustained above 70–80%.
• Temperature — a consequence, not a target. If the CPU isn't chronically overloaded and inference runs on the NPU, temperatures stay in a reasonable range even without extreme cooling.

A healthy system has stable inference times, headroom on the CPU, a container that doesn't restart, and handles bursts of activity without slowing down. Stepping in makes sense if inference keeps trending up over time, the CPU stays constantly high, or the container keeps restarting. Mild fluctuation in the graphs is normal — instability isn't.

Snapshots vs. continuous recording

In real production use, snapshots turned out to be the most useful output of the whole system — created the moment something's detected, giving an instant overview, taking up little space, and easy to archive. 90% of events get reviewed the same day; old footage mostly never gets watched again, so long video retention is usually just wasted storage.

If you already have a separate NVR, let it record continuously and use Frigate purely for AI detection and snapshots — the two systems don't get in each other's way. If you don't have an NVR, Frigate can record video too, but it's best to record only on detection, not continuously, with sensible retention (e.g. snapshots kept for 1 day, short clip retention).

The same principle applies to storage as to the OS: the operating system on the SD card, footage/cache ideally on NVMe or an external drive — it reduces SD card wear and improves I/O performance.

Backups: peace of mind

A lot of working systems don't fail because they were badly built — they fail because something eventually breaks and there's no way back to a known-good state. The right moment to take a backup is when Frigate is running, detection works, and the system is stable — not after an experiment, or once something's already broken.

There are two levels:

1. Configuration backup (the minimum) — docker-compose.yml, config.yml, any scripts. Enables a fast restore or migration to another device.
2. A full bit-for-bit image of the SD card (recommended) — the safest option, an instant return to a working state with no reconfiguration needed.

# find the device
lsblk

# create a backup (CAUTION: the wrong device can overwrite a different disk)
sudo dd if=/dev/sdX of=opi_backup.img bs=4M status=progress

# restore
sudo dd if=opi_backup.img of=/dev/sdX bs=4M status=progress

You don't need a backup every week — one stable restore point after the initial working setup, or after a major configuration change, is enough. And if the system works, don't update it without a reason — stability beats chasing what's new.

Most common mistakes

A summary of what causes the most problems in practice:

1. The NPU isn't active, inference is too high — check the detectors section in config.yml, confirm you're on the vendor kernel, and that device mapping in docker-compose.yml is correct.
2. Mainline kernel for no reason — stick with the vendor kernel unless you have a specific reason to experiment.
3. FPS set too high — 8 FPS is fully sufficient for most scenarios.
4. H.265 without a real need for it — if the camera lets you choose, use H.264.
5. A wrong RTSP path — always test in VLC first.
6. A YAML indentation error — use consistent spaces, never tabs.
7. Adding several cameras at once — add them one at a time and watch the logs after each change.
8. Updating without a backup — always back up at least the configuration before a major change.
9. Chasing temperature instead of configuration — cooling addresses the consequence, not a bad configuration's root cause.
10. Ignoring the logs — docker logs -f frigate is the first and most important source of information. Read first, then change things.

Remote access

In this setup, Frigate is only reachable on the local network (http://IP_ADDRESS:5000). Exposing the port directly to the internet isn't recommended. For access from outside the house, a VPN (e.g. WireGuard) is the safer option — low overhead, and a fast, reliable solution even on SBC-class devices.

Frigate can also integrate with Home Assistant — push notifications with a preview image, automations (e.g. turning on lights when a person is detected), and hooks into other devices. It's not required for basic operation — it's an extension for more advanced users.

Legal notice

A camera system is meant to protect property and safety, not to conduct unauthorized surveillance of people. Operating a camera system means complying with data protection law (GDPR), the right to privacy, and the obligation to disclose that an area is being monitored. When using advanced features like face recognition, license plate recognition, or audio detection, local legislation needs to be respected even more carefully. Full responsibility for how the system is used rests with whoever operates it.

Conclusion

The reference system — 5 cameras, 1440p + 1080p, 8 FPS, the rknn detector — runs stably 24/7, with an inference time around 23 ms and CPU load of 22–31%. That's proof the Orange Pi 5 is capable of running a reliable local AI camera system with no cloud and no monthly fees.

The goal here wasn't to write the most exhaustive Frigate guide out there. It was to show how to build a system that works, has headroom, generates relevant events, and doesn't need daily babysitting. If, after a while, you stop thinking of the system as something you need to deal with, it's done its job — technology is supposed to bring peace of mind, not another thing to worry about.