YOLO Deployment on RK3588 NPU with RKNN

Why Onboard Inference?

Running object detection directly on the drone's onboard computer eliminates the latency and reliability issues of streaming video to a ground station. The RK3588's NPU (6 TOPS) can run YOLO models at real-time frame rates with very low power consumption.

Setup

1. RKNN Toolkit

The RKNN toolkit converts models from PyTorch/ONNX to the .rknn format:

# On x86 host (for model conversion)
pip install rknn-toolkit2

2. Model Conversion (YOLOv5s example)

from rknn.api import RKNN

rknn = RKNN()
# Load ONNX model
ret = rknn.load_onnx(model='yolov5s.onnx')
# Build for RK3588
ret = rknn.build(do_quantization=True, dataset='calibration.txt')
# Export
ret = rknn.export_rknn('yolov5s.rknn')

3. Onboard Inference

from rknnlite.api import RKNNLite

rknn = RKNNLite()
rknn.load_rknn('yolov5s.rknn')
rknn.init_runtime(core_mask=RKNNLite.NPU_CORE_AUTO)

outputs = rknn.inference(inputs=[preprocessed_image])

Performance

On the Orange Pi 5 Pro (RK3588):

Integration with ROS

The yolo_rknn_target_detector package wraps the RKNN inference in a ROS node:

• Subscribes to /camera/color/image_raw (RealSense RGB)
• Publishes detections to /detections (bounding boxes + classes)
• Publishes annotated images to /detections/image

The detector runs in a separate process to avoid blocking the main navigation stack, communicating via ROS topics.

Common Pitfalls

1. Quantization: INT8 quantization can cause accuracy loss. Always validate against the FP32 baseline on a test set before deploying.

2. Image Preprocessing: RKNN expects NHWC format with specific mean/std normalization. Mismatched preprocessing is the #1 cause of silent accuracy degradation.

3. NPU Warm-up: The first inference after loading is slow (∼500ms). Run a dummy inference during initialization to amortize this cost.

4. Power: The NPU draws significant current at peak load. Ensure your power supply can handle simultaneous CPU, NPU, and peripheral loads.