11:06:43 AM NPT Arduino, Robotics & IoT tutorials for Nepal
Beginner

OpenCV on Raspberry Pi: Colour Detection for a Robot

Detect a coloured ball with OpenCV on a Raspberry Pi using HSV masks and contours, then steer an Arduino robot over USB serial. Code, tuning and FPS tips.

On this page
OpenCV on Raspberry Pi: Colour Detection for a Robot

A robot that follows a red ball looks like magic at a college exhibition, but the idea behind it is simple. A camera on a Raspberry Pi looks for pixels of one colour, finds the biggest blob, and works out whether it is on the left, in the centre or on the right. The Pi then tells an Arduino which way to drive.

In this guide you will install OpenCV on the Pi, detect a colour using HSV, find the ball's position with contours, tune the colour range with sliders, and send steering commands to an Arduino over USB serial.

What you need

Scroll sideways to see all columns.

Item Qty Notes
Raspberry Pi 4 or 5 with Raspberry Pi OS 1 Pi 3B+ works but slower
Official or good-quality Pi power supply 1 Weak supplies cause random problems
Pi Camera Module or USB webcam 1 Code below covers both
Arduino Uno + L298N + 2-motor chassis 1 set The robot base
Motor battery pack 1 Separate from the Pi supply
Brightly coloured ball 1 Red, blue or green; avoid colours found in the room
USB cable (Pi to Arduino) 1 Carries serial commands

Why HSV and not RGB

A camera gives pixels as Blue, Green, Red values (OpenCV uses BGR order). The trouble is that the same red ball gives very different RGB numbers in sunlight, under a tube light or in shade, because brightness is mixed into all three channels.

HSV separates colour from brightness:

Scroll sideways to see all columns.

Channel Meaning OpenCV range
H (hue) The colour itself, as a position on a colour wheel 0 to 179
S (saturation) How strong the colour is (grey = low) 0 to 255
V (value) How bright it is 0 to 255

For a blue ball you can say "hue between about 100 and 130, saturation above 120, value above 70", and that works across much more lighting. Red is special: it sits at both ends of the hue wheel, so you need two ranges (about 0–10 and 170–179) and combine them.

Step 1: Install OpenCV

On current Raspberry Pi OS, install from the package manager. It is faster and more reliable than building OpenCV yourself.

example-1.shDownload code
sudo apt update
sudo apt install -y python3-opencv python3-serial python3-picamera2
python3 -c "import cv2; print(cv2.__version__)"

If the last command prints a version number, OpenCV is ready. python3-picamera2 is already present on most full Raspberry Pi OS images and is needed only for the Pi Camera Module.

Step 2: Find your HSV range with sliders

This tool shows the mask live while you move sliders. Hold the ball in front of the camera and adjust until only the ball is white. This code is not tested yet; it uses a USB webcam. For the Pi Camera, see the camera lines in Step 3.

example-2.pyDownload code
# HSV tuner - not tested yet
import cv2

cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 320)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 240)

cv2.namedWindow("sliders")
for name, maxv, start in [("Hmin",179,100),("Hmax",179,130),("Smin",255,120),
                          ("Smax",255,255),("Vmin",255,70),("Vmax",255,255)]:
    cv2.createTrackbar(name, "sliders", start, maxv, lambda x: None)

while True:
    ok, frame = cap.read()
    if not ok:
        break
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    v = [cv2.getTrackbarPos(n, "sliders") for n in ("Hmin","Smin","Vmin","Hmax","Smax","Vmax")]
    mask = cv2.inRange(hsv, tuple(v[:3]), tuple(v[3:]))
    cv2.imshow("frame", frame)
    cv2.imshow("mask", mask)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        print("lower =", v[:3], " upper =", v[3:])
        break

cap.release()
cv2.destroyAllWindows()

Press q to print your values. Run this on the Pi desktop (or over VNC), since it opens windows. Retune at the place where you will demo, because exhibition halls often have very different lighting from your room.

Step 3: Detect the ball and steer

The program below finds the largest blob of your colour, calculates its centre, and decides:

  • Ball in the left third of the frame → send L
  • Ball in the middle → send F (forward)
  • Ball in the right third → send R
  • No ball, or the ball is very close (large area) → send S (stop)

This code is not tested yet.

example-3.pyDownload code
# Colour tracker that sends steering commands - not tested yet
import cv2, serial, time
import numpy as np

LOWER = np.array([100, 120, 70])     # paste your tuned values
UPPER = np.array([130, 255, 255])
MIN_AREA = 300                       # ignore tiny specks
STOP_AREA = 25000                    # ball very close: stop
W, H = 320, 240

ser = serial.Serial('/dev/ttyACM0', 115200, timeout=0.1)  # may be /dev/ttyUSB0
time.sleep(2)                        # Arduino resets when the port opens

# --- USB webcam ---
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, W)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, H)
def grab():
    ok, f = cap.read()
    return f if ok else None

# --- Pi Camera Module instead: comment the webcam block, uncomment this ---
# from picamera2 import Picamera2
# picam = Picamera2()
# picam.configure(picam.create_preview_configuration(main={"format": "RGB888", "size": (W, H)}))
# picam.start()
# def grab():
#     return picam.capture_array()   # "RGB888" arrives in BGR order, ready for OpenCV

kernel = np.ones((5, 5), np.uint8)
last_cmd = None
last_send = 0.0

while True:
    frame = grab()
    if frame is None:
        ser.write(b'S')
        break
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    mask = cv2.inRange(hsv, LOWER, UPPER)
    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)   # remove noise

    contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    cmd = 'S'
    if contours:
        c = max(contours, key=cv2.contourArea)
        area = cv2.contourArea(c)
        if MIN_AREA < area < STOP_AREA:
            M = cv2.moments(c)
            cx = int(M["m10"] / M["m00"])
            if cx < W / 3:
                cmd = 'L'
            elif cx > 2 * W / 3:
                cmd = 'R'
            else:
                cmd = 'F'

    now = time.monotonic()
    if cmd != last_cmd or now - last_send >= 0.25:
        ser.write(cmd.encode())
        last_send = now
        if cmd != last_cmd:
            print(cmd)
            last_cmd = cmd

The Pi sends each command immediately when it changes and repeats it every 250 ms while the camera stream is healthy. Open Arduino Serial at 115200 baud. The Arduino receives each letter and drives the motors through the L298N. Use the motor functions from our L298N motor driver guide: F = both forward, L = left slow and right fast, R = the opposite, S = stop. Add a safety rule on the Arduino: if no command arrives for about one second, stop the motors. That way a crashed Python script does not leave the robot driving into a wall.

Wiring and power

Scroll sideways to see all columns.

Connection Notes
Pi USB port → Arduino USB port Serial commands; also powers the Arduino logic
Arduino D5 / D6 → L298N ENA / ENB Remove the ENA and ENB jumper caps for PWM
Arduino D7 / D8 / D9 / D10 → IN1 / IN2 / IN3 / IN4 Same mapping as the line follower
Arduino 5 V → L298N 5V logic terminal Remove the separate 5V-EN jumper first
Motor battery → L298N 12V and GND Motors never run from the Pi
L298N GND → Arduino GND Common ground for the control signals
Pi power Its own good supply or a suitable power bank

Using USB between Pi and Arduino avoids connecting the Pi's 3.3 V GPIO to the Arduino's 5 V pins directly. The robot base is the same as in our line follower and obstacle-avoiding robot builds.

Realistic speed (FPS) expectations

A Pi is far slower than a laptop with a graphics card. Colour detection at 320×240 is usually smooth enough for a slow robot, but heavier work like deep-learning object detection can drop to only a few frames per second. To keep it fast:

  • Use low resolution. 320×240 is plenty for one ball.
  • Skip frames. Process every second or third frame if needed.
  • Avoid imshow while driving. Showing windows costs time; turn them off for the demo.
  • Use small models if you later move to object detection.
  • Measure your own FPS by timing the loop. Report the real number in your project, not one from the internet.

Troubleshooting

Scroll sideways to see all columns.

Symptom Likely cause Fix
cap.read() fails or black frame Wrong camera index or Pi Camera used with VideoCapture Try index 1; use the Picamera2 lines for the Pi Camera
Mask is all white HSV range too wide Raise Smin and Vmin
Ball detected on and off Lighting changes, range too tight Retune on site; widen hue slightly
Red ball not detected Red needs two hue ranges Combine masks for 0–10 and 170–179
could not open port /dev/ttyACM0 Different name or no permission Check ls /dev/tty*; add your user to the dialout group
Robot turns the wrong way Motor wires or L/R logic swapped Swap motor wires or swap 'L' and 'R'
Pi reboots when motors start Shared or weak power Separate supplies, common GND only

FAQ

Can I run this on an ESP32-CAM instead?

Simple colour detection is possible, but the ESP32-CAM has far less memory and speed, and OpenCV is not available in its normal form. The Pi is much easier for learning.

Can it detect objects like people or bottles?

Yes, with pre-trained models, but expect low FPS on a Pi. Colour detection is the best first step.

Is this enough for a final-year project?

On its own it is a good minor project. For a major project, add path planning, multiple targets, or a comparison of methods. Check your department's format.

Python or C++?

Python is easier to write and debug. C++ can be faster, but on a Pi the camera and resolution usually matter more.

Related tutorials

See all Robotics →