OpenCV on Raspberry Pi: Colour Detection for a Robot
Detect a coloured ball with OpenCV on a Raspberry Pi using HSV masks and contours, then steer an Arduino robot over USB serial. Code, tuning and FPS tips.
On this page
A robot that follows a red ball looks like magic at a college exhibition, but the idea behind it is simple. A camera on a Raspberry Pi looks for pixels of one colour, finds the biggest blob, and works out whether it is on the left, in the centre or on the right. The Pi then tells an Arduino which way to drive.
In this guide you will install OpenCV on the Pi, detect a colour using HSV, find the ball's position with contours, tune the colour range with sliders, and send steering commands to an Arduino over USB serial.
What you need
Scroll sideways to see all columns.
| Item | Qty | Notes |
|---|---|---|
| Raspberry Pi 4 or 5 with Raspberry Pi OS | 1 | Pi 3B+ works but slower |
| Official or good-quality Pi power supply | 1 | Weak supplies cause random problems |
| Pi Camera Module or USB webcam | 1 | Code below covers both |
| Arduino Uno + L298N + 2-motor chassis | 1 set | The robot base |
| Motor battery pack | 1 | Separate from the Pi supply |
| Brightly coloured ball | 1 | Red, blue or green; avoid colours found in the room |
| USB cable (Pi to Arduino) | 1 | Carries serial commands |
Why HSV and not RGB
A camera gives pixels as Blue, Green, Red values (OpenCV uses BGR order). The trouble is that the same red ball gives very different RGB numbers in sunlight, under a tube light or in shade, because brightness is mixed into all three channels.
HSV separates colour from brightness:
Scroll sideways to see all columns.
| Channel | Meaning | OpenCV range |
|---|---|---|
| H (hue) | The colour itself, as a position on a colour wheel | 0 to 179 |
| S (saturation) | How strong the colour is (grey = low) | 0 to 255 |
| V (value) | How bright it is | 0 to 255 |
For a blue ball you can say "hue between about 100 and 130, saturation above 120, value above 70", and that works across much more lighting. Red is special: it sits at both ends of the hue wheel, so you need two ranges (about 0–10 and 170–179) and combine them.
Step 1: Install OpenCV
On current Raspberry Pi OS, install from the package manager. It is faster and more reliable than building OpenCV yourself.
sudo apt update
sudo apt install -y python3-opencv python3-serial python3-picamera2
python3 -c "import cv2; print(cv2.__version__)"
If the last command prints a version number, OpenCV is ready. python3-picamera2 is already present on most full Raspberry Pi OS images and is needed only for the Pi Camera Module.
Step 2: Find your HSV range with sliders
This tool shows the mask live while you move sliders. Hold the ball in front of the camera and adjust until only the ball is white. This code is not tested yet; it uses a USB webcam. For the Pi Camera, see the camera lines in Step 3.
# HSV tuner - not tested yet
import cv2
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 320)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 240)
cv2.namedWindow("sliders")
for name, maxv, start in [("Hmin",179,100),("Hmax",179,130),("Smin",255,120),
("Smax",255,255),("Vmin",255,70),("Vmax",255,255)]:
cv2.createTrackbar(name, "sliders", start, maxv, lambda x: None)
while True:
ok, frame = cap.read()
if not ok:
break
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
v = [cv2.getTrackbarPos(n, "sliders") for n in ("Hmin","Smin","Vmin","Hmax","Smax","Vmax")]
mask = cv2.inRange(hsv, tuple(v[:3]), tuple(v[3:]))
cv2.imshow("frame", frame)
cv2.imshow("mask", mask)
if cv2.waitKey(1) & 0xFF == ord('q'):
print("lower =", v[:3], " upper =", v[3:])
break
cap.release()
cv2.destroyAllWindows()
Press q to print your values. Run this on the Pi desktop (or over VNC), since it opens windows. Retune at the place where you will demo, because exhibition halls often have very different lighting from your room.
Step 3: Detect the ball and steer
The program below finds the largest blob of your colour, calculates its centre, and decides:
- Ball in the left third of the frame → send
L - Ball in the middle → send
F(forward) - Ball in the right third → send
R - No ball, or the ball is very close (large area) → send
S(stop)
This code is not tested yet.
# Colour tracker that sends steering commands - not tested yet
import cv2, serial, time
import numpy as np
LOWER = np.array([100, 120, 70]) # paste your tuned values
UPPER = np.array([130, 255, 255])
MIN_AREA = 300 # ignore tiny specks
STOP_AREA = 25000 # ball very close: stop
W, H = 320, 240
ser = serial.Serial('/dev/ttyACM0', 115200, timeout=0.1) # may be /dev/ttyUSB0
time.sleep(2) # Arduino resets when the port opens
# --- USB webcam ---
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, W)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, H)
def grab():
ok, f = cap.read()
return f if ok else None
# --- Pi Camera Module instead: comment the webcam block, uncomment this ---
# from picamera2 import Picamera2
# picam = Picamera2()
# picam.configure(picam.create_preview_configuration(main={"format": "RGB888", "size": (W, H)}))
# picam.start()
# def grab():
# return picam.capture_array() # "RGB888" arrives in BGR order, ready for OpenCV
kernel = np.ones((5, 5), np.uint8)
last_cmd = None
last_send = 0.0
while True:
frame = grab()
if frame is None:
ser.write(b'S')
break
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, LOWER, UPPER)
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel) # remove noise
contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cmd = 'S'
if contours:
c = max(contours, key=cv2.contourArea)
area = cv2.contourArea(c)
if MIN_AREA < area < STOP_AREA:
M = cv2.moments(c)
cx = int(M["m10"] / M["m00"])
if cx < W / 3:
cmd = 'L'
elif cx > 2 * W / 3:
cmd = 'R'
else:
cmd = 'F'
now = time.monotonic()
if cmd != last_cmd or now - last_send >= 0.25:
ser.write(cmd.encode())
last_send = now
if cmd != last_cmd:
print(cmd)
last_cmd = cmd
The Pi sends each command immediately when it changes and repeats it every 250 ms while the camera stream is healthy. Open Arduino Serial at 115200 baud. The Arduino receives each letter and drives the motors through the L298N. Use the motor functions from our L298N motor driver guide: F = both forward, L = left slow and right fast, R = the opposite, S = stop. Add a safety rule on the Arduino: if no command arrives for about one second, stop the motors. That way a crashed Python script does not leave the robot driving into a wall.
Wiring and power
Scroll sideways to see all columns.
| Connection | Notes |
|---|---|
| Pi USB port → Arduino USB port | Serial commands; also powers the Arduino logic |
| Arduino D5 / D6 → L298N ENA / ENB | Remove the ENA and ENB jumper caps for PWM |
| Arduino D7 / D8 / D9 / D10 → IN1 / IN2 / IN3 / IN4 | Same mapping as the line follower |
| Arduino 5 V → L298N 5V logic terminal | Remove the separate 5V-EN jumper first |
| Motor battery → L298N 12V and GND | Motors never run from the Pi |
| L298N GND → Arduino GND | Common ground for the control signals |
| Pi power | Its own good supply or a suitable power bank |
Using USB between Pi and Arduino avoids connecting the Pi's 3.3 V GPIO to the Arduino's 5 V pins directly. The robot base is the same as in our line follower and obstacle-avoiding robot builds.
Realistic speed (FPS) expectations
A Pi is far slower than a laptop with a graphics card. Colour detection at 320×240 is usually smooth enough for a slow robot, but heavier work like deep-learning object detection can drop to only a few frames per second. To keep it fast:
- Use low resolution. 320×240 is plenty for one ball.
- Skip frames. Process every second or third frame if needed.
- Avoid
imshowwhile driving. Showing windows costs time; turn them off for the demo. - Use small models if you later move to object detection.
- Measure your own FPS by timing the loop. Report the real number in your project, not one from the internet.
Troubleshooting
Scroll sideways to see all columns.
| Symptom | Likely cause | Fix |
|---|---|---|
cap.read() fails or black frame |
Wrong camera index or Pi Camera used with VideoCapture | Try index 1; use the Picamera2 lines for the Pi Camera |
| Mask is all white | HSV range too wide | Raise Smin and Vmin |
| Ball detected on and off | Lighting changes, range too tight | Retune on site; widen hue slightly |
| Red ball not detected | Red needs two hue ranges | Combine masks for 0–10 and 170–179 |
could not open port /dev/ttyACM0 |
Different name or no permission | Check ls /dev/tty*; add your user to the dialout group |
| Robot turns the wrong way | Motor wires or L/R logic swapped | Swap motor wires or swap 'L' and 'R' |
| Pi reboots when motors start | Shared or weak power | Separate supplies, common GND only |
FAQ
Can I run this on an ESP32-CAM instead?
Simple colour detection is possible, but the ESP32-CAM has far less memory and speed, and OpenCV is not available in its normal form. The Pi is much easier for learning.
Can it detect objects like people or bottles?
Yes, with pre-trained models, but expect low FPS on a Pi. Colour detection is the best first step.
Is this enough for a final-year project?
On its own it is a good minor project. For a major project, add path planning, multiple targets, or a comparison of methods. Check your department's format.
Python or C++?
Python is easier to write and debug. C++ can be faster, but on a Pi the camera and resolution usually matter more.