TinyML on ESP32 with Edge Impulse: Your First On-Device Model
Train a gesture recognition model with Edge Impulse and run it on an ESP32 with an MPU6050. Data collection, training, int8 deployment and testing.
On this page
Most people think AI needs a powerful computer or a cloud service. TinyML is the opposite idea: a small, trained model runs directly on a microcontroller, with no internet connection at all. The board reads its sensor, runs the model and makes a decision in milliseconds.
In this guide you will build a gesture recogniser. Hold an ESP32 with an MPU6050 accelerometer, draw a circle or a wave in the air, and the board tells you which gesture it was. You will collect data, train a model in Edge Impulse, export it as an Arduino library and run it on the ESP32.
Edge Impulse updates its website often, so menu names may differ slightly from what is written here. The workflow stays the same.
What you need
Scroll sideways to see all columns.
| Item | Qty | Notes |
|---|---|---|
| ESP32 dev board | 1 | Any common ESP32-WROOM board |
| MPU6050 module (GY-521) | 1 | Accelerometer source |
| Breadboard, jumper wires | 1 set | Keep the wiring firm; it will be waved around |
| USB data cable | 1 | Long enough to move your hand freely |
| Computer with Node.js | 1 | For the Edge Impulse CLI data forwarder |
| Free Edge Impulse account | 1 | Sign up on the Edge Impulse website |
Wiring
Scroll sideways to see all columns.
| MPU6050 pin | ESP32 pin |
|---|---|
| VCC | 3.3V |
| GND | GND |
| SDA | GPIO21 |
| SCL | GPIO22 |
Fix the MPU6050 firmly to the breadboard, with the ESP32 alongside, so the sensor moves with your hand and not by itself on loose wires.
How the Edge Impulse flow works
- Collect raw accelerometer samples while you perform each gesture.
- Label each recording:
circle,waveandidle. The idle class is important; without it the model will force every movement into a gesture. - Design an impulse: cut the data into windows (for example 2 seconds), extract features with a Spectral Analysis block, and feed them to a small neural network classifier.
- Train and test: Edge Impulse keeps a test set aside that the model never sees during training.
- Deploy as an Arduino library, which you include in a normal sketch.
Step 1: Stream data to Edge Impulse
The Edge Impulse data forwarder reads comma-separated numbers from the serial port and sends them to your project. Install the Edge Impulse CLI on your computer (it needs Node.js) by following the official installation page.
// Data forwarder sketch: MPU6050 accel at 50 Hz - not compiled yet
#include <Wire.h>
const int MPU = 0x68;
const float SCALE = 9.81 / 8192.0; // +/-4 g range: 8192 LSB per g
const unsigned long INTERVAL_MS = 20; // 50 Hz
void setup() {
Serial.begin(115200);
Wire.begin(21, 22);
Wire.beginTransmission(MPU);
Wire.write(0x6B); Wire.write(0); // wake up
Wire.endTransmission();
Wire.beginTransmission(MPU);
Wire.write(0x1C); Wire.write(0x08); // ACCEL_CONFIG: +/-4 g
Wire.endTransmission();
}
void readAccel(float &x, float &y, float &z) {
Wire.beginTransmission(MPU);
Wire.write(0x3B);
Wire.endTransmission(false);
Wire.requestFrom(MPU, 6, true);
int16_t ax = Wire.read() << 8 | Wire.read();
int16_t ay = Wire.read() << 8 | Wire.read();
int16_t az = Wire.read() << 8 | Wire.read();
x = ax * SCALE; y = ay * SCALE; z = az * SCALE;
}
void loop() {
static unsigned long last = 0;
if (millis() - last >= INTERVAL_MS) {
last = millis();
float x, y, z;
readAccel(x, y, z);
Serial.print(x); Serial.print(',');
Serial.print(y); Serial.print(',');
Serial.println(z);
}
}
We use the ±4 g range because a quick wave can exceed 2 g and the readings would clip.
Close the Arduino Serial Monitor (only one program can use the port), then run edge-impulse-data-forwarder in a terminal. Log in, choose your project, and name the three axes accX,accY,accZ. The tool detects the frequency automatically.
Step 2: Collect varied data
In the Edge Impulse Data acquisition page, record 10-second samples for each label. Aim for several minutes per class to start with.
The most common reason a TinyML model "works for me but fails for my friend" is narrow data. Collect from:
- Different people. Ask classmates to perform the gestures too.
- Different speeds and sizes. Small circles, big circles, fast and slow.
- Different positions. Sitting, standing, arm up, arm down.
- Real idle. Holding still, walking, scratching your head, picking up a phone. Idle is "everything else".
Keep the classes roughly balanced. Let Edge Impulse split the data into training and test sets, and do not move test samples into training to boost the score.
Step 3: Design the impulse and train
- Create impulse: time-series input with a window of about 2000 ms and a window increase of about 200 ms to 500 ms.
- Add a Spectral Analysis processing block, then a Classification learning block.
- Open Spectral Analysis, keep the defaults, and click Generate features. The feature explorer shows whether your classes form separate clusters. If two classes overlap heavily, your gestures may be too similar.
- Open Classifier and train with the defaults first. Watch the accuracy and the confusion matrix.
- Run Model testing on the test set. This number is more honest than training accuracy.
Step 4: Quantised int8 model
Edge Impulse can export the model as float32 or quantised int8. The int8 version stores weights as 8-bit integers instead of 32-bit floats, so it uses much less memory and runs faster on a microcontroller. Accuracy often changes only a little, but it can change, so compare both versions in the deployment page and test on real gestures, not only on the website's numbers.
Step 5: Deploy to the ESP32
In Deployment, choose Arduino library, select the int8 option, and build. You get a ZIP file. In Arduino IDE use Sketch → Include Library → Add .ZIP Library. The library name follows your project name, for example gestures_inferencing.h.
// Run the gesture model on ESP32 - not compiled yet
#include <gestures_inferencing.h> // change to your project's header
#include <Wire.h>
const int MPU = 0x68;
const float SCALE = 9.81 / 8192.0;
float buffer[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE];
void setup() {
Serial.begin(115200);
Wire.begin(21, 22);
Wire.beginTransmission(MPU); Wire.write(0x6B); Wire.write(0); Wire.endTransmission();
Wire.beginTransmission(MPU); Wire.write(0x1C); Wire.write(0x08); Wire.endTransmission();
Serial.println("Ready. Perform a gesture.");
}
void readAccel(float *out) {
Wire.beginTransmission(MPU);
Wire.write(0x3B);
Wire.endTransmission(false);
Wire.requestFrom(MPU, 6, true);
for (int i = 0; i < 3; i++) {
int16_t v = Wire.read() << 8 | Wire.read();
out[i] = v * SCALE;
}
}
void loop() {
// fill one window of samples at the model's sampling interval
for (size_t i = 0; i < EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE; i += 3) {
unsigned long next = millis() + EI_CLASSIFIER_INTERVAL_MS;
readAccel(&buffer[i]);
while (millis() < next) { }
}
signal_t signal;
numpy::signal_from_buffer(buffer, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, &signal);
ei_impulse_result_t result;
if (run_classifier(&signal, &result, false) != EI_IMPULSE_OK) return;
const char *best = "";
float bestScore = 0;
for (size_t i = 0; i < EI_CLASSIFIER_LABEL_COUNT; i++) {
if (result.classification[i].value > bestScore) {
bestScore = result.classification[i].value;
best = result.classification[i].label;
}
}
if (bestScore > 0.8) { // only act when the model is confident
Serial.print(best); Serial.print(" "); Serial.println(bestScore, 2);
}
}
The sample rate and axis order must match what you used during data collection. If you recorded at 50 Hz with X, Y, Z, the board must feed the model exactly that.
Edge vs cloud AI
Scroll sideways to see all columns.
| On-device (TinyML) | Cloud API | |
|---|---|---|
| Internet needed | No | Yes |
| Privacy | Raw data stays on the board | Data leaves the device |
| Delay | Milliseconds | Depends on the network |
| Model size | Very small | Very large models possible |
| Power | Low | Wi-Fi uses much more |
For places with weak or no connectivity, which includes many rural areas of Nepal, on-device decisions are a real advantage.
Troubleshooting
Scroll sideways to see all columns.
| Symptom | Likely cause | Fix |
|---|---|---|
| Data forwarder finds no device | Serial Monitor still open | Close it, unplug and replug, retry |
| Frequency detected wrongly | Uneven printing | Keep the fixed 20 ms timing; no extra prints |
| Great test accuracy, poor live results | Data from one person or one way of holding | Collect more varied data and retrain |
| Always predicts one gesture | No idle class, or unbalanced data | Add idle data; balance classes |
| Library will not compile | Wrong header name or old board package | Check the header in the ZIP; update the esp32 core |
| Readings flat at a maximum value | Range too small, data clipping | Use ±4 g or ±8 g, the same in both sketches |
FAQ
Is this a good final-year or semester project?
Yes, if you go beyond the demo: collect a proper dataset, report a confusion matrix on unseen people, and compare float32 vs int8. See our machine learning project ideas. Check your department's format for the report.
Can the ESP32 recognise voice or images?
Simple keyword spotting and very small image models are possible, but memory is tight. Start with accelerometer data, which is the easiest.
Do I need to understand neural networks first?
Not to start. But for a viva you should be able to explain windows, features, training vs test data and quantisation in simple words.
What can I build with gestures?
Wearable remote controls, sign-language prototypes, fall detection ideas (learning prototype only) or exercise counters. Our ESP32 IoT project ideas and final year project ideas list more.