Running AI Inference on an ESP32 — Smaller Than You Think, More Useful Than You Expect

The phrase 'edge AI' used to mean running inference on a Raspberry Pi or Jetson Nano instead of a cloud server. It now means something more interesting: running inference on microcontrollers like the ESP32, with sub-milliwatt power consumption and millisecond latency, completely offline.

This isn't theoretical. I've deployed two systems in the last year that run AI models on ESP32-based hardware. One detects anomalous vibration signatures in a small motor. The other classifies audio events — not speech recognition, but detecting specific sounds in an environment.

What's Actually Possible on an ESP32

The ESP32 has 520KB of SRAM and 4MB of flash (on most development boards). This is not much. But TensorFlow Lite for Microcontrollers (TFLite Micro) is designed to operate in exactly these constraints — models must fit in RAM, inference runs in kilobytes of working memory.

Models that work well: keyword detection (wake word), gesture recognition from IMU data, vibration anomaly detection, simple image classification (with ESP32-S3 and camera), time-series pattern recognition on sensor data.

• TensorFlow Lite Micro — the standard framework for microcontroller inference
• Models are quantized to INT8 (8-bit weights) — 4x smaller than float32
• Inference time for a small model: 10–100ms on ESP32
• Power consumption: ~80mA during inference, back to sleep mode otherwise
• No internet connection required — fully offline inference

The Training vs Deployment Split

You don't train models on the ESP32 — you train them on a PC or cloud service (Google Colab is free and sufficient for most small models), then convert and quantize for deployment.

Edge Impulse is the easiest end-to-end platform for this workflow specifically — it handles data collection, training, quantization, and generates Arduino or ESP-IDF compatible code. For someone new to TinyML, this is the fastest path to a working system.

Train in the cloud. Deploy on the edge. The ESP32 never sees the training data — it only runs the finished model.

A Real Use Case: Motor Anomaly Detection

The most immediately useful AIoT application I've deployed is motor health monitoring. An ESP32 with an accelerometer (ADXL345 or MPU6050) samples vibration at 400Hz. A small TFLite model trained on normal and anomalous vibration signatures classifies each window as healthy or anomalous.

When anomaly is detected, it publishes an MQTT alert. The whole system runs on 3.7V LiPo, uses under 100mA during active monitoring, and has been running for four months without intervention. This is an actually useful system, not a demo.

Getting Started With TinyML

The book 'TinyML' by Pete Warden and Daniel Situnayake is the most comprehensive resource. Edge Impulse provides the fastest practical path. The ESP32-S3 specifically (vs original ESP32) has hardware-accelerated inference instructions that make it significantly faster for ML workloads.