Edge AI and TinyML: Running Machine Learning on Microcontrollers and Embedded Devices

Most conversations about AI focus on enormous models running in data centres. But a quieter revolution is happening at the other end of the scale. TinyML refers to machine learning on extremely resource-constrained devices, typically microcontrollers with kilobytes of RAM and power budgets measured in milliwatts. The community around tinyML has helped turn this from a research curiosity into a practical discipline.

The motivations are straightforward:

• Connectivity: a sensor in a farm in Kwara or on a pipeline in the Niger Delta may have no reliable network. On-device inference keeps working regardless.
• Power: transmitting raw data over radio usually costs far more energy than processing it locally. Sending a single "anomaly detected" message instead of a continuous audio stream can extend battery life dramatically.
• Latency: safety-critical responses, such as stopping a motor, cannot wait for a round trip to the cloud.
• Privacy: audio or images processed on-device and immediately discarded never become a data protection liability.
• Cost: microcontrollers are cheap, which makes dense sensor deployments affordable.

My interest here comes from the intersection of embedded systems and geospatial work. Environmental monitoring at scale needs many cheap, autonomous sensors, and those sensors become far more useful when they can decide locally what is worth reporting.

The workflow differs from cloud ML mainly in its obsession with size. A typical pipeline looks like this:

1. Collect data on the target sensor. Models trained on lab-quality data often fail on cheap sensors with different noise profiles, so capture data with the actual hardware.
2. Engineer compact features. For audio, that usually means spectrograms or MFCCs; for motion, windowed statistics from accelerometers. Good features let you use far smaller models.
3. Train a small architecture. Depthwise-separable convolutions, tiny dense networks and even classical models like decision trees are common. Research such as MCUNet co-designs neural architectures and inference engines specifically for microcontroller memory limits.
4. Quantise. Converting 32-bit floating-point weights and activations to 8-bit integers cuts model size by roughly four times and enables fast integer arithmetic. The integer-arithmetic-only approach described by Jacob et al. underpins much of today's tooling.
5. Convert and deploy. Frameworks such as TensorFlow Lite Micro provide an interpreter designed for bare-metal environments with no operating system and no dynamic memory allocation.
6. Profile on device. Measure latency, peak RAM (the "tensor arena"), flash usage and energy per inference on the real board.

A minimal inference loop on a microcontroller is refreshingly simple:

// allocate a fixed memory arena once at startup
static uint8_t tensor_arena[20 * 1024];

void loop() {
  read_sensor_window(input->data.int8);
  interpreter.Invoke();
  if (output->data.int8[ANOMALY] > threshold) {
    send_alert();   // radio wakes only when needed
  }
  deep_sleep_until_next_window();
}

Platforms like Edge Impulse wrap much of this pipeline in a guided workflow, which is useful for teams new to embedded development.

Applications that work well on tiny devices share a pattern: they detect a specific, bounded event in a continuous sensor stream.

• Keyword spotting: the classic example, where a device listens for a wake word without streaming audio anywhere.
• Predictive maintenance: vibration sensors on pumps, generators and motors flag unusual signatures before failure. Given how many Nigerian businesses depend on diesel generators, early fault detection has obvious value.
• Agriculture: detecting pest sounds, monitoring soil conditions or recognising livestock behaviour from collar sensors.
• Environmental and climate monitoring: classifying air-quality events, detecting chainsaw sounds in protected forests or monitoring flood-prone drainage channels.
• Health wearables: fall detection and activity recognition on low-power devices, where keeping raw data local supports privacy.

A step up from microcontrollers, edge gateways and single-board computers with small neural accelerators can run object detection or compact language models locally, aggregating data from many tiny sensors.

This tiered design, sometimes described as a cascade, is worth highlighting. A microcontroller runs a tiny, always-on model that decides whether something interesting is happening. Only then does it wake a more capable processor or transmit a short clip for heavier analysis. The always-on stage is optimised for energy, the second stage for accuracy. Combined, they deliver behaviour that neither stage could achieve alone within the same power budget, and they keep the vast majority of raw data on the device where it was captured.

TinyML is powerful within its envelope, but the envelope is tight.

• Model capacity: you cannot fit general-purpose intelligence into a few hundred kilobytes. Problems must be narrowly defined.
• Updating models in the field: over-the-air updates require secure bootloaders, signed firmware and careful rollback strategies. A failed update on a remote device can mean a truck journey.
• Security: physical access to devices enables firmware extraction and tampering. Secure elements, encrypted storage and disabled debug ports matter, and cybersecurity should be designed in from the first prototype.
• Silent drift: sensors degrade, environments change and devices rarely phone home with rich telemetry. Plan for periodic sampling of raw data to detect drift.
• E-waste: cheap, disposable smart devices carry an environmental cost. Design for longevity and repairability where possible.

For teams in emerging markets, a few practices make a big difference:

1. Start with the power budget. Decide how long the device must run on its battery or solar panel, then work backwards to duty cycle and model size.
2. Choose widely available hardware. Import delays and customs costs are real. Prefer boards and sensors you can source reliably, and design for substitutes.
3. Prototype the data pipeline first. Getting labelled data from the actual environment is usually the slowest step. Build a simple logger and deploy it before you train anything.
4. Keep a cloud companion. Edge does not mean isolated. A lightweight backend, perhaps a Django API receiving compact event messages, gives you fleet visibility, alerting and a channel for model updates.
5. Test in harsh conditions. Heat, dust, humidity and voltage fluctuations affect both electronics and sensor readings.
6. Measure energy per inference. It is the metric that most directly determines field viability.

• TinyML runs inference on microcontrollers, enabling offline, low-power, privacy-preserving intelligence.
• Quantisation, compact architectures and good feature engineering are the core techniques.
• Best-fit problems detect specific events in continuous sensor streams.
• Security, field updates and drift monitoring must be designed in from the start.
• In regions with unreliable power and connectivity, edge AI is often the most practical architecture.
• Pair edge devices with a lightweight cloud backend for fleet management and model updates.

1. tinyML Foundation — tinyML
2. TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems — arXiv
3. MCUNet: Tiny Deep Learning on IoT Devices — arXiv
4. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference — arXiv
5. Edge Impulse — Edge Impulse

Cover image: original illustration created for this article. Cited works remain the property of their respective authors and publishers.