On-Device AI in Android: MediaPipe Tasks & Gemini Nano

Running Artificial Intelligence models directly on mobile hardware is one of the most transformative advances in modern software engineering. Unlike cloud-based AI that incurs server costs and network latency, On-Device AI executes locally on device NPUs and GPUs — delivering instantaneous zero-latency inference, zero server bills, and 100% user privacy.

Figure 41.1: Android On-Device AI Stack — MediaPipe Tasks and Gemini Nano Accelerated by Hardware NPUs and Neural Networks API (NNAPI).

MediaPipe Tasks: Computer Vision in 10 Lines of Kotlin

Google's MediaPipe Tasks API provides pre-trained on-device vision, text, and audio models ready for production:

import android.content.Context
import android.graphics.Bitmap
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.objectdetector.ObjectDetector
import com.google.mediapipe.tasks.vision.objectdetector.ObjectDetector.ObjectDetectorOptions
import com.google.mediapipe.framework.image.BitmapImageBuilder

class LocalObjectDetector(context: Context) {

    private val detector: ObjectDetector

    init {
        val options = ObjectDetectorOptions.builder()
            .setBaseOptions(
                com.google.mediapipe.tasks.core.BaseOptions.builder()
                    .setModelAssetPath("efficientdet_lite0.tflite")
                    .build()
            )
            .setScoreThreshold(0.5f)
            .setRunningMode(RunningMode.IMAGE)
            .build()

        detector = ObjectDetector.createFromOptions(context, options)
    }

    fun detectObjects(bitmap: Bitmap): List<String> {
        val mpImage = BitmapImageBuilder(bitmap).build()
        val results = detector.detect(mpImage)

        return results.detections().map { detection ->
            val label = detection.categories().firstOrNull()?.categoryName() ?: "Unknown"
            val score = detection.categories().firstOrNull()?.score() ?: 0f
            "$label (${(score * 100).toInt()}%)"
        }
    }
}

Android AICore & Gemini Nano

Starting in modern Android versions, Android AICore is a system-level service that provides direct access to Gemini Nano — Google's most efficient multimodal LLM designed for on-device silicon. Apps can execute Smart Reply, text summarization, proofreading, and audio transcription completely offline without making a single network call.