On-Device AI for Mobile Apps: When It Fits and When It Won't
On-device AI for mobile apps means the AI model runs on the phone itself instead of on a server. Apple (since iOS 26) and Google (through a system service on recent Android phones) now ship a small language model on the phone that any app can call. There is no per-request bill, it works with no signal, and the user's text never leaves the phone. The catch is that it only runs on newer phones, it has a short memory, and it is built for small, focused jobs rather than open-ended chat. If your feature is "summarize this note", "sort this message into a category" or "pull the fields out of this receipt", on-device is often the best choice. If your feature is "answer any question about our 200-page manual", it usually is not.
What on-device AI for mobile apps gives you now
Until recently, on-device AI meant shipping your own model file inside the app and hoping it fit. That still works, but most small-business apps no longer need it, because the platforms now include a model for you.
On iPhone, Apple's Foundation Models framework gives apps direct access to the roughly 3-billion-parameter model behind Apple Intelligence. Apple's September 2025 announcement says it is available with iOS 26, iPadOS 26 and macOS 26, works offline, and uses inference that is "free of cost". It supports guided generation, where the model fills in a structure your code defines, and tool calling, where the model can ask your app for data.
On Android, Google's ML Kit GenAI APIs run on Gemini Nano through a system service called AICore. Google offers ready-made task APIs for summarization, proofreading, rewriting and image description, plus a Prompt API for custom requests. Google's page says input, inference and output stay on the device and there is "no additional server cost" for each call. In a July 2026 developer post, Google said Gemini Nano runs on over 140 million devices.
In plain terms, a feature that would carry a per-use charge on a cloud API can now cost nothing per use, and it keeps working in a basement, a barn or a hospital corridor.
!A smartphone with a glowing chip inside it, linked to a distant cloud by a broken dotted line The model ships with the operating system, so your app does not carry the weight or pay per request.
The limits that decide whether it fits your app
The free model comes with conditions, and four of them shape most real projects.
Only newer phones run it
Apple's Apple Intelligence support page lists iPhone 15 Pro, iPhone 15 Pro Max, iPhone 16 models and later, and iPhone Air, among others. A regular iPhone 15 or anything older does not qualify, and the user must also have Apple Intelligence turned on. On Android, Google keeps a device list on the ML Kit GenAI page that covers recent Pixel phones, recent Samsung Galaxy flagships and some other brands. Budget Android phones are mostly not on it. Your app has to check availability every time and do something sensible when the model is missing.
The memory is short
Google's Prompt API guide says input must fit in a context window of under 4,000 tokens, roughly 3,000 English words, and warns against use cases that need long output. Apple's WWDC26 session on its cloud model gives the on-device model a 4K context in the same range. That is plenty for one email or one form. It is not enough to read a long document in one pass.
Android only runs it in the foreground
Google's overview says GenAI inference is allowed only when your app is the top foreground app, and AICore enforces a per-app quota, including longer limits tied to battery use, such as a daily cap. A feature that quietly processes a thousand records overnight will not work this way.
It is a small model
A model this size is good at rewriting, classifying, extracting and short summaries. It is weaker at facts about the world, at math, and at long reasoning. Do not ask it to give medical, legal or pricing answers from its own knowledge.
Where on-device AI is the right call
Here are the shapes of features where we would reach for the built-in model first. These are illustrative examples, not client projects.
- Turning messy input into clean fields. A contractor photographs a supplier receipt, and the app fills in vendor, date and total. Google's July 2026 post walks through this exact receipt-parsing pattern with structured output.
- Short summaries of the user's own content: today's job notes, or a long customer message, cut to three lines.
- Sorting and tagging, so the app can decide whether an incoming request is a booking, a complaint or a question and route it to the right list.
- Writing help, such as proofreading or softening a reply before the user sends it.
- Anything private or offline: health notes, field inspections, or work in places with no signal.
In each case the job is small, the input is already on the phone, and a slightly imperfect answer is easy for the user to fix.
!Colored cards spilling out of a phone and sorted into neat colored bins and rows Small, structured jobs like sorting and extracting are where a compact model is strongest.
Where a cloud model is still better
Pick on-device for the wrong job and you get a feature users stop trusting. These are the cases where we would start in the cloud.
- Your users have older or budget phones. If a large share of your customers carry phones that are not on the support lists, most of them never see the feature. You end up building the cloud version anyway.
- The answer depends on your data, not theirs. A chatbot that answers from your price list, policies or inventory needs that data. A cloud model with retrieval handles this far better than a 4K window.
- You need the same answer on every phone. Apple and Google update their models with the OS, so behavior can shift between versions. If consistency matters, a pinned cloud model is easier to control.
- The task is long or hard. Long documents, multi-step reasoning and anything needing current facts belong in the cloud.
Apple now offers a middle option. Its WWDC26 session describes a larger model on Private Cloud Compute with a 32K context, no token costs to the developer, and a daily limit per user. It requires an internet connection and an Apple Intelligence device, and Apple says it is for apps with fewer than 2 million downloads that apply for access. It is worth a look for iPhone-only apps, but it does nothing for your Android users.
The hybrid design most apps end up with
In practice, the strongest pattern is on-device first with a fallback. The app checks whether the local model is available. If it is, the feature runs free and offline. If it is not, the app either sends the request to a cloud model you pay for, or hides the feature and offers the plain version of the screen.
Most of the build effort goes into that fallback. You design two code paths, test on phones that have the model and phones that do not, and decide what the user sees in each case. It is also why on-device AI rarely makes a build cheaper. It makes the feature cheaper to run, which matters more as you grow. If you are working out what that does to your budget, our guide to what an app costs per month to run breaks down the hosting, service and store fees an AI feature adds to.
!A path looping inside a phone and an arrow rising to a cloud, with a check mark beside them Most apps check for the local model first and fall back to the cloud, so every user gets the feature.
When US APP Team is the wrong fit
If your app's main product is the AI itself, such as a custom model trained on your own data, a voice agent, or a research tool, you want an ML specialist, not a small-business app studio. We build apps where AI is one useful feature inside a product people already need. Our builds run from $3,500 to $12,000 on our pricing page, and an AI feature with a proper fallback usually puts a project in the upper part of that range. For a minimum version of a new idea, our MVP app development page explains how we keep the first release small.
If you are unsure whether your feature fits on the phone, write down the one job you want the AI to do, the longest input it would see, and which phones your customers carry. Those three answers settle most of the decision, and you can put them straight into the app brief wizard.
Frequently asked questions
Is on-device AI free for app developers?
The built-in models are. Apple says Foundation Models inference is free of cost, and Google says ML Kit GenAI calls carry no additional server cost. You still pay to design, build and test the feature and its fallback.
Which iPhones support on-device AI apps?
Apps using Apple's Foundation Models framework need an Apple Intelligence device with Apple Intelligence turned on. Apple's support page lists iPhone 15 Pro, iPhone 15 Pro Max, iPhone 16 models and later, and iPhone Air, among others.
Does on-device AI work without internet?
Yes. Apple's on-device model and Google's Gemini Nano both run locally once the model is on the phone. Apple's larger Private Cloud Compute model is the exception, because it needs a connection.
Can on-device AI replace a cloud chatbot?
Rarely. On-device AI for mobile apps runs on small models with a context window of about 4,000 tokens and limited knowledge, so a chatbot that answers from your own documents or prices usually needs a cloud model.
Your app. In the stores. Done for you. US APP Team designs, builds, and launches your iOS and Android app for a fixed price. Start your app brief or book a free intro call — both are free.
Want to poke around first? Every tool in our free tool library runs in your browser with no signup: an app cost calculator, an app name generator, and a privacy policy generator.
