Core ML vs MLX vs Foundation Models: Choosing Apple's On-Device AI Stack

Apple now ships four distinct ways to run a model on your users’ hardware, and picking the wrong one costs a rewrite. On the M2 Max I am writing this on, MLX sustains 11.2 TFLOPS of fp16 matrix math and moves data at 351 GB/s of measured unified-memory bandwidth¹. For most apps that power is the wrong choice, because the built-in Foundation Models framework does the job in six lines of Swift with zero megabytes added to the download. The four frameworks are not competitors. They are four different answers to one question: whose model is it, and who maintains the execution?

TL;DR

• Foundation Models is the default. If Apple’s on-device ~3B-parameter LLM can do the task (summarize, extract, classify, generate structured output, call your tools), you ship zero model weights, pay zero inference cost, and inherit every OS-level model upgrade for free.²
• Core ML is for shipping your trained model inside an app. It is the only path with first-class Neural Engine dispatch, App Store-friendly packaging, and a compile-ahead contract that trades flexibility for battery life and predictability.³
• MLX is for when the model is the product: research, fine-tuning, running open-weights LLMs, or any workload where you need eager control over tensors. It is a developer-machine and Mac-app tool, not an iOS shipping vehicle.⁴
• Core AI (iOS 27) is the new floor under all of it: a low-level model-execution framework with NDArray tensors and explicit compute-unit targeting, for teams that today abuse Core ML as a tensor runtime.
• The decision usually takes one question: built-in model → Foundation Models; your model on iOS → Core ML; your model on your Mac → MLX; tensor-level control on iOS → Core AI. The rest of this post is the 20% of cases where that heuristic breaks.

The Question Behind the Question

Every “which framework” debate I have watched collapses into two axes. (I have written a deep dive on each of these: Foundation Models, Core ML, MLX, Core AI.)

Axis one: whose weights? Apple’s built-in model, a model you trained or converted, or open weights you pulled from a hub. Foundation Models only ever runs Apple’s model (plus your LoRA-style adapters).⁵ Core ML and MLX only ever run weights you provide. That single fact eliminates half the option space before performance enters the conversation.

Axis two: who owns execution? Foundation Models hides everything: no tensors, no compute units, no context-window management beyond a session object. Core ML hides most things: you hand it a compiled .mlmodelc and hint at compute units; the framework decides dispatch. MLX hides nothing: you write the array math, and lazy evaluation plus unified memory are your tools. Core AI sits between Core ML and MLX: explicit tensors and compute-unit targeting, but Apple-managed execution.

Plot any real feature on those two axes and the framework picks itself.

Foundation Models: The Zero-Megabyte Default

The framework most teams should try first is the one that requires no model at all. Foundation Models exposes the on-device LLM behind Apple Intelligence through LanguageModelSession, and its killer feature is not the model. It is guided generation: annotate a Swift struct with @Generable and the framework constrains decoding so the model’s output is your type, not a string you parse and pray over.²

@Generable
struct Itinerary {
    let destination: String
    let days: Int
    @Guide(description: "Budget in USD") let budget: Double
}

let session = LanguageModelSession()
let itinerary = try await session.respond(
    to: "Weekend in Kyoto under $800",
    generating: Itinerary.self
).content

That is the entire integration. No download step, no quantization decisions, no thermals to reason about; the OS schedules the model, shares it across every app on the device, and upgrades it with each release. Tool calling lets the model invoke functions you define, which is enough to build genuinely agentic features without any server.²

The constraint is symmetrical: the model is a ~3B-parameter generalist. It summarizes, extracts, classifies, and roleplays structured output well; it does not do open-domain world knowledge, long-context document analysis, or anything a frontier model does. When the built-in model is not enough, Apple’s answer is adapters (LoRA-style fine-tunes you train with Apple’s toolkit against the base model), but an adapter is a commitment: you retrain it every time Apple revs the base model.⁵ My full teardown, including session management and the failure modes reviewers hit, is in the Foundation Models deep dive.

Choose it when: the task is language-shaped, the built-in model clears your quality bar, and shipping zero weights matters. That is more features than most teams assume; try it before you reach for anything below.

Core ML: Your Model, Apple’s Execution

Core ML answers a different question. You already have a model (a vision classifier, an embedding model, a speech pipeline, a distilled LLM), and it needs to run inside an iOS app, on battery, within App Store size limits, this year.

The contract is compile-ahead. You convert weights with coremltools, the model compiles to .mlmodelc, and at runtime the framework dispatches across Neural Engine, GPU, and CPU; the ANE is the part nothing else on this list gives you on iOS today. The conversion step is where the real engineering lives: operator coverage, quantization (palettization, linear 8/4-bit, sparsity; the coremltools optimization stack is genuinely deep⁶), and the compute-unit hinting that decides whether your model runs at 2 watts or 8. The patterns that actually survive production (dispatch hints, latency budgets, the quantization ladder) are the subject of the Core ML pillar.

What Core ML does not do is also load-bearing: it does not train, it does not fine-tune, and it does not give you tensors mid-graph. It is an inference appliance. Teams that need “run my graph AND poke at intermediate results” have historically contorted Core ML into a tensor runtime; that contortion is exactly what Core AI now exists to absorb.

Choose it when: the weights are yours, the target is iOS/iPadOS (or cross-platform Apple), and battery/size/ANE matter. Which is to say: almost every shipped custom-model feature on the platform.

MLX: The Research Machine in Your Mac

MLX is the odd one out: an open-source array framework from Apple’s ML research group, not an OS framework.⁴ It looks like NumPy, evaluates lazily, and treats Apple silicon’s unified memory as the feature it is: no host-to-device copies, because there is no separate device memory.

The numbers from this machine make the case concretely. An M2 Max sustains 11.2 TFLOPS of fp16 matmul at 4096×4096 and 351 GB/s of measured streaming bandwidth under MLX with no tuning¹: enough to run quantized 30B-parameter open-weights models interactively, fine-tune 7B models with LoRA overnight, and iterate on training runs that would otherwise mean renting cloud GPUs. The mlx-lm ecosystem means most open models are a pip install and a download away.

The boundaries are equally concrete. No Neural Engine: MLX programs the GPU (and CPU), not the ANE. No iOS shipping story: you can technically embed it, but weight distribution, thermals, and review guidelines make it the wrong tool; MLX is for your Mac, your users’ Macs, and your research loop. And nobody manages execution for you: memory pressure, quantization choices, and eval scheduling are yours. The MLX pillar covers when “your own model, not Apple’s” is worth that ownership.

Choose it when: you are training or fine-tuning, running open-weights LLMs locally, or building Mac-native tools where model control is the product. If your daily driver is an Apple silicon Mac with real RAM, MLX turns it into a legitimate ML workstation.

Core AI: The New Floor

iOS 27 added the layer Apple had been missing: Core AI, a low-level model-execution framework with NDArray tensors, an asset-versus-model split, explicit compute-unit targeting, and callable inference functions.⁷ It is what you use when Core ML’s “hand me a compiled model, receive predictions” contract is too rigid: pipelines that interleave tensor math with model calls, custom pre/post-processing that belongs on the ANE, multi-model graphs with shared buffers.

Two clarifications save most readers a wrong turn. Core AI is not a Foundation Models alternative (there is still no built-in LLM at this layer; you bring compiled assets). And it is not MLX-on-iOS: no training, no autograd, no eager research loop. It is Core ML’s engine room with the hatch opened. If you have never fought Core ML’s abstraction, you do not need Core AI; the moment you have shipped MLMultiArray gymnastics or chained models through unnecessary conversions, you already do. The tensor model and targeting API are covered in the Core AI pillar.

The Cases That Actually Get Argued

“We need an LLM, but ours/bigger.” Foundation Models cannot load external weights, full stop. A distilled small LLM through Core ML works and gets you the ANE; a quantized open model through MLX works on Mac. Be honest about the alternative nobody on-device wants to say aloud: if the task needs frontier quality, the answer is a server call, and the on-device layers handle fallback and privacy-sensitive subsets.

“We want one stack for iPhone and Mac.” Foundation Models and Core ML are both cross-platform within Apple’s ecosystem; MLX is Mac-only in practice. Teams shipping a Mac pro app plus an iOS companion commonly land on MLX for the Mac (big models, user-provided weights) and Core ML or Foundation Models on iOS: same feature, two engines, one honest admission that the devices are different.

“We already have a Core ML pipeline; is Core AI a migration target?” Only where the pipeline hurts. Core AI absorbs the tensor-shaped edges; the compiled-model core that works today keeps working. Migrating a healthy Core ML deployment wholesale buys nothing.

“Does SwiftData/App Intents change any of this?” Adjacent, not overlapping: your model layer feeds features that App Intents expose to the system, and whatever you persist obeys SwiftData’s schema rules. The frameworks here decide how inference runs, not how the app around it is built.

A Decision Tree

Is the task language-shaped and within a ~3B generalist's reach?
├── YES → Foundation Models. Ship zero weights. Done.
└── NO → Whose weights?
    ├── Apple's, but specialized → Foundation Models + trained adapter
    │     (accept: retrain per base-model rev)
    └── Yours →  Where does it run?
        ├── iOS/iPadOS app → Core ML
        │     └── unless you need tensor-level control → Core AI (iOS 27+)
        ├── Mac, models-as-product / training / open weights → MLX
        └── Frontier-quality required → server, with on-device fallback

Key Takeaways

For app teams shipping features: - Prototype against Foundation Models first; six lines of Swift is the cheapest possible experiment, and “the built-in model is enough” is the most common surprise. - Budget Core ML conversion time separately from inference time; operator coverage and quantization are where the weeks go.

For ML engineers: - An Apple silicon Mac with 64GB+ is a real fine-tuning rig under MLX; measure before renting cloud GPUs. The 11.2 TFLOPS / 351 GB/s numbers above came from a laptop chip two generations old.¹ - The ANE remains Core ML’s (and now Core AI’s) moat. If battery draw per inference matters, no GPU path competes.

For architects: - Treat the four frameworks as layers, not rivals: Foundation Models for language features, Core ML for shipped custom models, Core AI for the tensor edges, MLX for the Mac and the lab. Mixed deployments are the norm in serious apps, not a smell. - Model ownership is the long-term cost center: Apple maintains Foundation Models’ weights, you maintain everything else’s. Price that into the choice.