Core AI: Running Models on Apple Silicon

Apple’s on-device AI stack has had a missing rung. Foundation Models gives you the system LLM, sealed and free. Core ML runs a fixed converted model with the converter making the hardware decisions for you. MLX ships an array framework you embed and a model you select. iOS 27 adds the rung below all three: Core AI, a framework whose one-line abstract is “Run AI models in your app on Apple silicon.”¹ It is the model-execution surface, the place you reach when you want to drive specialization, caching, and inference scheduling yourself instead of accepting a higher layer’s defaults.

In session 324, Apple positions Core AI as the same inference framework that powers on-device Apple Intelligence, now opened up for your own app’s intelligence.¹⁵

Core AI is the inference framework behind on-device Apple Intelligence, now available to your app.

The framing matters because Core AI sits under the abstractions most apps should use. Apple describes it as designed with Apple silicon in mind, letting your app use the latest model architectures and inference techniques across the CPU, GPU, and Neural Engine, with a Swift API that makes common tasks simple while giving you more control over model specialization, caching, and inference performance when needed.¹ The thesis of this post: reach for Core AI when you have a model you want to run with explicit control over where and how it executes, and stay at Core ML or Foundation Models when you do not. The framework rewards a specific need, not a default preference.

TL;DR / Key Takeaways

• Core AI separates an unspecialized AIModelAsset (inspect a model’s structure and metadata cheaply) from a specialized AIModel (run inference on a device), with AIModelCache holding the device-specific artifacts and AssetError for asset operation failures.²³⁶⁴
• Inference data flows through NDArray, a multidimensional array of scalar values, described by an NDArrayDescriptor that fixes shape, scalar type, and memory layout expectations.⁵⁷
• You target hardware with ComputeUnitKind (CPU, GPU, or Neural Engine) through SpecializationOptions, and schedule asynchronous work onto a ComputeStream.⁸⁹¹⁰
• An InferenceFunction owns the weights and buffers and runs inference; an InferenceFunctionDescriptor lets you inspect its input, output, and state signature first. The function is Sendable, so you can run it concurrently.¹⁴¹³
• Models load from an .aimodel bundle on disk, and Core AI ships preparation, conversion, and debugging tools alongside the framework. Reach for Core AI when you need explicit control over specialization and scheduling; otherwise stay at Core ML or Foundation Models.²¹

Two Words That Run the Whole Design: Asset and Model

The first thing Core AI asks you to internalize is that a model on disk and a model that runs inference are different objects, and specializing one into the other is expensive. The framework gives each a type.

An AIModelAsset is “an unspecialized source model asset.”² You create it from the URL of an .aimodel bundle on disk, and you use it to inspect a model without paying the specialization cost. Apple is explicit about why the split exists: a model asset lets you query model information without performing specialization, which is an expensive operation. From an asset you can read function signatures, input and output descriptions, compute and storage types, and author-provided metadata. What you cannot do is run inference; an asset is for inspection only.²

// Call shape is illustrative; confirm the exact initializer against Apple's docs.
let asset = try AIModelAsset(url: bundleURL)   // an .aimodel bundle on disk
// Inspect signatures, input/output descriptions, compute and storage types,
// and author-provided metadata — without specializing.

The AIModel is the other half: “a specialized model for running inference on a device.”³ An AIModel represents a specialized .aimodel asset optimized for the current device’s hardware, and you create one by loading the asset from disk.³ The asset answers what is this model?; the model answers run it here, now. The cost asymmetry between the two is the reason the API forces you to name which one you want. Inspecting a hundred candidate models to pick one is cheap if you only build assets; it would be ruinous if every inspection specialized.

Specialization produces device-specific artifacts, and those artifacts have a home: AIModelCache, “a cache that stores the specialized model artifacts for inference.”⁶ The cache holds the optimized, device-specific artifacts that a model loads to execute its inference functions, and Apple notes that each cache entry contains a specialized asset formed from a specific .aimodel or .aimodelc and a specialization combination.⁶ The practical reading: specialization is not something you want to repeat on every launch. The cache is how Core AI lets the expensive step happen once and the cheap step (loading cached artifacts) happen thereafter.

When asset operations go wrong (a missing bundle, a malformed .aimodel, an unreadable file), Core AI surfaces an AssetError, “an error that occurs during model asset operations.”⁴ Treat it the way you treat any I/O boundary: the asset lives on disk, disk operations fail, and the type system tells you exactly where to put the catch.

Tensors: NDArray and Its Descriptor

Inference moves numbers in and numbers out, and Core AI’s container for those numbers is NDArray, “a multidimensional array of scalar values used for model inference.”⁵ If you have worked with NumPy ndarray, MLX arrays, or MLMultiArray, the shape of the idea is familiar: an n-dimensional block of scalars with a defined layout. An NDArray stores its data in a layout defined by its shape and the rest of its descriptive properties.⁵

The companion type is NDArrayDescriptor, “a description of an array’s shape, scalar type, and memory layout expectations.”⁷ A descriptor is the contract. Apple’s framing is direct: the descriptor contains the expectations for an array value you provide to an inference function, and most expectations are strict. If the descriptor specifies a scalar type of .float32, the array you provide must use .float32.⁷ You do not guess the shape and type a function wants; you ask the function’s descriptor and conform to it.

// Call shape is illustrative; confirm exact property/method names against Apple's docs.
let inputDescriptor = function.descriptor.inputs.first!   // an NDArrayDescriptor
// The descriptor fixes shape, scalar type, and layout; the array you build
// must satisfy those expectations (e.g. .float32 means .float32).

The design lesson here mirrors the asset/model split. Core AI consistently puts a cheap description object in front of an expensive value object. You read the descriptor to learn the contract, then allocate the NDArray that satisfies it, rather than allocating first and discovering a mismatch at inference time. For image inputs specifically, Core AI also defines an ImageDescriptor, “a description of an image’s dimensions and pixel format,” so a vision model’s pixel input gets the same descriptor-first treatment.¹¹

Choosing Where Inference Runs

Apple silicon has three places to compute: CPU, GPU, and Neural Engine. The reason Core AI exists rather than only Core ML is that Core AI lets you say which of those the framework targets, instead of inferring it.

ComputeUnitKind is “a type of hardware compute unit available for model inference.”⁸ You use compute unit kinds with the specialization options to control which hardware the framework targets when specializing a model, and by default specialization uses all available compute units on the device.⁸ The default is the right answer for most work, which is the point: you only override it when you have a reason (a latency-sensitive path you want pinned to the Neural Engine, a debugging pass you want forced onto the CPU, a GPU-heavy pipeline you are coordinating with other GPU work).

You pass that intent through SpecializationOptions, the structure that carries the choices made at specialization time.⁹ Specialization is the expensive step from earlier, and SpecializationOptions is where the compute-unit targeting and other specialization decisions live. Because a cache entry keys off a specific asset and specialization combination, changing your options changes which cached artifact you get back, which closes the loop between targeting and caching.⁶

Scheduling is the other axis of “how it runs,” and Core AI models it as a ComputeStream, “a stream of work to be run asynchronously.”¹⁰ A compute stream is what you provide to encode work onto the stream, and Apple notes that multiple inferences encoded to the same stream are serialized as needed based on the values read and written.¹⁰ Two implications follow. First, a stream is your ordering primitive: encode dependent inferences to one stream and Core AI sequences them by data dependency. Second, work is asynchronous by default, so the stream is also how you keep the calling thread free while the Neural Engine or GPU does the work.

Inference Functions: The Thing That Actually Runs

A loaded .aimodel is not a single callable. Models expose named functions (an encoder, a decoder, a vision tower, a prefill versus a decode step), and Core AI’s unit of execution is the InferenceFunction: “a function that performs inference on input values and produces output values.”¹⁴

Before you call one, you inspect it. InferenceFunctionDescriptor is “a description of an inference function’s signature,” and you use a descriptor to inspect the names and types of a function’s inputs, outputs, and states before running inference.¹³ States are the detail worth pausing on: a function with state is how a stateful model (a KV cache in a transformer decode loop, for instance) keeps information between calls, and the descriptor tells you a function has them before you try to drive it.

The InferenceFunction itself owns the resources inference needs, including model weights and intermediate buffers. You load a function from a model and call run(inputs:states:outputViews:) to perform inference.¹⁴ The signature of run is named in Apple’s own discussion, so the three things a call needs are explicit: the input values, the state values, and the output views you want written.

// run(inputs:states:outputViews:) is named in Apple's docs; surrounding
// loading/value-construction shapes are illustrative — confirm against Apple's docs.
let function: InferenceFunction = /* load from an AIModel */
let outputs = try function.run(
    inputs: inputValues,        // InferenceValue per input
    states: stateValues,        // any stateful values the function declares
    outputViews: outputViews
)

Two properties make the function pleasant under load. It is Sendable, so you can run it concurrently from multiple tasks, and Apple notes it automatically allocates additional intermediate buffers as needed to support that concurrency.¹⁴ You do not serialize calls behind a lock to protect shared scratch space; the function manages its own buffers per concurrent caller. That is a meaningful difference from APIs where a single inference handle is effectively single-threaded.

The values flowing through run are InferenceValue instances, “a value that an inference function accepts as input or produces as output.”¹² An InferenceValue wraps either an NDArray or a pixel buffer, and you retrieve a result after inference using its value property.¹² The wrapper is what lets one run signature carry both tensor inputs and image inputs without separate overloads: a text model passes NDArray-backed values, a vision model passes pixel-buffer-backed ones, and the function reads the descriptor to know which it expects.

When to Reach for Core AI

The hardest part of Core AI is not the API. It is knowing that you should be here at all rather than one layer up. The honest decision tree:

• Foundation Models when Apple’s system model does the task. Summarize, classify, extract, rewrite, structured output: these belong to Foundation Models, which costs you no weights, no memory budget, and no specialization step. If your feature fits, stop there. Dropping to Core AI to re-implement what the system model already does is wasted work.
• Core ML when you have a fixed, converted model and want the converter to make the hardware and optimization decisions for you. Core ML targets the Neural Engine with tight power and latency for a locked-down production model, and it asks nothing of you about specialization or scheduling. If you do not want to think about compute-unit targeting or compute streams, that is the signal to stay at Core ML.
• MLX when you want a research-grade array framework you embed and iterate on: your own training loop, quantized open-weight models, LoRA fine-tunes, fast experimentation. MLX is a library you ship with weights, not a system model-execution surface. It wins on flexibility and iteration speed.
• Core AI when you have a model to run and you want the framework’s explicit handles: an AIModelAsset you inspect before committing, SpecializationOptions that pin compute units, an AIModelCache you manage, a ComputeStream you schedule onto, and InferenceFunctions you call concurrently. You reach here when the higher layers’ defaults are the thing in your way, and you can name which default you need to override.

The throughline across the whole stack: each layer down trades a default away for a handle. Foundation Models hands you everything and asks nothing. Core AI hands you the levers and asks you to know which to pull. If you cannot name the specialization, caching, or scheduling control you need, you do not need Core AI yet.

How a Model Gets to Core AI

The framework is the runtime half of a larger workflow. Apple notes that Core AI includes additional tools for model preparation, integration, and debugging alongside the framework: you prepare your models for Apple silicon, convert them into the .aimodel format, and use a companion app that supports visualization and numeric debugging.¹ The fact-sheet’s description of those tool names is truncated, so confirm the exact tool names and their invocation against Apple’s Core AI documentation rather than trusting any reconstruction.¹ What is verified is the pipeline shape: a source model is prepared, converted to .aimodel, loaded as an AIModelAsset for inspection, specialized into an AIModel, and run through its InferenceFunctions, with AIModelCache keeping the specialized artifacts so the expensive step happens once.¹²³⁶¹⁴

FAQ

What is Apple’s Core AI framework?

Core AI is iOS 27’s low-level framework for running AI models on Apple silicon, summarized by Apple as “Run AI models in your app on Apple silicon.”¹ It runs model inference across the CPU, GPU, and Neural Engine through a Swift API that makes common tasks simple while giving you control over model specialization, caching, and inference performance when you need it.¹ It sits below Foundation Models and Core ML as a model-execution surface.

What is the difference between AIModelAsset and AIModel?

AIModelAsset is an unspecialized source asset you create from an .aimodel bundle’s URL on disk; you use it to inspect a model’s function signatures, input and output descriptions, compute and storage types, and metadata without specializing, because specialization is expensive, and an asset cannot run inference.² AIModel is the specialized model optimized for the current device’s hardware that does run inference; you create one by loading the asset from disk.³ The split lets you inspect cheaply and specialize only when you commit.

How does Core AI choose between the CPU, GPU, and Neural Engine?

You control hardware targeting with ComputeUnitKind through SpecializationOptions. A compute unit kind names a type of hardware compute unit available for inference, and you use it to control which hardware the framework targets when specializing a model; by default specialization uses all available compute units on the device.⁸⁹ You override the default only when you have a specific reason, such as pinning a latency-sensitive path to one compute unit.

What is an InferenceFunction and how do I run it?

An InferenceFunction performs inference on input values and produces output values, owning the model weights and intermediate buffers.¹⁴ You inspect its signature first through an InferenceFunctionDescriptor, which describes the names and types of the function’s inputs, outputs, and states, then load the function from an AIModel and call run(inputs:states:outputViews:).¹³¹⁴ The function is Sendable and allocates intermediate buffers automatically to support concurrency, so multiple tasks can run it at once.¹⁴

Should I use Core AI instead of Core ML or Foundation Models?

Use Foundation Models when the system model does the task, and Core ML when you have a fixed converted model and want the converter to make hardware and optimization decisions for you. Reach for Core AI when you want explicit control over specialization (SpecializationOptions, ComputeUnitKind), caching (AIModelCache), and scheduling (ComputeStream) that the higher layers handle on your behalf.⁸⁹⁶¹⁰ If you cannot name the control you need, stay one layer up.

The full Apple Ecosystem cluster: MLX on Apple Silicon for the array framework you embed when you want your own model and training loop; Apple Silicon’s TBDR and unified memory for the hardware substrate that makes CPU/GPU/Neural Engine sharing work; Core ML on-device inference for the fixed-model layer above Core AI; and Foundation Models for Apple’s sealed system LLM at the top of the stack. The hub is at the Apple Ecosystem Series. For broader iOS-with-AI-agents context, see the iOS Agent Development guide.

References