Foundation Models from Python: the fm CLI

For a year, Apple’s on-device large language model lived behind a wall: you reached it only from Swift, only inside an app you built in Xcode¹. macOS 27 tears that wall down. Apple now ships a command line tool called fm pre-installed with the OS, and a Foundation Models SDK for Python you install with pip¹. The model that used to require a project, a build, and a LanguageModelSession in compiled Swift now answers a one-line shell command and runs inside a Jupyter notebook. Eric Gourlaouen, an engineer on the Foundation Models Framework team, framed the shift plainly in WWDC26 session 334: “until now, those models were only available from Swift code”¹. The change is not a new model. The change is that the same on-device model is suddenly scriptable, automatable, and evaluable from outside an app, with no API key and no cloud cost¹.

TL;DR

• macOS 27 ships fm, a pre-installed command line tool for the on-device Apple Foundation Model. Its subcommands include respond (one-shot prompt to stdout), chat (interactive session), and schema (define structured output)¹.
• fm respond takes options for the model (switch to Private Cloud Compute), an image input, and a schema for structured output; --help lists the rest¹.
• The Python SDK reaches the same on-device model from Python, requiring Python 3.10 or later, Xcode installed, and an Apple Silicon Mac, installed through pip or another package manager¹.
• The SDK mirrors the Swift framework: a LanguageModelSession you call respond on, tool calling, and guided generation via the fm.generable decorator passed to fm.respond as the generating argument¹.
• Both surfaces default to the always-available on-device model and can opt into the larger Private Cloud Compute model, which is more capable but carries usage limits¹.
• The payoff is prototyping and automation: shell scripts that sort files by meaning, and Python evaluation pipelines that grade prompt variants with Pandas and matplotlib¹.

The fm command line tool

Open Terminal on macOS 27 and type fm, and the tool prints the commands it supports¹. Apple highlights three. fm respond prompts the model and returns a response. fm chat starts an interactive conversation. fm schema creates a schema for structured output¹. The simplest possible use is the one Eric demonstrated first: type fm respond, type a prompt, press enter, and read the model’s answer in the terminal a moment later¹.

The two top-level commands split along a clean line: exploration versus scripting. fm chat is for getting a first pulse of the model. You ask a question, ask a follow-up, and the conversation holds, with its own slash commands: /model switches the conversation to the Private Cloud Compute model, and /save saves the conversation to resume later¹. When you would rather have an inline response you can capture, like in a script, you reach for fm respond instead, which writes the model’s output to stdout¹.

fm respond is where the options live. Eric named three explicitly. A model option prompts the Private Cloud Compute model rather than the on-device default. An image option includes an image in the prompt. And a schema option pairs with fm schema object to constrain the output to a structure you defined¹. He noted there are more, and pointed at the help option to list them all¹. The transcript names the options by their role rather than their exact flag spelling for each one (the one literal form shown on screen is fm schema object), so where I describe an option below I am describing the documented behavior, not inventing a flag string.

The model choice is the decision that matters most. By default fm uses the on-device model that comes with macOS, which is always available¹. You can switch to the Apple Foundation Model on Private Cloud Compute, which Eric described as “a much bigger model than the on-device model, so it will perform better on complex problems,” with the tradeoff that it carries usage limits¹. The default is the right starting point: it is free, local, and has no cap. You opt up to Private Cloud Compute when a task is genuinely hard enough to need it.

Building an automation script

The file-sorting demo is the clearest argument for why a CLI matters. Eric had a project folder full of draft and final asset versions and wanted a repeatable script that keeps the finals, backs them up, and moves the drafts to an archive disk¹. The hard part is not moving files. The hard part is deciding which file is a draft when the names are messy. As he put it, calling into a language model from the script lets it “sort draft versus final files” even when “the names are messy and are difficult to sort predictably”¹.

The shape generalizes to any “judgment over a list” automation. The script loads the files in the working directory, then prompts the model through fm respond to sort that list into two groups: final files and draft files¹. To make the output usable, it defines a schema up front with fm schema object describing two fields, a list of finals and a list of drafts, and passes that schema to fm respond through the schema option¹. The model returns its answer as JSON, and the script reads it to copy the finals to a backup and move the drafts to the archive¹.

The structured-output step is the load-bearing one. A free-text answer would force the script to parse prose, the brittle part of every shell pipeline that talks to an LLM. By declaring the schema with fm schema object and getting JSON back, the script gets a contract it can act on directly¹. The pattern is the same one Swift developers know as guided generation, exposed here as a CLI option¹. Any task that ends in “do something deterministic with the model’s decision” wants exactly this shape: prompt, schema, JSON, act.

The Python SDK

The second surface is for a different person at a different moment. As Eric put it, “if you’re a machine learning engineer, you might use more Python than Swift,” and the SDK “makes it easy to use the on-device model in your Python code”¹. The pitch rests on Python’s ecosystem: “Python has a rich ecosystem of open-source packages for machine learning and data science,” which means you can write evaluation pipelines and “leverage those packages to quantify the quality of your feature”¹.

Installation has four requirements, all stated in the session. You need Python 3.10 or later, Xcode installed, and an Apple Silicon Mac, and you install the SDK through pip or any other package manager of your choice¹. The Apple Silicon and Xcode requirements are the tell that the package is a binding to the same on-device model the OS runs, not a hosted API.

The API will feel familiar to anyone who used the Swift framework², by design: “the APIs and abstractions will quickly feel familiar”¹. You prompt by creating a LanguageModelSession, optionally passing instructions, then calling session.respond with your prompt; the result contains the model’s output¹. The SDK carries the framework’s core features across: text and image inputs, streaming responses, tool calling so the model can interact with your code, and guided generation for structured output¹.

Two of those features got concrete treatment. For tool calling, Eric defined a tool the model can call to fetch a user’s last few orders, “so that it can provide more personalized information,” the same pattern as the Swift framework’s Tool protocol¹. For guided generation, he used the fm.generable decorator to define the desired output structure, an ItemsSuggestion object, and passed it to fm.respond as the generating argument¹. The decorator is the Python analog of Swift’s @Generable macro, and the generating argument is how you hand the model the shape you want back. Because the transcript shows these by role and object name rather than printing the full class body, treat ItemsSuggestion as the example’s name for a structure you would define yourself.

Evaluation pipelines: the real reason to use Python

The case study is where the Python SDK stops being a convenience and becomes a method. Eric was building a feature to predict what a user wants to add to their grocery cart, and he had three different prompt implementations: a minimal one, a more descriptive one, and a detailed one that spelled out a full list of rules¹. The question every prompt engineer faces is which one is actually best, and the honest answer requires measurement, not taste.

Apple is explicit that Swift developers have their own answer here. The Evaluations framework ships with Xcode 27 and makes it easy to create evaluations and track feature accuracy across iterations¹. The Python SDK is the parallel path for data scientists who live in notebooks. Eric ran the whole analysis from Jupyter¹.

The pipeline reads like a standard ML evaluation loop pointed at the on-device model. First, he used a large server model to generate evaluation data, giving him inputs and an expected output for each¹. Then, for every input, he generated outputs from each of the three prompt implementations and stored the inputs and outputs as rows in a Pandas DataFrame¹. Next, judge functions backed by a server model scored each output against criteria he chose, and those metrics went back into the DataFrame¹. Finally, matplotlib turned the grades into charts¹.

The charts told a story no amount of staring at prompts would have: the detailed prompt hit a high rate of generation errors, which Eric attributed to reaching the model’s maximum context window size; the two less-detailed prompts added excess items to the cart while the detailed one added fewer; the detailed prompt missed more expected items; and the minimal prompt hallucinated the most items¹. Every prompt had a different failure mode, and only the measurement surfaced them. That is the argument for the whole approach. “With Python, I can make those iterations quickly right from my notebook without having to rebuild the whole project,” Eric said¹.

When to reach for each

A few rules follow from the contracts above.

Reach for fm respond when a shell script needs a judgment call. Sorting messy filenames, classifying a line of input, extracting a field from unstructured text. Pair it with fm schema object and the schema option so the script acts on JSON instead of parsing prose¹.

Reach for fm chat when you are exploring, not scripting. It is the fastest way to get a first pulse of how the model handles your prompts, with /model to escalate to Private Cloud Compute and /save to keep a session¹.

Reach for the Python SDK when you want to measure, beyond simply calling. The moment you have more than one prompt and need to know which is better, the notebook-plus-Pandas-plus-matplotlib loop is the tool, because the on-device model is free and local enough to run an entire evaluation set without a bill¹.

Default to the on-device model; opt up to Private Cloud Compute deliberately. The on-device model is always available and has no usage limit. Private Cloud Compute is bigger and better on complex problems but carries usage limits, so save it for the tasks that earn it¹.

Prototype here, ship in Swift. Eric’s own framing is that you can use these tools “alongside your Xcode project, as a way to prototype and evaluate prompts,” or “on their own, to use the model in novel ways”¹. The grocery-app example prototypes prompts in Python “before implementing them in Swift”¹. The CLI and SDK shorten the loop between idea and evidence; the app is still where the feature lands.

FAQ

What is the fm command line tool?

fm is a command line tool that comes pre-installed with macOS 27 and reaches the on-device Apple Foundation Model from the Terminal app¹. Its subcommands include respond to prompt the model and print a response, chat to start an interactive conversation, and schema to define structured output. You run it without an API key and without cloud cost, because the default model runs on-device¹.

How do I get structured JSON out of fm?

Define a schema with fm schema object, then pass that schema to fm respond through its schema option. The model returns its answer as JSON matching the schema, which a script can act on directly instead of parsing free text¹. The mechanism is the CLI version of the framework’s guided generation¹.

What does the Foundation Models Python SDK require?

Python 3.10 or later, Xcode installed, and an Apple Silicon Mac¹. You install it through pip or another package manager of your choice. The Apple Silicon and Xcode requirements reflect that the SDK binds to the same on-device model the OS runs, rather than calling a hosted API¹.

How is the Python SDK different from the Swift framework?

It is the same model and a deliberately familiar API in a different language. You create a LanguageModelSession, call respond, expose tools, and use guided generation through the fm.generable decorator passed to fm.respond as the generating argument¹. The reason to choose Python is the ecosystem: Pandas, matplotlib, Jupyter, and the rest of the data-science stack for evaluation pipelines that Swift does not reach as directly¹.

When should I use Private Cloud Compute instead of the on-device model?

Both fm and the SDK default to the on-device model, which is always available and has no usage limit¹. Switch to Private Cloud Compute, via fm respond’s model option or fm chat’s /model command, when a problem is complex enough to need the bigger model, accepting that it carries usage limits¹.

The full Apple Ecosystem cluster: the Foundation Models framework explainer for the Swift foundation these tools mirror; the iOS 27 tool-calling controls for how the model uses tools; the Foundation Models agentic workflow for the on-device-versus-larger-model choice; and Xcode 27’s coding agents for the in-IDE side of an agent-heavy workflow. The hub is at the Apple Ecosystem Series. For the broader picture of building iOS with agents, see the iOS Agent Development guide.