RealityKit And Reality Composer Pro 3 In 2026

Reality Composer Pro 3 left the Xcode bundle and shipped as its own app you launch from the Applications folder. The detail reads like a packaging note, but it is the clearest signal in the whole WWDC26 spatial story: the authoring tool is now a destination, not a sidecar, and the engine underneath it grew to match.¹

The 2026 release moves both halves of the pipeline at once. RealityKit gains the rendering and simulation features that make a scene look real: soft shadows, baked lightmaps, projective textures that spill onto your actual walls, cloth, navigation meshes, gaussian splats, and ray-traced acoustics. Reality Composer Pro 3 gains the workflow that wires those features together without a build cycle: live preview on a headset, prototypes and instances, node-based graphs, and an AI assistant that generates 3D content. A third thread connects them, code: Xcode plug-ins run your custom components inside the authoring tool, and a new structured-model toolkit powers collaborative design-review apps. The pieces stopped being separate demos and became a coherent way to work.

The post walks the five WWDC26 sessions that define the spatial authoring pipeline this year, each with the clip that grounds it and the insight that changes how you build.

TL;DR / Key Takeaways

• RealityKit’s lighting and simulation matured: soft shadows driven by lightSize, baked lightmaps, projective textures plus physical space lighting onto your real room, cloth, navigation meshes, gaussian splats, and ray-traced reverb.²
• Reality Composer Pro 3 ships as a standalone app (no longer an Xcode developer tool) built for fast iteration: a dockable simulation tab, prototypes and instances with overrides, on-device live preview, baked lightmaps, and a generative AI assistant.³
• You extend the editor with Xcode plug-ins: components and systems run inside it through the RealityComposerProPlugin protocol, custom animation actions slot into the sequencer via EntityAction, and the @Scriptable macro exposes components to Script Graph.⁴
• Visual node-based tools (Animation Graph, Behavior Tree, Script Graph, Compute Graph, Shader Graph) let designers author character behavior, GPU particles, and materials without a build cycle.⁵
• visionOS 27 adds a structured-3D-model toolkit for collaborative design review: hierarchy-driven ManipulationComponent placement, a new ClippingComponent for cross-sections, and volume-weighted variance to auto-expand assemblies.⁶

RealityKit Got The Features That Sell Realism

RealityKit has been Apple’s cross-platform 3D engine since 2019, the layer you build once and deploy to visionOS, iOS, iPadOS, macOS, and tvOS.² The 2026 additions close the gap between “renders correctly” and “looks real,” demonstrated inside one game, Chaparral Village, which gives the session a through-line instead of a feature checklist.

Start with shadows, because the API choice tells you how the team thinks. RealityKit shadows default to hard edges, correct only when the light source is infinitesimally small. A real light has area, and area produces a penumbra, the soft falloff where only part of the light is blocked. RealityKit models that with one intuition-shaped knob: a shadow’s lightSize, the diameter of the light in meters. Set it to 0.7 and the edges soften; the larger the light, the wider the penumbra. A companion quality setting trades samples for performance (low forces a hard shadow; medium suffices at typical viewing distance).² The controls map to physical quantities, the pattern worth internalizing.

Two features push light past the virtual boundary. Projective textures emulate shining a flashlight through a piece of film, casting a pattern onto whatever the spotlight points at, animated caustics or a planetarium’s stars and nebulae. Physical space lighting then lets those virtual lights interact with the real room through RealityKit’s scene understanding mesh, so the projected stars conform to your actual walls. The activation is almost anticlimactic: you add a SurroundingsLight component to the spotlight (currently spotlights and point lights).² Turning a virtual light into a real-room light is a component you attach, not a separate rendering path.

The rest of the session tours simulation and capture. Cloth simulation describes fabric as a mesh of particle-vertices connected by spring-edges; pinning a curtain marks specific vertices kinematic. Navigation meshes define traversable regions with per-area traversal costs and off-mesh connections, fed through a NavigationComponent and queried by a NavigationController. Gaussian splats render real-world captures as ellipsoids defined by position, scale, rotation, opacity, and spherical harmonics, assembled into a GaussianSplatComponent. Immersive audio gains a custom reverb mesh built on ray-traced geometrical acoustics, where a ReverbMeshResource defines room geometry and per-material acoustics.² The throughline: each capability is a component you compose onto an entity, exactly the entity-component-system shape RealityKit has always had.

Reality Composer Pro 3 Closes The Iteration Loop

The most consequential change in the editor has no API at all. Reality Composer Pro 3 is no longer an Xcode developer tool; you launch it from Applications, and the whole release is organized around cutting the time between an edit and seeing it.³

The editor keeps the entity-and-component model. You import a USD file as an optimized bundle, drag it into the viewport as an entity, and shape it through components in the inspector. What changes is the feedback. Press Play and the scene runs; dock the simulation tab beside the scene tab and you keep authoring while the game is live, tweaking a graph and watching the result without any deployment step. JP’s framing is the one to remember: the simulation tab removes the deployment process from the inner loop, so the friction between you and the final experience nearly disappears.³

Three systems extend that loop. Prototypes turn an entity into a reusable asset dragged into the Project Browser; you instantiate it many times, override any instance, reset an override, or propagate it back to the source, and nothing changes permanently unless you decide it does. Live preview targets a connected Vision Pro through a companion app (shipping later this year), so you author on the Mac and see updates instantly on device, where physical space lighting actually has to be judged.³ Lightmaps handle static lighting: a Lightmap component pre-calculates indirect lighting, ambient occlusion, or beauty into a texture, with a preview tab showing the impact before a full bake. The session closes on the Reality Composer Pro Assistant, an AI panel that generates 3D objects and materials on demand.³ The pattern repeats: see the consequence of a change before you pay for it.

Extending The Editor With Xcode

The standalone editor would be a dead end for serious teams if artists could only use built-in components. The plug-in system is the answer, reframing Reality Composer Pro from a tool you use into a tool you program. It ships later this year.⁴

The team structure is the premise. A Chaparral Village-style project pairs a Reality Composer Pro project (artists and designers) with an Xcode project (engineers) in the same git repository. Imported files convert to internal JSON on disk, so standard git tooling merges them, and the editor adds a custom merge tool with fewer conflicts than plain git merge.⁴ Engineers build two schemes from shared code: the app, and an RCPCustomComponents.framework plug-in exposing their components and systems to the editor. Scenes export to a Reality File, RealityKit’s serialization format, which the app loads. Code flows into the editor as a plug-in; content flows into the app as a Reality File.

The mechanism is a registration handshake. You write a class conforming to RealityComposerProPlugin (from the RealityComposerPro Swift package, added automatically when you link the project), and in its setup method you register your components and systems with the context the editor hands you. A createRealityComposerProPlugin() function, exported as a C symbol so the loader can find it, returns the plug-in. Build the plug-in scheme, open the project, accept the trust prompt, and your custom component appears in the inspector, an artist drags the water-level slider and the surface reacts because your Swift system runs inside the editor.⁴ You can even attach the Xcode debugger and hit breakpoints in plug-in code. The session’s honest aside is worth keeping: Script Graph and code do similar things, and the real reason to reach for code is that big Script Graphs get hard to maintain and code can call Apple APIs (SwiftUI among them) that Script Graph cannot.⁴

The plug-in surface reaches three editor systems. Custom components and systems are the base case. Custom animation actions implement EntityAction (and Codable), then drag onto the sequencer with editable parameters. And the @Scriptable macro expands a component into a schema you register so designers wire it into Script Graph nodes directly.⁴ The same custom data, reachable from code, the timeline, and the visual scripting layer at once.

The Node Graphs That Build Behavior Without Code

If session 281 is the code story, session 393 is its mirror: how far a designer gets with no code, using the editor’s visual graphs. The 2026 answer is most of the way to an interactive character.

The build stacks four graph types. The Animation Graph is a node-based runtime editor with a Final Pose node at its root; Vincent drops in a State Machine, defines Idle and Walk states with conditioned transitions, and drives the blend with a single isWalking boolean toggled at runtime. The Behavior Tree authors the autonomous routine: a Sequence composite runs Rotate To Face, Move To, and Wait nodes in order, Parameter Setter nodes flip isWalking around the movement, and a Precondition keeps the alchemist waiting until a readyToBrew flag turns true. Script Graph supplies the event-driven glue, an On Tap node sets readyToBrew so a tap sends the character to the cauldron.⁵ The insight that matters for team velocity: because the graphs are visual and event-driven, anyone on the team builds and tests behaviors directly in the editor without a build cycle.⁵

The session fills in two pieces session 279 described from the code side. The Navigation Mesh component is the authoring counterpart to that API: a Shapes bounding box selects which geometry to sample, Off-Mesh Connections link disjoint regions, and Generation Parameters like cell size control sampling resolution. Once authored, the mesh feeds a Behavior Tree, an Animation Graph, or a custom Swift system.⁵ Compute Graph builds Metal-backed GPU particle simulations across four phases (Emitter, Initialize, Simulate, Output), and the cauldron’s smoke uses a negative gravity force so steam drifts upward. Shader Graph adds RealityKit PBR Surface 2 (sheen and subsurface scattering), a dedicated hair surface, and editable portal surface and geometry.⁵

Collaborating On Structured 3D Models

The last session is the payoff, what you build when the engine and editor are this capable. visionOS 27 ships a toolkit for collaborative design review, where people in a SharePlay call manipulate the same complex assembly, open it, cross-section it, and pull parts free, at the same fidelity in the same space.⁶

The foundation is structure, and the session’s sharpest point is that geometry without hierarchy is useless to code. An engine block flattened to the root renders fine but offers nothing to grab or animate, the pistons buried in a flat list of InteriorPart_47-style names. A nested hierarchy makes each part findable, the precondition for everything else.⁶ On that hierarchy, manipulation is a placement decision: put a ManipulationComponent at the root and the whole assembly moves as one object; move it down to the children (with InputTargetComponent and CollisionComponent alongside, releaseBehavior set to .stay) and every part becomes individually grabbable while the rest stay put. Bill’s framing is the cleanest statement of the ECS philosophy in the five sessions: the shift from “thing to look at” to “thing to explore” happens entirely because of where the component lives in the tree.⁶

Clipping is the genuinely new RealityKit capability. The ClippingComponent discards geometry outside an axis-aligned bounds each frame (shouldClipChildren defaults to false, the common gotcha; shouldClipSelf defaults to true), exposing the interior as readily as the exterior.⁶ The interaction model is deliberately small, six faces of the bounding box become six draggable planes, each controlling one scalar. Making those drags feel natural is the session’s math lesson: a drag delta transforms through world to the model frame, then projects onto the plane’s normal so the bounds change only along the axis you grabbed. Auto-expansion uses the same statistical honesty, computing a volume-weighted position variance along each axis and fanning the sub-assemblies apart along the one with the largest variance.⁶ Bill’s closing lesson is unusual for a graphics talk: get comfortable with variance, vector projection, and coordinate-frame transforms, because the interactions that feel effortless are built on that math.

Where To Start

Adopting all of it at once is the wrong move. Sequence by what unblocks the rest.

1. Download the standalone editor first. Reality Composer Pro 3 leaving the Xcode bundle is the change every other workflow depends on; re-import a scene so the simulation tab, prototypes, and lightmap preview are in hand.³
2. Get the iteration loop before the features. Dock the simulation tab to author while the scene runs, then wire on-device live preview to a Vision Pro. Judging physical space lighting or soft shadows on a flat Mac screen will mislead you.³²
3. Reach for RealityKit features as components, not rewrites. Soft shadows, a baked lightmap, and a SurroundingsLight on a spotlight are each one component or property, and each buys a disproportionate amount of realism.²
4. Add the Xcode plug-in only when artists need your data. The RealityComposerProPlugin registration earns its keep the moment a designer wants to tune a custom component live; until then built-in components and Script Graph cover a lot of ground.⁴⁵
5. Build on hierarchy for design review. The structured-model toolkit only works on a well-nested asset, so fix the hierarchy before writing interaction code.⁶

The throughline across the five sessions: the spatial pipeline this year rewards composition over construction. Realism is a component you attach, iteration a tab you dock, collaboration a hierarchy you respect.

FAQ

Is Reality Composer Pro 3 still part of Xcode?

No. Reality Composer Pro 3 is now a standalone app you download from the developer site and launch from your Applications folder. It still links to an Xcode project for building the app and loading plug-ins, but the editor runs independently.³

How do soft shadows work in RealityKit in 2026?

A shadow’s lightSize property is the light’s diameter in meters; the default 0 produces a hard shadow, and a larger value (the session uses 0.7) widens the penumbra. Set quality to medium or high, since low forces a hard shadow regardless of light size, with medium recommended at typical viewing distance to manage cost.²

Can I run my own Swift code inside the Reality Composer Pro editor?

Yes. Build a plug-in conforming to the RealityComposerProPlugin protocol and register your components and systems in its setup method; the editor runs that code live, so an artist adjusts a property and sees the system react without rebuilding the app. You can attach the Xcode debugger to hit breakpoints in plug-in code. The feature ships later in 2026.⁴

What is the new clipping feature in visionOS 27?

ClippingComponent discards any geometry outside an axis-aligned bounding box (bounds) each frame, letting you see through a complex assembly to its interior. Note that shouldClipChildren defaults to false, so a parent assembly won’t clip its children until you set it to true.⁶

Do I need to write code to make a character animate and respond to taps?

No. Reality Composer Pro 3’s visual graphs cover it: an Animation Graph state machine blends idle and walk animations, a Behavior Tree sequences the routine and gates it on a boolean, and a Script Graph wires an On Tap node to flip that boolean, all authored without a build cycle.⁵

The full Apple Ecosystem cluster: the spatial mental model explaining why RealityKit is an entity-component-system rather than SwiftUI in 3D; the visionOS spatial patterns for structuring scenes and immersion styles; what’s new in visionOS 27 for the platform features around these tools; and the Apple platform matrix for how RealityKit deploys across visionOS, iOS, iPadOS, macOS, and tvOS. The hub is at the Apple Ecosystem Series. For broader iOS-with-AI-agents context, see the iOS Agent Development guide.

References