VisionUI 2.0: spatial layouts and immersive spaces for visionOS, rebuilt from scratch

Table of Contents

• The problem
• Design decisions and alternatives
• What went wrong, and what was learned
• Usage
• Limits

The problem

A visionOS app keeps running into the same small jobs. Five cards should sit on an arc that bends toward the viewer. A button should open an immersive space, and pressing it twice while the space is still opening should not leave the button and the system disagreeing. A model should be draggable, but not out of its box, and a pinch should not scale it to nothing. None of these is hard on its own, but each one hides geometry or state that is easy to get wrong: which way an item faces, how SwiftUI's points relate to RealityKit's meters, what a gesture reports on every frame, and what happens when a NaN reaches a position.

VisionOS-UI-Framework is a Swift package, imported as VisionUI, that collects these pieces. Version 2.0.0 has five layouts (arc, ring, grid, sphere and helix), a SpatialCarousel built on the ring, an ImmersiveSpaceCoordinator, a constrained ManipulationState, and a few components: SpatialCard, a toolbar that becomes an ornament on visionOS, and hit targets sized to Apple's guidelines. It supports visionOS 2, iOS 18 and macOS 15 and has no dependencies.

Design decisions and alternatives

A layout is a value that produces data. SpatialLayout has one requirement, placements(count:) -> [SpatialPlacement], which returns a position in meters and an orientation for each item. SwiftUI's Layout protocol was the obvious alternative, but it works in two dimensions: it proposes sizes and places points, and it has no way to express depth or a rotation, and it does not apply to RealityKit entities at all. Writing a view and a separate RealityKit helper for every layout would have duplicated the math. With a pure function in the middle, SpatialLayoutView and Entity.arrangeChildren(using:) consume the same output, and the tests check the geometry without creating a view. The cost is that a layout does not take part in SwiftUI's measuring pass: it does not know how large an item is, so the spacing is an argument.

Meters and RealityKit's axes. Positions use meters with +y up and +z toward the viewer. RealityKit already works in meters, and on visionOS the environment's physicalMetrics converts meters to points. SwiftUI's y axis points down, so the conversion lives in one place (SpatialUnits) instead of being repeated in every view.

A flat projection instead of nothing on iOS and macOS. Behind #if os(visionOS) the views get real depth through offset(z:) and rotation3DEffect. On iOS and macOS, FlatProjection draws the layout as seen from the front: an item z meters closer than the origin is drawn distance / (distance - z) as large, clamped to 0.2...3, and its depth becomes its zIndex. This is an approximation, not 3D, but it lets the same API run on an iPhone or a Mac, and it lets the tests render views with ImageRenderer and read pixels without a headset.

A state machine for immersive spaces. A hand-kept Bool drifts away from the system as soon as someone taps twice during a transition. ImmersiveSpaceCoordinator is a @MainActor @Observable class with one phase: .closed, .opening(id), .open(id) or .closing(id). A request during a transition returns .busy and changes nothing. Asking for a different space while one is open dismisses the current one first, and asking for the open one returns .alreadyOpen. Every request increments a generation counter, and when the system answers, the coordinator checks request == generation before it touches the phase, so a late answer to a replaced request is ignored. When the person closes a space with the Digital Crown, the app calls spaceDidDisappear(_:) from the space's onDisappear. The system calls go through ImmersiveSpaceActions, a pair of closures, because OpenImmersiveSpaceAction comes from the environment and cannot be created in a test. The tests use a fake system that holds a request open on purpose, which makes the race reproducible.

Manipulation as a value type. ManipulationState holds the position, scale and orientation; ManipulationConstraints holds the bounds, the scale range, the rotation mode (free, yaw only or locked) and an optional yaw snap angle. The spatialManipulation(_:) modifier only translates SwiftUI gestures into drag(translation:), magnify(by:) and rotate(by:) calls. Keeping the rules in a struct means they are tested without gestures. SwiftUI gestures report the total change since they began, so the state keeps a separate starting value for each gesture (dragStart, magnifyStart, rotationStart); a drag and a pinch at the same time do not overwrite each other's starting point.

Defensive input, documented. A count or a length can come from a file or a server. Item counts are capped at 4096 and lengths at 10 km, and NaN or infinite arguments fall back to documented defaults. A view modifier cannot throw, so correcting the value and saying so in the documentation was the practical choice; the risk is that a silent correction hides a caller's mistake.

What went wrong, and what was learned

The repository is older than the package it holds now. It was opened in August 2025 and its first month went mostly into documentation pages and workflows. Version 1.0.0 was tagged in February 2026. As the 2.0.0 changelog states, its manifest declared .visionOS(.v1_0), a platform version that does not exist, so Swift Package Manager could not resolve the package, and with that fixed its sources still called APIs that are not in the visionOS SDK, such as RealityViewContent.physicsWorld and SpatialDragGesture. In June 2026 the components were removed and placeholders took their place: a version constant, a modifier that only drew a shadow and a VolumetricWindowManager.present() that printed a line. Commit 5c17403 on 15 October 2026 removed all of it, and the package was written again. Because every public name changed, the result is a new major version. The lesson is in the new CI: the tests, the example app and the documentation build on every pull request, and the tests and the example app do so on all three platforms.

The rebuild has its own bug worth telling. ManipulationState checked simd_length(vector) > 1e-6 before normalizing a quaternion. For a finite but huge value such as [3e38, 3e38, 0, 0], the length overflows to infinity, the check passes, and simd_normalize returns a zero quaternion. The initializer stored it, the first three columns of matrix became zero, and the object collapsed to a point. The documentation said zero quaternions were ignored, so the code contradicted it. Commit 5f6c475 added an internal unitQuaternion(_:) that divides the vector by its largest component before measuring it, which keeps the length between 1 and 2, and returns nil for NaN, infinite or near-zero input. The initializer, constraints changes and rotate(by:) all use it. Checking for NaN and infinity was not enough: finite values that overflow are a separate case.

A second fix taught something about tests. SpatialLayout is public, so a custom layout can return a NaN position. Commit f3f8c0e added SpatialPlacement.sanitized and a pixel test that drew a broken layout. As commit 75e0e4e records, that test still passed with the fix reverted, because FlatProjection already replaced a non-finite position with the origin. The layout in the test now returns [.nan, 0.025, 0]: sanitizing turns only the NaN into 0 and draws the swatch 25 points above the center, while the raw placement lands in the middle. The test now fails without the fix. A new green test proves little until it is seen failing without the change it guards.

Usage

Add the package in Xcode with File › Add Package Dependencies…, or in Package.swift:

1dependencies: [
2    .package(url: "https://github.com/muhittincamdali/VisionOS-UI-Framework.git", from: "2.0.0")
3],
4targets: [
5    .target(name: "YourApp", dependencies: [
6        .product(name: "VisionUI", package: "VisionOS-UI-Framework")
7    ])
8]

The quick start from the README places five cards on an arc:

1import SwiftUI
2import VisionUI
3 
4struct ContentView: View {
5    let planets = ["Mercury", "Venus", "Earth", "Mars", "Jupiter"]
6 
7    var body: some View {
8        SpatialLayoutView(.arc(radius: 0.6, spacing: 0.17), planets, id: \.self) { planet in
9            SpatialCard {
10                Text(planet)
11                    .font(.title)
12            }
13        }
14    }
15}

On the Apple Vision Pro simulator the middle card sits in the window's plane and the outer cards come closer, turned toward the arc's center. On an iPhone or a Mac the arc is drawn from the front. The README's samples live in Examples/SpatialGallery/SpatialGallery/ReadmeSamples.swift, which the example app compiles and the test target compiles through a symlink, so a sample that stops compiling fails the build. The constrained manipulation sample from the same file:

1struct ManipulationSample: View {
2    @State private var globe = ManipulationState(constraints: ManipulationConstraints(
3        bounds: SpatialBounds(center: .zero, size: [0.4, 0.3, 0.2]),
4        scaleRange: 0.5...2,
5        rotation: .yaw,
6        yawSnapAngle: .pi / 4
7    ))
8 
9    var body: some View {
10        Image(systemName: "globe.americas.fill")
11            .font(.system(size: 120))
12            .foregroundStyle(.blue)
13            .spatialManipulation($globe)
14    }
15}

The globe stays inside a 40 × 30 × 20 cm box, scales between half and double size, turns only around the vertical axis and snaps to 45° when a turn ends.

Limits

• Simulator only. The screenshots in the repository come from the Apple Vision Pro simulator, and there are no benchmarks. Gesture feel and performance on hardware are not documented.
• The visionOS view path has fewer automated checks. The 97 tests run on macOS and an iPhone simulator; on a visionOS simulator 87 run, because the pixel tests are compiled out there. The visionOS branches that use physicalMetrics and RotateGesture3D are covered by the state tests and the example app build, not by pixel tests.
• The coordinator is tested against a fake. The fake system drives every case of the state machine, but it is not the real openImmersiveSpace. ImmersiveSpaceToggle needs visionOS, or macOS 26 where SwiftUI has immersive spaces.
• The flat projection is not 3D. On iOS and macOS it approximates depth with scale and order.
• The scope is narrow. There is no hand tracking, eye tracking, ARKit or spatial audio. The roadmap lists layouts that follow surfaces found by ARKit and a carousel that scrolls along an arc.
• Toolchain. The manifest uses swift-tools-version: 6.2, so a project needs Xcode 26 or later.

The package, its DocC articles and the example app are on GitHub under the MIT license.