You want to let users place a 3D Pixar character on their desk and walk around it. That means your app needs to understand the physical world — detecting flat surfaces, tracking the device’s position in 3D space, and anchoring virtual content so it stays put when the camera moves. ARKit handles all of this with a handful of configuration objects and delegate callbacks.

This guide covers ARWorldTrackingConfiguration, plane detection, ARSCNView for SceneKit-based rendering, hit testing and ray casting, anchor management, and iOS 26’s Shared World Anchors for multiplayer experiences. We won’t cover RealityKit’s entity-component system (see RealityKit Introduction) or visionOS spatial computing.

Contents

• The Problem
• ARKit Architecture
• Setting Up ARSCNView with Plane Detection
• Placing Objects with Ray Casting
• Managing Anchors and World Understanding
• Shared World Anchors in iOS 26
• Performance Considerations
• When to Use (and When Not To)
• Summary

The Problem

Pixar’s marketing team wants an app that lets fans place Buzz Lightyear on any flat surface and take photos with him. The first instinct might be to drop a 3D model at a fixed position in front of the camera:

import SceneKit

func placeCharacterNaively(in sceneView: SCNView) {
    let scene = SCNScene(named: "BuzzLightyear.usdz")!
    sceneView.scene = scene

    // Fixed position — no understanding of the real world
    let characterNode = scene.rootNode.childNodes.first!
    characterNode.position = SCNVector3(0, 0, -1)
}
// Simplified for clarity

This fails immediately. The model floats in the air because there is no surface detection. It drifts when the user moves because there is no motion tracking. And it exists only on one device because there is no spatial coordination. ARKit solves all three problems.

Apple Docs: ARKit — Apple Developer Documentation

ARKit Architecture: Sessions, Configurations, and Anchors

ARKit’s architecture has three layers:

ARSession is the central coordinator. It manages sensor fusion (camera, IMU, LiDAR when available), runs visual-inertial odometry to track device position, and delivers world-understanding updates through its delegate.

ARConfiguration subclasses tell the session what capabilities to enable. ARWorldTrackingConfiguration is the most common — it provides 6DoF (six degrees of freedom) tracking, plane detection, image detection, and environment mapping.

ARAnchor subclasses represent positions in the real world. When ARKit detects a horizontal surface, it creates an ARPlaneAnchor. When you place content, you create a custom ARAnchor at that position. Anchors persist across frames, and ARKit continuously refines their positions as it gathers more data about the environment.

// The relationship between these objects
let session = ARSession()
let configuration = ARWorldTrackingConfiguration()
configuration.planeDetection = [.horizontal, .vertical]

session.run(configuration)
// ARKit now:
//   1. Tracks device position and orientation (6DoF)
//   2. Detects horizontal and vertical planes
//   3. Delivers ARPlaneAnchor updates through the delegate

Required Capabilities

Before writing any code, add these to your Info.plist:

<key>NSCameraUsageDescription</key>
<string>$(PRODUCT_NAME) uses the camera for AR.</string>

ARKit requires an A9 chip or later. Always check availability at runtime:

func checkARSupport() -> Bool {
    ARWorldTrackingConfiguration.isSupported
}

Setting Up ARSCNView with Plane Detection

ARSCNView bridges ARKit and SceneKit. It automatically renders the camera feed as the background and positions SceneKit nodes according to ARKit’s world tracking. Here is the full setup for Pixar’s character placement app:

import ARKit
import UIKit

final class PixarARViewController: UIViewController {
    private var sceneView: ARSCNView!
    private var placedCharacters: [UUID: SCNNode] = [:]

    override func viewDidLoad() {
        super.viewDidLoad()

        sceneView = ARSCNView(frame: view.bounds)
        sceneView.autoresizingMask = [
            .flexibleWidth, .flexibleHeight
        ]
        sceneView.delegate = self
        sceneView.session.delegate = self
        sceneView.showsStatistics = true
        sceneView.automaticallyUpdatesLighting = true

        // Debug visualization — remove for production
        sceneView.debugOptions = [
            .showFeaturePoints, .showWorldOrigin
        ]

        view.addSubview(sceneView)
    }

    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)

        let configuration = ARWorldTrackingConfiguration()
        configuration.planeDetection = [.horizontal]
        configuration.environmentTexturing = .automatic

        // Use LiDAR mesh when available
        if ARWorldTrackingConfiguration
            .supportsSceneReconstruction(.mesh) {
            configuration.sceneReconstruction = .mesh
        }

        sceneView.session.run(
            configuration,
            options: [.resetTracking, .removeExistingAnchors]
        )
    }

    override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)
        sceneView.session.pause()
    }
}

The environmentTexturing = .automatic setting enables environment mapping, which provides realistic reflections on 3D objects. This is what makes a metallic Buzz Lightyear visor reflect the user’s actual room.

Visualizing Detected Planes

When ARKit detects a surface, it adds an ARPlaneAnchor and calls the ARSCNViewDelegate. Render each plane as a semi-transparent overlay so users know where they can place content:

extension PixarARViewController: ARSCNViewDelegate {
    func renderer(
        _ renderer: SCNSceneRenderer,
        didAdd node: SCNNode,
        for anchor: ARAnchor
    ) {
        guard let planeAnchor = anchor as? ARPlaneAnchor else {
            return
        }
        let plane = createPlaneVisualization(for: planeAnchor)
        node.addChildNode(plane)
    }

    func renderer(
        _ renderer: SCNSceneRenderer,
        didUpdate node: SCNNode,
        for anchor: ARAnchor
    ) {
        guard let planeAnchor = anchor as? ARPlaneAnchor else {
            return
        }
        // ARKit refines planes over time — update visualization
        node.childNodes.first?.geometry =
            createPlaneGeometry(for: planeAnchor)
    }

    private func createPlaneVisualization(
        for anchor: ARPlaneAnchor
    ) -> SCNNode {
        let geometry = createPlaneGeometry(for: anchor)
        let material = SCNMaterial()
        material.diffuse.contents =
            UIColor.systemBlue.withAlphaComponent(0.3)
        material.isDoubleSided = true
        geometry.materials = [material]

        let planeNode = SCNNode(geometry: geometry)
        planeNode.position = SCNVector3(
            anchor.center.x,
            0,
            anchor.center.z
        )
        return planeNode
    }

    private func createPlaneGeometry(
        for anchor: ARPlaneAnchor
    ) -> SCNPlane {
        SCNPlane(
            width: CGFloat(anchor.planeExtent.width),
            height: CGFloat(anchor.planeExtent.height)
        )
    }
}

Tip: In production, remove debug plane visualizations once the user has placed their first object. Persistent blue overlays on every surface feel like a debug build, not a polished app. Use a coaching overlay (ARCoachingOverlayView) instead.

Adding the Coaching Overlay

Apple provides a built-in coaching view that guides users to move their device and helps ARKit initialize:

extension PixarARViewController {
    func addCoachingOverlay() {
        let coachingView = ARCoachingOverlayView()
        coachingView.session = sceneView.session
        coachingView.goal = .horizontalPlane
        coachingView.activatesAutomatically = true
        coachingView.translatesAutoresizingMaskIntoConstraints =
            false

        sceneView.addSubview(coachingView)
        NSLayoutConstraint.activate([
            coachingView.topAnchor.constraint(
                equalTo: sceneView.topAnchor
            ),
            coachingView.bottomAnchor.constraint(
                equalTo: sceneView.bottomAnchor
            ),
            coachingView.leadingAnchor.constraint(
                equalTo: sceneView.leadingAnchor
            ),
            coachingView.trailingAnchor.constraint(
                equalTo: sceneView.trailingAnchor
            )
        ])
    }
}

Apple Docs: ARCoachingOverlayView — ARKit

Placing Objects with Ray Casting

When the user taps the screen, you need to convert that 2D screen point into a 3D world position on a detected plane. ARKit’s ray casting API replaces the deprecated hit-testing API and provides more accurate results with tracked updates.

Loading the Character Model

First, load the USDZ model:

final class CharacterModelLoader {
    static func loadBuzz() -> SCNNode? {
        guard let url = Bundle.main.url(
            forResource: "BuzzLightyear",
            withExtension: "usdz"
        ),
        let scene = try? SCNScene(
            url: url,
            options: [.checkConsistency: true]
        ) else {
            return nil
        }

        let characterNode = SCNNode()
        // USDZ files may have complex hierarchies
        for child in scene.rootNode.childNodes {
            characterNode.addChildNode(child)
        }

        // Scale to a reasonable size
        characterNode.scale = SCNVector3(0.01, 0.01, 0.01)

        return characterNode
    }
}

Ray Casting on Tap

// Add this stored property to PixarARViewController's class body
// private var placedCharacters: [UUID: SCNNode] = [:]

extension PixarARViewController {
    @objc func handleTap(_ gesture: UITapGestureRecognizer) {
        let location = gesture.location(in: sceneView)

        // Create a ray cast query targeting horizontal planes
        guard let query = sceneView.raycastQuery(
            from: location,
            allowing: .estimatedPlane,
            alignment: .horizontal
        ) else {
            return
        }

        let results = sceneView.session.raycast(query)

        guard let firstResult = results.first else {
            showPlacementFeedback(success: false)
            return
        }

        placeCharacter(at: firstResult)
    }

    private func placeCharacter(at result: ARRaycastResult) {
        guard let buzzNode = CharacterModelLoader.loadBuzz()
        else { return }

        // Create an anchor at the ray cast position
        let anchor = ARAnchor(
            name: "BuzzLightyear",
            transform: result.worldTransform
        )
        sceneView.session.add(anchor: anchor)
        placedCharacters[anchor.identifier] = buzzNode
    }

    private func showPlacementFeedback(success: Bool) {
        let generator = UINotificationFeedbackGenerator()
        generator.notificationOccurred(
            success ? .success : .error
        )
    }
}

Then update the existing renderer(_:didAdd:for:) delegate method to handle both planes and character anchors:

// Updated renderer(_:didAdd:for:) — replaces the earlier plane-only version
func renderer(
    _ renderer: SCNSceneRenderer,
    didAdd node: SCNNode,
    for anchor: ARAnchor
) {
    if let planeAnchor = anchor as? ARPlaneAnchor {
        let plane = createPlaneVisualization(for: planeAnchor)
        node.addChildNode(plane)
    } else if anchor.name == "BuzzLightyear",
              let characterNode =
                  placedCharacters[anchor.identifier] {
        node.addChildNode(characterNode)

        // Add a subtle drop animation
        characterNode.opacity = 0
        characterNode.position.y += 0.1
        SCNTransaction.begin()
        SCNTransaction.animationDuration = 0.5
        characterNode.opacity = 1
        characterNode.position.y -= 0.1
        SCNTransaction.commit()
    }
}

Warning: Never position SceneKit nodes using world coordinates directly. Always attach them to ARAnchor-backed nodes. ARKit continuously refines anchor positions as it maps more of the environment. Nodes attached to anchors follow these refinements automatically; free-floating nodes will drift.

Tracked Ray Casting for Continuous Placement

For a “drag to position” experience where the character follows the user’s finger, use a tracked ray cast that updates across frames:

extension PixarARViewController {
    private var activeRaycast: ARTrackedRaycast?
    private var previewNode: SCNNode?

    func beginTrackedPlacement(from point: CGPoint) {
        let preview = CharacterModelLoader.loadBuzz()
        preview?.opacity = 0.5
        sceneView.scene.rootNode.addChildNode(preview!)
        previewNode = preview

        guard let query = sceneView.raycastQuery(
            from: point,
            allowing: .estimatedPlane,
            alignment: .horizontal
        ) else { return }

        activeRaycast = sceneView.session.trackedRaycast(
            query
        ) { [weak self] results in
            guard let result = results.first else { return }
            DispatchQueue.main.async {
                self?.previewNode?.simdWorldTransform =
                    result.worldTransform
            }
        }
    }

    func confirmPlacement() {
        activeRaycast?.stopTracking()
        activeRaycast = nil
        previewNode?.opacity = 1.0
        previewNode = nil
    }
}

Managing Anchors and World Understanding

Persisting AR Anchors Across Sessions

ARKit supports world map serialization, which lets users save their AR scene and restore it later. Imagine a Pixar collector who arranges multiple characters on their shelf and wants to see them again tomorrow:

extension PixarARViewController {
    func saveWorldMap(to url: URL) async throws {
        let worldMap =
            try await sceneView.session.currentWorldMap

        let data = try NSKeyedArchiver.archivedData(
            withRootObject: worldMap,
            requiringSecureCoding: true
        )
        try data.write(to: url, options: .atomic)
    }

    func loadWorldMap(from url: URL) throws {
        let data = try Data(contentsOf: url)
        guard let worldMap =
            try NSKeyedUnarchiver.unarchivedObject(
                ofClass: ARWorldMap.self,
                from: data
            ) else {
            throw PixarARError.invalidWorldMap
        }

        let configuration = ARWorldTrackingConfiguration()
        configuration.planeDetection = [.horizontal]
        configuration.initialWorldMap = worldMap

        sceneView.session.run(
            configuration,
            options: [.resetTracking, .removeExistingAnchors]
        )
    }
}

enum PixarARError: Error {
    case invalidWorldMap
}

Note: World map relocation works best in environments with distinctive visual features. A plain white room provides few tracking points and will struggle to relocalize. Advise users to save world maps in well-lit, textured environments.

Handling Session Interruptions

AR sessions can be interrupted by phone calls, Siri, or multitasking. Handle these gracefully:

extension PixarARViewController: ARSessionDelegate {
    func sessionWasInterrupted(_ session: ARSession) {
        showInterruptionBanner(
            message: "AR tracking paused"
        )
    }

    func sessionInterruptionEnded(_ session: ARSession) {
        let configuration = ARWorldTrackingConfiguration()
        configuration.planeDetection = [.horizontal]
        session.run(configuration, options: [.resetTracking])
        hideInterruptionBanner()
    }

    func session(
        _ session: ARSession,
        didFailWithError error: Error
    ) {
        guard let arError = error as? ARError else { return }

        switch arError.code {
        case .sensorFailed:
            showErrorAlert(
                message: "Camera sensor unavailable."
            )
        case .sensorUnavailable:
            showErrorAlert(
                message: "Camera is in use by another app."
            )
        case .worldTrackingFailed:
            session.run(
                session.configuration!,
                options: [.resetTracking]
            )
        default:
            showErrorAlert(
                message: arError.localizedDescription
            )
        }
    }

    // Placeholder methods
    private func showInterruptionBanner(message: String) {}
    private func hideInterruptionBanner() {}
    private func showErrorAlert(message: String) {}
}

Advanced: Shared World Anchors in iOS 26

iOS 26 introduces SharedWorldAnchor, which lets multiple devices share AR anchors without requiring a shared world map exchange. This opens the door for multiplayer AR experiences where two Pixar fans can see the same Buzz Lightyear on the same table from their own devices.

Note: Shared World Anchors require devices with a U1 chip or later and an active network connection for coordination through Apple’s relay service.

Setting Up Shared Anchors

@available(iOS 26, *)
final class SharedARExperience {
    private let session: ARSession
    private var sharedAnchors: [SharedWorldAnchor] = []

    init(session: ARSession) {
        self.session = session
    }

    /// Creates a shared anchor that nearby devices can discover
    func shareCharacterPlacement(
        transform: simd_float4x4,
        characterName: String
    ) async throws -> SharedWorldAnchor {
        let anchor = SharedWorldAnchor(
            name: characterName,
            transform: transform
        )

        session.add(anchor: anchor)
        try await anchor.waitUntilShared()

        sharedAnchors.append(anchor)
        return anchor
    }

    /// Discovers shared anchors from nearby devices
    func discoverSharedAnchors()
        -> AsyncStream<SharedWorldAnchor> {
        AsyncStream { continuation in
            let query = SharedWorldAnchor.Query()
            Task {
                for await anchor in session
                    .sharedWorldAnchors(matching: query) {
                    continuation.yield(anchor)
                }
                continuation.finish()
            }
        }
    }
}

Coordinating Character Placement

@available(iOS 26, *)
extension SharedARExperience {
    /// Host: Places a character and shares the anchor
    func hostPlacement(
        characterName: String,
        at result: ARRaycastResult
    ) async throws {
        let anchor = try await shareCharacterPlacement(
            transform: result.worldTransform,
            characterName: characterName
        )
        print("Shared anchor \(anchor.identifier)")
    }

    /// Guest: Listens for shared anchors
    func joinExperience(
        onCharacterDiscovered: @escaping (
            String, simd_float4x4
        ) -> Void
    ) {
        Task {
            for await anchor in discoverSharedAnchors() {
                guard let name = anchor.name else {
                    continue
                }
                onCharacterDiscovered(
                    name, anchor.transform
                )
            }
        }
    }
}

The key difference from world map sharing is that Shared World Anchors do not require both devices to have mapped the same environment. The U1 chip handles spatial alignment through ultra-wideband ranging, making the setup process invisible to users.

Apple Docs: SharedWorldAnchor — ARKit (iOS 26+)

Performance Considerations

AR apps push the hardware harder than almost any other app category. The camera, GPU, CPU, and motion sensors all run simultaneously.

Frame rate: ARKit targets 60 fps for tracking updates. Your SceneKit rendering should match. If your scene drops below 60 fps, ARKit’s tracking quality degrades because it relies on consistent visual frame timing. Use showsStatistics = true during development to monitor frame rate.

Polygon budget: On iPhone 15 Pro, aim for under 100K triangles in your AR scene. USDZ models from 3D scanning often exceed this — decimate them in Reality Converter or Blender before shipping.

Thermal throttling: ARKit is power-intensive. After 5-10 minutes of continuous use, the device will throttle CPU and GPU frequencies. Your frame rate will drop. Mitigate by reducing scene complexity when ProcessInfo.processInfo.thermalState reaches .serious:

func adaptToThermalState() {
    let observer = NotificationCenter.default.addObserver(
        forName: ProcessInfo.thermalStateDidChangeNotification,
        object: nil,
        queue: .main
    ) { [weak self] _ in
        let state = ProcessInfo.processInfo.thermalState
        switch state {
        case .nominal, .fair:
            self?.sceneView.preferredFramesPerSecond = 60
        case .serious:
            self?.sceneView.preferredFramesPerSecond = 30
            self?.disableEnvironmentTexturing()
        case .critical:
            self?.sceneView.session.pause()
            self?.showThermalWarning()
        @unknown default:
            break
        }
    }
    // Store observer to prevent deallocation
    thermalObserver = observer
}
// Simplified for clarity

LiDAR acceleration: Devices with LiDAR (iPhone 12 Pro and later, iPad Pro 2020+) achieve plane detection in under 1 second versus 3-5 seconds without it. They also provide scene reconstruction meshes that enable occlusion — virtual objects can hide behind real furniture. Check for support at runtime:

let hasLiDAR = ARWorldTrackingConfiguration
    .supportsSceneReconstruction(.mesh)

Metric	Without LiDAR	With LiDAR
Plane detection time	3-5 seconds	Under 1 second
Plane accuracy	~5cm error	~1cm error
Occlusion support	No	Yes (scene mesh)
Placement accuracy	Good	Excellent

When to Use (and When Not To)

Scenario	Recommendation
Placing 3D objects on surfaces	ARKit + ARWorldTrackingConfiguration.
Face filters and effects	ARFaceTrackingConfiguration.
Measuring distances	ARKit ray casting, especially with LiDAR.
Indoor navigation	ARGeoTrackingConfiguration (limited cities).
Simple 3D model viewing	SCNView or RealityKit without AR session.
Cross-platform (iOS + Android)	Unity or Unreal with ARKit/ARCore plugins.
visionOS spatial experiences	RealityKit.
Multi-user AR content	Shared World Anchors (iOS 26).

Summary

• ARWorldTrackingConfiguration provides 6DoF tracking and plane detection. Enable .horizontal and .vertical plane detection based on your use case.
• Always attach virtual content to ARAnchor-backed nodes — never position SceneKit nodes using absolute world coordinates, or they will drift as ARKit refines its environment map.
• Use raycastQuery(from:allowing:alignment:) for object placement, not the deprecated hitTest(_:types:) API. Tracked ray casts provide continuous, refined results.
• Persist AR experiences across sessions with ARWorldMap serialization. Users can save and restore their entire scene.
• iOS 26’s SharedWorldAnchor enables multiplayer AR without manual world map exchange, using U1 ultra-wideband ranging for device-to-device spatial alignment.
• Monitor thermal state and adapt your scene complexity. AR apps push hardware to its limits, and graceful degradation is the difference between a polished experience and a force-quit.

For building entity-component architectures with richer 3D interactions, explore RealityKit Introduction, which builds on ARKit’s world understanding with a modern rendering pipeline.

Related Posts

• RealityKit Introduction
• VisionOS Primer