OKO: Designing for Invisible Users

“The best accessibility design is invisible — it simply works.” — AYES team philosophy

OKO is a crosswalk assistant for blind and low-vision users that won the Apple Design Award for Social Impact in 2023. The app uses the iPhone camera and on-device machine learning to detect pedestrian signal states (walk, don’t walk, countdown) and communicates the result through haptic feedback and spatial audio. Users point their phone toward a crosswalk signal, and OKO tells them when it is safe to cross without requiring them to see the screen at all.

This is design at its most essential. OKO has almost no visual interface in the traditional sense. There are no grids, no cards, no color palettes, no typography hierarchies. Instead, the design challenge is entirely about non-visual communication: how to convey safety-critical information through vibration patterns and sound in a noisy, unpredictable urban environment. OKO forces designers to reconsider what “interface” means when the screen is not the medium.

Why OKO Matters

Key achievements: - Won the Apple Design Award for Social Impact (2023), the highest recognition for impactful design on Apple platforms - Proved that the most impactful design work may have zero visual component - Created a haptic language that communicates binary state (safe/not safe) with nuance (confidence level, countdown awareness) - Demonstrated on-device ML can serve accessibility in real-time without network dependency - Addressed a genuine civic accessibility gap: the majority of intersections worldwide lack accessible pedestrian signals (APS)

Key Takeaways

1. The best interface may be no interface - OKO’s primary interaction is haptic and auditory; the screen exists for setup and sighted companions, not for the core user
2. Haptic feedback is a design language - Different vibration patterns communicate not just binary states but confidence levels, transitions, and urgency
3. Audio-first UX requires spatial thinking - Sound must convey direction and distance, not just presence, because blind users navigate a 3D environment
4. Civic infrastructure has design gaps - Most intersections lack accessible signals; OKO reveals how much public design assumes sightedness
5. Designing for extremes improves everything - The constraints of designing for blind users produce interaction patterns (haptic confirmation, audio feedback) that improve apps for everyone

Core Design Principles

1. Non-Visual Interaction Design

OKO’s core interaction has no visual component. The user holds their phone pointed toward a crosswalk signal, and OKO communicates through vibration and sound.

The interaction model:

USER ACTION:         SYSTEM RESPONSE:

Hold phone toward    Steady pulse: "I see a signal"
crosswalk signal     ┌──────────────────────────┐
                     │  ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙   │  Gentle repeating tap
                     └──────────────────────────┘

Signal detected:     WALK signal:
WALK                 ┌──────────────────────────┐
                     │  ━━━━━━━━━━━━━━━━━━━━━━  │  Strong continuous
                     │  + ascending tone         │  vibration + rising tone
                     └──────────────────────────┘

Signal detected:     DON'T WALK signal:
DON'T WALK           ┌──────────────────────────┐
                     │  ∙   ∙   ∙   ∙   ∙   ∙  │  Slow, spaced taps
                     │  + low tone               │  + descending tone
                     └──────────────────────────┘

Signal detected:     COUNTDOWN (time running out):
COUNTDOWN            ┌──────────────────────────┐
                     │  ∙∙ ∙∙ ∙∙ ∙∙ ∙∙ ∙∙ ∙∙   │  Accelerating double-taps
                     │  + urgency tone           │  increasing tempo
                     └──────────────────────────┘

Design decisions that matter: - Walk = continuous strong vibration. The positive state must be the most unmistakable. A blind user’s safety depends on correctly interpreting “it is safe to cross.” There is no room for ambiguity. - Don’t Walk = spaced gentle taps. The negative state is calm, not alarming. Alarming vibrations cause stress; the user simply needs to know “not yet.” - Countdown = accelerating tempo. Borrowed from the audible beeping of accessible pedestrian signals, the increasing speed communicates “time is running out” without requiring counting. - No signal detected = silence. If OKO cannot confidently identify the signal state, it communicates nothing rather than guessing. In safety-critical design, silence (no information) is safer than wrong information.

2. Haptic Feedback as Design Language

OKO’s haptic patterns are not random vibrations. They form a coherent language with grammar and vocabulary.

// Haptic pattern vocabulary (conceptual)

// WALK: Strong, continuous, confident
// Uses CHHapticEvent with sustained intensity
let walkPattern: [CHHapticEvent] = [
    CHHapticEvent(
        eventType: .hapticContinuous,
        parameters: [
            CHHapticEventParameter(
                parameterID: .hapticIntensity,
                value: 0.9    // High intensity = confidence
            ),
            CHHapticEventParameter(
                parameterID: .hapticSharpness,
                value: 0.5    // Medium sharpness = firm but not jarring
            )
        ],
        relativeTime: 0,
        duration: 2.0
    )
]

// DON'T WALK: Gentle, spaced, patient
let dontWalkPattern: [CHHapticEvent] = stride(
    from: 0, to: 3.0, by: 0.5
).map { time in
    CHHapticEvent(
        eventType: .hapticTransient,
        parameters: [
            CHHapticEventParameter(
                parameterID: .hapticIntensity,
                value: 0.3    // Low intensity = not urgent
            ),
            CHHapticEventParameter(
                parameterID: .hapticSharpness,
                value: 0.2    // Soft sharpness = gentle
            )
        ],
        relativeTime: time
    )
}

// COUNTDOWN: Accelerating tempo
func countdownPattern(secondsRemaining: Int) -> [CHHapticEvent] {
    // Interval decreases as time runs out
    // 10 seconds: tap every 500ms
    //  5 seconds: tap every 250ms
    //  2 seconds: tap every 125ms
    let interval = max(0.125, Double(secondsRemaining) / 20.0)
    // ... generate pattern with decreasing intervals
}

The haptic design principles: - Intensity = confidence. A strong vibration means OKO is certain. A weaker vibration means lower confidence. This maps to real-world intuition: a firm handshake conveys certainty. - Tempo = urgency. Faster patterns mean “act now.” Slower patterns mean “wait.” This is universal across cultures and does not require learning. - Silence = uncertainty. When OKO cannot determine the signal state, it does not vibrate. This is the most important design decision: in safety-critical contexts, no feedback is safer than wrong feedback. - Sharpness = attention demand. Sharp taps demand attention. Soft buzzes provide ambient awareness. The walk signal uses medium sharpness - firm enough to notice, soft enough to sustain without fatigue.

3. Audio-First UX with Spatial Awareness

OKO’s audio cues are designed for a user who is navigating a three-dimensional urban environment while wearing headphones (or using the phone speaker) amid traffic noise.

Audio design requirements: - Must be audible over traffic noise without being so loud it masks environmental sounds the user needs to hear (approaching vehicles, other pedestrians) - Must convey state change immediately - a half-second delay between signal change and audio cue could be dangerous - Must work with both speaker and headphones (many blind users keep one earbud in for navigation apps) - Must not be confused with other common app sounds (notification chimes, VoiceOver prompts)

The audio vocabulary:

WALK:
  ┌─ Rising two-tone ─────────────────────┐
  │  ♪ low → high  (major interval)       │
  │  Duration: 400ms                       │
  │  Character: Bright, clear, affirmative │
  └───────────────────────────────────────┘

DON'T WALK:
  ┌─ Single low tone ─────────────────────┐
  │  ♪ sustained low  (minor tone)        │
  │  Duration: 300ms                       │
  │  Character: Calm, not alarming         │
  └───────────────────────────────────────┘

COUNTDOWN:
  ┌─ Rhythmic clicking ───────────────────┐
  │  ♪ tick...tick..tick.tick.tickticktick │
  │  Duration: Matches remaining time      │
  │  Character: Accelerating, urgent       │
  └───────────────────────────────────────┘

SIGNAL LOST:
  ┌─ Descending tone ─────────────────────┐
  │  ♪ high → low  (gentle fade)          │
  │  Duration: 500ms                       │
  │  Character: "I lost track, repoint"    │
  └───────────────────────────────────────┘

Critical design choice: The walk signal is a rising interval (optimistic, affirmative) while the don’t-walk signal is a low sustained tone (calm, holding). These are not arbitrary. Rising intervals are cross-culturally associated with positive outcomes. Low sustained tones are associated with waiting. OKO leverages these universal associations so that a first-time user intuitively understands the meaning without training.

Design Patterns Worth Stealing

Designing for Zero-Visual Interaction

OKO’s approach to non-visual design produces patterns applicable far beyond accessibility:

Confirmation through haptics:

// Any app can use haptic confirmation
// Instead of only showing a green checkmark:

// Success: single strong tap
let successFeedback = UIImpactFeedbackGenerator(style: .heavy)
successFeedback.impactOccurred()

// Error: three rapid light taps
let errorFeedback = UIImpactFeedbackGenerator(style: .light)
for delay in [0, 0.1, 0.2] {
    DispatchQueue.main.asyncAfter(deadline: .now() + delay) {
        errorFeedback.impactOccurred()
    }
}

// Warning: two medium taps
let warningFeedback = UIImpactFeedbackGenerator(style: .medium)
for delay in [0, 0.15] {
    DispatchQueue.main.asyncAfter(deadline: .now() + delay) {
        warningFeedback.impactOccurred()
    }
}

The broader lesson: Most apps treat haptic feedback as an afterthought - a single buzz for any notification. OKO demonstrates that haptics can carry semantic meaning: different patterns for different states. This makes apps more usable for everyone, not just visually impaired users. A strong tap for “payment confirmed” feels different from a gentle tap for “message received,” and users learn these patterns unconsciously.

Safety-Critical Design Principles

OKO operates in an environment where design mistakes can cause physical harm. Its principles apply to any safety-adjacent interface (medical, automotive, industrial):

1. Fail silent, not fail loud. When uncertain, OKO provides no feedback rather than incorrect feedback. In safety contexts, false confidence is more dangerous than acknowledged uncertainty.

2. Binary before nuance. The primary communication is binary: safe or not safe. Nuance (countdown, confidence) is layered on top. The user’s first question (“Can I cross?”) gets a clear yes/no before any additional detail.

3. Redundant channels. OKO communicates through both haptic and audio simultaneously. If the user misses one channel (noisy environment drowns audio, phone in thick gloves dampens haptics), the other channel still carries the message.

4. No required learning. The haptic and audio patterns leverage universal associations (rising = positive, fast = urgent) so that first-time users understand without tutorials. Safety-critical interfaces cannot assume training.

SAFETY-CRITICAL DESIGN HIERARCHY:

1. Is it safe?          → Binary answer (yes/no)
2. How confident?       → Intensity of feedback
3. How much time?       → Tempo of feedback
4. What should I do?    → Direction (cross/wait)
5. What changed?        → Transition cues

Each layer builds on the previous.
Never skip to layer 3 without establishing layer 1.

The Verdict

OKO is the most important design case study that most designers will never think to examine, because it challenges the fundamental assumption of the discipline: that design is visual. OKO’s core users will never see its app icon, never admire its color palette, never read its typography. The entire design challenge is haptic, auditory, and temporal. And yet, the design rigor required is at least as demanding as any visual interface, because the stakes are a person’s physical safety at a street crossing.

The lesson OKO teaches is that designing for the most constrained users produces the most universal insights. Haptic confirmation patterns, audio state communication, binary-before-nuance information hierarchy, redundant feedback channels, and fail-silent error handling are principles that improve every app, not just accessibility apps. When you design for someone who cannot see your screen, you discover what your interface actually communicates versus what it merely displays.

Best for learning: How to design non-visual interactions with the same rigor as visual ones, and how safety-critical constraints produce design principles (fail-silent, redundant channels, binary-first) that elevate all software.

Frequently Asked Questions

How does OKO detect crosswalk signals without an internet connection?

OKO uses on-device machine learning through Apple’s Core ML framework to analyze the iPhone camera feed in real-time. The model is trained to recognize standard pedestrian signal states (walk hand, walking figure, countdown timer) and runs entirely on the device’s Neural Engine. No network connection is required, which is critical for a safety tool that must work in areas with poor cellular coverage.

Why does OKO use silence when it cannot detect a signal?

In safety-critical design, false confidence is more dangerous than acknowledged uncertainty. If OKO vibrated “walk” when it was actually uncertain, a user could step into traffic. By choosing silence for uncertain states, OKO communicates “I don’t know” clearly, prompting the user to reposition their phone or use other cues (asking a nearby pedestrian, listening for traffic patterns) rather than trusting incorrect information.

How do OKO’s haptic patterns work for users with reduced tactile sensitivity?

OKO pairs haptic feedback with audio cues redundantly, so if a user cannot feel the vibration (due to neuropathy, thick gloves, or a phone case), the audio channel still carries the full message. The app also allows users to adjust haptic intensity and audio volume independently. The dual-channel approach ensures that losing one feedback method does not compromise safety.

What can non-accessibility apps learn from OKO’s design?

OKO demonstrates that haptic feedback can carry semantic meaning beyond “something happened.” Different vibration patterns for different states (success, error, warning, progress) create an unconscious vocabulary that makes any app more usable. Additionally, OKO’s fail-silent principle (no feedback is better than wrong feedback), binary-first information hierarchy, and redundant feedback channels are applicable to any interface where user trust matters.

Why did OKO win the Apple Design Award for Inclusivity?

OKO addressed a genuine civic infrastructure gap: the majority of intersections worldwide lack accessible pedestrian signals. Rather than advocating for infrastructure changes (a decades-long process), OKO used the iPhone’s existing camera and haptic hardware to create an immediate solution. The award recognized both the technical achievement and the design philosophy of building technology that serves users who are typically invisible to the design industry.

Resources

• App Store: OKO - Crosswalk
• Developer: AYES (accessibility-focused development studio)
• Apple Design Awards 2024: Inclusivity category winner
• Core ML: Apple’s on-device machine learning framework that enables real-time signal detection