2024 Experience Design Carnegie Mellon ETC

AR Mixtory

A mixed reality experience that memorializes Pittsburgh's historic Chinatown — bringing a century of forgotten history back to life through browser-based augmented reality.

My role — Experience Designer. I shaped the narrative, interactions, and emotional arc of the 8–10 minute experience; led the in-experience UI; ran user tests with tourists, immigrants, and students; and worked with historians to keep the recreated world accurate.

Role

Experience Designer

Team

6 Carnegie Mellon Students

Client

WiSELab @ Carnegie Mellon University

Platform

ARENA (browser-based AR/VR/XR)

Duration

14 weeks (Jan 2024 - May 2024)

Tools

Unity, ARENA, Figma, Blender, ElevenLabs

01 · Overview

Preserving a lost neighborhood through mixed reality

Pittsburgh's Chinatown once thrived in the heart of downtown — a vibrant community of immigrants, merchants, and families. By the 1960s, urban redevelopment had erased nearly every trace of it. Today, most Pittsburghers don't know it ever existed.

AR Mixtory is a mixed reality experience that brings this forgotten history back to life. Built on Carnegie Mellon's ARENA platform, it lets users walk through a recreated 1890s Chinatown, interact with historical characters, and uncover the story of a community that shaped the city.

The experience is fully browser-based — no app downloads, no headsets required. Just open a link and step into history.

Visit Project Website ↗

Two objectives, one experience.

Build a location-based mixed reality experience on ARENA that both showcases the platform's capabilities and creates genuine emotional engagement with a forgotten neighborhood — designed for school-district students ages 6–15.

Educate Users About ARENA
Demonstrate storytelling, multi-user interaction, and cross-device accessibility through a compelling experience
Preserve Lost History
Make users care about Pittsburgh's Chinatown — a neighborhood erased by urban redevelopment in the 1960s
Browser-Based Access
No app downloads, no headsets required — open a link and step into history on any device
Engage Students Ages 6–15
Target audience is school-district learners spanning elementary through early high school — the experience must hold attention for both young kids and older teens

Emotional design on an evolving platform.

The core challenge wasn't technical — it was emotional. Passive observation wouldn't be enough. We needed users to feel like participants, not spectators.

Emotional Connection
Creating genuine empathy for people and places from over a century ago
Platform Constraints
ARENA was still evolving — browser-based XR has real performance limitations
Zero VR Experience
Target users had never used AR/VR — the experience needed to be instantly intuitive

Presented at ETC Festival.

Target audiences reported feeling a genuine emotional connection to the historical characters — validating our agency-driven storytelling approach.

3 Eras

1890s · 1930s · 1960s

8–10 Min Experience
Optimized duration for meaningful engagement without fatigue
14 Week Sprint
From historical research to polished demo in one semester
Cross-Platform
Works on phones, tablets, desktops, and VR headsets via browser

02 · Process

14 weeks, from mission to demo

Week 1

Identify the Mission

Unpacked the client's dual goal — showcase ARENA's capabilities and preserve Pittsburgh's lost Chinatown.

Week 2 — 3

Research & Discovery

Historical research, ARENA onboarding, persona development, and a scan of comparable MR experiences.

Week 4 — 6

Narrative Design & Prototyping

Story structure and interaction design; first playable with a basic environment and character beats.

Week 7 — 10

Production & Iteration

3D environments from Smithsonian assets and photogrammetry, ElevenLabs narration, multi-user support.

Week 11 — 14

Polish & Presentation

User testing with target audiences, visual polish, performance optimization, and the ETC Festival demo.

03 · Research

Grounding the story in primary sources

A neighborhood that no longer exists can't be designed from assumptions. Before any interaction was sketched, we rebuilt Chinatown from two primary sources — the people who remembered it, and the records that documented it.

Primary · Oral history

Interview with Shirley Yee

We interviewed Shirley Yee — a Carnegie Mellon professor and daughter of Yuen Yee, the last informal mayor of Pittsburgh's Chinatown — for first-hand accounts of the community and access to her family's photo collection.

Archival · Field research

The Heinz History Center

We studied historical documentation at the Heinz History Center to map the real street layout and the shops that filled Chinatown across its different eras — the blueprint for every environment we rebuilt.

Photograph from Shirley Yee's family collection

From Shirley Yee's family collection — the primary-source photographs and documents behind our reconstruction.

The takeaway — these sources gave us what fiction can't fake: real people to anchor the story in, and an accurate map of a place we had to rebuild from nothing. They set up the two principles that shaped everything next — emotional anchoring and temporal layering.

04 · Experience Design

Immersive storytelling through active participation

Rather than creating a museum-style walkthrough, we designed the experience as an exploration tour with storylines. Users don't just observe — they play a role. They arrive as two brothers, newly immigrated to Pittsburgh's Chinatown in the 1890s, and are given tasks that reveal the neighborhood's history through action.

"We wanted users to feel like they're discovering history, not being lectured about it."

Design Principles

Agency over observation — Every historical fact is revealed through a player action (moving, tapping, exploring), not a wall of text
Emotional anchoring — We used personal stories of individual immigrants rather than abstract historical data
Temporal layering — The experience spans three time periods (1890s, 1930s, 1960s), showing the neighborhood's rise and fall through the same physical space
Zero-friction access — Browser-based, no downloads, works on phone or desktop

Story architecture

The experience is a two-player co-op story — you and your brother arrive as immigrants and build a life in Chinatown across two interactive chapters, before a final chapter confronts you with the present.

Chapter 01 · 1890s

Settling down

The two brothers support each other to put down roots — renting a home in downtown Pittsburgh, transferring money, and decorating the place together. Players tap each other to start interactions and place the materials they need directly in augmented reality.

Chapter 02 · 1930s

Serving the community

Established now, the brothers give back — providing free food to fellow immigrants. The two players communicate to divide the work, grab ingredients, and cook side by side.

Chapter 03 · Present

The present day

The story leaves the past behind. Through sound narration and 2D animation, the final chapter reveals what became of Pittsburgh's Chinatown today — the quiet, deliberate turn the whole experience builds toward.

05 · Technical

Building on ARENA

ARENA is CMU's open-source augmented reality platform that enables browser-based XR experiences without requiring app downloads. This was both our biggest advantage (accessibility) and our biggest constraint (performance limitations).

3D Assets: Sourced from Smithsonian Institution's 3D Digitization Collections, supplemented with custom photogrammetry
Voice Acting: AI-generated narration via ElevenLabs with historically-appropriate accents
Multi-user: Real-time collaborative functionality — multiple users can explore simultaneously
Cross-platform: Works on phones, tablets, desktops, and VR headsets through the browser

Because ARENA is a mixed-reality platform, the experience also had to be location-based — every virtual element is pinned to a real-world position. So we physically built and decorated the room to mirror the digital scene, placing props and markers so the AR layer lined up precisely with the space.

06 · Usability Testing

What playtesting changed

Playtesting ran through every milestone — the client first, then instructors and a subject-matter expert, and finally 21 players in open sessions, with pre- and post-experience surveys measuring each against our goals. Three findings reshaped the design:

We observed

Players only collaborated when they already knew each other — strangers stayed quiet and missed the point of a two-player experience.

We changed

Added a tutorial that makes both players see each other's avatar before starting — so collaboration is the very first thing they do.

We observed

The interface was too obtrusive, and the in-experience text was hard to read.

We changed

Shrank the UI, reworded the prompts, and rebuilt the background for legibility.

We observed

Players wanted to connect the virtual neighborhood to the real one — "how does this map to the actual Pittsburgh Chinatown?"

We changed

Added source citations and descriptive text to the historical photos, and an end video tying them to present-day map imagery.

07 · Outcome

The finished experience, and how it landed

The result is a polished, two-player experience that premiered at the ETC Festival. Across testing, the surprise wasn't just that people enjoyed it — it was the educational pull: players said they learned about Pittsburgh's Chinatown and wanted to learn more, and parents were caught teaching their kids mid-experience. That's the whole goal — making people care about a neighborhood they'd never heard of — landing in real time.

08 · Reflection

What I learned

This project changed how I think about design as preservation. Technology is often used to create new things — but it can also be used to recover what's been lost. The most powerful design decisions weren't about UI patterns or interaction models. They were about empathy — deciding which stories to tell and how to make people feel them.

Working with ARENA also taught me to design within constraints. Browser-based XR can't compete with native apps on raw performance. But by focusing on storytelling quality over graphical fidelity, we created something that felt more meaningful than many polished VR experiences.