How to Build a Mixed Reality Prototype: Agile Software Development

Prototypes are a powerful way to validate hypotheses in product development. They provide an exploration sandbox that serves as a bridge between a vision of a product and an actual finalized product. A prototype enables a team to:

• Discover and validate core mechanics and features.
• Validate user value before implementing the full project.
• Assess the business potential of an idea before investing in the development of a full-featured product.

But is it always necessary to build a prototype? Not really. In software development, there are two primary methodologies when it comes to the process of developing digital products. These are:

• Agile, which focuses on prototyping and evolving a product.
• Waterfall, which begins with a thorough design phase followed by linear execution.

There isn't one methodology that's inherently better than the other; it really depends on the specific characteristics of each project. At Treeview, we work with both methodologies, depending on the specifics of each project scenario.

🆕 Starting Point

In our recent post: Finding Your Mixed Reality Idea: A Guide to Idea Generation, we showcased an example of creating an idea for a spatial computing physics education app. The product description came to:

Now, we’ll take an agile approach and create a prototype to further explore this idea. We’re going to organize the execution in three phases:

1. Phase 1: Explore mechanics in a controlled development environment.
2. Phase 2: Explore mechanics in mixed reality with user mechanics.
3. Phase 3: Create a feature-complete prototype.

The key priorities in this prototyping phase are:

• Prioritize agility, which is the flexibility to pivot based on learnings.
• Build on tangible insights by exploring the prototype from a user's perspective.
• Emphasize the use of abstract placeholders. These are visual components designed to represent what a final product would look like.

⛵ Phase 1: Exploring Mechanics in Unity Editor

We began by exploring different mechanics in the Unity Editor, guided by the high-level vision and core mechanic hypotheses of the application we were building. These experiments were coded as sandbox tools, enabling us to continue iterating and building systems based on the outcomes of these experiments.

Our initial tinkering provided insights that shaped the direction and high level product decisions. Based on initial learning guided by trail-and-error we implemented the full set of mechanics to cover the full vision for the prototype.

After finalizing the core system mechanics, we focused on user mechanics, enabling the full functionality of the app through abstract placeholders all inside the Unity Editor.

🥽 Phase 2: Exploring Mechanics in Mixed Reality

With a complete set of mechanics developed and interacted with via the Unity Editor, our focus shifted to start building the mixed reality components. We started to develop and deploy the app on Meta Quest, implementing user mechanics, and experimenting with the prototype as if it were an early-stage product.

🧑‍💻 Building for Meta Quest

Meta’s Presence Platform SDK is the most advanced and mature mixed reality development platform for AR/VR software developers.

The Presence Platform features we would need for the project are:

1. 👁️ Passthrough API

Meta’s Quest passthrough API uses the device camera feeds, allowing apps to merge the physical world with virtual content. The Quest 3 and Quest Pro provide color passthrough, while the Quest 2 offers black and white.

This key capability enables us to create highly realistic mixed reality experiences where users feel that the digital objects and physics are part of a cohesive reality.

2. 🏠 Scene SDK

The Meta Quest Scene SDK captures the user's environment in detail, including the floor, walls, ceiling, and furniture. This data can then be used to make virtual objects responsive to real-world elements. A key element in creating mixed reality experiences is to really make the environment part of the core gameplay of the application.

3. 🤏 Interaction SDK

The Interaction SDK enables developers to use hand tracking as the primary interaction system between the user and the virtual content. These natural input methods allow us to develop interactions that are analogous to physical world interactions, like grabbing objects and moving them. This results in a more natural and intuitive mixed reality experience.

🚲 Phase 3: Create Feature Complete Prototype

With the core mixed reality functions in place, we can assemble a full prototype to test our vision of Newton’s Room. This prototype focuses on testing a feature-complete vision of the project while using abstract placeholder visual assets.

We now have a fully working prototype of Newton’s Room that includes:

• A complete, usable prototype with all app mechanics in mixed reality.
• 15 levels of the theory learning module.
• 12 levels of problem-solving modules.
• Abstract placeholders representing all app assets.

This prototype enables us to preview what a final product could feel like, represented by abstract placeholders. There are also key value outcomes from this prototype phase which include:

• We identified and validated the key gameplay modes.
• We identified and validated key user mechanics.
• We designed and validated puzzles that require users to understand and apply Newton’s three laws of motion to succeed.

⏭️ Next Steps

Now that we have our working prototype, the next stage is to design a product around this preview we’ve created. This next phase will focus on:

1. Designing and implementing the Art Benchmark for the project.
2. Polishing and iterating the core features based on user feedback.
3. Validating and iterating the learning modules with subject matter experts.

Continue reading at: Best Practices in Mixed Reality Design: An In-Depth Guide

Until next time!

Horacio Torrendell