Powen Yao 姚博文

Powen Yao 姚博文

Computer Science Ph.D. at the intersection of spatial user interface, games, and innovation

About Powen

My specialty is best described by:

Spatial User Interface

My Computer Science PhD dissertation is “Design Lenses for 3D User Interfaces in Extended Reality”

Concept Image for Virtual Equipment System.
An user is shown interacting with Virtual Headphones to adjust audio settings. Other Virtual Equipment shown include utility belt, bracer, `and` microphone

What started as a desire to “get more buttons for XR interactions” has since turned into a framework for “bringing fantasy and science fiction into reality.” 1

The theoretical portion includes:

  • Design Lenses - Hyperphysical User Interface, Whole-Body Interaction, and Extramdimensional Space
  • Multi-dimensional Taxonomy of Gestures - classification improves understanding and allows generation of novel techniques in under-utilized areas
  • Spatial Interaction Model - for use in combination with design lenses and taxonomy to create new interaction techniques

The practical portion includes:

  • School of Spatial Sorcery - A sandbox containing a series of exploratory prototypes for novel 3D UI/UX
  • Virtual Equipment System
  • Extradimensional Space Storage

Together, they demonstrate a path forward to design space for Extended Reality User Interface and User Experience beyond the traditional 2D UI/UX. “Any sufficiently advanced technology is indistinguishable from magic,”1 and I’m here to push the boundary.

Games

Arkology Concept Poster for the western market.
A commander holding holographic planet surrounded by spaceships in her hand.

My passion for games, building games, and my experiences teaching games in various game project courses have provided me with a wealth of experience. Other than art, I have dabbled in most areas of game creation, such as engineering, design, and production. I have led many teams to create games, often serving as the team lead, design lead, engineer lead, or all of the above.

My role at USC has evolved as the game industry and the USC game program changed and evolved. From a computer science student to restructuring and lecturing for a class of 90, I have been, learned, and taught in many different roles.

In particular, my time teaching and working with students of different backgrounds and experience levels has helped me grow tremendously as a game developer.

I have mentored and guided students to take some wild combinations of genres, themes, and ideas to fruition.

Consider these student games:

Innovation

User accessing spells through the use of Cloak Space
User accessing spells through the use of Sleeve Space. Both techniques utilize the extradimensional space anchored below the user's forearm, but can retrieve different items based on the direction of the entry

Innovation is an often thrown-around term that’s highly sought after by many.

  • In industry, it’s in meetings, marketing, branding, etc.
  • In academia, scholars need to examine prior works and produce novel works.
  • In game class at USC, we ask students to create games with innovative mechanics.

But how does one innovate? When is something innovative and when is it just different, but for the worse?

Often, an innovative work may reference some source of inspiration, such as the Go-Go Gadget of Inspector Gadget inspiring the go-go interaction technique. What if all your inspiration has been done by someone else? How can you take it further? Alternatively, how do we encourage those without inspiration to innovate?

To innovate is a recurring challenge in my teaching, PhD, and personal work. Bolstered by my academic background, my approach and weapon of choice is to truly understand the problem:

  • Break the problem down to its fundamental components
  • Seek commonalities and differences by use of qualitative analytical techniques (e.g., Grounded Theory analysis),
  • Create design tools (e.g., design lenses, taxonomy, architecture, systems) to aid the ideation and exploration process
  • Develop iteratively
  • Evaluate and improve the process

Breaking down a genre and creating a game mechanics matrix can do wonders for identifying room for innovation. This was incorporated into the Mobile Game class at the University of Southern California, aimed at general computer science students not specialized in games. I created lectures and exercises to help them catch up and innovate.

See the lectures on innovation at 2019-2023-teaching

Footnote

  1. From Arthur C. Clarke’s Three Laws ↩ ↩2