(This is a lightly edited script for a talk given at UX Camp 2023 earlier this year)
I have a confession to make: I play way too many video games.
In fact, I play so much that I feel the need to somehow justify that. Like saying that I do this for research. That it actually helps me understand UX or product design better in some way.
This type of justification is dangerous, as it easily spirals out of control. Because the next moment I'm thinking about the big questions: What is product design or UX, really? What are we doing? Like, in a broader sense?
Maybe you know this situation. A relative – mom, dad, your spouse – is asking: "So... what is it that you actually do?".
You could say, "Hey, I'm working on this app so that everyone can understand what they are supposed to do" or "I make sure that people using this website actually get the information they need".
This is all valid. (They will ask again in a year anyway.)
But fundamentally, I think what we designers do is arrange the Lego blocks in such a way that if people see what we have built, they immediately know that yes, this is meant for driving. That you can push it so that it moves forward. That it will boost to top speeds thanks to the rocket thrusters attached to the back. We make what were just a few blocks of Lego moments before recognizable. We make the abstract understandable.
This is a skill that comes in handy, especially if we, as a society, encounter new things. Like new technology. Or new forms of expression. New cultural values.
So to be good at this job, I think you always have to be on the lookout for these kinds of shifts. And one of the best ways to do this, in my opinion, is through play.
Play is society's way of prototyping cultural shifts.
What fascinates me about playing games is their cultural significance as a future sense making tool. This goes contrary to the common perception of games as a waste of time or just for children. In reality, I think play is a crucial instrument for figuring out new developments.
In other words: Play is society prototyping cultural shifts.
The reason for that is pretty simple: Play allows us to pretend, to explore what-if scenarios, to try things we would otherwise never do.
Or as Gregory Bateson puts it:
(...) play is an interchange of massages within a contextual frame, which carries the label "all messages within this frame are not literally true"
- If I kill you in Fortnite, I don't really mean you any harm. It's just a game! But it is an outlet for my competitive side.
- If I play a dumb Orc in Dungeons and Dragons it allows me to act in ways I could never in real life - and experience consequences I wouldn't experience either.
- If I join a guild in World of Warcraft, I will get to work towards a common goal with people I would normally not engage with.
The framing of "this is not true" can be liberating. And we learn things that can be applied to non-play situations. So while playing, we are, in a way, "prototyping" behaviors that, if taken together, form culture.
This comes in handy all the time – but especially if we experience big cultural shifts that confront us with new problems that we, as a society, have no answers to.
So it's not only that we figure out new culture through play but also how to deal with the fallout of these shifts: Play is society prototyping responses to cultural shifts.
Play is society prototyping responses to cultural shifts.
Ok, but what do I mean by cultural shifts?
To explore this, let's go back in time roughly 5000 years and talk about the invention of writing.
Now, let me tell you: This was a big blow for society! It sucked!
Previously, you could trust that most of the things that happened would be forgotten over time: The mercy of our leaky memories meant that maybe Tom would stop remembering at some point that I owe him a beer! And if communication happened, it happened in the here and now - only between the people who were actually present at that moment.
But with the invention of writing, you were able to – you know – write things down. We obviously take this for granted now. We don't even really think about it anymore. But it was revolutionary!
Passing on knowledge became way easier - your ancestors could simply read all of your "big thoughts" and didn't have to rely on oral traditions. Planning big projects was suddenly possible. Suddenly, you did not have to personally attend every meeting. Instead, you could write down some instructions for others to follow. But worst of all: Tom could make a list of all the beers that I ow him. The birth of accounting.
In short: All things that make complex organization possible became feasible with the invention of writing.
How do you deal with such an increase in complexity?
I like to imagine that people actually freaked out a bit. Just like people are freaking out right now because they simply can not handle the internet.
Now, I would argue that games played a crucial role (pun intended) in teaching people how to respond to that increase in complexity.
Take this game shown above, called Senet, for example. The exact rules are lost – but we have a good enough understanding, that we can at least talk about it in general.
Developed some 5000 years ago, this game actually feels quite modern. You move your pieces over the squares by rolling some dice. You can block the movement of your opponent, obstructing their plans. Some of the squares even let you perform a "special action" if you move your piece over it.
I mean, it already sounds better than "Mensch Ärgere Dich Nicht", if you ask me!
This game had some important lessons for the people playing it. Like: Complexity is reduced when breaking it down into manageable squares. Or, that when you define a place for all things, everything becomes more manageable and predictable.
Just like in the game, where certain pieces could only go to certain squares, for example, all objects, animals and people have their own defined roles.
This would philosophically become a defining characteristic of the epoch: Everything and everyone had their place; there was a known order. This became known as "Teleology", based on the Greek word "telos" which means goal/ aim.
An early tool to make complexity bearable.
In play we discover what's next
At this point you are either asking yourself why on earth I'm talking about this – or you are probably already asleep.
I get it.
I'm just talking about this, because I think this thing – the computer – messed us up pretty bad. Just like writing messed us up all these thousand years before.
And we are just now learning how to deal with it.
And yet, every day I go to work with the expectation that I will develop and design new ways of interacting with a computer. To allow other people to make sense of it. People trust my expertise, believing that I know what I'm doing.
But I don't.
I think we all do not really understand what this thing is doing, how to best interact with it, or how to deal with all the challenges that it brings about.
But if you believed all the things that I was telling you earlier, playing video games would be a great place to start figuring this out. Because just as people playing Senet could learn ways of dealing with complexity, maybe we can too. By playing the games of our time.
Because, as I said earlier: In play we discover what's next.
So here are three things that I discovered while playing. Things that could maybe teach us how to best deal with the computer. To make us better at our jobs, especially when it comes to making the abstract understandable to regular humans.
1. The best manual is an invisible one.
Video games deal with a lot of complex systems and, oftentimes, even new input methods. So developers were forced to get good at explaining all of this to users.
In contrast, "onboarding" as we know it in the app-focused UX world, is still very much rooted in the old ways of writing and printing manuals.
This shows up when I type in "onboarding" in Dribble search. Colorful, creative screens to be shown right after the app launches. They are fun to look at!
And the idea behind screens like these is also quite rational: A user reads through them and, afterward, knows how the app works and what to expect. They are now equipped with the basic knowledge to fulfill their task.
In reality, the user sees this before they even have any idea what the actual app looks like. They have no context. All they have done so far is to tap on an icon on their screen! And what they are presented with at the start is an obstacle blocking their way to whatever goal they had in mind.
It is also something that just takes the computer as a medium not serious.
One of the defining characteristics of the technology is that it's _always watching_, always computing in the background, and constantly responding to new inputs.
Inputs like, "What is a user doing?" and "How could I help them in this specific context?".
Just like what you can find in modern games like "Zelda - Tears of the Kingdom".
After a brief story introduction, players of Tears of the Kingdom get to experience what's in store for them right away - albeit in a safe "learning playground".
As a player you don't realize at first is that this area - all of these floating islands - is basically a sandbox that introduces all the major mechanics of the game. By interacting with it and spending time in this place, you will eventually learn everything that you need to know to beat the game. There are some goals provided—like, here are four shrines to visit—but no one stops you in your path and tells you how to get there exactly.
And the things you learn while trying to reach these shrines is not simple stuff, like "press A to jump"; they're about how elements like fire, water, ice, etc. interact with each other. How to reverse time for an object. How to build things with the objects lying around. And how to cook food with the ingredients you find.
There is basically a full physics system behind the game, and after completing this first area you will know how to interact with it and how it affects you as a player.
It's really complex. But at no time does it feel like reading a manual. You were just playing the game. The learning - the onboarding - happens almost accidentally.
To me this is the holy grail of how to design an onboarding experience: Input and action blending together naturally in a safe space that allows you to make mistakes and learn from them.
In pedagogy and the context of teaching there are special terms for this, like kinesthetic or action-oriented learning. But I think that the computer is especially suited to provide such learning experiences.
If we understand our users' motivations at least to some extent, we should design our systems in a way that they are inviting to be explored. And to gently guide and help the user when their specific input or actions require it. Not before, and never without context.
2. Trusting a human is difficult. Trusting a computer is too.
The second thing I want to share is about trust.
Our society has long breached the limits of where you'd have at least a vague sense for everyone around you and how trustworthy they are. In a small village, you would hear stories about this and that person, see firsthand how they behaved at the town's festival, etc.
But once our little villages became towns, we had to come up with ways to allow a person to trust another unknown person. Because without trust, there is no collaboration, no progress, and no commerce.
So trust was externalized. Outsourced to organizations like the church and even bestowed upon people by the magic of certificates and diplomas.
But now, we have to deal not only with trust between humans, or humans and organizations, but also trust between humans and machines.
The big reason trust regarding computers is suddenly on everyones mind again, is this: ChatGTP.
How do you trust something that is called a "large language model"? A computational model so complex that even the creators don't really know how it produces the exact results it shows you? How do you know if it speaks the truth? Or at least tries its best?
In games, we are already confronted with this problem a lot. Here, AI is actually given form in something that we gamers call "a NPC".
NPC: Non Playable Character – an AI controlled game character that is often relevant to the story of the game and tasked with helping the player in various situations
In games like Bioshock, you often encounter characters that will join you on your journey. Like Elizabeth in the picture above. You cannot control them, so they are called "non playable characters".
For a long time, players dreaded moments like these. NPCs were just too dumb. They would walk the wrong way. Get into the enemy's line of fire. Make too much noise in a situation that requires stealth.
But by improving the NPCs AI and applying some tricks like always beaming them right behind you when you're not looking, players' feelings started to change. These NPCs are now often regarded as a boon.
They help out in difficult combat situations or, in the latest trend, hint at solutions when they notice you struggle with a puzzle.
As a player, you learn to trust them just like you would learn to trust another human: By observing their behavior, seeing how consistent they are in their responses, and by connecting with them emotionally.
Yes, you can never be 100% sure. But that is also true for other humans.
I believe that we will think about NPC and how to establish trust a lot more in the coming years. The moment we start to recruit AI-Teammates into our Figma-Files or our Excel-Sheets, we need to make sure that they are designed in a way that allows us to establish trust.
Just as we learned to trust Elisabeth not to hit us with that book again.
3. Low resolution does not mean low imagination.
I often think about how we construct what we perceive as "normal" for us.
Recently I have read this article about how computer science teacher struggle with the fact that many students have no clear understanding of a computer's file system. For them, everything is accessed through the search function and not by going through a hierarchical folder structure.
And it makes sense, right? If you grew up with Google as the gateway to all of earth knowledge, all accessible only through a search box, why would you expect a computer to work any different?
But for a professor who had to work with filing cabinets in their youth, sorting things in neat boxes is just as natural.
We take what we have learned and shape the world accordingly.
So what will a generation that grew up with Minecraft expect from their tools?
Minecraft is, as you can see, very low-res. Everything is literally a pixelated block.
And yet, people build amazing things in that game. There is so much creativity on display that there is now a YouTube genre that exists purely to show off other people's amazing builds.
The other thing to know about this game is that it's so easy and fun to play together. That is actually one of the main drives for a lot of people: It's just an endlessly customizable space to hang out with your friends.
It is very funny to me when people start to hype up the Metaverse as this crazy revolutionary thing when the truth is that video game players have been living in some form of it for a long time.
And it's not that Minecraft—just as one example—is a niche thing. It sold over 238 million copies, it has nearly 140 million monthly active players, and Microsoft paid 2.5 billion dollars to acquire it!
Now that we're all excited to build for Apple's Vision Pro Headset, we need to make sure to remember that it is about giving people the freedom to create a space for themselves and their friends. That the expectation is that everything is customizable and multiplayer, but not that it needs to look super shiny.
---
I think I did it! I justified spending all these hours playing video games!
And maybe I've convinced you to give games another look if you haven't already. Maybe there is something to discover and learn for you, too.
And if you are a player of video games, I would love to hear what you noticed and learned while playing.