Context

In August 2015, my research group (The Communications Design Group or CDG) had a game jam — an event where participants create games together over the course of a few days. The theme was to make hybrid physical/digital games using a prototype research system Bret Victor and Robert Ochshorn had made called Hypercard in the World. This system was like an operating system for an entire room — it connected cameras, projectors, computers, databases, and laser pointers throughout the lab to let people write programs that would magically add projected graphics and interactivity to physical objects. The point of the jam was to see what playful things you could make with this kind of system. We ended up making more than a dozen new and diverse games.

I made Laser Socks, a game about jumping around and shooting a laser pointer at an opponent's feet. It was fun, ridiculous, and simple to make. In some ways, Laser Socks became one of the highlight demonstrations of what could be done if there was a medium of expression that integrated dynamic computational elements into the physical world.

There were several interesting things that my lab members and I noticed about the process of making and playing Laser Socks that are worth explaining — I've tried to capture them below.

Insights

I could throw together a new type of game the world had never seen before in an afternoon.

This isn't to say that I'm some kind of genius, but rather that once the conceptual possibility of making a physical/digital game became available and a technical system lowered the barriers to making it, I could make something brand new and pretty fun! In our research group we viewed this as a positive sign that if many other people were given the same opportunities, a huge wave of creativity in this new expressive medium could be unleashed, and not just for games but across all kinds of human activities.

Laser Socks was made of posterboard, trash bags, and tape.

Since the cameras, projectors, and computers were already built into our lab space, all I had to do was use some stuff lying around in our lab to make the game. There is something magical and very humane about being able to use physical materials to make a "UI". Compare this to making a computer game: I would have had to laboriously code the appearance of each game element. When working in the dynamic medium the physical world is its own renderer.

Laser Socks was basically two lines of code.

Conceptually the game is extremely simple: "when a laser is detected in a play area, increase the nearest health meter". That's it. The Hypercard in the World system detected lasers and did projection mapping (which was only a few pages of code anyway), so really all I had do was draw a blue rectangle for the health meter. It still amazes me that there was so much fun and richness in such simple programming.

Laser Socks was multiplayer by default.

Though it wasn't given as an explicit instruction at the beginning of the game jam, all the games we made including Laser Socks were multiplayer. It seems that when making games in the physical world the natural impulse is to make them social. Compare this to current computer interfaces, which almost all assume a single user in their own private world.

In fact, I designed Laser Socks for two players, but much to my surprise we discovered that the game supported 2 vs. 2 or even 3 vs. 3 games with no modifications. The game's code was just looking for lasers — it didn't care how many people there were. Here is a 3D video of the first 2 vs. 2 game:

For comparison, imagine trying to modify a video game to support more players. Even if you had access to the source code, it would probably be quite difficult. Since the game took place in the physical world with pretty "dumb" code, it could easily support more social experiences.

Additionally, in January 2016 we had a lab anniversary party. We invited around 200 people into our lab space and had several demonstrations around the room all running on the Hypercard in the World system, including Laser Socks, which turned out to be a great party game. In contrast to virtual reality or augmented reality systems, these demos were naturally integrated into the party atmosphere.

Improvisation is encouraged in the physical world.

Not only were we able to add more players to Laser Socks with no problem, we were also able to add game modes or rules on a whim. Since many parts of the game are socially-mediated instead of mediated by a computer, they can be modified just by having the players agree to them. There is little or no change in code required to experiment with these new ideas.

For example, when testing the first game of Laser Socks we only had one pair of white socks so Michael Nagle and I each wore a single sock and hopped around on one foot (it was pretty hard!). This was just an idea we had and could try immediately. There have also been games where each person holds two laser pointers, one in each hand, or games where players kneel down to try to block a person's lasers.

In the video below, we can see this kind of improvisation in action — Bret "cheats" after he's technically lost by throwing a sock on Michael's game pad. Using that idea as inspiration, I can imagine a game mode where spectators are allowed to throw socks on either game area as an added challenge. This idea could be tried immediately, no code needed.

It has been a dream of many computer visionaries that people without an extensive technical background can create, adapt, and modify computer programs for their own particular uses. Somehow this dream hasn't come to fruition yet, but maybe with a type of computing integrated into the social world there is more possibility for this kind of authorship.

We can exploit the properties of the physical world.

The code for Hypercard in the World, the system that powered Laser Socks, was not fancy. It didn't use machine learning or advanced computer vision — it just looked for laser dots in certain areas. The reason Laser Socks worked is because a laser appears very dim when it bounces off black material and much brighter when it bounces off white material. This is why players needed to wear white socks while playing Laser Socks — the system would not detect the laser unless it bounced off bright material. It's wonderful that we could use this basic property of color and physics to our advantage instead of trying to make some advanced algorithm for sock detection which is out range for most people.

The Dream

In the 1960s, when computers took up entire rooms and academic researchers had to sign up for time slots to use them, some MIT students invented arguably the first real-time computer game, dubbed Spacewar. As Stewart Brand wrote later in Spacewar: Fanatic Life and Symbolic Death Among the Computer Bums, the game functioned as a harbinger of some of the new possibilities the computing medium might bring:

Yet Spacewar, if anyone cared to notice, was a flawless crystal ball of things to come in computer science and computer use:

1. It was intensely interactive in real time with the computer.
2. It encouraged new programming by the user.
3. It bonded human and machine through a responsive broadband interface of live graphics display.
4. It served primarily as a communication device between humans.
5. It was a game.
6. It functioned best on, stand-alone equipment (and diarupted multiple-user equipment).
7. It served human interest, not machine. (Spacewar is trivial to a computer.)
8. It was delightful.

Similarly, Laser Socks is a fun demo by itself, but what we as researchers hope it points toward is a new type of computing that instead of isolating humans in artificial digital worlds, provides a medium of expression that is continuous and integrated with our physical and social worlds. It's a type of computing that's hard to imagine for most of us now, but may one day be as much a part of our lives as laptops, mobile phones, or even the written word are now — an entirely new medium that becomes invisible to us like water is for fish. Laser Socks is just one small glimmer in this vast ocean of possibility.

The researchers at Dynamicland have been trying to simulate such a future to see what it would be like. Using dozens of prototypes made in earlier systems as examples, the group has developed a more advanced system dubbed "Realtalk" to support their continuing experiments. They've installed this system into their entire lab space and are beginning to open the space to the wider community. They're hoping to make the space into a new type of institution that is somewhere between a library, makerspace, lecture hall, and research lab, all centered around developing a new type of media the world has never seen before. Best luck to the team.

✨   🌙
      
🚀   ✨