In this week’s class, I decided to dive into what happens when the gamer player first starts the app. We talked about Angry Birds as the model for a good game, and tried to do what they did.

Angry Birds starts as with a short video, so we decided to do a short audio tutorial. We would tackle what the tutorial would say in a future class.

One child asked if she could control the game by talking to the iPad, the same way she talks to SIRI. It won’t work. Not only does Apple prohibit app developers from accessing SIRI’s features, but often SIRI (and other speech recognition systems) don’t understand everything you say. Trying to use SIRI to play again would be a very frustrating.

First we would need a way for someone to pause the game by making some gesture on the iPad – tap the screen, swipe the screen – and be told how to operate the game. We made a list of all possible gestures that the iPad allows, and talked about all of the actions a game player would want to do:

Each student would come up to the whiteboard, explain the gesture and draw out exactly what the game player would see. By repeating this for every gesture, the students began to understand the limitations of gestures, which gestures were easy and hard, and what the game playing activity would feel like. We rejected a lot of gestures that were ridiculous (such as tap 5 times and swipe to restart the game).

Some gestures would cause really big things to happen – such as restarting the game – and in the visual world, the game would confirm that’s the action you really meant; letting you CANCEL if you accidentally tapped RESTART. To emulate this on a black screen, some of the gestures would ask if that’s what you really wanted to do. We studied two alternatives:

• repeating that gesture for YES, and some other action for CANCEL
• one action to always mean “confirm yes” (such as tapping the screen once with one finger) and one action to always mean “cancel” (such as tapping with 3 fingers once)

We created the concept of an audio menu, analogous to a screen menu, where the options would be read to you (restart game, change settings, start tutorial) along with the gestures that would invoke that menu option.

They tried on these gestures on their iPhone, pretending the game was running, and after some “real-world” testing and revising, we had a pretty good set of gestures to control the game.

One of the children asked an obvious question: “What if the speaker is off?”. We would solve that by showing the words on the screen “Use your headphones to play this game”, and we would say that when the game started. If the game player is blind, they would always have their speakers on. If the game player is sighted, they would read those words, and plug in their headphones.

Now that we knew more or less how the game would operate, it was time for some software development.

I looked at several game development systems, such as Corona, but decided that since the screen would be blank, and only speakers are required, most of the features of those development systems would be useless; I would built the app the way most apps are created on iOS – using Apple’s xCode.

I started with just a straight road with a virtual fence on both sides of the road, and wrote a small program that would move the car (a small red box) on the screen, and it would respond to the movements of the iPad – turning left and right, and going faster and slower. I then added an audio library to it, so as the car got closer to the left fence, the sounds in the left speaker got louder; likewise with the right fence.

This was great for explaining the concepts of the apps to the students, and let them experience the game while it was being developed. In the next class, I had them try to drive on the road, using only their ears:

With that, we were able to design other levels. In level 3, for example, we put a chicken in the middle of the road, and the game player would have to avoid hitting the chicken. The other question was: what would happen when the car hit the fence. The choice was to lose the level, and have to restart the level, or bounce the car off of the fence, and put it somewhere. We chose the latter, since it made the game easier – the car would “bounce” to the middle of the road.

The students wanted prizes – like a pot of gold – on the road as well, and if you drive your car into a prize, you would get points. We don’t know what to do with those points yet, but that wasn’t important to the students. We started laying out each level: shape of the road, number of animals and number of prizes:

By the time we finished the class, we had all agreed that in each level, the road would be different and the items on the road would be either prizes (to be hit) or animals (to be avoided), and each item would make a sound (that would identify the item).