Programming a Blindfold Game

I’ve been working on a new game – Blindfold Piano Tiles.  This game will, amongst other things, teach you single octave melodies, such as Mary had a little lamb.

To play the game, you lay the phone in landscape mode on a table, and press one of eight keys.  There are 4 keys on the bottom, numbered 1 to 4, and 4 keys on the top, numbered 5 to 8.  The game tells you which keys to press, such as 3, 2, 1, 2, 3, 3, 3 and then you press each key in sequence.

iPhone piano keys

So here’s the problem.  When you press a key, it should sound the musical note while your finger is pressed and stop sounding when your finger lifts from the screen.  And if the game was saying something when you start playing, it must stop whatever it was saying or sounding, and respond to the key you are pressing.  Without that, the game feels unresponsive.

Here’s one way to program this:  when you press a key, anything that was being spoken stops.  If the game starts saying “Press each of these keys: 3, 2, 1, 2, 3, 3, 3”, as soon as you press the first key, all speaking stops.

When you press the first key, “3”, it stops speaking and starts sounding the musical note that corresponds to key “3”.  However, when you press the second key, “2”, it also stops speaking and starts sounding the musical note that corresponds to the key “2”.  Which means you don’t hear the first musical note.  And if you move through the keys quickly, you won’t hear any of the musical notes, except for the last one.

Another way to program this is to only stop speaking before the first musical note is keyed, and not have the second key affect the sounds of the first key.  That works, but then you encounter the next problem: remember, when you press the key, the musical note starts.  When you lift your finger, the musical note stops.

If you press and lift your finger too quickly, you don’t hear the entire musical note.  For example, if you quickly tap the “3” key, the note is only played for that length of the tap.  A quick tap might be as short as one-tenth of a second, and that’s not long enough to hear the musical note; you actually need about one-third of a second.

So, to program around that problem, you could make each musical note last at least one-third of a second.  But now, if you quickly tap 3 keys, your tapping will be ahead of the notes, so that the first musical note is still playing even though you’ve typed 3 keys.  And if you tapped the wrong key, it’s not clear which of the three keys was wrong.

So, to program around that problem, the game needs to prevent the user from typing faster than one key every one-third of a second.

To solve that, the game now says “Don’t type so fast”, and honks at you.  Of course, you won’t want to listen to the entire sentence of “Don’t type so fast” over and over again, so that the second time you type too fast, you just want to move on, and try again.  Which means the solution to only stop speaking before the first musical note is no longer valid – that sentence can occur whenever you press keys too quickly.

You wouldn’t think that something so simple – sounding a musical note when a key is pressed – would be so hard to program.  These are just some of the complexities of building a game, and why we test the games with so many different visually impaired people before publishing it to the App Store.

Advertisements

Blindfold Games on Podcasts

I would like to thank all of the podcasters and radio hosts who interviewed me over the past few weeks and motivated the community to contact Apple.  Please support them by becoming a regular listener.

podcast logo

Tap each of the podcast names to get more info.

The Blind Side Podcast – Jonathan Mosen

INSIGHT ON disAbility – Michael E. Gerlach

Life After Blindness – Tim Schwartz

Blind Bargains Podcast – J.J. Meddaugh

Blind and Beyond Radio Show – Michael and Lynn Goldner

Typical Confusion Podcast – Jim Holliday

Blindfold Slide Puzzle

About two years, I created Blindfold Tile Puzzle, based on the game 2048.

2048 is played on a 4 by 4 grid, where you combine identical numbers to produce their sum.  Hence, in the above puzzle, with line three reading: 2, 2, 4, 32, you can combine 2 plus 2 to generate a 4, resulting in 4, 4, 32, open-space.  Then you can combine 4 plus 4 to generate an 8, resulting in 8, 32, open-space, open-space.  You win the game when you combine 1024 plus 1024, yielding 2048.   I’ve heard the game called “Candy Crush” for math geeks.

Once Blindfold Tile Puzzle was published, fans starting asking for the sliding tile puzzle known as 15, also called the Boss Puzzle, Game of Fifteen, and Mystic Square.   It’s played on a 4 by 4 grid of numbered square tiles, from 1 to 15, in random order with one tile missing.   The 16th space is empty.  The object of the puzzle is to place the tiles in order by making sliding moves that use the empty space.

slidepuzzle

Originally, I thought this game was too boring, but getting several requests each month changed my mind.  I built the game, and started testing it recently.  Here’s the Wikipedia history:

  • It was invented by Noyes Chapman, a postmaster in Canastota, New York, who is said to have shown friends, as early as 1874.
  • Copies of the improved Fifteen Puzzle made their way to Syracuse, New York, by way of Noyes’ son, Frank, and from there, via sundry connections, to Watch Hill, Rhode Island, and finally to Hartford, Connecticut, where students in the American School for the Deaf started manufacturing the puzzle and, by December 1879, selling them both locally and in Boston, Massachusetts.
  • The game became a craze in the U.S. in February 1880, Canada in March, Europe in April, but that craze had pretty much dissipated by July. Apparently the puzzle was not introduced to Japan until 1889.

Some facts for math geeks:

  • In 1879, two mathematicians proved that half of the grid layouts for the game are unsolvable for the 15 puzzle.  Larger puzzles, such as 5 by 5, do not have this limitation.
  • For the 15 puzzle, the optimal solution takes from 0 to 80 moves.
  • For the 9 puzzle based on a 3 by 3 grid, the puzzle can be solved in less than 31 moves.
  • For the 24 puzzle based on a 5 by 5 grid, the number of possible positions is 7.65 times 10 to the 24th power, which is absurdly large.  In 2011, it was computed that the puzzle could be solved in as little as 152 moves, and as many as 208 moves.

You can download Blindfold Sliding Puzzle here:

https://itunes.apple.com/us/app/blindfold-slide-puzzle/id1276044630?ls=1&mt=8

Blindfold Games, Advocacy and Apple

I’ve been asked for a timeline of what happened with the Blindfold Games, so here’s a summary.

timeline

In late October, I submitted updates for 3 apps: Blindfold Craps, Blindfold Horserace and Blindfold Hopper, to accommodate changes in iOS 11.

A few days later, they were rejected for violating a new section of the Apple App Store Review Guidelines, section 4.3 which reads:

4.3 Spam – Don’t create multiple Bundle IDs of the same app. If your app has different versions for specific locations, sports teams, universities, etc., consider submitting a single app and provide the variations using in-app purchase. Also avoid piling on to a category that is already saturated; the App Store has enough fart, burp, flashlight, and Kama Sutra apps already. Spamming the store may lead to your removal from the Developer Program.

They reason for reject specifically is:

“We noticed that your app appears to be created from a template. Your app provides the same feature set as many of the other apps you’ve submitted to the App Store; it simply varies in content or language.”

I write back:

“There is no commonality between this app and the other apps. Each game is different.  Read the user guide.  The menu format is the same because blind people need a common interface but the functionality is far different.  It would be nice if you evaluated things prior to jumping to conclusions.”

A few days later, Apple responds:

“Thank you for providing this information. We ask that you consolidate your existing apps, as well as any new apps that you submit, as your app provides the same feature set as other apps you’ve submitted to the App Store, only varying slightly in content or language.”

I write back:

“Each app does not provide features of other apps. Each game is unique unto itself.   For example, of the three you just rejected, one is a game where you walk your fingers on the screen to race horses. The second is the casino game craps. The third is a game similar to the video game frogger.  Did you read the user’s guide before giving me the generic response above?”

Two days later, Apple responds:

“Thank you for your response.  An Apple Representative will call you on the number provided within the next 3 to 5 business days from today to discuss your app.”

About 10 days later, I talked with Adam, a reviewer in Apple’s App Group, and Apple’s decision is that unless I merge the 80 Blindfold Games into a handful of apps, they will no longer allow new games to be released or allow updates to be made.  I mentioned that there may be many in the blindness community who will be disappointed.  The following day, I blogged and tweeted about it.

Over the course of the next three days, thousands of people contacted Apple, several podcasters invited me to speak about the issue, and a few advocacy organizations for the blind reached out to their contacts at Apple.

I received an email from a woman who works with Apple Developer relations who has an interest in accessibility.  We talked for a while, and she recommended I submit a formal appeal, which I did.  I also sent to her a dozens of comments that the game fans sent to Apple or posted publicly.

On Friday night, three days after the original call, I heard from Adam again.  He mentioned how he and his team had a chance to explore many of the Blindfold Games over the past few days, and now realizes the intended audience is different from a sighted audience, and the needs of the blindness community must be considered.  We talked about how to improve the situation going forward, and came to a good understanding.

I told him that there are about 50 blind and low vision people worldwide that test each of the games for about two months prior to releasing the game.

I agreed that for new games, we’ll try something new.  If there’s a sensible way to combine it with other new games that does not affect its play or discoverability, I would talk to these testers, and proceed based on their recommendations.

What I’ve learned from this experience is that people in this community – or any community – when working together, can successfully advocate for change.  I also have more respect for Apple – very competent people made an initial assumption without full understanding.  They realized more research was required, and they invested their time to do that research.    And that led to doing what’s best for everyone.

 

 

 

Blindfold Barnyard Wackiness

Some of you may have played Blindfold Barnyard.  It’s an easy game, where you drag your finger around on the screen until you find an animal, such as a goat, and then drag it to the fence where you’ve dragged other goats.  When you have collected enough goats at the fence, you swipe your finger to move the animals into a barn.

upside down barnyard

The challenge is to remember which animals are tied at each fence: north, south, east and west.  If you drag an animal to the wrong fence, all the animals scurry away.  The more animals you collect on a fence, the more points you earn.

Last year, I added a game where there’s a pond in the middle of the barnyard.  When dragging an animal to the fence, you have to avoid the pond, or the animal will drown.  The game fans liked it, and wanted more complexity.

A month ago, I created the variant “Rotating Fences” that’s even harder.  Before the Rotating Fences variant, you simply swipe one finger up to drag the goat to a north fence, swipe right to the east fence, and so on.

In this harder variant, compass directions are rotated 90 degrees clockwise.  What was east, is now north.  What was south, is now east. To drag your goat to the east fence, you swipe your finger down.  Keeping that in your head makes the game quite challenging.  You can set the game to rotate CLOCKWISE, COUNTER-CLOCKWISE or SWAP.

If you want to really go crazy, set the rotation to CHANGE.  Each time you move animals from the fence to the barn, all compass directions are rotated.  For example, if the game starts with North being up, after you move goats to the barn, North is now to the left.  When you move the next set of animals into the barn, North is now down.  After the third set of animals, North is to the left, and after the fourth set of animals, North is back to the top.

You can get Blindfold Barnyard at: https://itunes.apple.com/us/app/blindfold-barnyard/id1069200628?mt=8

Blindfold Games: Too Stupid?

During the last week, I received a handful of nasty emails, and politely responded to many of them.

star of mr.robot tv show

Some thought the games are too expensive – I asked a blind colleague to post an article to AppleVis.com explaining the economics and business model for the games.

One person said “Your games are stupid and too easy.  I still refuse to purchase and will prove the point later.”

My only response is that if the games are too easy for you to play, you should invest your time into learning how to program a computer.  Several of the testers for the Blindfold Games are programmers and IT experts, and they can tell you what it’s like.

Building an app is not unlike playing a game, but far harder.  First, you need to conceptualize in your mind how the game will be played, and then you need to turn that into instructions for a computer to follow.  That’s the easy part.

Next, you must ensure the game is playable, and that all the gestures to operate the game make sense, and all game actions are consistent.  That’s a little harder; for example, I maintain a chart on my wall of all of the gestures of each of the games, to make sure they are consistent across games.

Now the fun part – debugging.  There are studies that indicate there are between 15 and 50 defects, called bugs, for every 1,000 lines of code.  My average game has about 10,000 lines of code.   Some of those bugs are easy to find – the app does the wrong thing, and it’s obvious.  Or the app crashes.

Some bugs won’t get found until hundreds of people play the game, and one person encounters unusual situation, such as resuming the game after not using it for weeks, and a new version of the game was just released.  Most of the game behavior can be recorded in the cloud, and I can look at the diagnostics log and investigate what caused the problem.

Finally, you need to keep enhancing the game based on the feedback of dozens of people, all with their own ideas and agendas.  And, if you want the game to be popular, it must have a “wow” factor and “playability”.

The “wow” factor means it must make a good impression in the first 90 seconds.  If it doesn’t, your users will reject it.  Playability means the game is as much fun in 12 months as it is now.  Not all Blindfold Games do that perfectly; but it’s always a goal.

But, to get back to my point, programming an app is much like playing a very complex RPG game.  You need to keep in your head exactly how the game works, what problems may be lurking somewhere, and have the patience and diligence to find and fix problems.

So again, my advice for you, if the games are too simple, learn how to create apps.  It will be challenging, inspiring and exciting, and it may even lead you to a new career.

Why combining games is a bad idea

Many of the comments I’ve read over the past week encouraged me to simply combine the games and follows Apple’s dictum.  The general inclination was that I chose to play in Apple’s world, so I need to follow their rules.

jigsaw puzzle

I’ll talk about the business issues about this later, but combining games is a disservice to many people in the blindness community.  Most of you are so experienced on mobile devices, you may have lost the perspective of people with less expertise, or who have mild mobility restrictions or mild cognitive issues.

Take Blindfold Bop Gesture Game for example.  I created it at the urging of the Braille Institute in Los Angeles.  When I met with Ben, he said they were getting more and more seniors who were losing their vision, and needed a better way to train them.  We discussed the idea for the game, and it’s now being used to first train seniors on a handful of gestures, and then to help people become competent in using all gestures.

Luke, one of the Blindfold Game testers, who is both an IT expert and a teacher of the visually impaired, evaluated Blindfold Bop Gesture, and suggested an alternative main menu that was even simpler.  He wanted a mode for his students where they didn’t even have to use any voice-over gestures in this simpler main menu, but could get to the game easily on their own, after a training session.  I added a simpler menu, and now his students can practice without his assistance.

Consider what happened with Blindfold Word Games.  At the suggestion of some testers, I put five different games into this app when I first released it: Word Ladder, Hangman, Unscramble, a variant of Boggle, and 7 Little Words.  On the main menu, you can scroll through the games to pick the game you want.  However, only first two or three games were ever played, and the other games were rarely evaluated or upgraded.

Initially, I thought only the first three games were fun.  To test this theory, I split off versions of Boggle and 7 Little Words into their own games.  The downloads and upgrades of these two games now matched all of the other games, and that showed me that some people have, as Apple described, a discoverability issue.  If something’s not on the main screen, it’s ignored.  People don’t always scroll down.

The analogy to the discoverability issue is a newspaper.  If the story is below the fold – the place where the newspaper is folded in half – then the story is considered less important.  Even on most websites, people read what’s on the screen when they visit the website, and never scroll down.

Now onto the business issues.

Several of you wondered why combining games would cost money.  In actuality, it would cost time – hundred or thousands of hours – to combine old games into a merged application.  Had I started from the beginning to build the games into groups, it might have had less of an impact, but at this point, it would require restructuring many of the games, changing how the help, settings and upgrades screens work, and how control flows between screens.   I would have to come up with ways to convert your in-app upgrades to the new app because the new app would have a different id in the app store from the original game.  Otherwise, people would have to purchase the in-app upgrades all over again.  And no one would agree to that.  In general, it would unleash a nightmare of unintended consequences.

And those hundreds of hours would simply be lost time – most of the downloads and upgrades occur in the first 3 months.  No one would benefit.  Doing all of the engineering work would result in no new revenue and new no downloads.  That same time could be used to build more games, or work with other programmers to move the games to other devices.

My discussions with Apple on Friday night went through these issues; Apple is keenly aware of accessibility issues and they want the best possible user experience on the iPhone.

Blindfold Games: The games are back!

I received a call from Adam on Friday night, and he said the review team had a chance to look at the Blindfold Games again, and understood why they are separate apps, and that the games can continue.  He mentioned games address a need that’s not normally considered by most app designers, and acknowledged how the games are focused on the needs of the visually impaired community.

New updates will be processed, and new games will be reviewed as before.   I’ll go into more details about the discussion in a blog next week, but I wanted to get the word out to everyone.

I want to thank the everyone for contacting Apple and spreading the word about the games.  I have heard from so many people – far more than I had hoped – telling me how much the games mean to them, and how they appreciate my efforts.  It has meant a lot to me, and I truly thank you.

Have a great weekend!

 

 

#keepBlindfoldGames

For those of you not following this issue: Two weeks ago, Apple rejected 3 updates: Blindfold Hopper, Blindfold Craps and Blindfold Horserace, by saying it violates a new rule of the App Store: apps that vary only by content (such as video or audio or text) must be merged into one app.  I explained that these apps are all different, and that only their menus and settings screens are similar.  I requested a phone call to discuss this further.

Apple’s decision as of Wednesday evening is that unless I merge the 80 Blindfold Games into a handful of apps, they will no longer allow new games to be released or allow updates to be made.  Tap here for details.

One person suggested adding the hashtag #keepBlindfoldGames to all of your tweets and facebook posts. You might pass that on.

I just received a comment asking why a hashtag.  A hashtag is tracked in twitter and Facebook. If thousands of people include that hashtag in their Facebook and twitter posts, then it is considered “trending”, and then more and more people will start talking about it. At some point, TV and radio stations may see the trend and discuss the issue of accessibility on the iPhone.

 

No more Blindfold Games or Updates

For those of you not following this issue: Two weeks ago, Apple rejected 3 updates: Blindfold Hopper, Blindfold Craps and Blindfold Horserace, by saying it violates a new rule of the App Store: apps that vary only by content (such as video or audio or text) must be merged into one app.  I explained that these apps are all different, and that only their menus and settings screens are similar.  I requested a phone call to discuss this further.  Tap here for details on Apple’s rejection.

On Wednesday night, I talked with an Apple representative, and Apple’s decision is that unless I merge the 80 Blindfold Games into a handful of apps, they will no longer allow new games to be released or allow updates to be made.

From a technology perspective, that’s extremely hard and time-consuming.  From a business perspective, that would mean spending hundreds of hours recoding the games, with no possible return-on-investment.  Most of the games generate sales in the first three months of the game being released, and I’ve been building these games for 4 years.

From a usability perspective, that means the main menus would be ridiculously complex, and the settings screens would be confusing and almost unusable.

If you are unhappy with this decision, you can express your opinion to Apple.  The accessibility desk is at accessibility@apple.com or you can call 1-800-MY-APPLE.  Thanks to everyone for enjoying my games.