Article
Swift Playgrounds – The VoiceOver Challenge
In this article we look at the usability of the popular coding program with the VoiceOver screenreader.
Then …
The look of horror on my wife’s face said it all. “How much did that cost?!” she exclaimed, as I unboxed my the iMac 24inch computer, to which I mumbled a figure under my breath, never to be repeated.
It was 2007 and I was excited. I had been working in the field of accessible technology for a few years by this stage and had become frustrated by the so-called ‘blind-tax’. It was discouraging that those for whom I was employed to create digital materials were burdened with the high-costs of ‘specialist alternative software’ just so they could access those materials. But rumors were abounding that these Apple computers were now shipping with a built-in screenreader free of charge! It was called VoiceOver and it was going to change the game forever.
Now …
Fast forward ten years and the game is indeed different. The pervasiveness of the iPad in schools all over the world has contributed greatly to sweeping change. Similarly, VoiceOver, as part of the suite of assistive features on the iPad, has helped greatly to ensure that, as change abounds, those who are reliant on this type of assistive technology are not left behind.
The increase in courses, programs and learning materials all focused on learning how to code is also a notable feature in this era of advancement. Industry has recognised that today’s kids are part of the first generation which merges the digital and physical world. This transformation requires new skills, skills that have not been taught in schools until very recently. The understanding of the concepts of sequencing, logical operations and conditional statements (the basics of computing) have become ever more valuable. The lingua franca is no longer English, or French or Arabic; it’s code. Technology has the potential to be the great equaliser, promoting innovation and meritocracy, amongst all people of every ability.
Apple and the Hour of Code
To that end, the Hour of Code movement been extremely successful. This global action by Computer Science Education Week and Code.org has reached millions of students in 180+ countries through a one-hour introduction to computer science and programming. The byte-sized lessons are mostly carried out via graphical block-based drag and drop applications. Unfortunately, most of these applications are not accessible with a screenreader. With Apple however, and their own contribution to the Hour of Code library with Swift Playgrounds, the story is a different one. And really, would we expect anything less? The Tech Giant has kept inclusion and usability at the forefront of their design mentality for many years. It was with high hopes that I triple-clicked that home button, launched VoiceOver and attempted to navigate the Swift Playgrounds Hour of Code.
Do Swift Playgrounds and VoiceOver Play Well Together?
For the most part, yes. As long as you are prepared for a lot of trial and error … which to be honest is part of every programmer’s daily life. Things work but not always in the ways you think they should.
For example, in the introductory section entitled “Simple Commands” the interactive graphical instructions do not change in unison with the screenreader as we swipe right. An individual solely using VoiceOver may not have a problem with this. But what if you are a teacher or a parent, working with your child, trying to maintain focus as you progress through the lesson? There may be a small bit of frustration here. Also, some of the interactive popups, which do not appear sequentially in a visual timeline, created confusion when using VoiceOver. In the interests of full disclosure, let me just say that this does improve with the later lessons on “Functions” and “Loops”. You must be prepared to be resilient from the outset though.
Gameplay
I love how activating VoiceOver transforms the graphics from a 3D output to an aerial view of the game board. The main objective is to guide a character, known as “Byte”, around the board using a coordinate system. The user is given audio feedback on the layout of the game board (how many rows and columns there are), the location of the Sprite you are controlling, the positioning of gems and obstacles relevant to the task at hand. In the initial lessons these notifications are short but as you progress, they become more and more detailed. Cognitive overload may be another issue to bear in mind here. Audio is presented in the format, for example, “Column 0, Height 1, Row 5”. There are one or two mistakes announced regarding how many rows and columns there are in certain tasks. And yes, you read that correctly, Row Zero. In computer- and programmer-speak the first numeral is not one but zero. You’ll get used to it. Trust me.
Thirdly, and this applies to both users who may have some sight and those who are primarily VoiceOver masters, I’d advise running the application in landscape mode. Lessons are just a little more buggy in portrait mode. Turning your iPad on it’s side just makes it more usable – instructions and code blocks on the left, game board on the right with an even 50/50 split in terms of screen real estate.
In terms of entering code (which are accessible and selectable icons in their own right) it’s quite good. It would be nice if there were options to indicate which direction Byte is facing throughout the lesson as opposed to just at the beginning. You can actually swipe to place screenreader focus on the entire game board and VoiceOver will provide feedback, but this is not intuitive. Also, if the user gets the code wrong, it would be helpful to bring the character back to the starting position. Currently, you need to figure out how to code him back and start again or work from the broken code and continue on. This was slightly frustrating at times, although I am still very impressed with how well it responds to swipe gestures.
Finally, when you believe you have the correct code and click the “Run My Code” button, the real-time audio feedback of Byte’s movements is very impressive. It reminded me of the type of feedback from electronic checkers or chess games of the late 80s, early 90s. And I really love the announcement “Byte nearly fell off the edge … Byte looks sad” when you have coded incorrectly.
I won’t use the word “fail” here because with coding “failure” is the wrong word. Coders don’t fail. We just find lots of ways that don’t work…
By Dan
