Story
Seeing AI – How About that Handwriting Feature?
This article reviews the handwriting recognition feature of the Seeing AI app.
Introduction
I remember when OCR (optical character recognition) was a relatively new thing. Of course, then you needed a scanner and special software that cost an arm and a leg and computers weren’t cheap either. Anyway, my TVI had invited someone to come speak to some of her students as a sort of peer mentoring program idea. I don’t recall the woman’s name or even much of what we talked about, but I do remember her talking about using what was then an Arkenstone product to read print documents. One of my peers asked whether it would recognize a handwritten note. She seemed kind of sad when she said “that’d be nice, but computers aren’t that smart yet”. Or it was something to that effect…I must admit my exact recall of a conversation that took place more than 20 years ago is a little dim.
Fast forward to 2017. Seeing AI, an app made available by Microsoft, has launched an experimental component that can recognize handwriting.
For an overview of Seeing AI, please see this article from July of this year.
General Guidelines for OCR Recognition
As we all know, technology can be finicky at times. There are a number of strategies that will assist you in making the most of OCR technology.
- Take pictures in a well-lit area. This helps improve the quality of the captured image.
- Try to keep your hands steady when taking the picture. If you have difficulty with this, there are several stands made for scanning that you can utilize.
- If you notice that your OCR is not working as well as it once was, try cleaning the camera lens with cleaning solution and a small microfiber cloth or cotton swab.
The Seeing AI Handwriting Interface
The handwriting recognition feature is in the “channel’s area” located at the bottom of the screen. With voiceover on, you can switch between channels by swiping up with one finger when focused in that area.
The first time you attempt to use the handwriting feature, you will see a disclaimer that this feature is experimental and must exit that screen prior to proceeding.
Once you’ve arrived at the handwriting area, you will have a camera screen in the upper 70% or so of the screen with the channel list taking up the remaining 30% of the screen. This assumes that you are in portrait mode. If you are in landscape mode, the channel list will appear on the right side of the screen and take up 30% of that screen.
Once you have chosen a piece of handwriting to test, you will need to use your preferred method to ensure that it is within the view of the camera. There is no real way to test this other than trial and error.
To capture the image for processing, you will need to double-tap the upper portion of the screen. As of this writing, you cannot capture the image by pressing one of the volume buttons.
Samples of Handwriting Scanned with Seeing AI
There were three separate types of handwriting sampled with seeing AI. They are a cursive written recipe card, a printed note on an index card, and note printed in large letters such as one might find on a door.
Each sample was scanned more than once and scanned in a well-lit area. The provided samples are the best of those collected.
Sample 1: A Recipe Card 
The first sample tested was a cursive written recipe card. The card is something that you might find in Aunt Mables recipe box.
The results on this particular sample were not promising. Much of the text was cut off and some of the measurements were not read correctly.
As you can see from the side-by-side comparison of the image and recognized text, much of the information was cut off or left out. It’s doubtful that if one tried to follow this chocolate chip cookie recipe, it would turn out very well.
Sample 2: A Note on a Notecard
The second sample tested was a note. In this case, it was a short note to Santa from a little girl wanting a zebra for Christmas. 
As you can see from the provided image, this sample turned out significantly better than the recipe. The whole note cannot be seen in the preview, but when it was read, it was almost completely perfect, with the exception of the fact that the note is not from “Penny” but from “Renney”. Since we are not Santa and do not need to be sure of who exactly wrote this note, it seems safe to say that you can get the gist of the message. It is supposed that if someone who is blind is handed this note, they would be able to decipher it well enough and use context to fill in the rest.
Sample 3: A Note in Larger Writing
This third sample was provided by a student who wanted to provide her teachers with a bit of humor. She printed it on a plain sheet of paper with a dark marker. The message says simply “‘Be strong’, I whispered to my WIFI signal”.
This note was recognized with complete accuracy, including the small signature on the bottom, which had to be removed for confidentiality reasons, but it was still impressive that the OCR caught it. This is the sort of note a friend might leave on a door or table that says “went to the store, be back soon” or “there’s dinner in the fridge, help yourself”.
An obvious caveat would be that it’s great that the app can read the note, but unless it is left in a location where the visually impaired individual will find it, it’s still as waste of paper!
Conclusions
This is a very good start to handwriting recognition. It does much better with printing than cursive writing and obviously the less crowded the writing is, the better. It also does well with larger text and shorter notes based on these samples. So you might not want to use it to read Aunt Mable’s recipes, it is useful in other functional ways.
There were other samples taken and tested, but the conclusions shared here seem to be true across the board. The app can recognize cursive, but only a well-spaced sentence or two. It recognizes writing on lined paper or on a standard blank sheet. As with the human eye, the larger and clearer the text is, the easier it is for the app to recognize it.
Is this app ready for the big time? Probably not. But it’s certainly a good start and demonstrates how far OCR and AI in general has progressed over the years.
By Snowflake_tvi
