A new AI development promises to change the way employees with visual impairments function at work, but could also massively increase their participation in the workforce.

Imagine the last time you sat in a meeting where you and a group of colleagues were discussing a graph that was displayed on a screen. Now, imagine that you can’t see the graph nor can you fully make out the faces belonging to the voices. This is the experience of many employees with visual impairment. But technology has the potential to change that.

Microsoft’s new app, named Seeing AI, offers a comprehensive navigation system. It’s a “talking camera” app that can read text and barcodes, describe images, help locate products, and recognises faces and emotions. While it has been on the market overseas since earlier this year, it has just launched in the Australian app store, and is part of a suite of developments from Microsoft which aim to create a more seamless working environment for people with a range of disabilities, including auditory and mobility impairments.

While Australia is on a par with the rest of the world in terms of disability employment rates, businesses here are far from perfect. Employment rates for people with disabilities remain almost half of their non-disabled counterparts, suggesting that a large proportion of the population, with valuable skills and insights, is kept out of the workforce. A loss, not just for people with disabilities, but for workplaces who are missing out on talent.

Jenny Lay-Flurrie, chief accessibility officer at Microsoft, has been involved with developing the Seeing AI technology, and has severe auditory impairment herself. “Disability is over 70 per cent invisible,” she says. “You may be sitting there going ‘I don’t know have people with disabilities in my company, on my team.’ How do you know? You would never know looking at me that I’m severely, profoundly deaf.”

Unfortunately, too, it’s often the case that well intentioned efforts to accommodate their needs often fall short.

“For folks with blindness, alternative text are descriptions behind pictures that explain what they are,” she says. But even when these are included Lay-Flurrie has found that most alternative text fails to provide meaningful information about the visuals in question, meaning that people with visual impairment are at risk of missing an important aspect of the conversation at work.

Does it work?

In their review of the app, Mashable said the Seeing AI app “really could be life changing”. They found that the app identified most barcodes and scenery easily, describing a “desk with a laptop monitor and keyboard” and “a sofa in a room”.

Identifying people was slightly more challenging. While it recognises the people looking directly at the camera, it struggled to detect those who weren’t, and, in what was perhaps a uniquely tech-nerd problem, didn’t register that a life-sized cutout of Chewbacca was not a person.

These are pitfalls that Microsoft seems to be aware of. When you open the app it’s careful to point out that “Seeing AI is not always accurate”. But, even if the app can’t be fully relied upon, it goes some way to alleviating the burden of translating the visual world into a non-visual one.

Microsoft Cloud solution engineer, Kenny Sing, who is severely vision impaired explains, “Until this point, over the last 20 years, I’ve had an incredibly killer app, she’s my wife. I have been relying on her to bridge that information gap between the real world and me. But now she has very serious competition.”

Photo credit: Ars Electronica