There are countless moments in life when communication barriers crop up – from traveling overseas and attending business meetings with people from different countries to interacting with those with hearing impairments. Staff at Mitsubishi Electric are no exception – with one particular communication challenge in the workplace inspiring the creation of game-changing new tablet technology: welcome to the world of User Interface for Voice-activated Drawing.

Talk Technology

Mitsubishi Electric’s mission is to enhance the quality of life for all, and support for the United Nations’ Sustainable Development Goals (SDGs) underscores this commitment. The goal to reduce inequalities particularly resonated with Masato Hirai, a developer at the Mitsubishi Electric Integrated Design Center in Kamakura, when he met with an intern who was hard of hearing. The challenges he faced in interacting with her gave him an idea for an app. "This project started from my desire to talk to her more, even though I didn’t know how to use sign language," he said.

With his colleagues at the "Design X" project – a team of designers that works with different business units to solve society’s problems through design – Hirai developed a completely new user interface that combines voice recognition, swiping on a tablet and drawing pictures. To use the User Interface for Voice-activated Drawing, all a person has to do is speak into the tablet’s microphone and swipe their finger across the screen – then voila! – the spoken words sprout out on the screen as text from the person’s fingertip.

A Simple Swipe

It’s an intuitive approach to make conversations with people who have hearing impairments smoother, richer and more profound. With conventional sign language and written communication, those with hearing difficulties struggle to follow explanations that involve pointing to documents and diagrams. That’s because it’s difficult to watch a speaker’s lips and hands at the same time. With the app, however, spoken words from the speaker’s fingertip can be displayed next to text and illustrations, all on the same screen. On top of that, the app does not require any special skills, and saves a lot of time since text doesn’t need to be written or typed.

A Global Conversation

As leisure and business travel around the world increases, the app could also help break down language barriers thanks to software that translates text and enables illustrations. For example, a hotel concierge could give accurate directions to a guest by drawing a map and combining it with translated text that is inserted into the display by fingertip.

From start to finish, the development of this next-generation communication tool was guided by Hirai’s desire to better connect people on an emotional level and break down communication barriers. "What we saw as the most important things when making the app were an intuitive and enjoyable ease of operation, and warm connections among people," he explained. In a world that’s becoming increasingly connected and diverse, this is an inclusive technology that everyone can look forward to.

The content is true and accurate as of the time of publication.Information related to products and services included in this article may differ by country or region.