Meet Sign4all: the first sign language translator app

Sign language is an essential communication system for deaf people, which plays a crucial role in promoting inclusion in modern society and which, like other tools for people with other types of disabilities, deserves visibility for that are worked on and are in the public domain.

By recognizing and supporting sign language, a path is opened towards equality and full participation of all people in various fields, and promoting its use in daily life, as well as in companies and organizations, not only guarantees access to information and communication but also enriches entities, encourages team collaboration and strengthens the most humane values ​​of the corporation.

In this post, we will see how technology and, specifically, Sign4all, the first sign language translator app, are setting real milestones for inclusion.

Meet Sign4all: the first sign language translator app

It is incredible how technology is activating change in all our facets of life and developing improvements for social aspects that, perhaps, were not so visible to the bulk of society, such as the importance of sign language.

Currently, in Spain, 98% of deaf-mute people use regulated sign language, which means that, in order to communicate with them, we should all learn it just as we learn English and another language. In fact, why learn a foreign language and not another one that gives us the possibility to communicate with people who are much closer?

Sign4all, the first sign language translator app capable of recognizing and interpreting the alphabet of the Spanish sign language in real-time, was born from the Robotics and Three-Dimensional Vision Group of the University of Alicante with the aim of breaking down barriers between deaf and listeners.

How does Sign4all work?

Thanks to deep learning technology and different computer vision techniques, this app captures and interprets signs to translate them into the majority language.

So that we can understand how the application to translate sign language works, its mechanism consists of capturing the person and extracting every detail of the movements they make with the skeleton of their arms and hands. Then it codes the left part of the body in blue and the right part in red, maintaining the user’s anonymity at all times.

Of course, we cannot forget that effective communication is always two-way, so the application also works the other way through a virtual avatar that signs when the listener types words in Spanish.

And the best of all are two things: first: sig4all can translate the conversation in real-time; and second: thanks to the low cost of the tool, it is accessible to everyone who needs it when they cannot be accompanied by an interpreter.

How does deep learning work?

As we have mentioned, the first sign language translator app is based on deep learning technology, but could you tell what it consists of? Attentive!

Deep learning, or deep learning in Spanish, is an artificial intelligence technique inspired by how the human brain works to learn and recognize patterns.

For example: Imagine that you are teaching a computer to identify cats in photos.

Instead of directly telling him how to recognize a cat, you give him lots of pictures of cats over and over to get him to take in their image. Thus, the system begins to look for common patterns, such as shapes, colours and textures, so that as it sees more and more photos of cats, it adjusts its internal connections (called “artificial neurons” or “nodes”) and improves its ability. to identify cats by herself.

Like the learning process of a child, in which more and more complex concepts are coupled and more neural connections are generated, in deep learning, layers of these “artificial neurons” are used to process information at ever deeper levels. Each layer detects specific features and passes that information to the next layer for a more complete and detailed analysis.

When the system has seen enough cats, it is able to generalize and recognize them in images it has never seen before, which is because it has learned to detect the essential characteristics that define the animal.

Going back to Sign4all, which is the subject at hand, the mechanism would work the same way: the more signs you see and words you hear, the better able you will be to act as a translator and interpreter between deaf and hearing people.

Learn about the latest advances so far in Sign4all

Now, the developers of this sign translator application are working on a collaborative project with the Spanish Language and Signed Languages ​​Research Group (Griles) of the University of Vigo, who are enriching the database with which to work in the near future Sig4all.

We hope you found this post on inclusion, deep learning and the digital revolution interesting. Subscribe to Educa.Pro for more professional training!