Smart Device Translates American Sign Language To English

By

CCSS NAS-1
Photo Credit: www.rit.edu

American Sign Language (ASL), has been one of the primary means of communication for the deaf in the United States and many parts of Canada since the 1800s. It is estimated that between 500,000 to 2 million people use the language on a regular basis.

But popular as it is, there are millions of people with normal hearing that do not understand the language. And while most deaf Americans learn to speak with the help of speech therapy, many find it easier to communicate through sign language. Now a team of researchers at the Texas A&M University has created wearable technology that will make it easy for ASL and non-ASL users to converse.

Photo Credit: Texas A&M University

The smart device is the brainchild of a team led by Biomedical Engineering Associate Professor Roozbeh Jafari. It uses two separate sensors to translate intricate ASL gestures into simple English. The first that is fitted with an accelerometer and gyroscope keeps track of the significant movements - the user's hand and arm as he/she tries to communicate.

The second sensor helps distinguish the smaller movements that follow, the larger ones. Called an electromyographic sensor (sEMG), it can recognize various hand and finger movements based on muscle activity. The two sensors working in tandem help provide an accurate interpretation of the gesture.

Image Credit: Quora.com

For example, when an ASL user is gesturing the words “please” or “sorry," the first sensor will pick up the hand drawing circles to the chest, while the second will ascertain if the fist is open ("please") or closed ("sorry").

Once the device that is worn on the user's right wrist has captured the gesture, it transmits the appropriate signals to a laptop via Bluetooth. A complicated algorithm translates them into English and displays the word on the computer screen.

Jafari, who unveiled the prototype at the Institute of Electrical and Electronics Engineers (IEEE) 12th Annual Body Sensor Networks Conference this past June, says there is still some work to be done before the technology can be used in the real world.

For one, it currently recognizes just 40 primary ESL signs, which means that it has thousands more to learn. Also, the smart device only translates one word at a time making ordinary conversations painfully slow.

The research team also realizes that not all communication takes place around a laptop. They hope to eliminate the need for one by incorporating a computer into the wearable. The computer will then send the translation to a smart device, allowing two people "speaking" different languages to have a coherent conversation.

Photo Credit: Texas A&M University

Additionally, each device has to be custom programmed, which means that the individual has to "train" the wearable by repeating every ASL sign a few times. This is a time-consuming process and can only get worse as the translator's vocabulary expands. Jafari hopes to reduce the learning time or eliminate this requirement altogether in the product's next release. Despite the numerous challenges the researcher is not worried. After all, it took his two graduate students just a few weeks to come up with the first impressive prototype.

Photo Credit: Motion Savvy/Indiegogo

The Texas team is not the only one working on making conversation between ASL and non-ASL users easier. MotionSavvy has a product that uses a smart device camera to translate gestures into speech. In China, researchers have created a motion sensing device that translates Chinese Sign Language into both spoken and written words. With so many brilliant minds focused on finding a solution, communication difficulties experienced by ASL users may soon be a thing of the past!

Resources: engineering.tamu.edu, smithsonianmag.com

Cite Article
Vocabulary List
accelerometeralgorithmappropriateascertaincoherenteliminategesturesgyroscopeincorporatinginterpretationintricateprototypetandemtherapyvocabulary
341 Comments
  • Sally Monday, September 16, 2019 at 5:01 pm
    great way to help those who need it!
    • JennyTuesday, May 21, 2019 at 11:29 am
      cool
      • girlWednesday, March 13, 2019 at 4:41 am
        this is cool
        • heyy kidsThursday, February 7, 2019 at 9:57 am
          this website is so so so so so amazing!
          • ehhTuesday, November 27, 2018 at 6:19 pm
            super
            • sadfolf
              sadfolfTuesday, November 27, 2018 at 12:42 pm
              This was a very nice thing bc my friend's mom is deaf so yaay
              • emojiuni
                emojiuniFriday, November 9, 2018 at 10:19 am
                AAAAAMAZING
                • ajg16
                  ajg16Thursday, August 9, 2018 at 2:54 pm
                  I'm learning ASL right now, I've pretty much got the alphabet down, I'm gonna start learning words and stuff in a few days
                  • austinxo
                    austinxoWednesday, March 7, 2018 at 8:23 am
                    cool.
                    • sports_star
                      sports_starThursday, December 21, 2017 at 12:46 pm
                      I'm not deaf but I'm nearly fluent in ASL. It's a very interesting language to learn and I think you should learn it too... it's tons of fun!