The Smart Letterboard enables communicating with others without the use of voice or hand movement. A laser pointer is attached to one’s ear or glasses. The laser pointer than points at the letter on the letter board. A camera sends the image to a computer (in our case it was the Jetson Xavier NX SBC). The computer processes the image to figure out which letter is being pointed at. This uses the laser tracker code developed here. Credit to @bradmontgomery for the initial implementation. Once we figure out the letter, we use the Happy Transformers library to predict the top 3 words (either completion if we are in the process of communicating a word or prediction of new words if we have completed). This is then voiced using the gTTS (Google Text To Speech) library. The letter and predictions are also sent to the Arduino so they can be communicated to the users via LED strips on each letter and an LCD screen that displays the sentence being built and predictions
1 Like