A project led by one of the key creators of Google Glass, the tech giant’s influential but ultimately ill-fated smart eyewear, aims to let people have conversations without talking or using their hands to type, sign, or gesture.
Called SilentSpeller, the project is a communication system that allows people to send texts using a high-tech dental retainer to spell out words without actually voicing them, according to a demo video and academic paper reviewed by BuzzFeed News. The device works by tracking the movement of the user’s tongue. Researchers claim the system identifies letters with 97% accuracy, and 93% accuracy for entire words.
The research is the brainchild of Thad Starner, a pioneer in wearable technology. Starner played a lead technical role in developing Google Glass, the much-hyped device that helped introduce the world to a new genre of gadgets beyond smartphones. But the device courted controversy and pushed the bounds of society’s relationship with technology when Google introduced it almost a decade ago. SilentSpeller, by contrast, is a research project out of the Georgia Institute of Technology, where Starner is a professor, so the goal for now is more academic than product road map.
However, the device could eventually be used to help people with movement disorders including Parkinson’s disease and essential tremor, Starner told BuzzFeed News in an interview. He also sees potential consumer applications, like hands-free communication in really quiet places, like a library, or really loud places where people would have to strain their voices to be heard.
To develop SilentSpeller, researchers didn’t create a whole new retainer from scratch. Instead, they jury-rigged an existing product called SmartPalate, which looks like the kind of retainer used by orthodontists but is loaded with tiny sensors to track tongue movement for speech therapy. Software creates a visual map of how someone’s tongue functions while speaking, but while SmartPalate’s primary use is to help people correct speech disorders, researchers working on SilentSpeller adapted the system to transform the retainer into a communications tool.
The device reads tongue movements through the retainer’s 124 sensors, according to the academic paper. The data is then sent to a personal computer or smartphone through a USB wire. For now that means that users of the prototype will have a wire dangling from their mouths, but researchers believe they could eventually create a wireless version that fits entirely in the mouth. The device requires a dental impression for a custom-fitting retainer. Although researchers have been working on the project since January 2020, progress has been slowed because the COVID-19 pandemic has limited the number of participants that could get fitted for custom retainers.
The effort is not a Google project, but it has the fingerprints of the company’s researchers on it. Starner is still a part-time research scientist at Google focusing on machine learning. Another author on the academic paper is Alex Olwal, a senior Google research scientist that leads the company’s Interaction Lab, an initiative aimed at intertwining digital and physical experiences. Olwal is also a technical lead for Google’s augmented reality division.
The demo video for the project, published on Olwal’s YouTube channel, features Olwal testing out the device. His eyes are blurred out, but a Google spokesperson confirmed it’s him.
Google Research, an arm of the company that focuses on pure science and technological breakthroughs, is credited as a collaborator in the demo video, but the company spokesperson, as well as Starner, said the effort was not considered a Google Research project.
The lead researcher of the project, Naoki Kimura, is a graduate student from the University of Tokyo and an exchange student studying under Starner at Georgia Tech, Starner said. Other researchers on the project come from universities including Carnegie Mellon and the University of Washington. The project was presented in May at the Conference on Human Factors in Computing Systems, which normally takes place in Yokohama, Japan, but was held virtually amid the pandemic. The project, however, hasn’t been widely known.
Devices worn on — or in this case inside — the body have become a bigger focus of research over the last several years as tech becomes infused in more aspects of people’s lives. An Apple Watch can take an electrocardiogram with sensors that track your heartbeat. Facebook’s parent company Meta makes virtual reality headsets for tapping into the so-called metaverse, which tech executives describe as an immersive world with a mix of both digital and physical elements. For years, Facebook was also developing a “brain-computer interface” with the goal of allowing people to communicate by imagining the words they wanted to type, but the company pulled the plug on that project in July. Google has created internet-connected jackets and bags, and for a time the company experimented with smart contact lenses with glucose readers for people with diabetes.
But as people start to use more of these devices, privacy concerns have cropped up. A device that fits into a person’s mouth could open the door to invasive types of data collection, said John Davisson, senior counsel for the Electronic Privacy Information Center, or EPIC, a Washington, DC–based nonprofit. A nightmare scenario, he said, might be a company gathering information about a person’s saliva or dental records. While that’s a long-term concern for a new generation of devices that deal with biometric data, there’s no indication SilentSpeller would have the capability to do anything like that.
“It’s sort of obvious, the sensitivity of a device that goes inside of your head,” Davisson said. “That’s going to present some privacy risks.” He added there would need to be strict safeguards on what information it could collect.
In the academic paper for SilentSpeller, researchers list a handful of use cases. In addition to helping people with limited dexterity and other consumers, there might even be applications on the battlefield. One use case, according to the paper, is “United Nations operations where silent communication is key.” The corresponding drawing is of two heavily armored soldiers holding assault rifles. In speech bubbles, the soldiers discuss tactical strategy, presumably through SilentSpeller.
This isn’t the first time Google scientists have worked with university researchers. Google has long had a tight-knit relationship with academia, stemming from the company’s start as a Stanford University project by Larry Page and Sergey Brin, then graduate students. For 15 years, the company awarded financial grants to academic researchers called the Google Faculty Research Awards. In 2020, the company discontinued that program and said it would instead focus on supporting up-and-coming researchers and faculty working on projects that serve underrepresented communities. That year, CNET reported on several projects funded by Google Research or developed by the Interaction Lab. They included temporary tattoos that transform a person’s body into a living touchpad when applied to the skin and sunglasses that project holographic icons.
