by Katie Geleff
CBC Radio
A PhD student in Ottawa has developed a device that allows blind and low-vision students to feel music cues, replacing the need to see them.
Instead of having to follow a conductor’s baton, or a teacher’s hand gestures during a lesson, music learners can receive instruction through a vibration in their right leg sent by the Tap-Tap device.
Leon Lu, the information technology PhD student at Carleton University behind the idea, says the wearable device is easy to use and cheap to make.
Two people wear paired devices around their right ankles. Then, by tapping their heels together in different patterns, they can send messages to each other.
“Imagine, kind of like a Morse code, but vibration,” Lu said. “What’s really cool about that is that you can get real-time communication happening through discreet methods.”
“The student can continue playing and they don’t have to stop and actually discuss [the instruction], but they can still be in that moment and in flow.”
“Music and creativity, some may argue, are not necessarily things that everyone needs to have access to. But in my mind, they are extremely important aspects of the human experience,” Lu said.
The nuts and bolts of the tap-tap devices are simple. Each one has a small circuit board with sensors and a radio component – which is how Lu programs them to communicate. A small motor is what prompts the vibration and other than that, it’s mostly just straps and Velcro.
The Tap-Tap device doesn’t come with a set communication code or language. That language is set by the participants, which Lu says is both a limitation and advantage of the device.
Piano teacher Nora Bartosik echoes that.
Bartosik is a faculty member at Manhattan’s Filomen M. D’Agostino Greenberg Music School, a school for blind and low-vision students, which teamed up with Lu to test the prototype and provide his team with weekly feedback about the device.
Bartosik used the Tap-Tap device with one of her piano students for about two months earlier this year and says there was a learning curve.
“It kind of feels like a phone buzz,” Bartosik said. “Actually, a little bit stronger.”
“[My student and I] decided to focus on the things that I’ve been saying a lot in lessons…. We decided we would use the buzz to indicate that I wanted him to play quieter in certain sections.”
As they got used to the device, they added more signals.
“Two buzzes might mean like get louder, you know, do a crescendo,” she explained. “One long buzz would mean this or one short buzz would mean that. We kind of expanded the vocabulary as we went along.”
Chase Crispin, a blind musician and teacher in Lincoln, Neb., who consulted on the Tap-Tap project, said in a written statement from Carleton University, that “many people don’t realize how much a musician is managing at once: posture, notes, rhythms, dynamics — the list goes on. For blind music learners, who memorize most of this, it adds even more layers.”
Crispin said getting involved with the study was a way to “blend my own interest in technology with the needs I had as a blind musician.”
Lu, who sings, plays guitar and is learning violin, said his passion for music and desire to be creative was part of the reason he took up this project.
He’s surprised that a device similar to the Tap-Tap isn’t already integrated into commercially-available devices, such as a smartwatch, and hopes that might be a possibility in the future.
Lu is now analyzing the data and feedback gathered from his study with the FMDG Music School at Carleton to refine the device and finish his thesis.
After graduating, Lu hopes to make the device open-source.
“I don’t want there to be a cost attached,” he says. “If people want it and if they find it useful, I want them to be able to use it – I’m happy to help with that.”
In addition to music learning, Lu says the device could be customized to fit other uses.
Notably, he is working with Caroline Pakėnaitė to potentially upgrade the device to help her summit Mount Everest. Pakėnaitė has Usher’s Syndrome, which degenerates both sight and hearing over time. Lu hopes the tap-tap will help her communicate with her climbing partner.
“This device has a high-floor, low-ceiling,” he says. “While it’s very simple to understand, the potential uses are countless.”
Page 7 