This summer, Jocelyn Dunkley ’20, a computer science and German double major, was one of only six students selected to participate in the National Science Foundation-funded International Research Experiences for Students (IRES) program, where she conducted human-computer interaction research.
Jocelyn worked at the Human-Centered Ubiquitous Media Lab at Ludwig Maximilian University in Munich, Germany. Her job? Helping to create an alternative to braille using tactile books, computer vision, and text-to-speech software.
“Only 10 percent of visually impaired people know how to read braille, and braille books are heavy,” says Jocelyn. “Most blind people use text-to-speech software, so the lab wants to help integrate it more into the classroom.”
There is a wide array of text-to-speech software available, but it's not very classroom friendly. It can only read entire pages aloud, so users can't skip around or choose only specific passages to listen to. The lab was working on creating a tactile representation of textbooks that a visually impaired person could use to quickly navigate a page and choose what they wanted to have read aloud with different textured sections to represent headers, images, etc.
There’s already an algorithm to convert books to a tactile representation, but “10 to 30 percent of the time there are errors, so my role was to create a web tool so someone could manually edit it,” says Jocelyn.
The coolest thing she learned, she says, was how to alter images through code. “[The books] are typically PDFs, which are not meant to be altered at all, so I had to figure out how to change it to an SVG (vector graphic) that could be altered programatically.”
Jocelyn didn’t spend all her time staring at a computer screen, though. Munich is consistently ranked as one of the best places in the world to live. It has a thriving cultural and intellectual scene, is home to some of the best museums in Germany, and boasts spectacular scenery and the best beer on the planet. Jocelyn got plenty of opportunities to practice her German, noting she had made several German friends outside of work.
Jocelyn also managed to get a fair amount of traveling in. She visited the Alps, went to Prague, and attended a research conference in Oldenburg for her internship—and built a 10-day loop around the trip, visiting Frankfurt and Berlin along the way.
What’s next for Jocelyn? “I’m working on a senior project for computer science. I’m partnering with a Swarthmore professor to make a virtual reality music education app to create an individualized learning experience and develop basic music literacy, so kids can learn at their own pace. VR equipment is a little bit cheaper than buying 30 instruments.”
Jocelyn has yet another feather in her cap as the founder and co-organizer of SisterHacks, a hackathon for the Seven Sisters Colleges, where participants gather together and build a tech product from start to finish within the span of a weekend. Attendees also get to attend a wide array of events, including tech workshops, an Alumnae in Tech panel, and even an a capella concert. “But,” Jocelyn says, “I’m graduating, so I’ll need to find someone to carry on my legacy.”