With the help of staff from Bryn Mawr’s Educational Technology Services (ETS), four Bryn Mawr students are developing an app that uses augmented reality to move budding musicians from air guitar to concert stage.
The app is made for the Microsoft HoloLens, a visor-like device with oversized lenses that gives users a “mixed reality” experience.
Taking advantage of the HoloLens’ ability to impose virtual objects into a wearer’s real environment, the app, dubbed HoloMusic, teaches users the fundamentals of every instrument in the standard Western orchestra.
The user turns around and the 13 instruments appear behind them. When the user taps on one, he or she learns what the instrument looks like, what it sounds like, how to hold it, as well as facts about it. Eventually, the developers want to include a feature for minigames in which users can guess which song is playing and which instrument is playing it.
“The app allows anyone who doesn’t want to spend the money on an instrument to try it out and see if they like it,” says Jocelyn Dunkley ’20, one of the app developers and a viola player in the Haverford orchestra. “This would be a good introduction to see which instrument you’d actually want to play and what sounds nice to you.”
Kara Breeden '18, Hyunjung Kim '19, and Linghan Mei '19, also worked on developing the app. The students worked with Palak Bhandari ’16, a Mellon Digital Curriculum Assistant with ETS. The team developed the app during a winter-break internship and recently presented their work at an on-campus event. They have also submitted the app to the EdSim Challenge, which seeks next-generation educational simulations that strengthen academic, technical, and employability skills. The winner will be announced in the summer of 2017.
The developers see the app as a possible way to improve and expand music education in cash-strapped school districts with limited arts funding.
“This opportunity inspired me to think about the social responsibility that we STEM majors have,” says Linghan, a biology major. “We can advance technology in practical ways and make a real difference.”
Bryn Mawr is one of a few educational institutions chosen to partner with Pearson Education to explore how the HoloLens might enhance college teaching and learning.
The College was introduced to the technology in September by Linda Bush '85 who works at Pearson’s Smarthinking Services as their mathematics and sciences director.
“Our team immediately understood that we could find exactly what we were looking for at Bryn Mawr—engaged and reflective students, innovative and collaborative faculty, and a campus culture open to merging traditional educational methods with high-tech approaches,” says Bush.
Jennifer Spohrer, director of ETS, sees a bright future for other students who wish to work with the HoloLens.
“The winter internships show the potential synergies between a liberal arts education and emerging technologies,” says Spohrer. “We hope this will be the first of many such opportunities for Bryn Mawr students to exercise their analytical, critical thinking, communication, metacognitive and collaboration skills while exploring the potential and current capabilities of new digital technologies.”
ETS is currently accepting applications for interns to work with the HoloLens over the summer. Another HoloLens tool, a chemistry application, is currently in development and will be tested at Bryn Mawr in the near future.
