Berklee Team Helped Create Program Pairing Photos and Music
Professor Jean-Luc Sinclair breaks down the science behind SoundSeeker in a video for Royal Caribbean.
Image courtesy of Royal Caribbean International
On August 7, Royal Caribbean International introduced a new product allowing customers to share their vacation memories. The technology, called SoundSeeker™, is a groundbreaking, patent-pending digital experience that uses artificial intelligence (A.I.) to create original soundtracks based on users’ photography.
Royal Caribbean worked with faculty from Berklee and technology experts from around the world to develop the song generator. SoundSeeker uses Google Cloud Vision to identify objects, expressions, and colors in users’ images to match the photos with a corresponding soundtrack. The customized tracks take inspiration from a wide variety of music, including '90s hip-hop, rock, modern, and electronic dance music. Berklee leveraged music theory to build a roadmap that helps the tool analyze music for pitch, tempo, and instrumental combinations, among other factors.
Exploring the Intersection of A.I. and Music
The contributors from Berklee’s Institute for Creative Entrepreneurship (BerkleeICE) included Michael Bierylo, chair of the Electronic Production and Design (EPD) Department; Michael Sweet, associate professor in the Film Scoring Department; Ben Houge, associate professor in EPD; and Jean-Luc Sinclair, assistant professor in EPD.
“The Royal Caribbean project gave us the opportunity to start thinking about some of the implications for musicians in A.I. and machine learning,” said Bierylo. “Faculty working on this project are experts in the field of game audio, and the insights they brought from that work helped frame some ideas on how music would respond to different types of data input.”
Added Sweet, “We were able to bring together Berklee's diverse expertise to the project including interactive music design, cognitive influence through musical perception, and systems design, culminating in the creation of a unique, individual, musical experience to each customer.”
Royal Caribbean and the team of musicians and technologists spent more than 600 hours reviewing hundreds of tracks, along with 10,000 photos, to match each of the 2.5 million combinations to one of 10 moods.
“To me, the most interesting aspect [of this project] was that it made us rethink how we approach music, both from a composition and production standpoint,” said Sinclair. “In a way, we were distilling the essence of each style of music we approached in order for the A.I. algorithm and the composers to be successful.”
Watch Royal Caribbean's video about SoundSeeker:
