deep learning” technology to generate voice audio, and Spotify has its sights set on more ambitious applications.”
Spotify has been pleased with the response to its new AI DJ feature, which utilizes the most cutting-edge “deep learning” technology to generate realistic-sounding voice audio to accompany the personalized selection of music. However, the company is looking to apply this same technology to more ambitious projects in the future. AI advances and large language models, as well as synthesized voices, have been developed and utilized in conjunction with Spotify’s prior investments in personalization and machine learning.
Spotify is now exploring other ways to use the technology it has developed, not just for a single purpose.
At Spotify’s Stream On event, the main focus was the revamped mobile app, which now has discovery feeds for music, podcasts, and audiobooks that are similar to TikTok. Additionally, Premium subscribers in the U.S. and Canada can now access the AI DJ, which is designed to get to know the user in order to play their desired music with the press of a button.
The application has been redesigned so that the DJ will be featured prominently at the top of the Music section of the home page for subscribers. This offers both a straightforward way to access favorite music, as well as a way to encourage those who don’t pay for the service to upgrade.
Spotify has employed their own music experts to curate the commentary heard while streaming music. To expand this to the app’s users, they have utilized OpenAI’s Generative AI. ChatGPT, which attempts to form answers from web-wide data, is not utilized in this process; Spotify’s restricted pool of music information guarantees that the DJ’s commentary is pertinent and exact.
The DJ selects music based on its knowledge of a user’s preferences and likes, which is similar to what is put together in personalized playlists such as Discover Weekly and other ones.
The AI DJ’s voice was created through technology that Spotify secured from Sonatic in 2020 and was modeled after Xavier Jernigan, the Head of Cultural Partnerships at Spotify, who hosted their discontinued podcast, “The Get Up”. Astonishingly, the voice is incredibly lifelike and not robotic in the slightest. At the Spotify event, Jernigan spoke beside his AI replica and it was difficult to distinguish the two. He jokingly commented that he could listen to his own voice for an entire day.
Ziad Sultan, Head of Personalization at Spotify, stated in a talk with TechCrunch after Stream On was finished that the purpose of the Sonatic technology the company obtained is for the sound to be pleasant. He went on to say that when you hear the Artificial Intelligence DJ, it will be possible to pick up on the pauses for breathing, the different tones and the enthusiasm for particular genres.
Google previously stunned the public with its AI technology that had an authentic human sound. This was integrated into Duplex, but it caused some controversy as it called businesses on behalf of the user without making it clear that it wasn’t a real person. This shouldn’t be a concern with Spotify’s new AI DJ feature, since it is labeled as such.
Jernigan entered the studio to produce recordings of his voice with the help of specialists in vocal technology. There, he was told to read multiple lines in different moods that were then used to train the AI. Spotify declined to give any specifics on the length of the procedure or any of the details, simply referring to it as their “special recipe”.
Sultan notes that from the high-quality source audio, Jernigan no longer needs to contribute anything and the voice is generated solely by AI. However, Jernigan sometimes visits the Spotify writers’ room to provide his opinion on how he would deliver a line, to ensure he can still participate.
Despite employing Sonatic and OpenAI technology to construct its AI DJ, Spotify is putting resources into internal research with the hope of becoming more informed on the most up-to-date Artificial Intelligence and Large Language Models.
Sultan informed TechCrunch that their research team consists of hundreds of members who are working on personalization, machine learning, and the OpenAI model for the AI DJ. Additionally, the research team is exploring the possibilities of large language models, generative voice, and personalization. He states that the situation is rapidly changing and they would like to be recognized for their proficiency in artificial intelligence.
It is uncertain if Spotify will employ its own AI technology for upcoming projects or collaborate with a partner, such as OpenAI, to do so. It is still too soon to make a definite decision.
Sultan has stated that their organization is regularly releasing documents and will be investing in the most recent technologies, such as legal language machine programs, to build up their abilities.
Using this core technology, Spotify can explore other fields related to artificial intelligence, natural language processing and generative AI technology. Regarding what those areas could be in terms of customer offerings, the company has not yet revealed any information. (It has been rumoured that a chatbot similar to ChatGPT is being tested. Nevertheless, no decisions have been taken on launching it as it is just one of many trials.)
Sultan states that they have not yet revealed when they might be entering into new markets or languages, but the technology allows for them to do it. He hopes to have more information soon regarding the progress of the expansion.
Spotify reports that initial customer reactions to artificial intelligence are positive.
The company was hesitant to develop a comprehensive range of AI products due to the uncertainty of customer response to the DJ. Would people be interested in an AI DJ? Would they use the feature? There was no definitive answer. This was especially true considering that Spotify’s voice assistant (“Hey Spotify”) had been discontinued due to a lack of use.
It appeared that the DJ feature could be successful, as it had been tested by Spotify employees prior to its launch and the results of the usage and re-engagement were incredibly positive.
The acceptance of the product by the public is in line with what Spotify has observed in their own research, Sultan states. This suggests that there is the possibility of creating more products based on the same basic infrastructure.
Individuals spend hours each day using this item… It assists them with making choices, finding new things, it tells them what music they should tune in to next and explains why… So the response – if you take a look at different web-based media, you will observe it’s very positive, it’s enthusiastic, Sultan states.
In addition, Spotify reported that people devoted 25% of their listening time to the DJ feature when they used it, and more than half of the first-time users came back to the feature the following day. Though the feature is not completely available in the US and Canada yet, the company is encouraged by these initial results.
Sultan believes that this is an incredible step forward in connecting remarkable products with their users. He does, however, warn that the difficult task of finding the perfect application and developing it correctly still lies ahead.
We declared that this was an AI DJ for music. We established a writers’ room for it and gave it to users so they could do what it was designed to do. It is performing really well. We do like to ponder what else it can do and how quickly we can make it happen, he went on to say.
