Inside the Lab That's Producing the First AI-Generated Pop Album

By Lucy Jordan, Seeker

Some 70 years ago, computer scientist Alan Turing famously set the bar for artificial intelligence: a computer that could convince a human conversation partner that it was a person.

On a recent spring afternoon in the Flow Machines laboratory, located on a quiet street in the Fifth Arrondissement of Paris, senior researcher Pierre Roy was more concerned with his music-making AI software's ability to create a convincingly catchy song.

"So far, from the technical standpoint, no one knows how to do a proper song, to tell a story," he said. "It's a hot topic in AI."  

Flow Machines, a project of Sony Computer Science Laboratories in Paris that receives funding from the European Research Council, is developing an AI program that can compose compelling, professional-quality music — an aim shared by similar ventures such as Jukedeck in the UK and Google's Magenta project.

Ever since Turing defined his test, popular culture has fixated on the idea of sentient AI, both benign and catastrophically malign. But AI today has become something of a catch-all term for software that augments human intelligence, usually by mining vast troves of data. It's become commonplace in everyday life — asking Siri to find the nearest Chipotle, listening to a Discover Weekly personalized playlist on Spotify, or letting Facebook auto-tag you or your friends in a photo.

What makes AI different from traditional data analytics programs is its capacity to teach itself to recognize patterns using machine learning algorithms, and then make autonomous decisions based on those patterns without being explicitly programmed to do so.

Flow Machines' technology is no different. The project's AI machine is not a self-aware robot that yearns to write a love song — not yet, anyway. It's a set of complex algorithms that has been given 15,000 songs to analyze.

When Benoît Carré, the project's artistic director, wants to compose a new song, he sets his program loose on a selection of music from their database — a bossa nova playlist, perhaps, or his favorite Charlie Parker tunes. He runs an analytical model known as a Markov chain that identifies patterns in those selections and then imitates and varies them to create its own original composition.

The computer calculates the probability of certain chord progressions, melodic sequences, and rhythms, and uses these probabilities to generate new, plausible variations.

"We don't give the machine musical rules or abstract musical knowledge," explained Roy. "It's only the machine producing music based on what it learned from the data."

RELATED: Artificially Intelligent Investors Rack Up Massive Returns in Stock Market Study

At this point in the process, the Flow Machines system requires the intervention of human expertise. Carré might keep an eight-bar phrase he likes and reject the rest, running the program again until he has a melody and a chord sequence that he's happy with.

Last September, Flow Machines unveiled its first single. "Daddy's Car" is a summery, upbeat pop song based on a selection of Beatles tunes. The song is tuneful, but undeniably a pastiche and somehow bewildering, triggering an odd nostalgia for a non-existent memory. It was accompanied by an irregular take on the Great American Songbook, the less accessible "Mr. Shadow."

All the stylistic flourishes — the harmonies, instrumentation, and the lyrics — are thanks to Carré. What the computer spits out is pretty basic. Songs are entered into the database in their most barebones form — a lead sheet with just chord labels and a melody — and that's how the computer composes as well. Carré is responsible for all the production touches and details that give a song its distinctive sound.    

This offers Carré a great deal of artistic license in the rendering and production phase, but also places constraints on what kind of music the program can interpret and produce.

"It only can be used for the type of music that can be represented by a lead sheet," said Roy. "Classical wouldn't make sense ... even techno or trance wouldn't make sense. Only music based on harmonized melodies, that is Western popular music, that can be reduced to notes and chords."

The limitations of this particular technology beg the question: When we thrill to a certain phrase of music, what is it that we're responding to? And can we expect a machine to ever recreate an expression we consider to be so fundamentally human?

Marvin Minsky, who founded MIT's AI Lab and is considered one of the forefathers of the field, notably hypothesized that humans take pleasure in self-contained, repetitive patterns and their variations because they allow us to comprehend and play with time.

Advances in neuroimaging have offered deeper insight into this idea. Neuroscience researchers from McGill University and the Rotman Research Institute have found that when people listen to music they enjoy, the parts of their brain linked to reward and motivation light up. Listening to passages of music that cause us to react with particular intensity — those that give us "chills" — initiates a rush of dopamine, the "reward" neurotransmitter associated with pleasure that has been linked to sex, food, and falling in love. 

Interestingly, dopamine is released in anticipation of the peak emotional moment as well as during it.

"We build expectations and delight when they are creatively violated, whether by composers or in the performer's interpretations," explained Jonathan Berger, a composer and professor of music at Stanford University.

By this logic, AI should, in theory, eventually be able to identify the patterns of expectation, disruption, deferral, and eventual satisfaction that we find the most pleasurable and produce melodies that give us chills — although "Daddy's Car" suggests they have some way to go yet.

Even if the next song produced by Flow Machines appeals to a number of people, Berger is skeptical that the program is capable of producing truly great art.

Beyond pattern repetition and variation, he said, "music (at least great music) is multilayered and unfolds in multiple time scales. Pattern replication generally lacks the large-scale contexts that constitute a significant aspect of art."

The Flow Machines team counter that this isn't necessarily the point. They see their machine not as a way to sideline artists but as a tool to augment human creativity, and hopefully send it off in novel directions that otherwise may not have been possible.

"What's really nice is that the songs sound very different from anything else, including what Benoît was doing before," Roy said of the material being produced by the lab.  

Flow Machines will release a full album before the end of the year — the first ever composed by AI — and are inviting a range of artists to collaborate, including Ash Workman, British synth-pop band NZCA Lines, and electronic duo Darkstar.

Carré hopes to bring together different musical styles but link them under an over-arching conceptual theme, taken from one of Hans Christian Anderson's darker fairy tales about a writer who loses, and re-discovers, his shadow.

"It's like a mirror, a play on identity, so it's interesting with what we are doing," he said, adding that there were also parallels to be drawn with his goal of AI contributing to stylistic diversity.

"The shadow, in the tale it is traveling, and the more it travels, the more it lives," Carré said.

It bears mentioning that in the fairy tale the shadow eventually becomes corrupt, exchanges places with, and then kills the idealistic writer. So perhaps an echo of the malign AI is with us yet.

Ed Newton-Rex, a founder of Jukedeck in London, says his company, which offers to make custom soundtracks for videos, is also partly motivated by the idea of increasing access to what has historically been a somewhat rarified industry — at least in the classical world in which Newton-Rex trained.

"I did nothing but music for 21 years, studied music at Cambridge, was going down route of becoming a classical composer," he said. "In terms of democratization, once computers can write music they can give tools to people who don’t have this huge musical education and let them get involved in the music-writing process."

But unlike Flow Machines, Jukedeck has begun experimenting with using AI for every step of the process, including synthesizing the computer's composition into a fully-realized track. The technology is still in its early days, but Newton-Rex sees no reason why computers couldn't one day make the leap from pastiche to truly creative art.

"It's hard to argue that even very disruptive innovations aren't built on some preceding model and preceding experience," he remarked. "Having said that, part of the problem with asking if computers can be creative is defining what creativity is in the first place. We haven't agreed as a society what it is and how it works."

Jukedeck is using slightly different tech from Flow Machines: neural networks that imitate the human brain. These require much more training data than Markov Chains but, once trained, can be more powerful.

Roy and Carré freely admit that some of the implications and consequences of their creation may be out of their hands — and that's part of the excitement.

"With rap, or disco, often it's a new technology that creates a new style," said Carré. He gestured to an Akai sampler sitting among an array of keyboards, headphones, and espresso cups. Launched in the 1980s, the Akai allowed artists in the emerging hip-hop scene to sample classic tracks and produce beats without a drum kit.

"At the beginning, a lot of people were afraid that the pianist and the drummers will be replaced, but it never happens this way," Carré added. "It's humans that find the ways to use [tech] to make interesting things."

Originally published on Seeker.