By Payal Dhar | Illustration by Dan Page
Ludwig van Beethoven’s Symphony No. 10 might be considered the longest freelance assignment in history—commissioned in 1817 by the Philharmonic Society of London and finally performed in October 2021. When the maestro died in 1827, he left behind an incredible musical legacy of nine symphonies, but the tenth existed only in the form of some random notes and scribbles. That is, until two centuries later, when historians, musicologists, composers and computer scientists came together to finish the piece with the help of artificial intelligence (AI).
sold in seven minutes. Photo courtesy of Christie’s Images LTD. 2022.
This wasn’t the first time someone had attempted to imagine what Beethoven’s unfinished symphony might have sounded like. In 1988, the English musicologist Barry Cooper, known for his work on Beethoven, put together his interpretation of the first movement. More recently, in September 2021, the Swiss orchestra Nexus performed an AI-created, four-minute extract of yet another interpretation. Because it was an artificial neural network (ANN) that “composed” the piece, they called it the BeethovANN Symphony 10.1.
Music isn’t the only form of art attempted by machine learning (ML)—a form of AI that attempts to replicate intelligent human-like behavior in machines. In September 2020, the Guardian published an essay titled, “A robot wrote this entire article. Are you scared yet, human?” It was written by GPT-3, a powerful new AI language generator, which was provided instructions and a 25-word intro. The resulting article, put together by an editor from eight different outputs generated by the AI, led to much excitement about whether AI would make writers and journalists redundant.
Even the theater has “collaborated” with AI. In August 2021, a play was co-created by humans and GPT-3 in London’s Young Vic theater. An example from the visual art world is the AI-created piece titled “Edmond de Belamy,” the first to go under the hammer at Christie’s in New York in 2018. It sold for almost half a million dollars.
Of course, this isn’t an exhaustive list.
The human element of AI
To imply that AI is capable of creativity or that it can understand and create art seems like an exercise in mixing oil and water, at least in terms of how we imagine both AI and art. Yet, from the above examples, it appears that ML is capable of human-like “imagination.” If a machine can replace Beethoven, how long before it makes all artists, musicians, poets and playwrights redundant?
Fortunately, the question itself is redundant. The element that makes AI appear to think and create is actually the humans—the artists and programmers—behind it. Art and its appreciation might be an entirely subjective representation and interpretation of beauty and meaning, but AI isn’t just the result of a machine’s impartial logic. These so-called intelligent systems come with the perspectives and information they were built and programmed with.
“Modern AI and modern ML is all about just taking small local patterns and replicating them,” says Matthew Guzdial, a researcher in machine learning and creativity. He was talking about Beethoven’s unfinished symphony on Scientific American’s 60-Second Science podcast. “And it’s up to a human to then take what the AI outputs and find the genius…The genius wasn’t in the AI. The genius was in the human who was doing the selection.”
In August 2021, Evgeniya Fedoseeva, founder and CEO of the knowledge management startup Generation KM and a researcher in AI/ML ethics, attended the screening of the play “AI.” The production was directed by Jennifer Tang and developed by Chinonyerem Odimba, Nina Segal and—of course—GPT-3. It was set up as a live workshop spread over three days, with the director, actors, production team and the AI working together to generate the dialogue for the script.
“It was almost like you were viewing a live production of the play,” Fedoseeva says, the play being cowritten with a “fair contribution” from the AI algorithm. Screenwriters would pose a question and GPT-3 answered back. At the end of the workshop, actors performed the whole play, accompanied by music suggested by the AI.
“The result,” Fedoseeva says, “was interesting.” Human-machine interaction is her research area, and what she found striking was the AI’s language selection. “Elements of biased answers were still there,” she adds, embodying almost every stereotype you could think of. It’s not surprising considering the data source for GPT-3.
“You have to remember one formula for all AI—the quality of input equals the quality of output. So the biased nature of the AI algorithm is purely based on the quality of data you feed in,” says Fedoseeva. Which, in GPT-3’s case, was the Common Crawl corpus collected over eight years of web crawling Reddit, books on the internet and the whole of English-language Wikipedia. As a result, it contains a fair amount of toxic language. “What GPT-3 produced in some cases was so bad, the actor was uncomfortable,” she says. (The show’s webpage warns against “strong language, homophobia, racism, sexism, ableism, and references to sex and violence.”)
“GPT-3 can generate impressively fluid text, but it is often unmoored from reality,” read a 2020 review in Wired. Fedoseeva also experienced this. “You [can] see it learning through itself, polishing its own answers,” she says, leading her to an ethics question. If you are feeding it certain data sources, she asks, and the machine gives you an assured answer, should you believe it? “A machine will answer you confidently all the time,” because it is programmed to do so—it can also answer you in the voice of Morgan Freeman if you so program it, she adds. But it doesn’t mean it’s the right answer.
To create like a machine
“Composing” Beethoven’s unfinished symphony started with an ML platform called Playform AI, developed by Ahmed Elgammal, Ph.D., and team at Rutgers University. To teach the AI to “think” like the German composer, they fed it Beethoven’s complete works, including his notes and sketches. Next, they taught it Beethoven’s creative process—“How he would take a few bars of music and painstakingly develop them into stirring symphonies, quartets and sonatas [from] his sketches and notes,” writes Elgammal in an essay in The Conversation, a nonprofit news organization.
It is similar to using predictive text in an email—the AI works well in short pieces but descends into gibberish if you continue long enough. This was where the human element came into the project: The Austrian composer Walter Werzowa was responsible for picking up fragments from the AI output and piecing it together with Beethoven’s notes to create the entire symphony piece by piece.
The genius wasn’t in the AI. The genius was in the human who was doing the selection.
—Matthew Guzdial, researcher in machine learning and creativity
But is it original art?
“If you have a large enough base of samples for a machine to learn from, or you have an expert artist who’s been able to guide the algorithm on what’s going to come out, you can have something that somebody will appreciate as art,” says Pete Herzog, hacker, analyst and researcher. Herzog is also co-creator of an experimental project called Invisibles, which uses AI to correlate research on sound, music, behavioral psychology and the physical effects of frequencies to determine a musical template.
Herzog was interested in exploring whether it’s possible to make music that primes our brains for focus. Along with professional musicians and using vast amounts of existing research, they made real music, “and not white noise, static, leaves crunching, waves rushing—which is typical for this kind of thing.” The ML element, he adds, “figures out how to make it better, or how to fix it so that it fits our template better.”
ML systems can be trained to spot patterns and perform calculations that inspire us, and augment our creativity and productivity—as Invisibles does. “[But] they’re not really creating themselves,” Herzog says. “I don’t think [AI is] going to replace [human creatives]; the only thing that maybe could happen is that [it] can make so much art so much faster.”
Want to read more stories like this? Check out Realize magazine.