A Spanish orchestra creates and performs a work made with AI for the first time: “It was a pastiche” |  Technology

A Spanish orchestra creates and performs a work made with AI for the first time: “It was a pastiche” | Technology

“Tell him to do a work like Shostakovich, with force, perennial, with metal and percussion instruments, with many dynamic contrasts,” suggests Rocío León, artistic sub-delegate of the RTVE Orchestra and Choir. The petition is in favor of artificial intelligence that creates music from text. After a few minutes, the machine returns a melody. Something very metallic begins to ring, then suddenly falls silent and the tone changes: “It sounds like a village orchestra,” says César Peña, member of the association. RTVE Innovation Laboratory. “Now she’s going rock, she’s figured out the metal genre, you can turn it off now,” he adds. We heard for a few minutes: “Do you have the essence of this man (Shostakovich)? » asks Isabella Rocafull, also a member of Lab a León. “No,” he answers without hesitation.

This request was only an additional test of the second session to create with artificial intelligence (AI) a work of a few minutes for symphony orchestra. So far, the result is bad. León then tried another route: “Tell him how Bach would write a piece for symphony orchestra in D major for action cartoons,” he suggests. The result is also strident: “I don’t like it, it doesn’t have a clear structure, which is fundamental in Bach. There was a moment after the introduction which was a disaster. It seemed like it started well, but then it got muddled again,” criticizes León.

Video: RTVE laboratory

The objective of the Lab and the RTVE Orchestra was to perform for the first time a symphonic piece created by artificial intelligence. EL PAÍS accompanied different stages of the process and spoke with several protagonists. The last two pieces were played at the start of a concert already scheduled at the Monumental Theater on November 17. “For the first time in the history of this theater, we are going to start with a small experiment, and at the end we will reveal the mystery,” announced to the newspaper the arranger Borja Arias, who also led the training of these two pieces. audience. The first was transcribed as it was created by the AI, the second was arranged to make more sense. Once finished, he revealed the trick: “The two works you just heard were composed by artificial intelligence,” he admitted. The audience then let out a long “ooooh”. They were surprised by the novelty, but not impressed by the artwork.

Music with emojis

Lab RTVE has documented the process called Lab Orquesta and he has been telling it since this Tuesday both on television, radio and on the web as well as in native pieces for his TikTok channel and other networks. They looked at various tools for creating music with AI. Only one supported elaborate text requests and yielded inconsistent results. There are others already available that only allow you to describe the query with labels or emojis, and which return something less elaborate.

Since 2022, AI has been a text and image creation tool alongside ChatGPT and other graphics tools such as Dall-E or Midjourney. Musicians have seen this process up close, but in their sector they have not experienced such an explosion: “What we are going to ask the machine is to construct, to the extent that it can and knows how to do so , a musical composition. » explained Manuel Ventero, general director of the Orchestra, at the beginning of the process. “We feel that the result will be, in the worst case, very good,” he added.

But the complexity of classical music poses an even more complicated challenge for the machine: It’s one thing to fill in the background of a YouTube video and another to create a work performed by dozens of musicians. The process showed that, for the moment, AI does not yet have the capabilities of Mozart or Bach, nor even those of a diligent student of composition: “It does very immature things,” laments Arias. They also tried an artificial intelligence tool to transcribe the music, but it went wrong.

“It’s more Indiana Jones”

Requests (or instructions) orchestra officials and journalists from the RTVE Lab first developed ideas, as is done for example with illustrations. The proposals aspired for the machine to combine composers, genres and instruments. But the AI ​​produced messy sounds, like a puzzle ungovernable. The first day of tests, last September on the stage of the Monumental Theater, already indicated that it was not going to be easy: “It’s more Indiana Jones”, “there are reggaeton influences here”, “it’s not is not a runaway. or “I don’t understand this language” were some of the comments made during this session.

After around thirty tests, those responsible for the project finally settled on a very standard request to obtain an acceptable result: “A piece of music to be orchestrated for a symphony orchestra by a human composer”. The music was mediocre and left composers alone: ​​AI won’t take away their jobs, just yet.

Human work remains essential. The arranger Borja Arias received approximately 20 minutes of AI-created music collected into 10 tracks. One transcribed it and another arranged it: “I tried to take what (the machine) wanted to tell me, I took a few elements, because there were a lot” , he explains. This is how Arias compares what he received from the machine with other arts: “Musical coherence is abstract and difficult to achieve. When you compose, you have to stick to a style. This room had a lot contributions and we didn’t know where they came from. It was as if he started to tell you the story of a protagonist who is a firefighter and then all of a sudden he continued to tell you the story of a carpenter, then a flower girl and finally ‘a judge,’ he describes.

The RTVE project anticipated that the orchestra musicians would not know the identity of the author until the end. They rehearsed and performed the piece as if it were anonymous. Lab journalists interviewed the orchestra members before they learned the truth. They felt something strange, without knowing what: “I didn’t like the first work (created by AI and only transcribed) but hey, it’s as true as we read it once”, said trumpeter Borja Antón after the first rehearsal. “Maybe with a little more reading and work we can give it a little more shape, but it’s a bit basic work.” When he was told who had composed the piece, he was not surprised: “A lot of things suit me. The first job was very bad. There was nowhere to take the concept. In the second (arranged by Arias), the melody is more catchy, the structure is better achieved,” he added.

Useful for inspiration

To the artists’ trained ears, it was obvious that something was wrong: “If that doesn’t sound familiar, it might sound like a mix of film scores that then try to be more classical then contemporary, but without be innovative. It’s a pastiche. It wasn’t unpleasant to listen to, it’s true. I imagined that the musicians would think that he was a mediocre composer,” says León, the artistic subdelegate.

For some reason, AI has a harder time creating something musically meaningful for an orchestra: “There are things that AI doesn’t know are very difficult to play. There’s a high C that stays there for a long time. It’s impossible to do. Physically, this is not possible. And then, there are no different shots, everything happens at once: you hear the melody with the harmony at the same time and you don’t recognize anything,” explains trumpeter Antón.

The difficulty of classical music does not necessarily have to be repeated in other styles. “To compose symphonic music, the truth is that you have a lot left,” says arranger Arias. “It left me quite calm and it also demonstrated the value of this type of music because of its harmonic and contrapuntal complexity and quite a deep message. With other styles, the AI ​​is a little closer to an optimal result. This leaves me calm because there is still room for improvement.

This lack of complexity does not prevent it from also being a useful tool to inspire composers in their work or to imagine less sophisticated works, according to León: “Composers could take it into account as a little help and not find themselves facing the blank page. To give an air of cinematic composition, AI can serve as a filling, atmospheric musical moment. Not for an orchestra, which is something deeper and richer.

You can follow EL PAÍS Technology In Facebook And X or sign up here to receive our weekly newsletter.

Subscribe to continue reading

Read without limits

_