AI Generated Music

If you thought that AI image generation was developing at a rapid pace, wait till you see music generation. It's progressing so fast that anything I post about it here will be out of date almost immediately.

The two main contenders in the race at the moment (as I understand it) are Udio and Suno, although there are others, as well as recent rumours about a third model that is as good as those two. Udio and Suno keep leapfrogging each other. I have only tried Udio, but it's very impressive. I found it was easy to slip into a creative high where I spent all day playing with it, very similar to the addictiveness of image generators.

By the time you read this, the generative music landscape will probably have changed, but at the moment, this is how Udio works:

You input a prompt such as "indie rock song about a dog in space", and then choose between three options:

1. Generated lyrics
2. Custom lyrics (you paste your own lyrics)
   
3. Instrumental
   

The system takes the lyrics, either ones it has generated for you or ones you have given it, and generates a 32 second segment of music with an appropriate vocal layer incorporating those lyrics. (Or without vocals if you chose Instrumental).

It's pretty mind-blowing if you think about what it's doing. Not only is it generating music in the style you specified, but it's creating vocals that match the music, using specified lyrics.

Say that a segment of music contains a verse and a chorus. It can figure out which lyrics go with which parts, and arrange them to match the song structure and timing. The vocal melodies also have to be in the right key and work with the song. And the vocal style of course has to be appropriate for the genre.

Udio does all this, and does it so well that for the most part, you can't tell it's AI generated. If you heard the song on the radio, you wouldn't know the difference unless you really knew what you were looking for.

Here is an example I created, of a heavy metal song:

And to show a different style, here's a trance dance track I also made with it:

Pretty convincing, right?

 

The Process

Although it sounds like you just push a button and music comes out, the process of making a song like either of the above is a little more involved. The heavy metal song took me two days to make. A musician might be able to write a song in less time than that. Here's why it took that long.

Udio generates music in 32 second chunks. You put in the prompt and choose one of the above options for the lyrics. It then generates two possible segments. I created about 20 before finding one that I liked enough to work with.

Next you can extend the 32 second segment in either direction. Again it creates two options each time. The generation time varies but was about 1-2 minutes at the time I did it with the free tier. Each extension took me about 10 attempts before I was satisfied with one. That then became the base for the next iteration.

There are also options to specifically generate an intro or an outro when extending, so that you can wrap the song up neatly at both ends. It also lets you crop bits you don't like. This is really useful because often I would extend by 32 seconds, and like some of it but not all of it.

Other Features

I've only used Udio's free plan. The paid subscription tier has two very tempting features I've not tried:

1. Inpainting. This means you can select a section of a song you've made and get it to regenerate that section. So if there was one lyric you wanted to change, or anything else at a particular point in the song, you can change it.
2. Music as input. As of yesterday, you can now upload music as a starting point and use Udio to extend it. This will be amazing for anyone looking to remix music based on a sample.
   

Exciting times. The main reason I haven't upgraded to the paid tier to get these features isn't the cost in money. It's that I could happily sink into a week of creativity churning out songs that no one but me will ever hear. Perhaps I will anyway when I need a break from studying.