Meta's 'MusicGen' is ChatGPT for music: How it works
Artificial intelligence (AI) has impacted various artistic endeavors, including music. Meta has now announced a music equivalent of ChatGPT. Dubbed 'MusicGen,' it is an AI model that can generate music. The tech giant has open-sourced the tool, which can create music from simple text prompts and melody. Let's take a look at what MusicGen is capable of.
Why does this story matter?
Meta began its AI journey later than its peers. The company, however, has been trying to make up for that. With MusicGen, Meta aims to take on Google, which released its own AI-based music generator, MusicLM, earlier this year. The last few weeks have seen multiple AI-related announcements from Meta, which has taken the open-source approach to challenge its Big Tech brethren.
MusicGen can create music from audio
MusicGen is a single-stage transformer language model (LM) that can create 12 seconds of audio. Users can also use reference audio to steer the LM. In that case, MusicGen will pay attention to both the text prompt and the reference audio. It can also take music and modify it. MusicGen is based on the EnCodec audio tokenizer.
The AI model was trained on 20,000 hours of music
According to a research paper on the model, Meta trained MusicGen on 20,000 hours of licensed music, which includes an internal dataset of 10,000 music tracks and about 390,000 instrument-only tracks from Shutterstock and Pond5. Considering the controversy surrounding training AIs with license material, Meta said the data used to train MusicGen was covered by "legal agreements with right holders."
The AI tool's demo is available on Hugging Face
Users can find a demo of MusicGen through Hugging Face's API. The speed of the process will depend on how many are using the API. On the other hand, users can download the code for MusicGen from GitHub and run the code. It will, however, require the right hardware, including a GPU with 16GB of memory.
Music Gen seems to be better than its rivals
Music generation is an area that lacked quality AI generators so far. With MusicGen, Meta aims to address that. Meta has a page showcasing how music generated by MusicGen fares against its rivals, including MusicLM, Riffusion, and Musai. Music-generating models like MusicGen are nowhere close to replacing musicians. However, MusicGen does seem to have an upper hand over other models.
