Enabling API for jetson container AudioCraft

Hello,

I would like to use the jetson container AudioCraft in order to run a server that can offer an API. I want do do something similar of what I’ve already done with stable-diffusion (see here).

In the official tutorial I see that it’s started a Jupyter Web Server, but I don’t understand the reason why it’s not possible to have a front-end equivalent to what we have with stable-diffusion (MusicGen: see image)

So, my 2 questions:

  1. Is it possible with the current jetson container of AudioCraft enable a server with EndPoints in a similar fashion of what is already possible with stable-diffusion?
  2. (Optional) Is it possible to enable the MusicGen front-end in a similar fashion to stable-diffusion front-end?

Any help would be greatly appreciated.

@esteban.gallardo just browsing through the AudioCraft code on github, I don’t see it supporting REST APIs. It has API documentation for Python here:

Note that the jetson-containers for stable-diffusion-webui and text-generation-webui just run those projects, and the projects themselves implement the REST APIs you are using. It runs inside the container when you start those apps with the corresponding flags, but I didn’t add code to those projects implementing the REST APIs. If a project doesn’t implement it, you would need to expose it yourself (i.e. via a Python script that loaded the model and used flask/fastapi/ect to serve your desired REST endpoints)

I tried to import the libraries to run it in my Flask application. Unfortunatelly, I’ve not been able to import any successfully. I’ve spent several days trying to compiling repos, installing wheels, etc, etc… to get the whole thing working without any luck. Torch libraries are a nightmare. So far I work with Torch without CUDA available because it’s impossible to do it any other way. When I manage to install Torch with CUDA nothing else is compatible with it, torchvision not compatible, torchaudio not compatible, nothing works. You can clone the repos, compile them. Nothing works. I will wait for the future until there is good integration of the torch libraries in the Jetson AGX Orin.

Try building your flask application on top of the audiocraft container which already has it installed and PyTorch working. PyTorch, torchvision, torchaudio, ect do work on Jetson, you just need to have the CUDA-enabled versions installed (or build them yourself). The containers make sure the correct versions stay installed.

2 Likes

Inside the container it’s possible to run the web front-end I previously mentioned with:

demos/python3 musicgen_app.py --share --listen 0.0.0.0

The issue is that in order to complete a sound creation request it needs from the package ffmpeg.

I’m not experienced with dockers so what I’ve tried to include that package has failed.

On the other side I’m trying to create an endpoint in the container but I’m not able to install Flask in order to create the service.

Hi @esteban.gallardo , sorry for the delay - if you are still stuck on this, try running the voicecraft container instead (it is based on the audiocraft container, but also installs ffmpeg). You can still run the original audiocraft in it (or voicecraft)

Yes, I’m stuck. I’ve a lot stuff to program, a tight deadline and having to face docker problems is draining a lot of time.

VoiceCraft also didn’t work.

@esteban.gallardo just made these fixes and rebuilt the audiocraft/voicecraft containers in commit updated audiocraft and voicecraft · dusty-nv/jetson-containers@49df6bc · GitHub

  • added ffmpeg to audiocraft
  • tried to get openai-triton working, it would not
  • added XFORMERS_FORCE_DISABLE_TRITON=1 to xformers
  • added python3 demos/musicgen_app.py --listen 0.0.0.0 to audiocraft start-up

Here are the updated container images you can pull:

dustynv/audiocraft:r36.3.0
dustynv/voicecraft:r36.3.0
1 Like

Thanks for your support. Unfortunatelly there are still missing libraries when trying to load and generate audio.

By any chance is there any information about how to create a custom container? I’m a front-end developer (Unity3D) but I would like to try by myself to derivate from jetson PyTorch container, in order to install audiocraft and voicecraft with CUDA support.