Welcome to NVIDIA Maxine, a GPU-accelerated SDK for building virtual collaboration and content creation applications.
Hi @TomNVIDIA could you point out if there a broadcast feature, please?
Which sdk is required to implement it? Or it is out of the box solution?
@Andrey1984 Maxine is a set of SDKs that can be engineered into solutions. The SDKs are focused on applying AI to audio and video, so these SDKs do not provide broadcast capabilities themselves, but could be tied into broadcast solutions. The Maxine SDKs can run on servers or on laptops and desktops. If you’re looking for an out-of-the-box solution, then you might try the NVIDIA Broadcast App or one of the solutions from partners who have integrated the Maxine capabilities.
3 Likes
@jkrinitt Thank you for following up.
However, at the Maxine Download page it is stated “All of the NVIDIA Broadcast Engine SDKs are now included in NVIDIA Maxine.”
My question is which SDK of the three “Video SDK, Audio SDK, AR SDK” has the broadcast option if at all? I was not able to find one in the documentation.
Regarding the pointed out broadcast app runs that runs only on Windows 10 with GPU >=2600RTX. Does it support 4K resolution broadcasting? or 2K only?
How about Ubuntu/ Linux? Which of the Maxine components will provide broadcast option on Linux? on x86_64 linux? 0n aarch64 tegra-linux? On either of the two?
So the broadcast SDK doesn’t have the broadcast per se as a feature, only in the name of the SDK? Could you confirm?
Hello @Andrey1984.
There are two ways to utilize the capabilities of Maxine. One is using the Broadcast App, which does not have a broadcasting capability in itself, but can be used with thousands of different applications as a source for broadcasting. The NVIDIA Broadcast App allows you to select a source and creates a virtual device (camera or microphone) that can be used in a variety of apps such as video conferencing, OBS, and others. The app is only available for Windows.
The other way to use the Maxine capabilities is via the SDKs. None of the SDKs have built in broadcasting capabilities, they provide the AI processing capabilities and can be integrated into broadcast applications. There are different input, processing and output resolution capabilities for the different SDKs, but in general, yes, 4k resolution is supported. One example of this is Super Resolution, which supports conversion of 1080 video into 4k. The SDKs are available for both Windows and Linux. They are currently only supported on x86 with Turing and Ampere GPUs with Tensor Cores.
You are correct, the Broadcast in the name is referring to use in broadcasting applications, not in providing the broadcasting capability itself. There are many organizations with whom we’ve collaborated to integrate these capabilities into their broadcasting apps.
1 Like
we are asked to look into project proposal that is based on Maxine
We need a cloud based web service that will remove background from webcam video streams by using the Maxine SDK.
We are currently thinking of using the Jitse broadcasting framework or otherwise building our own platform.
We are looking for someone with experience in body segmentation and handles high definition video feeds.
Could you advise on feasibility of the requirements? Will it be possible to implement it with Maxine? Will you support such development effort?
Thanks
Andrey, you can implement Maxine capabilities either on the client or the server. From your message, it seems that you want to implement Maxine on the server. This would enable any device, whether it has a GPU or not, to have the background removed through the Maxine SDK. To do this, you need a video conferencing infrastructure that is an MCU. An SFU infrastructure would not allow you to access the video to run it through the SDK and remove the background or add other Maxine quality improvement or effects.
@jkrinitt
thank you for following up
could you extend MCU/SFU meaning please?
what is MFU
what is SFU?
Given there is a cloud server with supported GPU how to start implementing?
@jkrinitt we can not install broadcast app on Windows 10 Pro with T4;
is that particular envirnment not supported? for the broadcast app? for video sdk? for both? neither of the two?
this one is Quadro RTX solutionu by Nvidia offered at Google Cloud
Seriously, someone use Deep Learning to create an automated cloth weight painting application…I just spent an entire week trying to perfect the weight map of one skirt…
Hi @Andrey1984 to know more about MCU and SFU architectures for video conferencing, you can check this link
Hi all,
Good to see you helping out each other here. Since we don’t find a Audio post we post here in the general topics here.
We are a start-up with a focus of deploying a new-age AR chatroom project on the cloud. We tried out several technologies and zeroed-in on NVIDIA Maxine and its siblings. We downloaded the resources and configured the binaries to our needs, but have been facing trouble with the NVIDIA Audio SDK. We tried the Samples codes on our GCP instances.Though the binaries write out the processed audio files almost all the time, the execution ends with an error “NvAFX_Run() failed”(Ref. tailing error log)
Since the execution control depnds almost entirely on configurations we just tried on other instances and found the same error. Our local machine is of a Pascal architecture and so unsuitable for the Maxine technology. We are in a confusion as to how to move ahead. We have enough know-how in CUDA and other matters but since the code is not available we could not proceed. We posted on the Developer’s forum but there is not much response.
We would like to request you to kindly show us some light with regard to this issue so that we can stick on to Maxine and its promising capabilities.
Best Regards,
Muthu
(For Kickback.Space Inc.,)
Hi @asawarkar
Thank you for your response.
I understand that nvidia won’t support / share an example for neither MCU, nor SFU, right?
Hello @muthu3
Thank you for trying out NVIDIA Maxine Audio SDK. Unfortunately, Maxine SDKs don’t work on Pascal architecture. I’d encourage you to try running the SDK on either of these hardware platforms: V100, T4, A10, A100, A30. Maxine SDK docker containers are now available with pre-packaged dependencies which are much more convenient to pull from NGC and run on your cloud instance. For Maxine Audio SDK, please check this link and download the containers from NGC. Let me know if this helps thanks!
@Andrey1984 There is an early access version of MCU reference pipeline with a sample video conferencing application with NVIDIA Deepstream container. You’d have to apply for early access here: link
@asawarkar the lins that is shared by you opens the download page
could you extend which exactly file to download to try the MCU?
Hi @asawarkar , Thanks a lot for the response. I had just mentioned that since my machine is a Pascal I did not try Maxine on it. I tried it on GCP with Tesla T4 GPUs. Thanks for your suggestion to try out the Docker versions. I will check out and get back with a hopefully positive note.
You may want to spin up a new topic to discuss this. I think others will be interested in this discussion.
1 Like