[Test Request]- Usage of POCL cuda backend combined with Jetson_FFMPEG library for rapid multimedia trancsoding and filtering

Autonomous Machines Jetson & Embedded Systems Jetson Nano
, , , ,
#1

Hi everyone!

I’m looking for help in a very bare bones initial test to use OpenCL ffmpeg filters on a jetson nano.

very specifically I’m hoping to achieve an ffmpeg command in the ball park of

ffmpeg -hwaccel cuda -init_hw_device opencl=ocl -filter_hw_device ocl -c:v hevc_cuvid -resize 1920x1080 -i INPUT.mp4 -vf "format=p010,hwupload,tonemap_opencl=tonemap=mobius:param=0.01:desat=0:r=tv:p=bt709:t=bt709:m=bt709:format=nv12,hwdownload,format=nv12" -c:a copy -c:s copy -c:v 264_nvenc OUTPUT.mp4

however because nvenc/nvdec hasnt been implemented on the nano yet, the command would use this library to support the hardware encoder/decoder in ffmpeg and use this library (POCL) to add openCL support.

ffmpeg -hwaccel cuda -init_hw_device opencl=ocl -filter_hw_device ocl -c:v hevc_nvmpi -resize 1920x1080 -i INPUT.mp4 -vf "format=p010,hwupload,tonemap_opencl=tonemap=mobius:param=0.01:desat=0:r=tv:p=bt709:t=bt709:m=bt709:format=nv12,hwdownload,format=nv12" -c:a copy -c:s copy -c:v h264_nvmpi  OUTPUT.mp4

This should be able to use the decoding block on any of the current jetson model, keep in gpu system ram, utilize openCL 1.2 filters, once again keep it in gpu ram, encode to h264 to an output file.

Would test myself on hardware, but don’t have the spare budget to commit to the device if this isn’t going to work. [Computer engineering student on a tight budget, looking for advice and if someone with hardware might be able to help me out.]

#3

Hi,
We have a package to enable hardware decoding in ffmpeg. Please check
Jetson Nano FAQ
Q: Is hardware acceleration enabled in ffmpeg?

Related topics:

#4

Hi Dane!

Thanks for the pair of links! I already knew about the decoder- but AFAIK the encoder has yet to be implemented outside of Jcover’s git repository correct?

Also, tho not an officially supported use case,do you believe that by using the POCL 1.6 library to serve as a OpenCL to Cuda backend, we would be able to utilize OpenCL ffmpeg filters?

#5

Hi,
Yes, in NVIDIA package, hardware encoding is not enabled.

Since we have independent hardware engines NVENC and NVDEC on Jetson platforms, we usually leverage the engines and leave GPU for doing deep learning inference. Don’t have experience to enable OpenCL ffmpeg. May see if others can share suggestion on this.

You may consider to use jetson_multimedia_api. Hardware encoding/decoding can be done through v4l2 interface. Please take a look at document:

#6

Hi Dane,

Thanks for the reply! I’ve looked through the docks, but unfortunately one of the requirements for my project is the use of FFMPEG.

The pipeline would be to decode in the hardware decoder, do the filtering/image processing on the gpu (tone mapping and resizing) then pass that to the hardware encoder to be encoded to the relevant format. [Which would then be sent to a different device]

Essentially, between both the current nvidia ffmpeg build and the version created by @jocover , I don’t think I need to worry about being able to leverage the build in decode/encode engines.

it also seems that both @girgink and @znmeb have gotten OpenCL 1.2 support via the POCL/Cuda backend (per this thread)

Since POCL creates a OpenCL 1.2 compliant device. If I can somehow confirm that using POCL works with ffmpeg open CL filters, then life will be pretty sweet!

#7

I have POCL working in a Docker image - the Docker context is here if you want to fork it:

I also have CSound and ChucK on that image but the build does POCL first. It shouldn’t be too hard to add FFMPEG.

#8

Hey mate!

Thanks for replying!

Have you noticed any issues with the OpenCL implementation? I’m a student, so trying to get a general idea of if what I want to do is even possible before spending part of my budget!

the Image buffer size looks large enough (8096*8096 in 2D) from the clinfo-nano.txt output file.

if you don’t mind terribly, would it be possible to test this set of commands on this file or any video file you might have on hand?

ffmpeg -init_hw_device opencl=ocl -filter_hw_device ocl -i INPUTFILE format=p010,hwupload,tonemap_opencl=tonemap=mobius:param=0.01:desat=0:r=tv:p=bt709:t=bt709:m=bt709:format=nv12,hwdownload,format=nv12" -c:a copy -c:s copy -c:v libx264 OUTPUTFILENAME.mp4

so long as the output is different in colour than the input, that means the filter is being initialized and that I should be rocking a rolling!

#9

Issue is open: https://github.com/edgyR/edgyR-containers/issues/23

1 Like
#11

Tracking the issue! Thanks for taking the time mate!

Super excited to see the results

FCLC