Hi,
There is a possible solution in the post:
Hardware Accelerated JPEG encode/decode on Jetson Xavier JP 5.1.3 - #4 by DaneLLL
Please take a look and see if it can be applied to your use-case. Since hardware JPEG encoder does not support BGR, the color conversion is required.