DRIVE OS Version: 6.0.6.0-32441545
Issue Description: We’re trying to encode processed images with H265 or H264.
Hello Nvidia Forum Support,
we are developing an application with an Nvidia Drive AGX Orin device to read out multiple cameras, do some processing like scaling and rectifying. We then want to send the processed images to another device (Nvidia Drive AGX Thor) which should display the multiple processed images as videos.
Currently, we are struggling with the encoding part. We want to use H265 (H264 would be acceptable too but we prefer H265), it’s not clear to us how we can encode the images.
At first, we wanted to use the API like in the Sensor Serialization / Camera Encoding Tutorial, but unfortunately, this tutorial does not match the actual API, since the dwSensorSerializer_serializeCameraFrameAsync function does not take a dwImageHandle_t but a dwCameraFrameHandle_t. But our processed images work with dwImageHandle_t so we can’t use this serialization. We’re also not really trying to serialize a sensor but to encode our processed images.
Because of this, we are now trying to use nvmedia’s image encoder, but it’s not quite clear to us how to actually integrate this into our application. After creating the NvMediaEncodeInitializeParamsH265, using NvSciBufModuleOpen and creating a NvSciBufAttrList and filling it, the NvMediaIEPCreate returns a nullpointer. I believe the error my in not filling that NviSciBufAttrList correctly, but don’t know how to do it correctly.
We use this method to try and create the nvmedia encoder:
void Encoder::initialize_nvmedia_encoder() {
// The documentation says getting the version is a precondition. That
// makes no sense to me but whatever.
NvMediaVersion \_version; // unused
CHECK_NVMEDIA(NvMediaIEPGetVersion(&\_version),
“could not get nvmedia IEP version”);
// We create an encoder of type HEVC, this requires a
// NvMediaEncodeInitializeParamsH265 as the parameter struct.
NvMediaEncodeInitializeParamsH265 params = {}; // TODO: create params
// this is some nv sci buffer container that we need
this->nv_module = nullptr;
CHECK_NVSCI(NvSciBufModuleOpen(&this->nv_module), “could not open nv_module”);
// this is a list of attributes which tells nvmedia how to allocate some
// kind of memory.
this->nv_attrlist =
nullptr; // this is a pointer type, set by the next function call
CHECK_NVSCI(NvSciBufAttrListCreate(this->nv_module, &this->nv_attrlist),
“could not create nv_attrlist”);
CHECK_NVMEDIA(NvMediaIEPFillNvSciBufAttrList(NVMEDIA_ENCODER_INSTANCE_0,
this->nv_attrlist),
“could not prepare NvSciBufAttrList”);
// BUG: for some reason this currently always returns a nullpointer
NvMediaIEP\* encoder = NvMediaIEPCreate(
NVMEDIA_IMAGE_ENCODE_HEVC, ¶ms, this->nv_attrlist,
settings::encoding::ENCODING_MAX_BUFFERING, NVMEDIA_ENCODER_INSTANCE_0);
if (nullptr == encoder) {
throw std::runtime_error(“Encoder could not be initialized!”);
}
this->nv_encoder = encoder;
}
It would be very appreciated if you could help me understand how to encode image data from a dwImageHandle_t .
Greetings,
Christoph
