deserializeCudaEngine fail after changing GPU scheduling mode

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi experts,

Recently I encountered a technical issue when changing the GPU scheduling mode through cudaSetDeviceFlags to cudaDeviceScheduleYield and cudaDeviceBlockingSync. After the change, deserializeCudaEngine returns null directly without any further information like error code or message. I did not see any indication that this API could not live with other GPU scheduling modes, so I just wonder if this is not officially supported or I have missed anything.

Please feel free to let me know if you need more context on it.

Thanks!
Jianan.

2

hello,

I’m not able to reproduce the symptoms. I modified sampleMNIST.cpp with:

std::cout << "Building and running a GPU inference engine for MNIST" << std::endl;
    err = cudaSetDeviceFlags(0x4| 0x2 ) ;
    if (err != cudaSuccess){
         std::cout << "cudasetdeviceflags failed" << std::endl;
         printf("err %x\n", err);
         return EXIT_FAILURE;
    }
    if (!sample.build())
        return EXIT_FAILURE;

    if (!sample.infer())
        return EXIT_FAILURE;

    if (!sample.teardown())
        return EXIT_FAILURE;

and got a 0xb or “This indicates that one or more of the parameters passed to the API call is not within an acceptable range of values.”

3

Hi there,

Sorry for wrong explanation, but can you try set either cudaDeviceScheduleYield or cudaDeviceBlockingSync not both of them at the same time? I do not think it could allow setting both flags at the same time, which results in the cudaErrorInvalidValue you got.

Thanks!
Jianan.