Engine Creation and reading Doesnt work in windows

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi all,

I was trying for creating an engine from sample code

sampleUffMNIST
TensorRT-5.0.4.3
Visual Studio 2017
windows 10

I have used the code snippets from GitHub - dusty-nv/jetson-inference: Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson. as suggested in Loading of the tensorRT Engine in C++ API - Jetson TX1 - NVIDIA Developer Forums

However the engine was successfully created.

// This is the code of writing engines////
IHostMemory* trtModelStreamtry = engine->serialize();
std::stringstream gieModelStream;
gieModelStream.seekg(0, gieModelStream.beg);
gieModelStream.write((const char*)trtModelStreamtry->data(), trtModelStreamtry->size());
std::ofstream outFile;
outFile.open("outtest.engine");
outFile << gieModelStream.rdbuf();
outFile.close();

But I tried reading and restoring the engine using the following code

std::vector<char> trtModelStream_;
size_t size{ 0 };

std::ifstream file("outtest.engine", std::ios::binary);
if (file.good())
{
	file.seekg(0, file.end);
	size = file.tellg();
	file.seekg(0, file.beg);
	trtModelStream_.resize(size);
	std::cout << "size" << trtModelStream_.size() << std::endl;
	file.read(trtModelStream_.data(), size);
	file.close();
}
std::cout << "size" << size;
IRuntime* runtime = createInferRuntime(gLogger);
assert(runtime != nullptr);
ICudaEngine* engine = runtime->deserializeCudaEngine(trtModelStream_.data(), size, nullptr);

It throws Exception thrown at 0x00007FFD2336E8B0 (nvinfer.dll) in sample_uff_mnist.exe: 0xC0000005: Access violation reading location 0x0000019563E06FA0.
 during runtime->deserializeCudaEngine

Is engine creation and usage supported in windows ???

2

Hi,

Have you managed to solve this problem? I’m experiencing nearly exactly the same problem in terms of the access violation when trying to load the newly created engine.

3

Hi,

did you solve this problem yet? I coded a network definition using C++ API and am currently trying to build the CUDA engine. But it threw me an exception “Exception thrown at 0x00007FF828C282BB (nvcuda.dll) in Project2.exe: 0xC0000005: Access violation reading location 0x000000B828DA0000.” at:

auto engine = builder->buildCudaEngine(*network);

I really appreciate any suggestions! Thank you!

4

@he44 what you are facing is some other issue. if you follow sample codes from TensorRT you should not recieve those errors.

@jmirza no i havent solved it yet