RUNTIME_EXCEPTION : Non-zero status code returned while running Conv node

Accelerated Computing CUDA CUDA Programming and Performance
,
1

Hey everyone,
I’m using CUDA Toolkit version 11.8 and cuDNN version 8.9.6 with my RTX 4070 GPU.

I am getting this error that I haven’t been able to solve:

onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Conv node. Name:‘Conv_242’ Status Message: D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:121 onnxruntime::CudaCall D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:114 onnxruntime::CudaCall CUDA failure 2: out of memory ; GPU=0 ; hostname=LAPTOP-61H6U28U ; file=D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_allocator.cc ; line=48 ; expr=cudaMalloc((void*)&p, size);

Any hints as to why this might be?
Below the full error log:
File “C:\Users\User\Desktop\workspaces\roop\run.py”, line 6, in
** core.run()**
** File “C:\Users\User\Desktop\workspaces\roop\roop\core.py”, line 217, in run**
** start()**
** File “C:\Users\User\Desktop\workspaces\roop\roop\core.py”, line 169, in start**
** frame_processor.process_video(roop.globals.source_path, temp_frame_paths)**
** File “C:\Users\User\Desktop\workspaces\roop\roop\processors\frame\face_swapper.py”, line 100, in process_video**
** roop.processors.frame.core.process_video(source_path, temp_frame_paths, process_frames)**
** File “C:\Users\User\Desktop\workspaces\roop\roop\processors\frame\core.py”, line 79, in process_video**
** multi_process_frame(source_path, frame_paths, process_frames, lambda: update_progress(progress))**
** File “C:\Users\User\Desktop\workspaces\roop\roop\processors\frame\core.py”, line 57, in multi_process_frame**
** future.result()**
** File “C:\Program Files\Python39\lib\concurrent\futures_base.py”, line 439, in result**
** return self.__get_result()**
** File “C:\Program Files\Python39\lib\concurrent\futures_base.py”, line 391, in __get_result**
** raise self._exception**
** File “C:\Program Files\Python39\lib\concurrent\futures\thread.py”, line 58, in run**
** result = self.fn(self.args, self.kwargs)
** File “C:\Users\User\Desktop\workspaces\roop\roop\processors\frame\face_swapper.py”, line 81, in process_frames*
** result = process_frame(source_face, reference_face, temp_frame)**
** File “C:\Users\User\Desktop\workspaces\roop\roop\processors\frame\face_swapper.py”, line 72, in process_frame**
** temp_frame = swap_face(source_face, target_face, temp_frame)**
** File “C:\Users\User\Desktop\workspaces\roop\roop\processors\frame\face_swapper.py”, line 60, in swap_face**
** return get_face_swapper().get(temp_frame, target_face, source_face, paste_back=True)**
** File “C:\Users\User\AppData\Roaming\Python\Python39\site-packages\insightface\model_zoo\inswapper.py”, line 53, in get**
** pred = self.session.run(self.output_names, {self.input_names[0]: blob, self.input_names[1]: latent})[0]**
** File “C:\Users\User\AppData\Roaming\Python\Python39\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py”, line 217, in run**
** return self._sess.run(output_names, input_feed, run_options)**
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Conv node. Name:‘Conv_242’ Status Message: D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:121 onnxruntime::CudaCall D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_call.cc:114 onnxruntime::CudaCall CUDA failure 2: out of memory ; GPU=0 ; hostname=LAPTOP-61H6U28U ; file=D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_allocator.cc ; line=48 ; expr=cudaMalloc((void)&p, size);**

2

you’re attempting to use more memory than your GPU has available.

This is from a cudaMalloc call in ONNX:

This will require changes to the work you are doing that uses ONNX, which you haven’t fully shown here. In any event, this isn’t the right forum to ask about ONNX issues, or Deep Learning frameworks.