I am reaching out as a beginner developer with CUDA (integration with Unreal Engine), hoping to get some insights or assistance with this technical issue.
I have compiled GGML using CMake and CUDA 12.6 with the following configuration:
cmake -B build/release -DGGML_CUDA=1 -DGGML_OPENMP=0 -DGGML_VULKAN=0 -DGGML_CUDA_ENABLE_UNIFIED_MEMORY=1 -DGGML_CUDA_DMMV_X=512 -DGGML_CUDA_FA_ALL_QUANTS=1 -DGGML_CUDA_MMV_Y=16 -DGGML_LTO=1 -DGGML_CUDA_FORCE_CUBLAS=1 -DGGML_CUDA_F16=1 -DCMAKE_BUILD_TYPE=MinSizeRel
My goal is to build an application using Whisper ASR with GPU acceleration to reduce latency times. Following the build, I obtained .dll, .exp, and .lib files, which I then included in Unreal Engine through a custom plugin and modules. For the GGML module in Unreal Engine, I included all the .h, .hpp, and .cuh files from include and src in GGML.
The DLL files I included are as follows:
• ggml.dll
• cublas64_12.dll (from CUDA 12.6)
• cublasLt64_12.dll (from CUDA 12.6)
• cudart64_12.dll (from CUDA 12.6)
I also added the necessary .lib files for GGML, cublas, cublasLt, and cudart.
Following this, I compiled Whisper.cpp with CMake, also using CUDA 12.6. I created a module within my Unreal Engine plugin from Runtime Speech Recognizer for Whisper as well, including the respective .dll, .exp, .pdb, .lib, and .h files. In the plugin, I linked the dependencies such as the GGML and Whisper modules and added the definition GGML_USE_CUDA.
The issue I am encountering is that when I call whisper_full_parallel with n_processors set to 1, I receive the following error:
[2024.11.13-14.09.37:658][972]LogRuntimeSpeechRecognizer: Speech recognition progress: 0
[2024.11.13-14.09.37:761][979]LogRuntimeSpeechRecognizer: Error: ggml_cuda_compute_forward: IM2COL failed
[2024.11.13-14.09.37:762][979]LogRuntimeSpeechRecognizer: Error: CUDA error: invalid resource handle
[2024.11.13-14.09.37:762][979]LogRuntimeSpeechRecognizer: Error: current device: 0, in function ggml_cuda_compute_forward at D:\ggml\src\ggml-cuda.cu:2338
I am unsure if this issue stems from an incorrect structure or setup of headers, a misconfiguration in the DLL compilation, or another underlying issue. I would greatly appreciate any advice or guidance on resolving this error.
Thank you for your time and assistance.