I hava something wiht CMakelists on Liunx

my CMakelists :

cmake_minimum_required(VERSION 3.10 FATAL_ERROR)

project(baking_ao)

set(OPTIX_DIR …/NVIDIA-OptiX-SDK-6.5.0-linux64/include)

set(CUDA_DIR …/…/…/usr/local/cuda-10.2/include)

include_directories(${CUDA_DIR})

include_directories(${OPTIX_DIR})

aux_source_directory(./ SRC_MAIN)

#link_directories(…/NVIDIA-OptiX-SDK-6.5.0-linux64/lib64)

#set(optix optixu optix_prime)

find_package(CUDA QUIET REQUIRED)

add_executable(baking_ao ${SRC_MAIN})

but when i compile the project

i get a lot of error like this

[build] /home/s/baking_ao/…/NVIDIA-OptiX-SDK-6.5.0-linux64/include/optix_prime/optix_primepp.h: In member function ‘void optix::prime::QueryObj::setCudaStream(optix::cudaStream_t)’:
[build] /home/s/baking_ao/…/NVIDIA-OptiX-SDK-6.5.0-linux64/include/optix_prime/optix_primepp.h:682:48: error: cannot convert ‘optix::cudaStream_t {aka optix::CUstream_st*}’ to ‘cudaStream_t {aka CUstream_st*}’ for argument ‘2’ to ‘RTPresult rtpQuerySetCudaStream(RTPquery, cudaStream_t)’
[build] CHK(rtpQuerySetCudaStream(m_query, stream));
[build] ^

** this is content in optix6.5 file ,how can i resovle this problem . **

and my device:
linux : ubuntu 18.04
gpu:rtx 2060
optix : 6.5
cuda version : 10.02
cmake :3.10.2
driver version: 440.82

The error message describes that there is a namespace mismatch between optix::cudaStream_t and cudaStream_t.

But cudaStream_t is defined as typedef struct CUstream_st *cudaStream_t; in optix_prime.h outside of namespaces and it’s also included outside of namespace optix inside the OptiX headers.
I’m not sure why the member function inline void QueryObj::setCudaStream( cudaStream_t stream ) is putting an optix:: namespace in front of the cudaStream_t argument type then. That sounds incorrect by the compiler.

Possible fixes:

  • Try another GCC.
  • Change the function signatures to explicitly use the root namespace ::cudaStream_t stream.

Other than that, what exactly are you’re trying to build there? Is that the (deprecated) OptiX Prime ambient occlusion baking example at https://github.com/nvpro-samples/optix_prime_baking or something based on that?

Please note that OptiX Prime is not using the RT cores on RTX boards, means OptiX Prime based applications are not going to run at the possible maximum speed on your RTX 2060.
OptiX Prime is discontinued with OptiX 7 already. I would recommend to use the OptiX 7 API for these kinds of algorithms today.

yes,I have run successfully on Window,but when i use the same code in linux, i have lots of bug, except this problem,also like float3 error,i think is maybe system problem,I first tried to solve this problem.I have tried using OptiX 7 API,but is too difficult to me.thinks,and my english is not well,please understand.

bucause all error are when i use #include -something lib-,so i think my CMakeLIst maybe hava some mistake.

I fear I cannot help with CMake issues under Linux if the same is working under Windows. I’m not a Linux user.

Maybe try using the same CMake version first.

I’m using a different order of find_package and adding include directories from the found SDKs.
Check that the hard-coded paths you’re using are actually matching what CMake found.

Do the OptiX DK 6.5.0 “prime” examples compile on your Linux system?
If yes, that would indicate the issue is not about your installed SDKs.

Then it would be a matter of finding out what the different compiler settings are between the OptiX SDK “prime” examples and your project.

I’ll try.thinks.hava a good day.

I have another question,when i use prime on OptiX SDK 6.5,and my CUDA version is 10.2.
Are there some compatibility issues that can be caused by the OptiX version being too different from the CUDA version.

I tried running primeInstancing in OptiX6.5 prime under window, but it failed.

OptiX Prime is not running any of your CUDA code. The ray queries are hard-coded inside OptiX Prime and not programmable. All it needs is a display driver supporting the CUDA version used inside that OptiX version which is specified inside the respective OptiX version’s release notes.

Again, I wouldn’t recommend to use OptiX Prime on RTX boards at all. It should be faster to use OptiX (and esp. OptiX 7) and implementing such ray queries on top of that.
That will use the RT cores and provide a lot more flexibility for the scene graph, geometric primitives, hit result format, and some more possibilities.

Define “it failed”.
The program is not interactive and only renders a *.ppm image to disc and exits.

sorry,is my problem i modify source code,i‘ll try OptiX 7.thinks.

Have a look at the optixRaycasting example inside the OptiX SDK 7.0.0 which demonstrates that.
(That example exists since OptiX SDK 5.)

think you.