Trace time definitely depends on how many triangles the ray comes close to, the time is not independent of the scene geometry. Rays that almost hit a lot of triangles without actually hitting any will be slower than rays that hit something nearby, and slower than rays that aim away from your scene into empty space.
To configure OptiX Prime to give up after some distance, you can set the t_min and t_max value of your rays, which will perform traversal only within that interval, and it will stop checking for intersections beyond the t_max distance value. For that to help with your trace times, it would mean that you successfully prevent some intersection tests, which may mean that you’ll miss geometry you care about. It won’t help anything to set t_max to be slightly larger than your scene bounds.
I can’t tell if your hunch is correct without a lot more info, but I can give you some generic suggestions. Check how many rays you’re casting, and check what percentage of them are misses. You could also check to see if a lot of rays are being cast near some dense geometry. Lastly I would recommend stepping back and double-checking that all your data is valid. If some NaNs or INFs or unintended zeros snuck into your ray data or your scene data, that could end up causing noticeable slowdowns.