What’s the current suggested approach when working with big “.qdstrm” files? I’m trying to use profiling for analyzing an issue that occurs rarely, so I need to import a “.qdstrm” file around 20gb on my machine with 213gb ram. But the import fails with this error that I observed using “journalctl”. Is there a way to specify a range for this importing process?
Error:
kernel: Out of memory: Killed process 771384 (QdstrmImporter) total-vm:281775392kB, anon-rss:223358556kB, file-rss:26624kB, shmem-rss:0kB, UID:1000 pgtables:532496kB oom_score_adj:0
Unfortunately, we don’t have a solution for partially loading .qdstrm files. My standard recommendation would be to try to find a way to reduce the runtime or the number of collectors being used. If you tell me a little more about the situation I might be able to make more specific suggestions.
That’s very unfortunate because I’m trying to handle an issue that occurs very rarely. So I can’t just reduce the runtime. I thought Nvidia Nsight Systems was the right tool for this. If Nvidia could provide a way(s) to work with huge “.qdstrm” files that’d be awesome. I believe this tool should allow users to divide huge “.qdstrm” file into chunks when the “QdstrmImporter” tool fails.
We are working on a way to split long runs into time chunks, so, for instance, if you needed to run for days, you could have it output .nsys-rep files for 20 min increments. Once we have that, you would be able to use analysis to find the one where the issue occurred and then just open that one.
But that isn’t here yet, hopefully in the next year.