Is there a way to get a text list of sampling points from Nsight profile?
I am looking at the Timeline View but I have to mouse over every single sampling point to see the information associated with it. I’d like to see a text list of all sampling points with associated information (call stack). Is there a way to get such list?
The most common way to get access to raw capture data is the export command. The nsys export CLI command is the easiest way to convert an existing .nsys-rep file. There are several export formats available, but not all of them export exactly the same data. Exporting to JSON will export the raw event data, while formats like SQLite and HDF5 provide more comprehensive, structured data. Generally, we recommend SQLite, as it is a fairly universal format that is easy to explore, or to pull into other analytic environments. Please be aware that the capture files and exports contain fairly low level data. When a report is loaded into Nsys, quite a bit of calculation and correlation happens in order to generate the displays. Depending on the level of data you’re looking for, some of those correlations may need to be replicated. That will take some comfortability with SQL and/or a similar data environment, such as Pandas dataframes.
@jkreibich : Thank you. I exported the .nsys-rep file into .json (for now). As you said, this would need additional processing to map call-stack strings that seem to be located in the dictionary at the head of .json file into the events which seem to have ids that refer to the dictionary (I think). I’m going to try to hack a script that does such processing either from .json or from SQLite.
Two questions:
- Is there a doc that outlines/explains structure of the .json or SQLite file? A pointer would help.
- I might post the script to github or some other open repo. Any input, caveats, whatever if I decide to do this?
Thanks again
in the documentation at User Guide :: Nsight Systems Documentation there is the current sqlite schema as of the 2023.2 release. (I know the text is weird, I promise, that’s a direct link). The schema for a specific database can be obtained with the sqlite3 tool built-in command .schema. The sqlite3 tool can be located in the Target or Host directory of your Nsight Systems installation.
There is a description of the .json format at User Guide :: Nsight Systems Documentation (again, direct link), but unfortunately, I cannot guarantee how current that is.
The JSON export is a raw dump of just the core event data. The SQLite file has nearly all the same event data (although sometimes slightly reformatted to make it easier to understand), plus some of the auxiliary data, plus a bunch of enum and meta-data.
We mostly use the JSON exports for debug and verification. As someone that’s comfortable with SQL, I find it much faster and easier to explore and poke around the data within the SQLite environment, be it the sqlite3 command line or a more point and click database explorer type program. Mapping one data set to another is the kind of thing databases are designed to do, so I find it a better way to do exploratory, ad-hoc data exploration.
We’ve also tried to include comments in the SQL to help explain what the different fields mean. When using the sqlite3 CLI tool, you can use the command .schema to lookup the command used to create the table. The exports include comments (-- is the “comment to end of line” in SQL) that try to provide some guidance.
For example:
sqlite> .schema SAMPLING_CALLCHAINS
CREATE TABLE SAMPLING_CALLCHAINS (
-- Callchain entries obtained from composite events, used to construct function table views.
id INTEGER NOT NULL, -- REFERENCES COMPOSITE_EVENTS(id)
symbol INTEGER NOT NULL, -- REFERENCES StringIds(id) -- Function name
module INTEGER NOT NULL, -- REFERENCES StringIds(id) -- Module name
kernelMode INTEGER, -- True if kernel mode.
thumbCode INTEGER, -- True if thumb code.
unresolved INTEGER, -- True if the symbol was not resolved.
specialEntry INTEGER, -- True if artifical entry added during processing callchain.
originalIP INTEGER, -- Instruction pointer value.
unwindMethod INTEGER, -- REFERENCES ENUM_STACK_UNWIND_METHOD(id)
stackDepth INTEGER NOT NULL, -- Zero-base index of the given function in call stack.
PRIMARY KEY (id, stackDepth)
);
EDIT: please note, the forum syntax formatting seems to get the comments correct, but I have no idea why it decided to bold random key words. Please understand that’s the forum software, not something I put there.
Thank you for detailed additional information. My colleague @dilencevi wrote a script that extracts the timeline from sqlite file. She’ll post a link here.
Here is the link to the github repo.
https://github.com/dilencevi/NTimeLine
Please let me know if you have any comments or feedback.
Thank you
Perhaps I should have mentioned this earlier, but Nsys has a rather extensive stats/report framework, accessible via the CLI command nsys stats and in the GUI by accessing the drop-down menu in the lower-right panel. These reports are also written in Python, and focus completely on the data manipulation. The stats command takes output from report and can format it in a number of different ways (including pretty tables, CSV, and a few other formats) and route it to the console or a file or even pipe it to another process.
You can find the provided reports in <nsys_install_dir>/reports, and find out more about the stats command with nsys help stats.
I’ve attached a report called timeline_trace based off the query in the NTimeLine script. If you drop this file into the working directy when you call nsys stats you should be able to call it as nsys stats --report timeline_trace <report .sqlite|.nsys-rep>. To output to a CSV file, just add -o . (that says to write the output to the default filename (<report>_timeline_trace.csv), which also defaults to CSV format).
timeline_trace.py.zip (984 Bytes)