hi.
I found difference between Jetson orin direct writing and buffered writing
First of all, when I used direct = 0 for FIO testing, the cache I was using was not released with my fio imports and exits, and the files had to be deleted manually
Second, when I used direct = 1, I found that the cache was partially requested when I executed the FIO instruction. Unlike the first, the cache was completely freed when the Fio exited
I have two questions：
Is this normal ？
I need to get a specific list of files if I want to determine which file caused the cache increase when no cache was released
Hi, Good afternoon. To be precise, I’ve been testing and found that direct write and buffer write differ from the system in that the latter causes the cache not to be freed, and I’m not sure why
Watch what goes on using xosview (“sudo apt-get install xosview”). This GUI app can be enlarged to see bar charts better, and has a bit more “graphical” view of what you are doing.
There might be some dedicated cache, but for files, consider that cache and buffer is normally only taken from unused RAM. This is for performance reasons, and you won’t find that this is actually “bound” to your application for the non-dedicated uses. Those parts of RAM which are used for cache/buffer will typically persist until the RAM is needed for something else. This is intentional as it greatly increases performance the next time that content is accessed. Unless there is pressure from something else to give up that RAM, there is neither need nor benefit from doing so.
As an experiment, not necessarily using your app fio (because it is specifically changing access), on a freshly booted Jetson, time how long it takes to read all bytes of your app while xosview is running. You should use a rather large file, or maybe even read characters/bytes from an entire directory into /dev/null. Watch how cache and/or buffer grows naturally, and how long it takes.
Now repeat the test on the same exact file content. You should notice the second and later times run faster than the first case. This would be due to having buffer/cache filled during the first test.
As you begin to do this with more and more file content, you might find the oldest of the reads and/or writes are no longer speeding up from cache/buffer. This would be because some of this is removed from older content due to being less used, and being needed for cache/buffer on newer content.
That is filesystem I/O behavior. There might also be some sort of buffer tied to the I/O controller itself, but I don’t think the cache is visible if it isn’t via system RAM (and that cache is perhaps part of the controller and/or drive itself). I’m only guessing, but there is a strong likelihood that the cache/buffer you see is the system RAM and not some dedicated cache/buffer that is part of the hardware. Any system RAM version is something that is not actually allocated, it is just sitting there being useful, and as soon as something needs that RAM, it would be released (lazy release, not release after use).
The cache and buffer is somewhere in the kernel space, so although there might be some export somewhere of information on file/directory caching, I don’t know of this actually existing. Some of it will be part of the ext4 filesystem driver, and some of it is probably part of the storage device driver (e.g., eMMC or SATA or NVMe drivers, depending on the hardware). That RAM is automatically released when needed, so you are not really “consuming” that RAM.
What is it you specifically want to accomplish? Answers might differ depending on what you really want to know about, and RAM itself has a lot of complicated classifications.
I personally tend to think of buffer memory as having a specific layout and use, but cache as something which might be more “primitive”. For example, reading network traffic into memory which understands Ethernet might be something I’d consider buffer, but something which randomly memorizes registers and individual bytes without structure is something I might be more tempted to call cache. The memory which resides in a disk drive would be cache since the drive doesn’t care about the partition or filesystem type. The application which reads an ext4 filesystem and remembers the most recently accessed data I would personally think of as structured data I’d call buffered. I can’t say that this is how the rest of the world thinks of it.
One thing which is certain is that cache and buffer can change due to the system needing more RAM or having extra RAM which can be used for more performance.
Sort of related to this, the Linux kernel is PID 0, and init (systemd in semi-recent Ubuntu has a symbolic link to init) is PID 1. All other processes are child processes of PID 1. There are pseudo files in “/proc” for every PID. For example, if the shell you are using has PID 1234, then you will find “/proc/1234/” (often you will need root access to see what is in that). Those files are not real files, and exist only as part of drivers in the kernel pretending to be a file. Those files tend to include various “context” for a given process, including memory.
Pick a process that your user owns, or else use sudo and pick any process (e.g., look at some process ID found in top/htop/ps aux). Then cd to that PID at “/proc/<PID>/”. Now run this command: find . -iname '*cache*'
Some of those files can be read with the “cat <filename>” command. That one process has a lot of different cache classifications. Commands such as free -hw will use this information in some scheme and try to combine that information into a single value. The only way to know exactly what is what is to instead go to each PID and look at every listing of cache. Then you’d have to probably spend years of research figuring out exactly what a given driver or specific kernel code type means. In some cases you’ll even find that one classification of cache overlaps with some other classification, and thus it might be that adding up every cache in a given process exceeds total RAM (think of “vehicle” being a classification, and some are “cars” or “trucks”, and if you add “vehicle” plus “cars” plus “trucks”, then you get more than the actual total number of vehicles).
Cache is handled automatically, it isn’t something you need to “release”. Cache would normally not be “consumed” in any permanent way, although something like a disk drive might have its own physical cache RAM which shows up, and might not be part of the system RAM.
