You might want to install “xosview”, enlarge it to easily see the charts, and take a close look at the “used”. “Used” memory is what is actually mandatory for current applications. Watch buffer and cache…these may show as memory consumption in some applications, but these are just taking advantage of “spare” ram to improve performance and are essentially “free” ram if anything needs it.
There may also be some legitimate memory leak. You might want to install “htop”. The “RES” column is the physical memory requirement. If you run htop in a terminal in a GUI environment, then you could click on the RES column and have it sort via that. Other columns will not be particularly good at indicating what might be a memory leak, e.g., Xorg will show an enormous virtual, but in reality only uses a tiny fraction of actual physical memory.