I do not think that this is Linux's disk cache. In
htop, the memory bar is green (not orange for cache) and I removed the files stored in Zram. No processes seem to be using a lot of memory.
The load was compiling software with its build files stored in Zram (PORTAGE_TMPDIR(/var/tmp/portage)), with swapfile on Zram too. It had Zram writeback configured so that it would write to disk if there is not much RAM left. I compiled 2 softwares, after 1 software, there seemed to be still about 1/2 the memory used, zramctl said that total data used was near 0G, and no proccess is using much memory, and Linux disk cache wasn't the issue. With kswapd continuously at 100% CPU utilization the kernel OOM killed the process consuming too much RAM. After this there was still RAM being used but nothing I could find was using it. If it was disk cache, the kernel would have handed over the space to the memory consuming process. But it didn't, so this is most likely NOT a disk cache issue, I rebooted the computer and the 2nd software compiled quickly without an issue!
Does anyone know what could be the case, is there any way I could further identify what is using the memory?