The value is dynamically applied, but there are a couple things you have to consider beyond just that value, namely:
While Linux will swap data out opportunistically (that is, it will start pushing data out to swap before RAM is full), it only swaps data in on-demand (so it only pulls stuff in that's actually needed).
Even after data gets swapped back in, a copy is still kept around in swap until the program terminates. This theoretically improves system performance when the virtual memory subsystem is thrashing (that is, constantly shuffling data in and out of swap because everything that's running won't fit entirely in memory) because it avoids the need to write the data out to swap again if it would get swapped out.
Even if both 1 and 2 weren't the case, the system wouldn't visibly react to the change immediately, simply because it takes time to move data back in from swap.
Now, all that aside, there's something else to consider, namely that your machine was swapping well before memory exhaustion was a reasonable concern. That means one of three things in most cases:
- You're using memory control groups to limit certain programs' memory usage, and some of them are hitting the limits in the control groups.
- You have an insane amount of files cached that the system is for some reason not willing to evict from the cache to free up memory.
- Something that's very active has a huge amount of memory allocated (possibly using huge pages) that it's not actually using.
In the first case, tweaking
/proc/sys/vm/swappiness won't help, you need to look at the memory limits set up in the control groups and adjust those instead (or look at the applications you're running and figure out which one is using more RAM than you thought it needed).
In the second case, you will likely have better results increasing
/proc/sys/vm/vfs_cache_pressure than you will lowering
vfs_cache_pressure sysctl controls how aggressively the system reclaims space from the page cache. It's expressed as a percentage, with the default being 100. I find that bumping it to 200 will usually convince the kernel that you really care more about other memory usage. Setting this too high though will negatively impact overalll performance (because it will waste time trying to shrink the cache when it can't be shrunk).
The third case is an issue with the program in question, and you can't really do much in most cases other than complain to the developers.