High CPU usage issues

[NickJ]

When writing to a full cache with FancyCache, each thread uses significantly more CPU usage than a write to an empty cache, or a write with FancyCache disabled. This is causing a significant performance hit in high CPU usage multi threaded applications and benchmarks.

To clearly reproduce: Enable FancyCache with 128-256 MB of RAM on an SSD. Run AS-SSD benchmark on the SSD. Compare CPU usage and throughput with and without FancyCache.
I have seen the increased system CPU usage in other scenarios also, this is just the one that clearly shows why it is a problem.

Results:
http://i.imgur.com/v8GTk

Everything is higher with a 256MB cache (as expected, the test is 1 GB of data, writing 25% of it to RAM should return the results shown in every other area), except when enough threads are being used that the CPU usage impacts the program, which caused it to drop from 67 MB/s to 25 MB/s. 61% write performance dropped by enabling FancyCache in a high CPU usage scenario.

Fancy Cache disabled run (for 64 threads): http://i.imgur.com/53oMV

Fancy Cache run (with settings for 64 threads): http://i.imgur.com/ljQNf
I've tried many different settings (different Latency, Averaging Write Amount, Release After Write) and it reproduces on all of them.

CPU usage comparison (only for the 64 threads test): http://i.imgur.com/PWiSH
Note how on the run with FancyCache enabled (bottom graphs), the write (left of the orange line) has far lower CPU usage than when FancyCache was disabled (top graphs). This excess CPU usage shows up in the System process, which spikes up to 57% that I've seen (and sometimes causes the entire computer to become unresponsive for a few seconds), while the application itself is reduced CPU usage as compared to having FancyCache disabled.


Image collection:
http://imgur.com/a/ZunjT


Another easy scenario to reproduce: Copy 1.5 GB on the cached disk to the cached disk (larger amount copied than the cache size). With FancyCache enabled: 5-10% CPU usage by the System Process. With FancyCache disabled: 1-2% CPU usage by the System Process.

Suggested fix? I assume the issue is because all writes are forced through the cache even when it's full. Thus, if writes bypass the cache when the cache is full it should fix the issue.

OS: Win7 64 bit

Edits: Added better descriptions and an additional repro scenario/data.

[minhgi]

I think you made found a flaw in Fancy cache. Fancycache have the tendency to cache everything to the to the L1 and L2. If the smart cache is implemented for the read/write cache, only small files should be cache and all large files should be written directly to the harddrive.

minhgi

[NickJ]

Suggested fix? I assume the issue is because all writes are forced through the cache even when it's full. Thus, if writes bypass the cache when the cache is full it should fix the issue.

Thinking about this further, it seems like a short term naive band-aid solution that encourages data corruption. Hopefully you have some solutions in house to reduce the CPU usage that are better (such as throttling etc)

That being said, I doubt this is an urgent task