A problem when data is partially cached

[npelov]

I just tried caching a 500GB HDD with 60 GB L2 cache. The L2 cache read speed is 2000 mb/s. I left it for two days working and 50 GB of the cache was filled. the program was hitting 90% cache. Then I decided that it's too slow so I bought ssd. During the copy I froze the cache so it doesn't fill it unnecessary and get following results:


with cache at 60% hit rate - 40-50 MB/s (cache frozen)
without cache at that place - 90MB/s (cache paused)

later:
with cache at 28% hit rate - 30 MB/s
without cache at that place - 90MB/s

later:
with cache at 8% hit rate - 50 MB/s
without cache at that place - 87MB/s

It seems that cache is hurting sequential copy. For random access it may be beneficial, but not for sequential when it's not 100% cached.

Could you maybe experiment with different level of cached content at random places how it behaves and maybe see if you can optimize that - bigger queue, order reads to be sequential. Maybe when the IOPs get bigger delay a few ms to grow the read queue and reorder the blocks so they are in sequence. Or just log the packets and try different reading patterns to see what helps.

If you want real live experiments - try synchronizing a cryptocurrency (spacemesh in my case), Ravencoin is smaller, faster. Or any database that reads big portion of the database in random order, but not all of it.
Make sure to use 2.5" laptop hdd because it probably has higher access times.

Please, write me back if you find some optimization for this.

[Support]

Thanks for pointing out this problem! We'll experiment with this some more.