Faster than cache disk performance is possible:

Suggestions around PrimoCache
Post Reply
Logic
Level 5
Level 5
Posts: 47
Joined: Mon Oct 29, 2018 2:12 pm

Faster than cache disk performance is possible:

Post by Logic »

  • HDDs are good at large sequential reads.
  • Flash is good at small random reads where it can read multiple files in the time it takes just to seek just one such file on a HDD.
  • It's a known fact that the faster a drive is at low que depth random 4K; the snappier the system feels to the user.
Suggestion:
Filter/prioritise I/O requests so that random small files are read from the caching SSD, while
at the same time
large sequential is read from the HDD.

Obviously large sequential is also faster from the cache drive, so the above rule needs to be refined so that the fast cache disk takes over from the the HDD
after
the small files are read.
Perhaps this can be further refined, using NCQ data, so that less sequential files are prioritised from the cache drive, while the more sequential reads (= low seek time) continue from the HDD simultaneously.

ie:
Put the slow cached drive into a kind of RAID 0 with the cache drive using the above filtering rules.

I think adding these filtering rules to your current set may result in faster than cache drive I/O performance
and that would be a major selling point.

If feasible; I would like to be involved in testing and refining this idea plz.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Faster than cache disk performance is possible:

Post by Support »

Thank you very much for your suggestion!
We did consider this kind of caching policy before which shall benefit caching by usb flash drives in theory. However, real scenarios are quite complicated and Windows may break sequential reads/writes into series of small chunks. This brings difficulty, especially when there are simultaneous multiple reading and writing operations.
Of course, we still will evaluate this policy, as well as other methods, to improve cache performance and effectiveness.
Logic
Level 5
Level 5
Posts: 47
Joined: Mon Oct 29, 2018 2:12 pm

Re: Faster than cache disk performance is possible:

Post by Logic »

Thx for the reply.

WhyTF would windows want to break fast, large sequential I/O into small chunks for anything but the pagefile?
Links if you have any plz?

From researching ReadyBoost, which does this well, it became apparent that it was bought from a university.
(isn't everything worth having in Windoze..?)
You will have to search for this type of caching/filtering to find the paper and author. It may well be worthwhile doing so.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Faster than cache disk performance is possible:

Post by Support »

PrimoCache is a block-level (disk-sector-level) caching program, so it only analyzes IO requests on disk sectors. Windows processes file read/write at file level first, large sequential access requests might be divided into chunks requests at disk sector level.
We know that this caching policy is used in ReadyBoost because ReadyBoost was designed to use flash drives to speed up HDDs. Nowadays, SSDs are widely used and their sequential performance is also much better than HDD's.
Post Reply