Tiered L2 Cache

Suggestions around PrimoCache
Post Reply
JimieSitu
Level 1
Level 1
Posts: 2
Joined: Sat May 15, 2021 4:37 pm

Tiered L2 Cache

Post by JimieSitu »

I can't be the only one with this scenario:
I'm replacing my old system drive SSD with a fancy new super-fast NVMe SSD.

I'd like to be able to use a couple GB of the NVMe as a quasi L1 Cache and use my old SATA SSD as my L2 cache. There's a factor 5 in read/write speed and IOPS between the two, after all.

Unfortunately, PrimoCache as it is now doesn't seem to support this functionality.
Last edited by JimieSitu on Mon May 17, 2021 4:12 am, edited 2 times in total.
Reason: remove spam
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: Tiered L2 Cache

Post by Jaga »

If your NVMe is replacing your C: drive (currently the old SSD), then there are no other physical tiers of caching you can use against it other than actual RAM as a L1. The rule of thumb is: never cache a SSD with another SSD, unless it is magnitudes faster. i.e. you could cache your old SSD with a portion of the NVMe, but the gains would only be marginal overall.

You didn't mention if you were trying to cache a spinner drive (non-SSD) with either, so it's hard to figure out your true goals in this scenario. Either way, only one SSD would suffice as a L2 cache if you were - your assumption that there is no tiered L2 caching is correct.

You'd also never want to use a portion of a drive to cache the rest of the drive (i.e. a couple GB from the NVMe to cache the rest of it), as it would cause significant slowdowns.
Logic
Level 5
Level 5
Posts: 47
Joined: Mon Oct 29, 2018 2:12 pm

Re: Tiered L2 Cache

Post by Logic »

jaga:
I disagree:
99% of windows I/O is random 4K at low que depth. Writes mostly.

https://www.thessdreview.com/ssd-guides ... ers-bluff/

https://www.gamersnexus.net/guides/1577 ... ant/Page-2

NVME is around 3X faster at random I/O. So there is an extra snappiness to be had from caching a SATA SSD on a good NVME SSD.

Over and above that there are the small Optane drives that are hugely faster at low QD random 4K.
I would love to have a third tier of persistent cache for my 58GB Optane 800P.
Especially if I could specify the size of the data chunks that get cached on it as my NVME gets faster than it at a chunk size of around 64KB.

ie: Cache my HDDs to my:
256GB overprovisioned NVME SSD (L3),
then cache low QD, random I/O below 64KB on the Optane (L2),
then DRAM cache (L1)

I can only imagine the speed increase if:
Random I/O is read from the Optane at ~250MB/s (its around 60MB/s for NVME)
and large I/O is read from the NVME SSD simultaneously!
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: Tiered L2 Cache

Post by InquiringMind »

Logic wrote: Mon Jul 05, 2021 3:06 pm NVME is around 3X faster at random I/O. So there is an extra snappiness to be had from caching a SATA SSD on a good NVME SSD.
You seem to be overlooking the performance overhead of caching itself - for any read/write, a search has to be done on the index to see if that data has been cached (time taken being related to the index size, so a small index of larger blocks is quicker to search than a large index of smaller blocks). If the data is cached, it is read/written from/to the cache - otherwise it has to be read into the cache first.

Since the index search occurs in every case (cache hit or miss), you need a significantly faster media for your cache to make up for the time taken. An order of magnitude faster is a simple recommendation that is likely to show a benefit in almost every case. A cache that is just 3x faster on the other hand is less likely to be of benefit.
Logic wrote: Mon Jul 05, 2021 3:06 pm I would love to have a third tier of persistent cache for my 58GB Optane 800P.
Especially if I could specify the size of the data chunks that get cached on it as my NVME gets faster than it at a chunk size of around 64KB.
PrimoCache's L1 (RAM) cache allows you to specify a block size, and with Pre-Fetch enabled is almost persistent.
Logic wrote: Mon Jul 05, 2021 3:06 pm I can only imagine the speed increase if:
Random I/O is read from the Optane at ~250MB/s (its around 60MB/s for NVME)
and large I/O is read from the NVME SSD simultaneously!
You'll probably have to keep imagining then. Short of maintaining two cached copies of data on separate media (one for sequential and one for random access), there's no way to separate out random from sequential I/O (bear in mind that large files are often accessed in small chunks, so filesize can't be used as a criterion).

And having different caches for sequential/random I/O is not just wasteful of space, it creates a coherency problem with writes - programs that perform both sequential and random writes would have these going to separate caches which would then contain different data.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: Tiered L2 Cache

Post by Jaga »

InquiringMind wrote: Mon Jul 05, 2021 7:32 pmYou seem to be overlooking the performance overhead of caching itself - for any read/write, a search has to be done on the index to see if that data has been cached (time taken being related to the index size, so a small index of larger blocks is quicker to search than a large index of smaller blocks). If the data is cached, it is read/written from/to the cache - otherwise it has to be read into the cache first.

Since the index search occurs in every case (cache hit or miss), you need a significantly faster media for your cache to make up for the time taken. An order of magnitude faster is a simple recommendation that is likely to show a benefit in almost every case. A cache that is just 3x faster on the other hand is less likely to be of benefit.
Pretty much spot-on. The result of caching a SSD with a NVMe is typically a loss of IOPS due to lookup overhead. You'd see little gains unless the NVMe was magnitudes faster than the slower SSD. If it was however, the gains would be noticeable.


Logic wrote: Mon Jul 05, 2021 3:06 pm jaga:
I disagree:
99% of windows I/O is random 4K at low que depth. Writes mostly.

{snip}

NVME is around 3X faster at random I/O. So there is an extra snappiness to be had from caching a SATA SSD on a good NVME SSD.
Run some tests, do the math. There are only a few circumstances where this would apply. InquiringMind and I agree that the NVMe needs to be magnitudes faster for this to be the result. i.e. it needs to be a slower SSD, and a much faster NVMe. Additionally, the CPU and memory need to be rather fast themselves to mitigate any overhead associated. The major speed increase when using any kind of solid state drive (NVMe included) is the IOPS. When that is delayed due to lookups and cache changes, IOPS suffers similarly.
Post Reply