L2 Striped Reading

Suggestions around PrimoCache
Post Reply
steveb
Level 4
Level 4
Posts: 21
Joined: Fri Sep 17, 2021 7:58 am

L2 Striped Reading

Post by steveb »

For example, if the L2 storage is a 4x PCIe NVMe low latency storage and the normal storage is a 4x PCIe NVMe normal storage. If the L2 bandwidth is maxed out, performance could be improved by reading from both L2 and normal storage. Thus having a total of 8x PCIe bandwidth.
User avatar
Support
Support Team
Support Team
Posts: 3124
Joined: Sun Dec 21, 2008 2:42 am

Re: L2 Striped Reading

Post by Support »

Thank you for the suggestion. However, I think if you want to get double bandwidth, why not just use a raid?
steveb
Level 4
Level 4
Posts: 21
Joined: Fri Sep 17, 2021 7:58 am

Re: L2 Striped Reading

Post by steveb »

In this example the L2 storage is higher performance and lower latency than the normal storage, raiding the 2 drives would results in lower performance. The idea here is to know when L2 storage has hit its limit and reading additional info should be read from the normal storage. Thus increasing the overall performance.
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: L2 Striped Reading

Post by InquiringMind »

steveb wrote: Sat Sep 18, 2021 9:57 am ...raiding the 2 drives would results in lower performance...
What type of RAID did you use for your testing? Windows software, motherboard or dedicated RAID controller card?

Implementing this feature request would not only seem complex (each drive would need to be benchmarked to determine best speed/latency and drives of differing performance would need load balancing) but would only benefit a small number of users.

If you have 4 x NVMe drives, then setting them up as a RAID for direct data storage and ditching L2 caching (just use L1 if you have the RAM to spare) would seem to be a better choice.
steveb
Level 4
Level 4
Posts: 21
Joined: Fri Sep 17, 2021 7:58 am

Re: L2 Striped Reading

Post by steveb »

InquiringMind wrote: Mon Sep 20, 2021 10:57 am
steveb wrote: Sat Sep 18, 2021 9:57 am ...raiding the 2 drives would results in lower performance...
What type of RAID did you use for your testing? Windows software, motherboard or dedicated RAID controller card?

Implementing this feature request would not only seem complex (each drive would need to be benchmarked to determine best speed/latency and drives of differing performance would need load balancing) but would only benefit a small number of users.

If you have 4 x NVMe drives, then setting them up as a RAID for direct data storage and ditching L2 caching (just use L1 if you have the RAM to spare) would seem to be a better choice.
The feature would not be complex as its just off loading read operations to the lower tier when the active time on l2 is 100%. This would not just work for my extreme case but normal users with NVMe and Sata SSDs.

I am testing using Optane P5800x SSD that operate at around 10us for 4k random read and 10-11us for 4k random write. I have not personally tested raid with this drive but similar drives have been tested before with Intel VROC raid 0, etc. Sadly they increase the latency by a decent amount.
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: L2 Striped Reading

Post by InquiringMind »

steveb wrote: Mon Sep 20, 2021 2:56 pmThe feature would not be complex as its just off loading read operations to the lower tier when the active time on l2 is 100%...similar drives have been tested before with Intel VROC raid 0, etc. Sadly they increase the latency by a decent amount.
If Intel's VROC increases latency (with hardware support) then how exactly would PrimoCache implementing a software alternative avoid the same problem? And it would be more complex if it had to cope with a L2 volume being split across devices of varying performance.

If latency is your concern, boosting your RAM and setting up a large L1 cache is a more feasible (and available) solution.
steveb
Level 4
Level 4
Posts: 21
Joined: Fri Sep 17, 2021 7:58 am

Re: L2 Striped Reading

Post by steveb »

InquiringMind wrote: Mon Sep 20, 2021 10:54 pm
steveb wrote: Mon Sep 20, 2021 2:56 pmThe feature would not be complex as its just off loading read operations to the lower tier when the active time on l2 is 100%...similar drives have been tested before with Intel VROC raid 0, etc. Sadly they increase the latency by a decent amount.
If Intel's VROC increases latency (with hardware support) then how exactly would PrimoCache implementing a software alternative avoid the same problem? And it would be more complex if it had to cope with a L2 volume being split across devices of varying performance.

If latency is your concern, boosting your RAM and setting up a large L1 cache is a more feasible (and available) solution.
As it is not increasing the latency until the L2 drive is maxed out, at which point it would be better to use the other drive.
User avatar
Support
Support Team
Support Team
Posts: 3124
Joined: Sun Dec 21, 2008 2:42 am

Re: L2 Striped Reading

Post by Support »

steveb wrote: Tue Sep 21, 2021 2:29 am As it is not increasing the latency until the L2 drive is maxed out, at which point it would be better to use the other drive.
Software scheduling/processing also introduces additional latency. The implementation is not simple and ideal. Usually hardware solutions would be faster than software soultions.
steveb
Level 4
Level 4
Posts: 21
Joined: Fri Sep 17, 2021 7:58 am

Re: L2 Striped Reading

Post by steveb »

Support wrote: Tue Sep 21, 2021 12:59 pm
steveb wrote: Tue Sep 21, 2021 2:29 am As it is not increasing the latency until the L2 drive is maxed out, at which point it would be better to use the other drive.
Software scheduling/processing also introduces additional latency. The implementation is not simple and ideal. Usually hardware solutions would be faster than software soultions.
Thanks, i will keep looking into other hardware options.
Post Reply