Page 1 of 1

HDD will lose speed when L2 Cache is full

Posted: Tue Apr 23, 2019 4:52 pm
by LUnicornGUNDAM
Hi. just call me Cheung for easier spelling.

Primocache is a fantastic software and i have to say good job Romex Software. I'm currently having a Optane 900P as a Level-2 Cache to accelerate my HDD. Recently i have some big data to transfer(about 10TiB). during the process, I find that as long as the L2 Cache runs into full, the Speed of the HDD start to drop. this may the whole process becoming longer.

After further testing, i find the reason the HDD drops speed is that once the L2 Cache become full, primocache and the source disk simultaneously write into the target disk. This condition works fine in a SSD but no in a HDD since HDD has a poor multi Thread reading/Writing Performance.

The ideal writing method should be slowing down the source disk reading while keep the primocache writing to the target disk.

So does Primocache support this kind of ideal writing method? here is my configuration.
捕获.PNG
捕获.PNG (94.32 KiB) Viewed 5368 times
捕获2.PNG
捕获2.PNG (87.61 KiB) Viewed 5368 times

Re: HDD will lose speed when L2 Cache is full

Posted: Tue Apr 23, 2019 7:07 pm
by neatchee
It sounds like what you're asking for is:
- Deferred Write while the cache has free space remaining
- Effectively, writethrough once the cache is full (such that the data hitting the HDD is always FIFO through the cache)

I believe this is how it already works.
How did you confirm that "the reason the HDD drops speed is that once the L2 Cache become full, primocache and the source disk simultaneously write into the target disk"?

When using "deferred write" settings it is expected that transfer rate will decrease once the cache is full.
Deferred Write means that it will fill the cache as quickly as possible, based on the cache write speed.
It will then transfer data out of the cache and onto the disk at the latency you specified (60s)
If the cache is full, it will try to flush the cache to disk before writing more data to the cache. At this point, your transfer speed will be limited by the write speed of the disk, since that is the rate at which cache space is becoming available.

If the cache is getting full and then the transfer is bypassing the cache (writing directly from source disk to target disk) while simultaneously flushing the cache of deferred write data, that would be very surprising and a significant bug I think.

Re: HDD will lose speed when L2 Cache is full

Posted: Wed Apr 24, 2019 2:44 am
by Support
@Cheung, welcome to the forum!

For your question, even when cache space is full, source disk will not directly write data to the target disk. It will wait until cache space has room available. The workflow is just as neatchee said.

It is not easy to get a very good acceleration if your incoming write stream (from the source disk) keeps at a high speed. As you may have known, the cache works like a writing buffer, if incoming write speed much exceeds the write capability of the cache storage and target disk, no software can help.
I think you may try a smaller latency like 1 second, or 2 seconds, and see if this helps.

Re: HDD will lose speed when L2 Cache is full

Posted: Wed Apr 24, 2019 7:18 am
by LUnicornGUNDAM
neatchee wrote: Tue Apr 23, 2019 7:07 pm How did you confirm that "the reason the HDD drops speed is that once the L2 Cache become full, primocache and the source disk simultaneously write into the target disk"?
Here is my testing.

Source Disk: HGST HUH721010ALE600 10T HDD(Reading Speed: around 200MB/s)
Target Disk: HGST HUA723030ALA641 3T HDD(Writing Speed: around 140MB/s)
L2 Cache Disk:Optane 900P 280G(Faster than any of them)
Transfer File size: about 400GiB.

I use the same configuration as i mention at the top, but only changing the size of the level-2 Cache from 64GB to 8GB(so i don't have to wait too long).

Each of them is monitored by HWiNFO32 v6.02-3670.
屏幕截图(8).png
屏幕截图(8).png (226.26 KiB) Viewed 5327 times
Before the L2 Cache runs into full, the Target disk has a speed of 145MB/s, While the Source Disk has push the limit to 239MB/s thanks to the L2-Cache. Everything works fine.
屏幕截图(10).png
屏幕截图(10).png (219.62 KiB) Viewed 5327 times
But as the L2 Cache runs into full, the Target disk start to Drop Speed, Only has a speed of 100MB/s, the Source Disk also reduce speed to 100MB/s.
屏幕截图(12).png
屏幕截图(12).png (222.7 KiB) Viewed 5327 times
So i pause the transfer. after a few seconds, the Target disk start to regain speed(126MB/s).

that makes me wondering the condition of "primocache and the source disk simultaneously write into the target disk" since HDD has a poor multi Thread reading/Writing Performance and in the case, it happens.

Re: HDD will lose speed when L2 Cache is full

Posted: Wed Apr 24, 2019 7:45 am
by LUnicornGUNDAM
support wrote: Wed Apr 24, 2019 2:44 am @Cheung, welcome to the forum!

For your question, even when cache space is full, source disk will not directly write data to the target disk. It will wait until cache space has room available. The workflow is just as neatchee said.

It is not easy to get a very good acceleration if your incoming write stream (from the source disk) keeps at a high speed. As you may have known, the cache works like a writing buffer, if incoming write speed much exceeds the write capability of the cache storage and target disk, no software can help.
I think you may try a smaller latency like 1 second, or 2 seconds, and see if this helps.
Thanks for replying! I beileve the software works just fine, so there must be my configuration mistake. But after reading the manual, i still can't figure out the correct configuration.

There is also some confusing condition i find.

In the older version of Primocache, L2 Cache can't work as a Writable Cache. Only L1 Cache is Writable. So Enabling Defer-Write means L1-Cache will enable the Defer-Write. But in the latest version of Primocache, L2 Cache becomes Writable. What if L1 and L2 Cache simultaneously write to the target disk become they are both full? there seem to be no explanation of the priority between L1 and L2 Cache. Will it happens or not?

Re: HDD will lose speed when L2 Cache is full

Posted: Wed Apr 24, 2019 9:36 am
by neatchee
Very interesting test results! Thank you for sharing.

I think @support will have a better answer than me :)

A guess: when cache is full, there is latency added because instead of doing this...
(Read Source -> Write Cache) -> (Read Cache -> Write Target)

...It has to do this...
Read Source -> Wait for... -> (Read Cache -> Write Target) -> Lookup Most Stale Cache Block -> Write Cache

That is just a guess though :D

Re: HDD will lose speed when L2 Cache is full

Posted: Thu Apr 25, 2019 10:51 am
by Support
LUnicornGUNDAM wrote: Wed Apr 24, 2019 7:45 am What if L1 and L2 Cache simultaneously write to the target disk become they are both full? there seem to be no explanation of the priority between L1 and L2 Cache. Will it happens or not?
When each of them (L1 and L2 cache) is full of "dirty" (unwritten) data, it will trigger "Urgent" writing to flush certain amount of dirty data to target disks. However, the total threads to do the flushing is limited to avoid the multi-threading issue of HDDs.