A theory about a problem that has always existed

FAQ, getting help, user experience about PrimoCache
Post Reply
RobF99
Level 7
Level 7
Posts: 97
Joined: Fri Sep 19, 2014 5:14 am

A theory about a problem that has always existed

Post by RobF99 »

Hi, ever since I started using PrimoCache many years ago I have always experienced problems when writing to external hard drives when the L1 and L2 flush to disk slows right down to only a few blocks per second and the entire system does not respond well caused by this very slow flushing of cache on the external drive. I have mentioned it a few times here.

It tends to happen during backup routines where I write many large files to the external drive. Often it slows right down to the point where the backup will not copy a new file until the cache is entirely flushed. it is very frustrating.

In the past few days I had to setup my backup routines again and found that I experienced the problem a lot, and then I finally worked out what causes this problem, and it should be an item you look to address. I don't know if it is a problem with PrimoCache or a fundamental problem with Windows itself.

I found that the problem occurs when there is overlap in files that get written to the Windows write buffer. e.g. if two 10 Gb files are written in sequence then after the first file is written but the Primocache write cache is still filling from that file but the second file has started writing also then there is an overlap of the two files in the Windows write buffer that is being written to PrimoCache's write cache. It is when PrimoCache is flushing the write cache from the point where these two files overlapped that the slowdown occurs.

You can reproduce this yourself by copying some large files to an external USB hard drive. Copy the files so that there is some overlap in them being copied to the external drive. e.g. when one is half way written then copy the next file. If you monitor the Resource manager disk writes you will see that PrimoCache flushes nicely at around 100 to 200 Mb/s (the speed of the drive). When the flushing gets to the point where there was overlap in the original copying of the files you will see the flushing slow down a LOT to just a few blocks per second and the Resource monitor shows that the writing to the external drive slows to 1 to 2 Mb/s and the queue on the external drive also fluctuates wildly with queue length increasing to as much as 300.

I haven't tested on an internal drive but I am sure that the problem is there but to a much lesser degree. You may want to test yourself on an internal. So this is why and when this problem occurs. It probably happens with smaller files too. I think it is dependent upon the amount of overlap and amount of files overlapping in the original Windows write buffer that gets written to Primocache that eventually gets flushed to the hard drive. Maybe recalculate the flush order more regularly? I'm not sure how that would impact performance. I hope this helps to solve a long existing problem.
RobF99
Level 7
Level 7
Posts: 97
Joined: Fri Sep 19, 2014 5:14 am

Re: A theory about a problem that has always existed

Post by RobF99 »

I did further testing on this. The problem is not PrimoCache. The problem is Windows. With PrimoCache not running, Windows Modified Memory gets badly bottlenecked because it seems it does not handle large files well if they are being copied to an external hard drive if there is quite a bit of fragmentation on that external drive. So it appears that Primocache is just mirroring the behavior of Windows after the modified memory has been emptied into PrimoCache write cache. I would think though that PrimoCache might get past the issue since it just has to write to the hard drive blocks.

Note that when running with Windows only and you can see Resource monitor statistics, the Response Time can be as high as 3000 milliseconds. And the modified memory counts down very slowly. Also note that it is Windows 7. I am testing later with Windows 10.
User avatar
Jaga
Contributor
Contributor
Posts: 684
Joined: Sat Jan 25, 2014 1:11 am

Re: A theory about a problem that has always existed

Post by Jaga »

Primocache can try to alleviate write bottlenecking when it uses write coalescing, but on a fragmented drive that feature is going to be reduced in effectiveness quite a bit. Best practice on external spinners may be to set an automated defrag routine a few times a week to try and help writes later on down the road, when they are large like those you are seeing.

Another contributor to the cache flushing could be the external drive controller and how fast/capable it is. Be sure to check for driver software for W10 prior to doing your upgrade, and/or source a replacement external bus that has up to date drivers for W10. I found that one of my USB add-in controller cards was absolute crap under W10, and had to stop using it after the upgrade.
RobF99
Level 7
Level 7
Posts: 97
Joined: Fri Sep 19, 2014 5:14 am

Re: A theory about a problem that has always existed

Post by RobF99 »

FYI, I did install Windows 10 and the problem no longer exists even with the drive as fragmented as it is. Win 10 writes it all very smoothly.
User avatar
Support
Support Team
Support Team
Posts: 3113
Joined: Sun Dec 21, 2008 2:42 am

Re: A theory about a problem that has always existed

Post by Support »

Thanks for the checking and information! I'm glad that the problem has been addressed now.:)
InquiringMind
Level SS
Level SS
Posts: 476
Joined: Wed Oct 06, 2010 11:10 pm

Re: A theory about a problem that has always existed

Post by InquiringMind »

RobF99 wrote: Wed Oct 06, 2021 6:43 am FYI, I did install Windows 10 and the problem no longer exists even with the drive as fragmented as it is. Win 10 writes it all very smoothly.
As an alternative to changing OS, a third-party File Copy Tool may alleviate that problem, if anyone else encounters it.
Post Reply