Any way to limit defer-write speed/utilization?

FAQ, getting help, user experience about PrimoCache
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Any way to limit defer-write speed/utilization?

Post by Support »

tverweij wrote: Fri Oct 28, 2022 1:53 pm And there should be a maximum physical write (instead of the max speed of the drive itself) - user defined, to prevent the drive from being used 100% for writes at any time.
Yes, currently we use the drive max speed in order to flush asap. We will consider your suggestion. Thanks.
RobF99
Level 8
Level 8
Posts: 130
Joined: Fri Sep 19, 2014 5:14 am

Re: Any way to limit defer-write speed/utilization?

Post by RobF99 »

You are very courageous letting data sit in a cache for that long! Probably the closest you can get to what you are trying to do with the current feature set of PrimoCache would be set an infinite write delay and Idle-Flush. This way it will only flush if the system is idle and as far as I understand if it is doing a write flush and it detects activity it will pause the flush. (Romex could confirm this). If your system is doing nothing and therefore won't interfere with your work, then why not use that idle time to get data out of the cache. I am sure that you walk away from your desk from time to time. There's no harm and only benefit that in that idle time the software does some flushing. It would stop when you resume work. Then at the end of the day you could do a manual flush. This would be a better scenario than what you are trying to achieve.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Any way to limit defer-write speed/utilization?

Post by Support »

@RobF99, thank you. Yes, Infinite latency + Idle flush might be a workaround. The only problem is that if the cache space is too small, when the cache space is full of deferred write-data, urgent writes will be triggered, resulting in poor performance.
tverweij
Level 6
Level 6
Posts: 74
Joined: Thu May 10, 2018 9:27 am

Re: Any way to limit defer-write speed/utilization?

Post by tverweij »

RobF99 wrote: Sun Nov 06, 2022 10:47 am You are very courageous letting data sit in a cache for that long!
Double power, both battery and generator backed-up. The only thing that can happen is a blue screen, which I didn't see for years.
But the only data that is cached that long (real latency isn't 15 sec but 60 sec for this) is temporary data like SQL Server TempDb, swapfiles from VM's, chrome caching data, etc. All have to be written, but none is important - but all together a huge amount of data that has to be written.
RobF99 wrote: Sun Nov 06, 2022 10:47 am Probably the closest you can get to what you are trying to do with the current feature set of PrimoCache would be set an infinite write delay and Idle-Flush.
Yes. But in my case there will never be a Windows Idle Event as the system is always busy (this is a server, not a workstation).
tverweij
Level 6
Level 6
Posts: 74
Joined: Thu May 10, 2018 9:27 am

Re: Any way to limit defer-write speed/utilization?

Post by tverweij »

Support wrote: Mon Oct 31, 2022 9:19 am
tverweij wrote: Fri Oct 28, 2022 1:53 pm And there should be a maximum physical write (instead of the max speed of the drive itself) - user defined, to prevent the drive from being used 100% for writes at any time.
Yes, currently we use the drive max speed in order to flush asap. We will consider your suggestion. Thanks.
Can you make this a cache-task setting? This is because multiple volumes can be on the same controller / network connection, influencing each other.
By example, I have a B: and a T: drive that are both located on the same network storage. Together there is only 1Gbit speed available (one network cable) for both B: and T: combined.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Any way to limit defer-write speed/utilization?

Post by Support »

tverweij wrote: Mon Nov 07, 2022 1:13 pm Can you make this a cache-task setting?
Seems difficult to current design because currently flushing on each volume is independent. Things get quite complicated when write speeds on other volumes are taken into account.
tverweij
Level 6
Level 6
Posts: 74
Joined: Thu May 10, 2018 9:27 am

Re: Any way to limit defer-write speed/utilization?

Post by tverweij »

I would like to give another possible solution.

When writing to disk, you can fetch the AvgDiskWriteQueueLength from WMI to see if the write queue is above 0.7 - if so, the write speed needs to be reduced. When it is below 0.6, it can be increased. This way, the volume will always function properly.

Even when multiple volumes share the same physical disk, this value will tell if the disk is still able to handle the amount of writes.

This can be implemented as a new write mode: Steady write, where the cache is flushed continuously keeping the disk write queue below 0.7.
As it is continuously flushing, the chance that urgent writes are needed goes down, and when it is needed, the queue can go up to 0.9 (at 1.0, you are at the max the drive can handle, so this reserves a small parts for reads).
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Any way to limit defer-write speed/utilization?

Post by Support »

@tverweij, thank you very much for your suggestion.
Post Reply