(L1 &) L2 & defer write latency (Native & Average unaffected!)

Found a bug? Report here
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: L2 & defer write latency

Post by Support »

enetec wrote: Fri Feb 10, 2023 7:17 pm 1) "urgent writes" is probably triggered BUT, if you look with attention to written to disk value in video, for many minutes it doesn't change AT ALL... still fixed to some hundred of MB, while several GB are stored in L2... so there should be something else too...! :?:
"Urgent Writes" sends small-size write IO requests to the disk while the disk may be processing a lot of read requests, causing the value of "written to disk" to appear unchanged.
enetec wrote: Fri Feb 10, 2023 7:17 pm 2) disk, as I said is a RAID5 of HDD and, even if under heavy READ load, is capable of write (I've tested it now!). Writes are not asked AT ALL by PrimoCache driver IMHO, since write to disk queue is empty at all for most time of operations, and when something is sent to disk, it doesn't ingenerate high queue time (my little "LED" software is very sharp on it... I use it by a LOT of time, and it is very useful to check queue issues...), so RAID seems capable of handle eventual writes at that level...
This is possible. Because write cache is used, write IOs send by the clone app will be processed in the cache and completed very quickly and then the clone app will issue next read IOs to source disk immediately after write IOs are completed. Thus, the disk may be busy in processing lots of read IOs.
If you have time, you can do a comparison experiment by cloning source files from other disks.
enetec
Level 2
Level 2
Posts: 8
Joined: Sat Feb 04, 2023 11:19 am

Re: L2 & defer write latency

Post by enetec »

I've done some more tests as promised... some interesting discovers!

- the signaled BUG exists with L1 only too... BUT since it is waay smaller is limited to only some seconds... (NOT 1 anyway... I don't understand how defer write latency is "interpreted" by INTELLIGENT mode... and I write "INTELLIGENT mode" for a reason we'll see... ;) )

- enabling Flush L1 to L2 seems to make the BUG even worse! Quite no writes to disk AT ALL during operations!! Only difference is that Urgent writes are reduced to 0 (!!) and it's a bit faster! (Oh, well... it has still to write all to disk....)

- NATIVE mode seems to be unaffected. No urgent writes (even with L1 -> L2 disabled) and only minimal deferred block values. Quite all writes go through L1, with no use of L2... (same speed as L1 only in fact!). My disk is able to write with very short queue as expected by me.
This seems to respect defered write latency value at all! :thumbup:

- IDLE-FLUSH is affected by the BUG too BUT less than INTELLIGENT: deferred blocks skyrocked too, urgent/normal are about 50%/50% (tested without L1 -> L2 flush) BUT at least it tries to write to disk too. It started fast with writes to disk and then seems to "stall", anyway is able to write about 10/15% of writes to disk during operations. Latency is not respected anyway (minutes to flush all, like INTELLIGENT...)

- BUFFER is affected too with a behaviour not so different from IDLE-FLUSH: same initial writings to disk and same stall (which happens before!). Less than 5% of writes go to disk during operations with skyrocked deferred blocks.

- AVERAGE mode seems to be unaffected too. No urgent writes (even with L1 -> L2 disabled) and veery LOW deferred block values. Minimal use of L2 like in NATIVE mode. It's very similar to NATIVE as behaviour BUT waay faster (over 15% faster in operations of about 20 minutes!) and very sticky to very LOW deferred block values. Latency is respected for sure here! It is capable of doing operation timings close to INTELLIGENT or IDLE-FLUSH with the "difference" that they still have to write (flush) quite all to disk, and AVERAGE has already done it!!! Simply the best!

So, my hint for all worried by this BUG and interested in sticky latency deferred writes is to use ONLY NATIVE or AVERAGE, with the last one which is the best IMHO.

I hope my testing would be useful for developers... (and users too!) ;)

Now... I've some questions after these tests:

- how defer write latency should work by project? :think:

- sometimes I reached a defer write percentage of over 100% (like 104% or similar...!). What's the meaning of this value?!? :wtf: :?:

- if I enable "flush L1 to L2" BUT I have L2 set as 100% READ what happens? The option is ignored at all, right? :?:
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: (L1 &) L2 & defer write latency (Native & Average unaffected!)

Post by Support »

Very useful information, thanks! It seems that the "Intelligent"/"Idle-flush"/"Buffer" mode has a bug, we will do further inspection.
enetec wrote: Sun Feb 12, 2023 9:44 pm - if I enable "flush L1 to L2" BUT I have L2 set as 100% READ what happens? The option is ignored at all, right?
Correct.
enetec wrote: Sun Feb 12, 2023 9:44 pm sometimes I reached a defer write percentage of over 100% (like 104% or similar...!). What's the meaning of this value?!?
Which statistic item are you referring to?
enetec
Level 2
Level 2
Posts: 8
Joined: Sat Feb 04, 2023 11:19 am

Re: (L1 &) L2 & defer write latency (Native & Average unaffected!)

Post by enetec »

Support wrote: Mon Feb 13, 2023 9:29 am Very useful information, thanks! It seems that the "Intelligent"/"Idle-flush"/"Buffer" mode has a bug, we will do further inspection.
Great!. If I can help more, I'm here. :thumbup:
Support wrote: Mon Feb 13, 2023 9:29 am
enetec wrote: Sun Feb 12, 2023 9:44 pm - if I enable "flush L1 to L2" BUT I have L2 set as 100% READ what happens? The option is ignored at all, right?
Correct.
Probably this should be stated more clearly or "greyed out" in case... :think:
Support wrote: Mon Feb 13, 2023 9:29 am
enetec wrote: Sun Feb 12, 2023 9:44 pm sometimes I reached a defer write percentage of over 100% (like 104% or similar...!). What's the meaning of this value?!?
Which statistic item are you referring to?
Deferred blocks. Value in percentage. With Intelligent/Idle Flush I sometimes got values over 100% (like 104% or so...). I don't understand the meaning of these values... :?: :wtf:
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: (L1 &) L2 & defer write latency (Native & Average unaffected!)

Post by Support »

enetec wrote: Tue Feb 14, 2023 5:54 pm Probably this should be stated more clearly or "greyed out" in case...
We considered graying out this option when L1 or L2 cache is not enabled. However, since this option is in a sub-dialog, users may return to the main settings dialog and enable the both L1 and L2, but forget to reconsider this option. We will state this option more clearly.
enetec wrote: Tue Feb 14, 2023 5:54 pm Deferred blocks. Value in percentage. With Intelligent/Idle Flush I sometimes got values over 100% (like 104% or so...). I don't understand the meaning of these values...
Due to the asynchrony between receiving write data and flashing to the disk, a situation may occur: the data blocks which have been flushed to the disk have not been deducted from the statistics, while the new received write-data blocks have been counted, resulting to the value over 100%.
Post Reply