Page 1 of 1

Deferred writes, urgent writes, SMR drives, and crash recovery

Posted: Fri Jul 31, 2020 3:29 am
by phat
I want to put an SSD cache in front of some SMR (shingled) hard drives. Shingled drives are very slow handling random writes, and are fastest doing large sequential writes. I believe they are fastest when the writes sequentially fill up shingled zones, which I think are 256MB in size.

When PrimoCache flushes deferred write data to disk, does it try to sequentialize the blocks? Is there a way to control how much data to write to disk during each flush?

I've noticed that performance appears to degrade when there are urgent writes. Why? Do blocks bypass the write buffer if it's full? Or does PrimoCache stall the writing task to evict existing deferred-write blocks to make room for the new writes? Does the urgent write eviction algorithm attempt to gather large and sequential series of blocks to evict? Is there a way to control the number of blocks to evict when making room for new blocks? For SMR drives, I'd want eviction to occur in 256MB sequential chunks.

Finally, I understand that deferred write data not yet written to disk are lost on a crash. Is there a way to find the list of blocks that were in the cache? That way I can find the files that may have been affected (using Window's fsutil.exe).

Thanks in advance.

Re: Deferred writes, urgent writes, SMR drives, and crash recovery

Posted: Fri Jul 31, 2020 4:57 am
by Support
phat wrote:
Fri Jul 31, 2020 3:29 am
When PrimoCache flushes deferred write data to disk, does it try to sequentialize the blocks?
Yes
phat wrote:
Fri Jul 31, 2020 3:29 am
Is there a way to control how much data to write to disk during each flush?
PrimoCache will decide the amount based on disk's write capacity and best performance.
phat wrote:
Fri Jul 31, 2020 3:29 am
I've noticed that performance appears to degrade when there are urgent writes. Why?
Reduce the latency or increase cache size to avoid the urgent write which degrades the performance.
phat wrote:
Fri Jul 31, 2020 3:29 am
Is there a way to find the list of blocks that were in the cache? That way I can find the files that may have been affected (using Window's fsutil.exe).
Sorry, so far now. It's unknown because the index of cached data may also get lost on a crash. You may run chkdsk after a crash.

Re: Deferred writes, urgent writes, SMR drives, and crash recovery

Posted: Thu Aug 06, 2020 6:37 am
by phat
Thanks for the answers so far. Can you elaborate on the flush policy for urgent writes? Does PrimoCache try to flush blocks in sequential order, in as large batches as possible, during an urgent write? If it flushes blocks at random, it can greatly reduce the throughput of the SMR destination drive, and make the cache full condition worse.

Also, a couple of new but related questions:

1. If I have Defer-Write FreeWritten set, and the R/W size is set to shared, and the cache is full, does it mean that only a small number of blocks will ever get used for writes? When a write block is written to disk and marked okay-for-eviction, and the cache is full, is this class of blocks the first to be reused? (I'm referring to L2 cache, specifically. And just to reiterate, my desire is to use PrimoCache to hide the slow random-write performance of SMR drives.)

2. Is there a way to know if PrimoCache was not gracefully shutdown, so I can run chkdsk? My computer has crashed several times during some overclocking attempts and PrimoCache always restarted silently, as though nothing abnormal happened, when I know that it was not gracefully shutdown. Is there a log that I can look at to find out if there was an ungraceful shutdown?

Thanks again.

Re: Deferred writes, urgent writes, SMR drives, and crash recovery

Posted: Sat Aug 08, 2020 4:42 pm
by Support
phat wrote:
Thu Aug 06, 2020 6:37 am
Does PrimoCache try to flush blocks in sequential order, in as large batches as possible, during an urgent write?
Normal Write does, but Urgent Write doesn't. You shall tune the latency/mode or cache size to avoid urgent write as possible.
phat wrote:
Thu Aug 06, 2020 6:37 am
1. If I have Defer-Write FreeWritten set, and the R/W size is set to shared, and the cache is full, does it mean that only a small number of blocks will ever get used for writes?
No, all blocks are available for writes if needed.
phat wrote:
Thu Aug 06, 2020 6:37 am
When a write block is written to disk and marked okay-for-eviction, and the cache is full, is this class of blocks the first to be reused?
With FreeWritten, yes.
phat wrote:
Thu Aug 06, 2020 6:37 am
2. Is there a way to know if PrimoCache was not gracefully shutdown, so I can run chkdsk?
You may check Windows Events. There should be an event indicating ungraceful shutdown. PrimoCache also log events if there are writing errors.
And we are thinking to add txt logs for users or pop up a warning message when it detects abnormal things.