HDD acceleration questions Topic is solved

[aamadeus]

I hope this question is not a duplicate question.
I have searched a lot for a lot of time, including this forum, but didn't get a satisfactory answer.

I configured a PC like this:
-Intel i5-8500 cpu
-16GB RAM
-WD Black 2.5" 1TB HDD 2EA RAID 1 (Intel RST)
-Samsung 970 EVO SSD (500GB)

(I originally tried to configure a PC with SSD RAID 1, but the speed is severely slowed down when using it continuously due to the problem of not being trimmed.)
(RAID 1 is a required configuration for data availability.)

This PC will act as a gateway router for my company. (pfSense)
It is a router of more than 100 PCs and servers, and above all, it has to act as a VPN server, so a good specification was required.
(I have failed to install pfSense natively, so I plan to run pfSense using Hyper-V in Windows 10.)
(It is unlikely that you will add a VM to this PC. It will work for pfSense only.)

Operating system and the 'pfSense VM' run on 1TB HDD. (RAID 1)

In fact, most of pfSense's actions will be done in memory, so I thought it wouldn't have a big impact, whether it's an HDD or SSD.
By the way, when it comes to updating Windows for management and handling virtual machines, I was very frustrated because I was already familiar with SSDs. In the meantime, I got to know PrimoCache and I really like it.

For the correct answer, the background explanation was a bit long.

First of all, let me show you the PrimoCache settings for this PC now.
(The SSD capacity is 500GB, but only 250GB was set for OverProvisioning.)

0000.jpg (288.81 KiB) Viewed 2002 times

0010.jpg (123.63 KiB) Viewed 2002 times

0020.jpg (58.91 KiB) Viewed 2002 times

Following are my requirements.
1. I want fast performance, but stability is much more important. So Defer-Write is not considered.
2. If possible, I would like to accelerate HDD read/write.

Questions:
1. Is the above setting appropriate for my requirements? If not, how would you change something?
2. With PrimoCache, the read performance seems sufficient. However, I can't feel that the writing performance has improved so much. If you don't use Defer-Write, isn't the write performance improvement big?
3. (Most important question) If you use a write cache but not Defer-Write, can you say that it is stable enough for unexpected shutdowns?
(The problem that may occur even when PrimoCache is not used is aside. We only need to review the problem that occurs by using PrimoCache.)
4. For the above requirements, is it correct to turn off the 'Individual Read/Write Cache Space' option? Or is it right to turn it on? (If you need to turn it on, how much should you give the write rate?)

I look forward to a good answer.

[janusz521]

From my perspective in most cases, unexpected shutdowns are as harmful with Defer-Write enabled as without. I have DW of 3 mins, L1 cache only, and I experienced several hung-ups so far. The consequences were as usual. Only once I saw a message about corrupted Chrome settings. Anyway, I use my computer as an average desktop so usually, I do not have any mission-critical write operations. I think the biggest concern should be about avoiding crashes in the first place and keeping the router smoothly running. Having RAM of substantial size and using a cache to limit local disk I/O operations should greatly help with this.

Looking at your data I see that the disk usage on your server is rather minimalistic and should not affect the overall server performances. After all, I believe that running a router does not involve any intensive data storing, and most traffic passes in and out through the network.

I also doubt one may achieve any good speed-up results without using the L1 (RAM) cache. So you could experiment more with this. But 16 GB of the total RAM may be quite limiting here. I lack any authoritative knowledge about write caching details but I think that to have any meaningful usage of write caching you need an L1 cache as PrimoCache is unlikely to use the RAM not dedicated to it.

Btw, the concept of overprovisioning is about leaving some disk space for the internal usage of the SSD and is not related to PrimoCache.

[Support]

First of all, without Defer-Write enabled, the write performance will not be improved. Of course, without Defer-Write, PrimoCache will not introduce additional risk of data loss.

For SSD cache, if defer-write is not enabled, you may just set 100% read which is default for it in order to reduce the ssd write wear. A write cache without defer-write is not recommended for SSD cache.
PS. if you enable Defer-Write, we recommend that you use L1 write cache for defer-write instead of ssd write cache, unless L1 cache is insufficient.

In my experience, if you can rebuild the whole system, I would like to do following changes.
Divide the Samsung SSD into two partitions. One is for the Windows host system and the other is used as SSD read cache for the HDD raid. You may left 10%~20% space for over provisioning if you like. The reason is that the native SSD speed > SSD cache > HDD.

[luis]

-I have been a silent reader for many years of this board and have never posted anything.
- I own both, Primocache and Primo Ramdisk Ultimate.

Here is my 2 cents for those who keep asking about not using DEFER WRITE.

If you don't want to use defer write, then why bother with Primocache? Quit using it. As simple as that.

-i have a 22gb level 1 cache
-My defer write latency is 7200 seconds. Not an error. 2 hours.
-No good reason for my lazy hard drive to be awaken every few minutes/seconds with a write operation.
-The hard drive waking up every 2 hours to write something works fine.

-My computer is extremely stable. I turn it off every 5+ days. I cannot remember the last time it crashed.

So, if you guys are thinking of not using the defer write option because your system is unstable, please remember:

*** The reason your computer is unstable has nothing to do with Primocache. ***

You should spend some time figuring out why your computer crashes first.

And only after you have fixed this problem, then start using Primocache with a long defer write.

[luis]

And looking specifically at the issues presented by "aamadeus"...

-16gb ram- not enough nowadays. Get yourself 64gb ram or 128gb ram, aamadeus.

-Of the 16gb ram, aamadeus has used 13gb ram and only has 3gb left. IN SPITE of not using Primocache Level 1 cache. Not good.

-aamadeus is so short on available ram (3gb only), that even if he wanted to use Primocache Level 1 cache, he would be severely limited in the size of his Level 1 cache.

So the first step for aamadeus is to get more ram.

[aamadeus]

First of all, without Defer-Write enabled, the write performance will not be improved. Of course, without Defer-Write, PrimoCache will not introduce additional risk of data loss.

For SSD cache, if defer-write is not enabled, you may just set 100% read which is default for it in order to reduce the ssd write wear. A write cache without defer-write is not recommended for SSD cache.
PS. if you enable Defer-Write, we recommend that you use L1 write cache for defer-write instead of ssd write cache, unless L1 cache is insufficient.

In my experience, if you can rebuild the whole system, I would like to do following changes.
Divide the Samsung SSD into two partitions. One is for the Windows host system and the other is used as SSD read cache for the HDD raid. You may left 10%~20% space for over provisioning if you like. The reason is that the native SSD speed > SSD cache > HDD.

This is just the answer I wanted.
thank you.
I also appreciate the solution you provided. I'll try it.

[aamadeus]

-I have been a silent reader for many years of this board and have never posted anything.
- I own both, Primocache and Primo Ramdisk Ultimate.

Here is my 2 cents for those who keep asking about not using DEFER WRITE.

If you don't want to use defer write, then why bother with Primocache? Quit using it. As simple as that.

-i have a 22gb level 1 cache
-My defer write latency is 7200 seconds. Not an error. 2 hours.
-No good reason for my lazy hard drive to be awaken every few minutes/seconds with a write operation.
-The hard drive waking up every 2 hours to write something works fine.

-My computer is extremely stable. I turn it off every 5+ days. I cannot remember the last time it crashed.

So, if you guys are thinking of not using the defer write option because your system is unstable, please remember:

*** The reason your computer is unstable has nothing to do with Primocache. ***

You should spend some time figuring out why your computer crashes first.

And only after you have fixed this problem, then start using Primocache with a long defer write.

Thanks for the reply.
I don't know what Luis is using it for, but I am satisfied with the read cache alone.
There can be a lot of requirements in the world, and in my case, preventing data corruption due to 'cache' is paramount.
So even if I only use the read cache, I think PrimoCache is valid enough.

[InquiringMind]

This seems to be a most peculiar setup - performance is supposed to be a priority and yet the main software (using FreeBSD, a high-performing OS) is being hobbled by running in a Win10 VM (comparable to hiring an elite athlete to run a 100m race only to then insist they be carried by a Sumo wrestler, waddling from the start line).

In comparison, reliability seems to have been almost ignored - if this is going to be a VPN server for a 100+ person company, then where is your resiliency? RAID1 isn't going to help if your server crashes or your Internet connection goes offline. And as Janusz521 has said, your disk usage is/will be minimal (aside from logging, more on this later) so focusing on disk performance is a waste of time, as would be using PrimoCache.

The one aspect that should be seen as critical (after system availability) is logging. You need to track and record online activity (including URLs accessed) in order to detect (and deter) abuse. These logs may provide key evidence in company disciplinary procedures and, in the worst case, legal action so they need to be reliable, backed up and stored in a way that prevents easy tampering or modification.

So, first point, you say you couldn't get FreeBSD working on your system? Find and fix that problem - it's almost certainly a hardware compatibility issue and if it is due to motherboard LAN or wireless adapters not being recognised, it's easily worked around by fitting PCI-e cards with better supported hardware.

If this system is going to be your company's means of Internet access, then availability is critical - so don't rely on one system. Get at least two and set up VRRP (Virtual Router Redundancy Protocol) to ensure that when one fails, another takes over immediately. Make sure the systems are in different physical locations and connected to the Internet via different ISPs (a 100-person company should be able to cover multiple ISP fees). Ideally choose ISPs with different network types (one xDSL, one cable - consider 3/4G or satellite as a further backup).

After system availability, logging should be your next concern. You need to be storing enough data to be able to identify individual activity (and employees will need to be informed and consent to such monitoring - you should liaise with your HR department on this if you haven't already). RAID1 may be worth considering here, but the key requirement should be to have log files regularly backed up (or synchronised with a remote server) and at regular points, copied to write-once media (depending on logfile size, CD/DVD/BD-ROM). That will provide long-term archival storage which, being write-once, can't be modified and may have more value as court evidence (liaise with your legal department on this, if you haven't already).

What you have is more of a network design issue than a simple system setup and if your company has a network administrator, you should consult with them. If you are the network administrator, then you may have your work cut out, but hopefully the above suggestions will help you come up with a more appropriate solution.