Level 2 cache not increasing speed

FAQ, getting help, user experience about PrimoCache
rajdude
Level 4
Level 4
Posts: 22
Joined: Thu Mar 07, 2019 5:24 pm

Level 2 cache not increasing speed

Post by rajdude »

Hello!
So I am evaluating PrimoCache before buying it... specifically for the Level 2 cache functionality on my home server.

I set it up to cache both Read and Write using L2 but I do not see any improvement in before and after.

Can someone please advise me?
Maybe I am doing something wrong, or
Maybe I am expecting PrimoCache to do something which it is not designed to do?

Here is my setup:
Windows 10 Pro
6 HDD Disk RAID6 array controlled by a Dell PERC H730P RAID controller, 2GB cache
4 GB RAM
Atom processor
Old Supermicro server motherboard
120 GB SSD for OS, connected to a SATA port on the motherboard
250 GB Samsung EVO SSD for PrimoCache, connected to a SATA port on the RAID controller <<< this connection may be the problem!

PrimoCache's settings:
2021-08-12 11_19_15-Window.png
2021-08-12 11_19_15-Window.png (8.42 KiB) Viewed 1706 times
2021-08-12 11_18_15-Window.png
2021-08-12 11_18_15-Window.png (24.42 KiB) Viewed 1706 times
2021-08-12 11_17_31-Window.png
2021-08-12 11_17_31-Window.png (75.43 KiB) Viewed 1706 times





Thanks
Raj
Last edited by rajdude on Thu Aug 12, 2021 4:00 pm, edited 3 times in total.
rajdude
Level 4
Level 4
Posts: 22
Joined: Thu Mar 07, 2019 5:24 pm

Re: Level 2 cache not increasing speed

Post by rajdude »

This is how am I testing the speed:

I am using a free software called "LAN Speed Test"
I run it on my PC (not the server).
I ask LST to write a 1 GB file over the network to the server, and then read it
LST tells me the R and W speeds

These speeds are the same before and after installing PrimoCache

Maybe the network is the bottleneck?
But my PC and server are both connected to the same switch. Switch and NICs are all 1 Gbps
2021-08-12 11_27_40-Window.png
2021-08-12 11_27_40-Window.png (31.38 KiB) Viewed 1705 times
rajdude
Level 4
Level 4
Posts: 22
Joined: Thu Mar 07, 2019 5:24 pm

Re: Level 2 cache not increasing speed

Post by rajdude »

Uh Oh....I just checked the specifications of my RAID card
It has 2GB cache in itself....so I am guessing that anything below 2 GB will be cached by the RAID card anyway, right?

https://www.dell.com/en-us/shop/dell-pe ... d/405-aamy
rajdude
Level 4
Level 4
Posts: 22
Joined: Thu Mar 07, 2019 5:24 pm

Re: Level 2 cache not increasing speed

Post by rajdude »

Note that I was testing using a 1 GB file in LST. That is smaller than the RAID card's own cache.

To mitigate that and hope that the RAID card's cache gets filled and PrimoCache takes over......I just tested using a 4 GB file. I got a similar result. Also heard the mechanical HDDs whirring during the test (more during the reading part)....meaning that the data was written/read (more read) to the HDDs......I was hoping that the data will be read and written to the SSD (by PrimoCache) first and after 60 seconds it will be written to the HDD.

Enable Defer-Write is set to 60 seconds
rajdude
Level 4
Level 4
Posts: 22
Joined: Thu Mar 07, 2019 5:24 pm

Re: Level 2 cache not increasing speed

Post by rajdude »

Any thoughts?
Anyone?

I did some more testing....ran tests right on the server machine itself....that takes out the 1Gbps limitation of my network cards....but so far, I see no advantage of using PrimoCache in my particular use case.

Can someone please advise me?
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: Level 2 cache not increasing speed

Post by InquiringMind »

As you spotted, running a network test results in a network bottleneck (1Gbps Ethernet is unlikely to exceed 80MB/s - you may be able to tune yours to offer better performance by checking the Jumbo packet settings if you haven't done so previously). Your setup only allows 128MB for L1 which is pointless (in fact, it may even reduce performance since that cache will be constantly swapping data), so unless you can allocate 2GB or more (in your case, given your existing 2GB RAID cache I would suggest 8GB+ RAM) then I would suggest disabling L1.

Your system specification lists an Atom processor - given these were intended for (low power) laptop use so I'm not sure what one of these is doing in a server. However PrimoCache does require higher levels of CPU to operate and these can be significant with a lot of disk read/write traffic (like during a benchmark). Open Task Manager and check it during a benchmark to ensure that you are not maxing out your CPU.

Having your L2 SSD connected to a SATA port on the RAID controller is unlikely to be a problem since the limiting factor would then be the controller card's PCI-E bandwidth. However it may result in duplication of data in the RAID cache (since the SSD would be containing copies of the HDD RAID data). This may be a long shot but I would suggest checking to see if you could move the RAID cache RAM modules onto the motherboard instead (according to Dell's documentation the card uses 72-pin DDR3 RAM so it might just be compatible with your SuperMicro motherboard) as that would remove a level of caching (currently you have 4 levels - RAID, Primo L2, Primo L1, Windows file cache) and allow you to use that memory for a larger L1 cache or for applications (4GB isn't much, particular for Windows 10).

PrimoCache L2 by default only attempts to boost read performance in order to minimise SSD writes. You may therefore benefit by limiting it to read only and starting your evaluation from there.

Overall it looks like your hardware (4GB RAM, Atom processor) in conjunction with a memory-hungry OS (Win10) may be the underlying issue. An older version of Windows (or Linux) should be able to provide better performance (at the very least, ensure you have all non-critical services disabled to reduce memory/CPU load).
Last edited by InquiringMind on Wed Aug 18, 2021 2:06 am, edited 1 time in total.
User avatar
Support
Support Team
Support Team
Posts: 3622
Joined: Sun Dec 21, 2008 2:42 am

Re: Level 2 cache not increasing speed

Post by Support »

@rajdude, enable the "Flush L1 to L2" option in the Advanced Defer-Write Options. Otherwise when L1 cache is full of deferred write-data, it will flush data to HDDs.
rajdude
Level 4
Level 4
Posts: 22
Joined: Thu Mar 07, 2019 5:24 pm

Re: Level 2 cache not increasing speed

Post by rajdude »

Support wrote: Mon Aug 16, 2021 3:18 am @rajdude, enable the "Flush L1 to L2" option in the Advanced Defer-Write Options. Otherwise when L1 cache is full of deferred write-data, it will flush data to HDDs.
Thanks for your tip!

I enabled the "Flush L1 to L2" option in the Advanced Defer-Write Options, and did some more testing. Right on the server itself (taking out the network from the equation). Here is what I observed...

Before enabling:
2021-08-18 18_45_07-server - Remote Desktop Connection.png
2021-08-18 18_45_07-server - Remote Desktop Connection.png (50.52 KiB) Viewed 1616 times

After enabling:
2021-08-18 18_40_19-server - Remote Desktop Connection.png
2021-08-18 18_40_19-server - Remote Desktop Connection.png (50.79 KiB) Viewed 1616 times
So yeah, you can see there is some increase in speed by PrimoCache, but not huge.

I guess I should try to connect the SSD in the motherboard's port. Because right now, the first 2 GB send to the array would be cached by the RAID card's own cache. (Note that I am using 9 GB file size, so that should fill out the cache and then L2 should take over, it probably does)
rajdude
Level 4
Level 4
Posts: 22
Joined: Thu Mar 07, 2019 5:24 pm

Re: Level 2 cache not increasing speed

Post by rajdude »

Two more questions:

1. Is it possible to disable L1 and enable ONLY L2 cache?
I notice that I can set OS managed memory to zero. Does that mean I disabled L1 cache?

Also does it make sense to disable L1?
Asking because my machine has very little RAM - only 4 GB.
It also has a very low power CPU, an old Atom. This config was chosen for lowest possible power consumption (electricity)......since it runs in my house 24x7, 365 days a year :-) sucking electricity!

2. What should be the Latency set to, under "Enable defer-write"?
I have it at 60 sec right now.
rajdude
Level 4
Level 4
Posts: 22
Joined: Thu Mar 07, 2019 5:24 pm

Re: Level 2 cache not increasing speed

Post by rajdude »

ok, I moved the L2 cache SSD to motherboard's SATA port.
Ran some tests, here is the result:
2021-08-18 21_05_35-server - Remote Desktop Connection.png
2021-08-18 21_05_35-server - Remote Desktop Connection.png (80.59 KiB) Viewed 1614 times
So it seems to me that having the SSD connected to mobo is not giving better results...in fact RW speeds are both is slower.

I then paused the cache.....and now the test results are faster:
2021-08-18 21_28_40-server - Remote Desktop Connection.png
2021-08-18 21_28_40-server - Remote Desktop Connection.png (93.2 KiB) Viewed 1614 times
That means in my machine, PrimoCache is actually making RW slower!
Of course this is because of my CPU. SSD when connected to the mobo's SATA port which uses Intel RST which depends upon the CPU to do I/O.

In contrast, the Dell RAID card has its own CPU and cache RAM....whence causing less stress on the already slow Atom CPU!

I also notice that the CPU maxes out to 100% while RW....where as when the SSD was connect to the RAID card, the CPU was around 65% while doing RW

I deduce that until I upgrade the hardware of thsi machine........... PrimoCache L2 is not for me!
Any thoughts?
Post Reply