CHIA and PrimoCache

FAQ, getting help, user experience about PrimoCache
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: CHIA and PrimoCache

Post by Jaga »

MadMax requires 2 drives for plotting: Temp2 (your faster drive) requires about 110GB, and takes ~75% of overall plotting writes. Temp1 is for assembling the final plot parts, and takes ~25% of the total plot writes. That's why you "plot to a Temp2 RAMdrive" to fully accelerate your plotting speeds.

Primo RAMdisk will be a much better solution for you if you create a volume of ~110-112 GB in size, and use that as Temp2.
andriusst
Level 2
Level 2
Posts: 6
Joined: Wed May 12, 2021 10:05 am

Re: CHIA and PrimoCache

Post by andriusst »

Jaga - I appreaciate your suggestions - thank you. I do not use mechanical disks - only SSD for plotting. My plots are stored on a separate NAS. So perhaps with that in light that changes your recommendation?

But to be honest I am already considering changing my setup. I will be selling my SSDs and RAM and will get a server with 32cores/64threads and 384GB RAM. Apparently older servers like that are cheaper on ebay than the RAM upgrade I did couple days ago. I am just hoping that Madmax plotter can utilize all 64 cores and PrimoCache can properly use all RAM as it will be a quad-CPU system.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: CHIA and PrimoCache

Post by Jaga »

I think that CPU will be overkill, but I've seen some Threadrippers take full advantage of it (and proper memory) so it may work fine for you. Absolutely don't get an older server with DDR3, it'll hurt plot times with the slower RAM.

You'll still want one fast SSD/NVMe (yes, you will want it) as the Temp1 drive for the MadMax plots if you use that plotter. Temp2 will be the RAMdrive. That sort of setup can drop plotting times way down. I don't think I've seen anyone use two RAMdrives for both Temp2 and Temp1, partially because it'll start to bottleneck the RAM. Using two devices (RAM and SSD/NVMe) will take advantage of both equally well. i.e. it's faster to "dump from RAM to NVMe" than it is to "dump from RAM to RAM". I haven't tried using a L1 cache against the Temp1 drive, mostly because I'm currently limited to 64GB. But again, that's RAM->RAM, so I think it would slow things down.

Will be interesting to see what you end up with.
Trixter
Level 1
Level 1
Posts: 1
Joined: Sat Jul 17, 2021 4:04 pm

Re: CHIA and PrimoCache

Post by Trixter »

Hello, I'm brand new to PrimoCache and I think I understand what most of the terms are now but I could use some clarification. To start im looking at PrimoCache with priority on saving SSD lifespace with the least amount of writes. An improvement in plotting time would be nice to have but is not my main goal.

My first question is what what is the difference between Urgent and Normal writes? Yes, I know urgent writes happen when the cache fills up and normal is when the cache is flushed to disk. But I would like to know if avoiding urgent writes saves on SSD life or if in terms of SSD life (not performance) there is no difference between the two. Are urgent and normal writes written to the SSD the same way?

The screenshot below is from my PC creating a single plot with Defer-Write set to Infinite and 40GB RAM Cache. If I understand what the display is showing, PrimoCache saved me 40% of the writes I would normally have with a plot. I chose this thinking the more data I can keep in the cache the more blocks that may be trimmed and never written to SSD. But what i'm not sure on is if it would be better (for SSD life, not performance) to tune the latency to get as many normal writes as possible.
Post Infinite DeferedWrite_1Plot (Primo).png
Post Infinite DeferedWrite_1Plot (Primo).png (61 KiB) Viewed 1346 times
So I guess my over all question is, for SSD life would it be better to use Infinite or try and tune latency to get as many normal writes as possible?
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: CHIA and PrimoCache

Post by Jaga »

Urgent writes are those that Primocache essentially hands off to the OS without processing, because the cache is full.

Mostly with plotting you're looking to reduce Total Write (Disk) by utilizing the memory. I was able to get my cache up to ~50GB on the L1 for plots, if I minified Windows services and closed everything else down. Every GB you add to the cache size helps with the plotting process.

For the most part, what you're seeing is already optimal. Tons of trimmed blocks, only 60% hitting the actual drive (and those that do are usually coalesced together). It will definitely help drive lifespan, and you can track that by looking at TBW on the drive before/after plots. You won't see it anywhere near the total expected for creating a single plot.

All in all looks good, but consider raising the cache size if you can shut down other processes while plotting.
GoldenSun3DS
Level 3
Level 3
Posts: 15
Joined: Wed Mar 09, 2022 9:52 pm

Re: CHIA and PrimoCache

Post by GoldenSun3DS »

azzurro wrote: Sun May 23, 2021 11:35 pm Hi

after thinking about how to eliminate the need for SSDs for chia farming, I came across PrimoCache and registered here to make this post.
First of all, very cool software and very good value for price. I wonder why I didn't stumble upon this piece of gold earlier.

Regarding chia mining and generally about disk i/o issues of spinning drives:
Look at HPe Nimblestorage. They use NL-SAS (basically 7200rpm SATA disks with SAS interface) as primary storage and SSDs for caching. So far, nothing new you say? Well, then read carefully:
1) SSDs are only used for read cache. an intelligent algorithm keeps "hot" data in the cache
2) additionally, they have NVRAM for caching write I/Os and here is the magic sauce: they not only cache the write I/Os, they also sort them and transform any random I/O to sequential I/O which spinning disks are very well suited for.

So you issue a metric crapton of random read and write I/Os to that storage, it takes everything into NVRAM, re-sorts everything and then writes a wonderful sequential stream down to the disks. the read I/Os go to the SSD cache, of course as the NVRAM isn't very large and it wouldn't make sense to hold any data for reading there.

Wouldn't that be something that could be implemented in PrimoCache? Killer Feature! RAM for write caching and streamlining data, SSD (L2 Cache) for holding read cache data (not really necessary for chia, though, i think) and spinning drives for sequential I/O.

regards
azzurro
Correct me if I'm wrong, but that just sounds like Primocache. Point #1 is covered by setting a read cache, and point #2 is covered by Defer Write.
Post Reply