Swap L1-Cache to Disk-File on Pause/Shutdown/Manually

Suggestions around PrimoCache
Post Reply
CrypEd
Level 6
Level 6
Posts: 71
Joined: Mon Nov 11, 2013 11:04 am

Swap L1-Cache to Disk-File on Pause/Shutdown/Manually

Post by CrypEd »

Beeing able to pause and start a cache-task is a good function.

But to take full advantage, what about beeing optionally able to swap all the cached-block data back to disk, so it frees up memory for i.e. loading a RAMDisk.

Then after I unload the RAM-Disk for my game whatsoever I can click on Start cache, what invokes loading all the blocks from the swapped disk-file again.

This means reading all the ramdisk-data and all the chached-data again from disk plus using additional disk-space...sure but A) not loosing the data itself, so you not start new from scratch B) sequentially loading disk-"images" is considered "fastest" even for HDD and of course ultra-fast on SSD and C) this is a ONE-TIME process.... after that the old-state is restored with all its data/accelerations of the previous state.

I more and more think of what about "Manual Save State" or "Mirror on Swap on Shutdown" and really make use of a optional "Swaps" or "Images" of "Primo-Caches"... so we can start or PC...open op our regular not-changing programs and tools... doing or daily work. Then save this "blocks-state" manually or automatically and after a reboot and Cache is cleaned, it starts streaming in blocks from the Cache-Swap during inactivity. This way it would realize kind of "pre-emptive" caching. Sure, shutdown would be slower, depending on disk-write-speed... but I would not care.... because this gives CACHE the skill to present me my "first" read from RAM after i cold-boot my PC..because it is able to load it fast (seq) from yesterdays swap in the background...few seconds after start the PC with containing that on a SSD: all swap is loaded and ready to use. :D

Swap on pause is very useful, especially without a L2, because no matter how much RAM you have, with these products consider evething "limited"... sooner or later you run out of mem...even with 64GB or 128GB :D

We all know that block-devices are not aware of the files inside the blocks... Primo only knows to which disk the block belongs. We need to use this.... save the state of our block-storage into a swap/image manually and re-loading it quick sequentially gives us the the power to accelerate when our application access the data the FIRST time, from the original-disk ...see we don't know which file... but which blocks, so why not save the blocks and simply re-load them after reboot?

By that we could create powerful profile Cache-Images with "predefined-content"... lets say i want to work with plenty of complex big databases.... i load them all one times and all stuff i need to operate during these "environment" with a clean cache... so all reads-getting cached... now i save the State to an image.... and continue my work.

When i cold-boot my PC the next morning and want to continue in that environment, i simply tell PrimoCache to load thise Cache-image with all that blocks in one big sequential write-into RAM what takes only some seconds. The coup is that all my databases and apps are instantly ready to use in RAM.

Thinking far more ahead.. Primo could have a algorythm to MERGE multiple Swaps/image by frequent use and so on... this would give awesome new possibilities in optimising caches.... no matter if reboot occurs! Think of it.

What do you think about beeing able to swap L1 to disk and revoke the data later on?
CrypEd
Level 6
Level 6
Posts: 71
Joined: Mon Nov 11, 2013 11:04 am

Re: Swap L1-Cache to Disk-File on Pause/Shutdown/Manually

Post by CrypEd »

Argh im stupid... if you manually save and re-load it, then data might have been changed in between... so you load old data xD

So when i load my databases the next day from manual saved cache.... yesterdays work "seems" lost, because i did not auto-save swap on shutdown. xD

Im sry the hole manual idea seems pointless now from this perspective...

Also when you auto-save to a swap on shutdown... i would have my actual databases...but...
on the next boot some data might have changed BEFORE the cache-driver loaded... so swapping cache-data might be dangerous, i don't know.

Do the developers think auto-save cache to swap/image-file and revoke afte rboot would work?

I dont think so.

So maybe completely forget it! xD
User avatar
Support
Support Team
Support Team
Posts: 3627
Joined: Sun Dec 21, 2008 2:42 am

Re: Swap L1-Cache to Disk-File on Pause/Shutdown/Manually

Post by Support »

Well, actually we're also thinking such feature. You have already point out the advantage as below.
CrypEd wrote:B) sequentially loading disk-"images" is considered "fastest" even for HDD and of course ultra-fast on SSD
This swap can only be done on shutdown. As you know, "On Pause" and "Manually" are meanless because cache may be out of sync at such states.
Post Reply