Win 10 Compact feature: Better Compatible? compression!

Suggestions around PrimoCache
Logic
Level 5
Level 5
Posts: 47
Joined: Mon Oct 29, 2018 2:12 pm

Win 10 Compact feature: Better Compatible? compression!

Post by Logic »

Windows 10 comes with a new compression feature called Compact/CompactOS:

It's generally faster (multi threaded?) than Windows compression
BUT
the main advantage is that it does NOT:
1st write the uncompressed file to disk,
Then also the compressed file
Then compare sizes
Then erase the uncompressed version if its bigger, or the compressed version if it is not

That is normally NOT good for SSD lifespan and results in extreme fragmentation which slows down HDDs.
It also causes 'Out of space' issues/crashes with Primocache.
With Compact; files should remain compacted all through Primocache and only be decompressed into RAM once passed to windows' I/O stack..?

Windows 10 Compact comes with 4 new algorithms:
XPRESS4K: Fastest - least compression
XPRESS8K: Medium speed - more compression
XPRESS16K: Slowest - most compression.
LZX: Even slower and best used to save space on infrequently used files.

As a general rule (Average modern CPU):
E4K will speed up I/O on NVME SSDs.
E8K will speed up I/O on SATA SSDs.
E16K will speed up I/O on HDDS.

Unless PrimoCache were to integrate this, I guess choosing according to your L2 drive will be most performant for cached data, but E16K would give you better uncached performance and more space in the caches..?
Testing reqd..!

2 GUIs (and info) for this command line tool are available:
https://github.com/Freaky/Compactor
https://github.com/ImminentFate/CompactGUI

Last I checked Freaky Compactor was the better choice due to knowing and skipping previously compacted files.
That may have changed in the newer version of CompactGUI.


(FOR THE OLD WINDOWS COMPRESSION:)
IF the above
'Double Write - Delete'
can happen in RAM using Primcache's Defered write option, without causing the 'Out of space' issue/crash;
the old Windows compression may be worthwhile..?
It's still going to be single threaded however.

NB that files are compressed in chunks 16X the cluster size.
That's 8KB for 512B clusters and 64KB for 4K clusters.
Something to keep in mind when choosing block size..?
Testing reqd..?
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Win 10 Compact feature: Better Compatible? compression!

Post by Support »

I think CompactOS was introduced for devices with only 64GB or even 32GB SSD installed in old days. Today's devices have bigger-capacity SSDs installed, this feature becomes not very useful. Maybe VPS or old devices still can benefit from it.
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: Win 10 Compact feature: Better Compatible? compression!

Post by InquiringMind »

Logic wrote: Sat Aug 07, 2021 1:07 am Windows 10 Compact comes with 4 new algorithms:
XPRESS4K: Fastest - least compression
XPRESS8K: Medium speed - more compression
XPRESS16K: Slowest - most compression.
LZX: Even slower and best used to save space on infrequently used files.
Heh - this brought up a Drivespace-related flashback (the compression system introduced with MS-DOS 6.22) which, with the Windows 95 Plus! pack offered different algorithms also (Standard, HiPack, UltraPack) but with the highest compression done during idle time by a background task rather than on-the-fly.
Logic wrote: Sat Aug 07, 2021 1:07 am Unless PrimoCache were to integrate this, I guess choosing according to your L2 drive will be most performant for cached data, but E16K would give you better uncached performance and more space in the caches..?
Compression for PrimoCache (and the practical complexities it brings) has been discussed before. It may be better to focus discussion in one thread.
Logic
Level 5
Level 5
Posts: 47
Joined: Mon Oct 29, 2018 2:12 pm

Re: Win 10 Compact feature: Better Compatible? compression!

Post by Logic »

Support wrote: Mon Aug 09, 2021 4:42 am I think CompactOS was introduced for devices with only 64GB or even 32GB SSD installed in old days. Today's devices have bigger-capacity SSDs installed, this feature becomes not very useful. Maybe VPS or old devices still can benefit from it.
Yes, that is the origin of Windows 10 Compact, but things have evolved since then and various levels of compaction can be applied to just about all files, games etc on a windows system.
Said files REMAIN compacted on the drive/s, unless modified, in which case the are written uncompacted...
You have to run Windows 10 compact again to compact them.
ie: Compact is specifically for read often, modified seldom, .exe .dll, etc files.
:)

So
here's my logic/point:
if
the data is half the size
(to make the argument easy to explain)
then
It gets to RAM in half the time.
if
decompressing it in RAM takes less than the other half of the original time
(Which is the case with todays fast, multi core, idle at the time CPUs)
then
You end up with faster I/O.
(Which is what Primo is all about. Right..? :) )

Now with Primocahce and due to the nature of Win 10 Compact:
You end up with more data in L2 and L1 cache, amplifying the speedup...
Data will also move into and out of caches faster.

If you follow the supplied links, you will find EVIDENCE of the above statements.
As soon as I get back (doing diamond plant electrification and control in Angola atm) I will test this myself.
If Primocache proves to be compatible; I will very likely PROVE my point and post here..!
I am not some yob, farting into a can here! :)
I am a ex Network admin (700 workstation WAN, 14 servers) with a MCSE...
So do look into this a bit in the mean time!?
:)
Logic
Level 5
Level 5
Posts: 47
Joined: Mon Oct 29, 2018 2:12 pm

Re: Win 10 Compact feature: Better Compatible? compression!

Post by Logic »

InquiringMind wrote: Mon Aug 09, 2021 9:00 am Heh - this brought up a Drivespace-related flashback (the compression system introduced with MS-DOS 6.22) which, with the Windows 95 Plus! pack offered different algorithms also (Standard, HiPack, UltraPack) but with the highest compression done during idle time by a background task rather than on-the-fly.

Compression for PrimoCache (and the practical complexities it brings) has been discussed before. It may be better to focus discussion in one thread.
This IS NOT on the fly decompression/compression with the incumbent;
"enough space?" guessing,
double writing (as explained)
and fragmentation issues.

ie: The same issues as those you linked DO NOT apply.
Again:
You might also re-read the 'why it's an issue' in my original post as no one seemed to know about the double write - erase done by the old compression method.
Feel free to look that up.
Logic
Level 5
Level 5
Posts: 47
Joined: Mon Oct 29, 2018 2:12 pm

Re: Win 10 Compact feature: Better Compatible? compression!

Post by Logic »

"...my laptop [2 core Intel at between 2.5 and 2.7GHz] runs faster with Compact OS enabled. I ran the benchmark several times and continued to experience higher benchmark scores with compression enabled..."
https://www.makeuseof.com/tag/save-disk ... indows-10/
Logic
Level 5
Level 5
Posts: 47
Joined: Mon Oct 29, 2018 2:12 pm

Re: Win 10 Compact feature: Better Compatible? compression!

Post by Logic »

"...Here’s the result of compacting the Windows folder on an Acer dual core, 2.2 Ghz laptop with 6 GB of RAM and a SANDISK SSD (256 GB)....
Boottime = 12.5 seconds before compression
Boottime = 5.6 seconds after compression..."
https://ttcshelbyville.wordpress.com/20 ... even-more/
Logic
Level 5
Level 5
Posts: 47
Joined: Mon Oct 29, 2018 2:12 pm

Re: Win 10 Compact feature: Better Compatible? compression!

Post by Logic »

"...Benefits of using that on games are the following.
Lower Size
Shorter load times
Probably fixed stutter when the HDD is loading too slow

Great Examples are Killing floor 2 (halves Initial load time ), Apex legends , rising storm games ( halves Initial load time ), CSGO , CSS , and plenty more..."
https://www.reddit.com/r/pcgaming/comme ... inks_your/
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Win 10 Compact feature: Better Compatible? compression!

Post by Support »

On Windows 10, I remember that cache (L1/L2) stores compressed data.
steveb
Level 4
Level 4
Posts: 21
Joined: Fri Sep 17, 2021 7:58 am

Re: Win 10 Compact feature: Better Compatible? compression!

Post by steveb »

From my understanding PrimoCache works at the block level then it would use the XPRESS8K versions for example. The performance improvement will really depend on the hardware used. In some cases similar to Windows memory compression, it might be handy instead of writing it to the next tier storage. XPRESS4K from a quick test seems to improve storage only if the storage is slower than 1GB/s.
Post Reply