Win 10 Compact feature: Better Compatible? compression!
Posted: Sat Aug 07, 2021 1:07 am
Windows 10 comes with a new compression feature called Compact/CompactOS:
It's generally faster (multi threaded?) than Windows compression
BUT
the main advantage is that it does NOT:
1st write the uncompressed file to disk,
Then also the compressed file
Then compare sizes
Then erase the uncompressed version if its bigger, or the compressed version if it is not
That is normally NOT good for SSD lifespan and results in extreme fragmentation which slows down HDDs.
It also causes 'Out of space' issues/crashes with Primocache.
With Compact; files should remain compacted all through Primocache and only be decompressed into RAM once passed to windows' I/O stack..?
Windows 10 Compact comes with 4 new algorithms:
XPRESS4K: Fastest - least compression
XPRESS8K: Medium speed - more compression
XPRESS16K: Slowest - most compression.
LZX: Even slower and best used to save space on infrequently used files.
As a general rule (Average modern CPU):
E4K will speed up I/O on NVME SSDs.
E8K will speed up I/O on SATA SSDs.
E16K will speed up I/O on HDDS.
Unless PrimoCache were to integrate this, I guess choosing according to your L2 drive will be most performant for cached data, but E16K would give you better uncached performance and more space in the caches..?
Testing reqd..!
2 GUIs (and info) for this command line tool are available:
https://github.com/Freaky/Compactor
https://github.com/ImminentFate/CompactGUI
Last I checked Freaky Compactor was the better choice due to knowing and skipping previously compacted files.
That may have changed in the newer version of CompactGUI.
(FOR THE OLD WINDOWS COMPRESSION:)
IF the above
'Double Write - Delete'
can happen in RAM using Primcache's Defered write option, without causing the 'Out of space' issue/crash;
the old Windows compression may be worthwhile..?
It's still going to be single threaded however.
NB that files are compressed in chunks 16X the cluster size.
That's 8KB for 512B clusters and 64KB for 4K clusters.
Something to keep in mind when choosing block size..?
Testing reqd..?
It's generally faster (multi threaded?) than Windows compression
BUT
the main advantage is that it does NOT:
1st write the uncompressed file to disk,
Then also the compressed file
Then compare sizes
Then erase the uncompressed version if its bigger, or the compressed version if it is not
That is normally NOT good for SSD lifespan and results in extreme fragmentation which slows down HDDs.
It also causes 'Out of space' issues/crashes with Primocache.
With Compact; files should remain compacted all through Primocache and only be decompressed into RAM once passed to windows' I/O stack..?
Windows 10 Compact comes with 4 new algorithms:
XPRESS4K: Fastest - least compression
XPRESS8K: Medium speed - more compression
XPRESS16K: Slowest - most compression.
LZX: Even slower and best used to save space on infrequently used files.
As a general rule (Average modern CPU):
E4K will speed up I/O on NVME SSDs.
E8K will speed up I/O on SATA SSDs.
E16K will speed up I/O on HDDS.
Unless PrimoCache were to integrate this, I guess choosing according to your L2 drive will be most performant for cached data, but E16K would give you better uncached performance and more space in the caches..?
Testing reqd..!
2 GUIs (and info) for this command line tool are available:
https://github.com/Freaky/Compactor
https://github.com/ImminentFate/CompactGUI
Last I checked Freaky Compactor was the better choice due to knowing and skipping previously compacted files.
That may have changed in the newer version of CompactGUI.
(FOR THE OLD WINDOWS COMPRESSION:)
IF the above
'Double Write - Delete'
can happen in RAM using Primcache's Defered write option, without causing the 'Out of space' issue/crash;
the old Windows compression may be worthwhile..?
It's still going to be single threaded however.
NB that files are compressed in chunks 16X the cluster size.
That's 8KB for 512B clusters and 64KB for 4K clusters.
Something to keep in mind when choosing block size..?
Testing reqd..?