A solution to preloading game data for you gamers?

[dustyny]

Hi peoples,

I've been thinking on the problem that a lot of you have been having with game data not being preloaded in to cache. I think I might have a solution for you but it will take a while for the file to be read. I've done a quick test and it appears that the data has been loaded in to cache but I'm not a gamer (not on the PC anyway) so I'll need someone test this for me. Keep in mind this is a bit of a hack, so if your game files far exceed your cache then the benefits maybe negligible

1) Identify the game data files. Usually they end in a .dat but they can be just about anything the developer wants.. I'd focus on sounds, textures, game binaries & 3d models. Usually they are large, but sometimes it could be a whole directory of small files. (in that case use a wild card, findstr a c:\game\data\*.*)
2) Open the Command line (CMD) and run the findstr command on the file. This will read & search through the enitre file and it should load the file in to the cache.
3) create a bat script that loads the data and then launches the game.

Here is an example of what the command should look like.
Findstr (search term, what you use doesn't matter) (Filename)
Findstr a 3dmodels.bin

Let's say you have 4 files, totaling 5GB, if you have a 5GB RAM cache you should be able to load all the files in to the cache.
textures.bin - 1.5GB
3dModels.bin - 1GB
sounds.bin - 2GB
game.exe - .500MB

--- What the bat file should look like. ----

findstr a "c:\program files\Kick ass game\textures.bin"
findstr a "c:\program files\Kick ass game\3dModels.bin"
findstr a "c:\program files\Kick ass game\sounds.bin"
findstr a "c:\program files\Kick ass game\game.exe"
"c:\program files\Kick ass game\game.exe"

---------------------------------

Keep in mind that reading from the HDD is always going to be slow so it might take a while for this data to be read off of the drives.
Data size / drive speed = time to load in seconds
5000MB / 150MBs = 33 seconds (this would be an ideal, since data is RARELY read at max speed I'd expect it to be about 2-3 minutes or more).

[IchiGoQc]

Hi there,

I'll give it a try tonight and give you a feedback on if it's works or not. I was trying to figure a way to optimize the loading time of my game without creating a RAM DISK and forcing the loading from RAM instead of HDD, the fancy cache was a bit hard to understand at first but after reading forum I now understand the function of defer write and how the fancy cache use the file. So pre caching the needed file before lunching the game sounds interesting but how can I verify that the game is taking information from files in cache instead of HDD !?

I'm not very familiar with caching so ..... It's kinda my first time. I do have SSD and it improves a lot loading time but we all know RAM is blazing fast !!!

[IchiGoQc]

I tested your method and it doesn't work, the only way I found to cache a file manually is with the "copy" command. You have to copy the file to another disk which is not caching into memory. So Fancy is gonna read the file into memory. But using Process Monitor you can't imagine how many files are in use by the game ..... I mean if you have DLC it's not the same texture files etc etc ..... The best way is to let the program handle the files it needs to be cache !

Hope this help !

[Violator]

If you want to have faster loading times than with Fancy Cache, a huge ram disk and creating junctions is the only way to go.
Fancy cache has it's advantage in not being a file level cache, and to support LFU/LRU reads, for games read only caching is all you need when your using Fancy Cache.
This also means that Fancy Cache will cache streamed data from games that got files bigger than your amount of ram.
Block level caching ftw.

EDIT.
A bit more info about how you should be using a ramdisk for game file caching.
1. You create a ramdisk.
2. You copy files from the games directory to the ramdisk
3. You rename the moved files in the games directory
4. You use MKLINK to create directory junctions and file hardlinks in the game directory, those point to the files on the ramdisk.

Those links will be seen by the game as the original game files, since they will appear with the same names at the place where you had the originals (now renamed) files.
Since the links point to the real files placed on your ramdisk, those will be used, and will give a significant increase in loading times and overall smoothness when runnnig the game.
You should monitor which files that are used the most, and how they are compressed, the highest benefit is to place packet texture files on the ramdisk, since they usually got the highest I/O.

I'm not familiar with the ramdisk sold by Romexo, but I know that products like imdisk are able to create an image of the ramdisk which can be used in a script for reloading.
That means you can automate the process, just note that the renamed files in your games directory wont be update if there is any update to the game, updates will happen to your files on the ramdisk instead.

[dustyny]

I wonder why it didn't work? I'm thinking it must be reading the files in text mode instead of binary.
I'm thinking a copy to null would work, but I don't know how to do that in Windows. I'll look around for a solution..

[Manny]

you need to read all clusters of file, to get it fully cached, i would use PowerShell script for that

[dustyny]

Just spent some time researching this.. I can't seem to find any tool in Win that makes this possible. Windows doesn't create a null device like Linux or Unix, so can't copy to that..
Violator's solution is probably the best you can do until Fancycache supports persistent caching.

[Manny]

Just spent some time researching this.. I can't seem to find any tool in Win that makes this possible. Windows doesn't create a null device like Linux or Unix, so can't copy to that..
Violator's solution is probably the best you can do until Fancycache supports persistent caching.

You can use C# or any other PL that you are familiar with, to did that. You just need to read that file from start to the end, without keeping read info in memory, it would be your null device if you want.
Algorithm is pretty simple:

Open file.
while(not the end of file)
Read peace of data (64kb or 512kb)

here is some info about how to read data effectively
http://stackoverflow.com/questions/3206 ... ut-copying

[Violator]

With exception that C# isn't the optimal solution for it, to much overhead compared to C and even C++ for such a simple operation.

The best performance benefits that I have seen gamers get with anything related to caching is simply by making a batch script that does utilize a ramdisk and Mklink for the games most used texture files.
As example: http://www.swtor.com/community/showthread.php?p=3780109

[Manny]

please be more specific how much overheads it will add in percents?