RimWorld

RimWorld

70 ratings
Image Opt
3
3
2
   
Award
Favorite
Favorited
Unfavorite
Mod, 1.6
File Size
Posted
Updated
6.301 MB
7 Aug @ 2:36am
13 Aug @ 9:41am
11 Change Notes ( view )

Subscribe to download
Image Opt

Description
Translated from gemini flash

This is a mod that attempts to optimize texture image loading times.

Experimental
Because this mod changes how RimWorld loads images and involves ffi and multithreading, strange errors are expected. So if you don't have many mods, you don't need this one. If you encounter errors related to "texture," "image," etc., please try disabling this mod.

Currently only supports Windows 64bit. Linux can be supported if necessary.

This mod mainly has three features:
  • Loads .dds files (just like Graphics Settings+)
  • Converts .png/jpeg files into GPU-compressed files (like Rimsort's optimize texture), and stores them with zstd compression.
  • Utilizes native language image decoding and parallel decoding.

Recommended Usage
After enabling this mod, open the game and go to the mod settings page. Click "Open GPU Compression Tool," check all options, set the quality to "high," and then click the "Generate" button. Please wait for the multithreaded compression to finish and do not close the window. For reference: on my 5600 CPU, processing 23,600 images from 550 mods takes about 600 seconds.
If you have previously used Rimsort/RimPy to generate .dds files, you can click "Clear all dds files" first to reduce disk space usage.
This operation will not delete or modify any of the mod's original png images; it will only generate/delete .dds files and will not corrupt mod content.
 

When using this mod, if you:
1. Not using .dds files: You'll benefit from parallel decoding.
2. Use DDS files generated by this mod or rimsort: This will be no different than using Graphics Settings+ alone, but parallel file reading may provide a slight load speed boost.
3. Use .zstd files generated by this mod: This will significantly reduce the disk space used by the DDS files, but will slightly increase load time (very little, I can't think of a reason not to use zstd).

How It Works
Loading png/jpeg images involves a very time-consuming operation: decoding. RimWorld loads all textures sequentially on the main thread, so when a large number of mods with a lot of images are used, it becomes very slow.
However, Unity doesn't allow textures to be loaded on non-main threads, so it's not possible to simply put texture loading into a thread pool for parallel processing.
This mod solves this by placing the image file reading and decoding operations into a thread pool at the point where textures are loaded. After traversing all mod textures, it waits on the main thread for all decoding operations to complete, and finally loads the texture data and submits it to the GPU. This allows the decoding operations to be performed in parallel.
Additionally, the decoding is done using a native language library (the Rust image crate), which should make the decoding process faster than C#.

What are the benefits of GPU-compressed BC7 textures?
The GPU can read these textures directly without CPU involvement, so the loading speed is faster than with common PNG files, and VRAM usage is significantly reduced.
This will also reduce GPU stress when rendering.
(BC7 is the compression algorithm, and DDS is the container format.)

So... what's the catch?
BC7 is a lossy compression, but the visual quality difference is virtually nonexistent, so it's irrelevant. The key issue is that its size on the hard drive is much larger than highly-compressed PNG/JPEG images. (If you use Rimsort's "optimize texture" and then check a mod's `Textures/` folder, you will find that .dds files are usually larger than their corresponding .png files. This is due to their storage format.)

Can this be solved?
To mitigate this drawback, this mod's GPU texture generator has a **zstd** option. When this option is enabled (it is by default), after the .dds file is generated, it undergoes another round of zstd compression before being stored on the hard drive. During loading, it is then decoded. This significantly reduces the size of the .dds files, and thanks to zstd's fast decoding performance, it has no significant impact on image loading times. (To standardize the format, zstd-compressed texture images will have the name "name.dds.zstd").
In the textures of the mod I use, it is about 6 times smaller. (dds -> dds.zstd)

Can all images be converted to DDS and loaded?
No. The texture generator will exclude UI images in the UI directory. Some DDS images that cannot be loaded correctly will attempt to load the original PNG file instead. Unsupported formats, images that fail to load in Rust, and some unconventional images (e.g., images in a tar archive) will fall back to the vanilla texture loading method, loading the original PNG/JPG images.

Compatibility
This mod is theoretically incompatible with any mod that modifies the texture loading process/results, with the exception of Graphics Settings+.

This mod can be used with Graphics Settings+. The DDS loading and mipmap bias features of Graphics Settings+ will not be active, but its VRAM usage calculation will work correctly.

Because it exceeds the word limit of Steam Workshop, so...


Some Data
Turning off Windows Defender will improve loading speed because, in my observation, the Windows Defender process uses nearly 50% of the CPU when loading the game. However, the Steam Workshop is a place where anyone can submit files, so if you don't want to play RimWorld with a virus...
The timings below were all recorded with Windows Defender enabled.

Mod Content Loading Time (including audio/strings, etc.):
Data comes from the `Verse.LoadedModManager+<>c -> Void <LoadModContent>b__13_0()` entry in the DeepProfiler after enabling verbose logging.
There is a margin of error with each load, so these results are for reference only. To avoid the effects of file caching, this data represents the results of the second consecutive game launch.
Vanilla
164890.683ms (164s)
ImageOpt without DDS
40534.917ms (40s)
ImageOpt with DDS and zstd
25727.026ms (25s)

More Accurate Data
Vanilla:
164310.504ms (164s) (Similar to the data above due to a margin of error from different launches) - From the "Reload textures" entry in the DeepProfiler

ImageOpt without DDS:
23.804 seconds DDS images: 0 PNG/JPG images: 26476 - From this mod's statistics

ImageOpt with DDS and zstd:
5.114 seconds DDS images: 23264 PNG/JPG images: 3212 - From this mod's statistics

VRAM Usage
Data from Graphics Settings+ VRAM usage statistics
PNG/JPG images
17.7 GB
DDS images
4.77 GB

AI-generated content:
* English translation
* Logo image
62 Comments
soeur  [author] 1 hour ago 
mac没办法,因为我没有apple设备,没办法在windows上编译macos的native lib (
王面手雷玉 2 hours ago 
This comment is awaiting analysis by our automated content check system. It will be temporarily hidden until we verify that it does not contain harmful content (e.g. links to websites that attempt to steal information).
miragea 4 hours ago 
Mac M2用户留下了羡慕的泪水
956344097 4 hours ago 
可惜在mac上用不了:steamsad:
torsina 12 hours ago 
@soeur ah alright, thank you for the clarification ❤️.
soeur  [author] 12 hours ago 
@torsina OK, I explained the difference between not using DDS, using DDS, and using DDS & ZSTD on the page. Regarding your question, this mod uses the same compression algorithm as Rimsort, but this mod uses parallel reading of image files, so loading DDS will be slightly faster (even DDS generated by Rimsort).ZSTD does increase the loading time, but it is very fast and almost negligible in my tests.
torsina 13 hours ago 
I'm a bit confused. If I don't care about disk space usage, is there any benefit in loading time or performance from using this mod instead of using rimsort's compression? Does the zsdt compression improve loading times? I feel like if anything it would make them longer as you need to decompress the zsdt on each load?

You should make this clearer in the workshop page what the differences are between ImageOpt with just dss versus dss+zsdt, as right now you only show the later.
soeur  [author] 13 hours ago 
@Senattoja In fact, I compiled a .so library for Linux in the current version, but since I don't have a Linux system with GUI, there is no way to test it (:
soeur  [author] 13 hours ago 
@Syntax This is because the program reads an image with a size of 0x0. I'm not sure why this is the case, but it won't actually cause an error because no incorrect DDS will be generated and the game will use the original PNG file.
Senattoja 14 hours ago 
Can you please add Linux support ?