Blank 14 Jul, 2024 @ 1:15pm
Steam Compression is slowing down Download on modern Hardware
I downloaded Apex Legends with "only" 1,5Gbit/s my local CDN server can handle 5Gbit/s esaily. My Ryzen 5 5600 is too slow to decompress it. xz (lzma) is very slow but has high compression rations, but it is worth it? LZ4 or even ZSTD would be better choice, steam would save computing power and the people with faster internet connection can download their games faster without buying the newest high end Processor.
< >
Showing 1-7 of 7 comments
Satoru 14 Jul, 2024 @ 1:19pm 
Steam downloads are limited by

1) your cpu
2) your disk IO
3) your anti virus
4) your isp

Pick one
William Shakesman 14 Jul, 2024 @ 2:06pm 
Originally posted by Satoru:
Steam downloads are limited by

1) your cpu
2) your disk IO
3) your anti virus
4) your isp

Pick one
Did you read OP? This is completely irrelevant to his statement about compression methods. In addition, it is inherently silly to assume Steam can serve infinite bandwidth to any location.
cSg|mc-Hotsauce 14 Jul, 2024 @ 2:32pm 
Originally posted by William Shakesman:
In addition, it is inherently silly to assume Steam can serve infinite bandwidth to any location.

What do you think is the maximum theoretical download speed of Steam's servers?

:cool_seagull:
Last edited by cSg|mc-Hotsauce; 14 Jul, 2024 @ 2:35pm
aiusepsi 15 Jul, 2024 @ 3:04am 
The OP's point is a good one. Here's some data on compression algorithms:

Algo Decompression rate Ratio (out of 100, lower is better) lzma 19.00 -9 107 MB/s 22.98 fastlzma2 1.0.1 -10 105 MB/s 22.96 xz 5.2.4 -9 88 MB/s 23.00 zstd 1.4.3 -22 865 MB/s 24.88

Data from here: https://github.com/inikep/lzbench (Note: this data may be quite old, the most recent Zstd version is 1.5.6, and is still under active development, so it may have improved.)

Steam uses the LZMA algorithm for compression, which is represented in this table by the implementations lzma, fastlzma2, and xz. LZMA has a very good compression ratio, 22.96% for fastlzma2. On the other hand, zstd decompresses 8.2 times as fast, and only slightly sacrifices compression ratio, at 24.88%.

The upshot being that if you were downloading a file which was 1 GB uncompressed, you'd download about 19 MB more if it was compressed with zstd rather than LZMA, but it would decompress much faster (or use less CPU if decompression isn't what's bottlenecking).
Last edited by aiusepsi; 15 Jul, 2024 @ 5:29am
Blank 15 Jul, 2024 @ 4:31am 
Originally posted by William Shakesman:
In addition, it is inherently silly to assume Steam can serve infinite bandwidth to any location.

I know that steam cant server infinity bandwidth but im speaking about my local cdn server. I host an cdn server in my house for many PCs but im bottlenecked by the cpu performance sadly because of the poor decompression performance.
Blank 15 Jul, 2024 @ 4:41am 
Originally posted by cSg|mc-Hotsauce:
Originally posted by William Shakesman:
In addition, it is inherently silly to assume Steam can serve infinite bandwidth to any location.

What do you think is the maximum theoretical download speed of Steam's servers?

:cool_seagull:
https://www.peeringdb.com/net/4782
aiusepsi 30 May @ 3:53am 
News relevant to this suggestion:
Originally posted by Valve employee Taylor Sherman:
we recently enabled a new compression algorithm which is vastly better-performing, and will enable faster downloads with much lower CPU load on clients. It will also improve unlock times for preloads for users on SSD (for spinning disks, the disk writes are the bottleneck). This was good just to save CPU load but it is also necessary for the faster (> 1Gbps) internet speeds people have in some places now, and it also really improves the experience on the Steam Deck.
https://steamhost.cn/steamcommunity_com/app/3238670/discussions/0/594022859095939680/?ctp=2#c594023708174098589

Valve recently switched over to Zstd for compression for new games and updates to existing games, for exactly the reasons stated by the OP. Funny that, eh?
Last edited by aiusepsi; 30 May @ 3:54am
< >
Showing 1-7 of 7 comments
Per page: 1530 50

Date Posted: 14 Jul, 2024 @ 1:15pm
Posts: 7