Why did Valve change the way game assets are stored?...
So a couple of updates ago, Steam neatly stored all game store assets in appcache/librarycache/<gameid>.

So, in that folder, you'd have logo.png, library_600x900.jpg, and library_hero.jpg.

Now, for some god forsaken unholy reason, Valve decided this was too straightforward and decided to obfuscate it by putting these assets in subfolders with cryptic, generated names. Worse, it's not even consistent. Some games still do it the old way, but most now use the new structure.

Okay, so, perhaps these folder names are stored in the game's json data perhaps? Nope. They have been hidden away, perhaps in some of Valve's proprietary binary VDF format.

So now, instead of just expecting logo.png to be in the root of the assets folder, I now have to iterate recursively through all folders and hope to find the asset in any of them. What a mess.

Did Valve do this on purpose to make life harder for third party frontends or something?
< >
Showing 1-15 of 18 comments
Eocene 25 Aug @ 1:44am 
Worse still, should you choose to use Steam's 'clear download cache' option, it nukes all downloaded game assets, so they are downloaded again on demand.

The Steam client may do it's job, but holy moly, peek just a little under the surface and it's a absolutely horribly designed piece of software.
Zefar 25 Aug @ 2:43am 
Just curious, what would you use this for?


Originally posted by Eocene:
Worse still, should you choose to use Steam's 'clear download cache' option, it nukes all downloaded game assets, so they are downloaded again on demand.

You ask Steam to nuke it's download cache. Steam did exactly that.
It says that it might solve some issues with downloading.

What is the issue here?
Eocene 25 Aug @ 3:31am 
Don't get hung up on the 'clear download cache' thing.

(You'd think that option would clear out the temp folders that store downloaded game chunks. It shouldn't delete art assets.)

The topic of this thread is: why did Valve decide to obfuscate stored art assets in such a nasty and seemingly senseless way? It was much better before!

Incidentally, it also makes the client slower, because now it has to read/parse/debinarify some file somewhere just to get the path to the asset.

My goodness, do I ever loathe this POS software. I hate not being able to get rid of it without losing access to my library.

Gaben, fire your software engineers. They are incompetent.
Last edited by Eocene; 25 Aug @ 3:36am
Wolfpig 25 Aug @ 3:45am 
Originally posted by Eocene:

The topic of this thread is: why did Valve decide to obfuscate stored art assets in such a nasty and seemingly senseless way?


because 99.99% of steam users do not care in which folders the client stores some cached data, as it has not the slightest impact on using steam.
Originally posted by Eocene:
The topic of this thread is: why did Valve decide to obfuscate stored art assets in such a nasty and seemingly senseless way? It was much better before!

Because they wanted to.
Eocene 25 Aug @ 4:10am 
Even if you don't care where or how assets are stored, it's worse for end users because now every time Steam has to load an art asset, it has to run code that returns the path to this asset.

If this path is stored in a binary file - as I suspect it is, to prevent snooping - it has to load in this file, debinarify it, and then parse it to get the path.

Every time it needs to display this asset.

It's nonsensical and wasteful.

It makes Steam slower.
Last edited by Eocene; 25 Aug @ 4:31am
Originally posted by Eocene:
It makes Steam slower.

And unless you're playing a two-decade old mouldy potato, no one is going to notice any such potential slow down.
Eocene 25 Aug @ 4:44am 
Sure you would. But because you don't have the older Steam client to compare it to, you're just going to go 'eh, must be my imagination' and move on.

And so the enshittification continues, bit by bit.
Eocene 25 Aug @ 4:53am 
Try this.

Click on the first game in your list.

Now, step down by pressing the 'arrow down' key.

Tap it rapidly.

Steam completely chokes here trying to load all the assets here and there.

It feels very laggy, taking about a full second to load a page.

It doesn't even employ proper threading; it won't display the logo or banner (even if locally cached) until all the HTML fluff is done.

This should be instant.

The Steam client is a terrible piece of software. Stop trying to defend it. You are arguing against your own interests.
Zefar 25 Aug @ 4:53am 
Originally posted by Eocene:
Don't get hung up on the 'clear download cache' thing.

(You'd think that option would clear out the temp folders that store downloaded game chunks. It shouldn't delete art assets.)

The topic of this thread is: why did Valve decide to obfuscate stored art assets in such a nasty and seemingly senseless way? It was much better before!

Incidentally, it also makes the client slower, because now it has to read/parse/debinarify some file somewhere just to get the path to the asset.

My goodness, do I ever loathe this POS software. I hate not being able to get rid of it without losing access to my library.

Gaben, fire your software engineers. They are incompetent.

You had 2 points, I asked about the first one and then commented on the second one.

For what reason would you need to be in those folders?


Originally posted by Eocene:
Sure you would. But because you don't have the older Steam client to compare it to, you're just going to go 'eh, must be my imagination' and move on.

And so the enshittification continues, bit by bit.

Modern day CPUs can do what? millions to billions of calculations per second so Steam going to another directory isn't going to make it any slower.
Originally posted by Eocene:
Steam completely chokes here trying to load all the assets here and there.

I'm sorry you appear to have lacking hardware or a similarly poor internet connection, but Steam is loading the assets just fine for me when doing that -- even though there is no real reasonable use case for even needing to do that.
Last edited by Chika Ogiue; 25 Aug @ 5:31am
aiusepsi 25 Aug @ 5:36am 
The "cryptic, generated names" are almost certainly SHA1 hashes.

If I had to guess, it's probably related to when they started accepting / requiring higher-resolution artwork for library assets, with the SHA1 hash of the originally submitted asset used to tag all the things which their backend pipeline generates from that asset.

For example, this library hero art:

https://shared.fastly.steamstatic.com/store_item_assets/steam/apps/3674060/ee25c37e61184a35ae5e814aadf22fff98d14cbc/library_hero_2x.jpg

https://shared.fastly.steamstatic.com/store_item_assets/steam/apps/3674060/ee25c37e61184a35ae5e814aadf22fff98d14cbc/library_hero.jpg

https://shared.fastly.steamstatic.com/store_item_assets/steam/apps/3674060/ee25c37e61184a35ae5e814aadf22fff98d14cbc/library_hero_blur.jpg

All clearly derived from the same source image.

Originally posted by Eocene:
Even if you don't care where or how assets are stored, it's worse for end users because now every time Steam has to load an art asset, it has to run code that returns the path to this asset.

If this path is stored in a binary file - as I suspect it is, to prevent snooping - it has to load in this file, debinarify it, and then parse it to get the path.

Every time it needs to display this asset.

It's nonsensical and wasteful.

I'm virtually certain it's not to prevent snooping. For one, if you actually wanted to prevent snooping, or at least make it more difficult, there's lots of ways you could easily obfuscate the data. Encryption, for example. Or heck, just XOR the data with a fixed bit-pattern.

Also, the binary VDF format isn't complicated; you can figure it out easily with a hex editor and some patience, and people did that years ago. Pretty sure I did it myself at one point. Because lots of people have done this before, there's plenty of code out there for parsing VDF files, e.g. https://www.npmjs.com/package/binary-vdf-parser Other people have written similar code for other languages.

People bothered to do this years ago because Steam's always* kept the metadata it needs to work in binary VDF files, like the appinfo.vdf file that stores app metadata. The mapping from asset type to path is one of the things stored in that metadata, so is one of the things tracked by SteamDB**, see e.g.: https://steamdb.info/app/3674060/info/

Anyway: I would be extremely surprised if Steam was reading and parsing these files every time it needed any of this data. It is almost certainly reading it once at start-up and caching the result in memory.

I think a fair summary is that, at most, they've gone from hardcoded paths to paths which are controlled by the same metadata which governs things like: what name is displayed for the game in the library, what review score it has for the purposes of library sorting, what beta branches are available, what the current version of the game content is, etc.

* It's actually not always. Steam used to keep everything in a file called "clientregistry.blob", way back in the day; but that was all an older iteration of the Steam system.

** Although SteamDB doesn't work by parsing appinfo.vdf; instead they talk to the Steam backend directly using a reverse-engineered implementation of the protocol the Steam client uses to talk to the backend to read the metadata from a system called PICS.
Originally posted by Eocene:
Try this.

Click on the first game in your list.

Now, step down by pressing the 'arrow down' key.

Tap it rapidly.

Steam completely chokes here trying to load all the assets here and there.

It feels very laggy, taking about a full second to load a page.

It doesn't even employ proper threading; it won't display the logo or banner (even if locally cached) until all the HTML fluff is done.

This should be instant.

The Steam client is a terrible piece of software. Stop trying to defend it. You are arguing against your own interests.

I've just done what you suggested, I tapped down each of my games in my library (over 9000 of them) and I had no problems at all.

You say you have a ghetto PC, so do you mind sharing the specs of your system because from this end it seems that Steam is working fine and the problem is more than likely your end.
Eocene 25 Aug @ 5:37am 
Ryzen 7 5800X (8 cores running at 3,8Ghz)
16Gb ram
SSD
10/Mb sec internet connection, wired (not wifi)

Stop trying to gaslight. The problem is that Steam is designed all wrong, not my specs. It is a friggin' desktop app. A very bad one.
Eocene 25 Aug @ 5:45am 
Originally posted by aiusepsi:
The "cryptic, generated names" are almost certainly SHA1 hashes.

If I had to guess, it's probably related to when they started accepting / requiring higher-resolution artwork for library assets, with the SHA1 hash of the originally submitted asset used to tag all the things which their backend pipeline generates from that asset.

For example, this library hero art:

https://shared.fastly.steamstatic.com/store_item_assets/steam/apps/3674060/ee25c37e61184a35ae5e814aadf22fff98d14cbc/library_hero_2x.jpg

https://shared.fastly.steamstatic.com/store_item_assets/steam/apps/3674060/ee25c37e61184a35ae5e814aadf22fff98d14cbc/library_hero.jpg

https://shared.fastly.steamstatic.com/store_item_assets/steam/apps/3674060/ee25c37e61184a35ae5e814aadf22fff98d14cbc/library_hero_blur.jpg

All clearly derived from the same source image.

Originally posted by Eocene:
Even if you don't care where or how assets are stored, it's worse for end users because now every time Steam has to load an art asset, it has to run code that returns the path to this asset.

If this path is stored in a binary file - as I suspect it is, to prevent snooping - it has to load in this file, debinarify it, and then parse it to get the path.

Every time it needs to display this asset.

It's nonsensical and wasteful.

I'm virtually certain it's not to prevent snooping. For one, if you actually wanted to prevent snooping, or at least make it more difficult, there's lots of ways you could easily obfuscate the data. Encryption, for example. Or heck, just XOR the data with a fixed bit-pattern.

Also, the binary VDF format isn't complicated; you can figure it out easily with a hex editor and some patience, and people did that years ago. Pretty sure I did it myself at one point. Because lots of people have done this before, there's plenty of code out there for parsing VDF files, e.g. https://www.npmjs.com/package/binary-vdf-parser Other people have written similar code for other languages.

People bothered to do this years ago because Steam's always* kept the metadata it needs to work in binary VDF files, like the appinfo.vdf file that stores app metadata. The mapping from asset type to path is one of the things stored in that metadata, so is one of the things tracked by SteamDB**, see e.g.: https://steamdb.info/app/3674060/info/

Anyway: I would be extremely surprised if Steam was reading and parsing these files every time it needed any of this data. It is almost certainly reading it once at start-up and caching the result in memory.

I think a fair summary is that, at most, they've gone from hardcoded paths to paths which are controlled by the same metadata which governs things like: what name is displayed for the game in the library, what review score it has for the purposes of library sorting, what beta branches are available, what the current version of the game content is, etc.

* It's actually not always. Steam used to keep everything in a file called "clientregistry.blob", way back in the day; but that was all an older iteration of the Steam system.

** Although SteamDB doesn't work by parsing appinfo.vdf; instead they talk to the Steam backend directly using a reverse-engineered implementation of the protocol the Steam client uses to talk to the backend to read the metadata from a system called PICS.

Thank you. This is a reply that can be worked with.

Ultimately, Steam is designed so badly it barely benefits from storing resources locally. Open a page. It won't display the header/logo until after it loaded in all the HTML fluff in the lower half.

What it should do is display this straight away, and load the rest of the HTML in the background - and cancel it when you select another game. It won't allow you to do that, the UI is basically frozen until it's done loading.
< >
Showing 1-15 of 18 comments
Per page: 1530 50