Hi Cirion,

First of all, I'd like to thank you for all the effort that you've put into your mods. I had a blast playing through the Antumbra Saga and the Caldecott Caper and I think it's not an overstatement to say that your mods are most likely the best for Shadowrun so far.

Now, I actually discovered only recently that you've already completed a third mod and thus I'm late to the discussion about linux-related technicalities. While I do have a linux machine, I usually play Shadowrun via wine. The reason for this is that there's still a bug on the linux port whereby the game can't be exited from the menu. Emulation with wine on the other hand works flawlessly.

I understand, though, that many linux users probably want to avoid sullying their hands with anything even remotely close to windows. Because of this, I've experimented with various solutions to the problem you described above. If this issue is still relevant, I have a script that should be able to provide the linux gaming community with similar functionality like your music replacer.

Anyway, I'm very thrilled to return to CalFree for one final run...

@Klotz Van Ziegelstein: Wow, that sounds awesome! If you don't mind sharing a link to your script, I'll update the above instructions to help out any other players who would prefer to run the game in native Linux.

Thanks for the update on wine as well - I hadn't considered that before, but it does make a lot of sense. I'll add that suggestion as well.

Enjoy your final run in CalFree!

Right, here is the source code:

https://ptpb.pw/MZXd/pl

The script was written in Perl and doesn't come with a GUI. I believe, though, that Linux users will be used to running tools like this from a terminal. Some additional explanation:

- Why Perl?

In contrast to Windows or Mac, Linux distros vary greatly in architecture, configuration and ideological goals. Nearly all of them ship with Perl pre-installed, however, despite it being a less used language nowadays. Thus in the ideal case, running this script will not require the installation of any additional packages. To further ensure of this, only core modules were used. Additionaly, Perl's regex capabilities made it the ideal solution for the task at hand.

-What the script does

The script uses Perl's pack and unpack functions combined with some regex magic to parse the resources.assets and the resources.assets.resS files. No offsets are hardcoded (except the one where content loading starts, for efficiency reasons). Instead, the script dynamically generates a mapping of the relevant offsets before replacing the resources.assets.resS file and updating resources.assets. A backup copy of resources.assets.resS is also created. Because of the dynamic nature of the script, though, the only real difference between a "replace" and a "restore" operation lies in the resources.assets.resS file being used.

-Limitations

In theory, the fact that the script is able to parse the resources.assets file means that it may work platform independent, regardless what kind of Shadowrun Hong Kong install is present on the local machine. All the script really requires to run is the presence of a Perl interpreter, which I think Mac probably still has.

That said, so far only installation paths for Linux have been added to the list that the script checks automatically upon invocation. Users can point it to a specific directory, though. I have no experience with running such scripts in a true Windows environment, however, and i guess the different pathing style (\ instead of /) will most likely cause issues without the presence of some kind of Unix compatibility layer.

Second, the script is still "dumb" in the sense that it does not provide a way to modify individual music tracks. It will simply chew through the resources.assets.resS file and map the tracks in the same order as it finds them to the corresponding entries in resources.assets.

And last but not least, in its current form the script is limited to Shadowrun Hong Kong. It might be possible to extend its functionality to Dragonfall and Returns in the future, but some additional effort will be necessary.

-What has been tested

I have tested the script for my Shadowrun Hong Kong Steam install as well as for my other Steam install via wine (which uses the Windows resource files). I was able to replace the soundtrack for both with the one you provided and revert back afterwards. A full playthrough hasn't happened yet, but I loaded some different saves and entered various locations and everything seemed to work fine. The Steam integrity check also didn't complain after the restore operation.

Below are some pictures from the script at work:

https://ptpb.pw/EyQD.png
https://ptpb.pw/I46S.png
https://ptpb.pw/9wtc.png


Unfortunately, I currently don't have any capacity to host the script myself. I may open a Github repo and also add some nice documentation documents some day but for the time being, I'd be happy to leave this tool in your hands.

@Klotz Van Ziegelstein: This is fantastic, thanks so much for taking the time to create this & write it up! I really like how you dynamically find the offsets, that's a lot better than the hard-coded approach I'm taking in the Mac/Windows versions.

I've made copies of the script and images, so feel free to clean up your versions if you want or need. I'd be happy to host them for now, let me know if you'd prefer me to point back to you in the future.

I can credit you by your Steam username, let me know if you'd prefer another attribution. Cheers!

For posterity / any future researchers, here was my original description for what the script should do. I like the actual solution better, as it uses more flexible regex searches instead of relying on hard-coded offsets.

===========

The replacer has to do two things.
1. The game’s music is stored in the file resources.assets.resS. We need to replace the current version with the new version. Ideally, keep the original around as a backup so players can easily switch back and forth between the original and new music.
2. Values inside the file resources.assets need to be updated with new information explaining where the new music can be found within resources.assets.resS. In particular, for each track of music, we need to provide the offset within resources.assets.resS where the music starts, and tell it how many bytes long that music track is.

This is probably easiest to explain via the source code. Here’s the Windows version[pastebin.com] (crappy C# Windows Form code). And here’s the Mac version[pastebin.com] (slightly-less-crappy Swift code).

Porting to Linux will require the following steps.
1. Open resources.assets in a hex editor.
2. Search for HongKong-Hub-TeaHouse[i.imgur.com]
3. Scanning forward, you’ll see[i.imgur.com] 0x02, multiple 0x00, a 0x0E, more 0x00, and then 0x020000.
4. After this is a 4-byte integer describing the track size. For Windows, this is 0xA6AB2F00.
5. Reversing the endian, we get 0x002FABA6. That translates to 3124134, about 3MB, which is a reasonable size for an audio track a minute or two long. The value in Linux will probably be close but not exactly the same. Make a note of both this value and its position. For Windows, it is at 0x778075CC.
6. After this is a 4-byte integer describing the track position[i.imgur.com]. Because this track comes at the very start, it is 0x00000000. This position will probably be 4 bytes greater than the previous position; in the screenshot, it is at 0x778075D0. Make a note of this value.
7. Repeat these steps for the next track, which will probably be HongKong-Hub-Exterior.
8. Then continue for all remaining tracks.
9. If you get this far, but don’t want to continue, please post your results - this is one of the most time-consuming parts of the process, and it will be a help to others.
10. Now, let’s experiment with changes. Find the size and position values corresponding to TitleTheme-UI . Update them to match another track, like the values you recorded for Club88-MainRoom.
11. Start Shadowrun Hong Kong. You should hear the music from Club 88 playing at the title screen.
12. Now it’s time to actually switch to new music. Download the new version[drive.google.com] of resources.assets.resS and overwrite the existing one.
13. Update the TitleTheme-UI values so the offset is 6075686 (0x26B55C00) and the length is 7148981 (0xB5156D00). (I THINK that’s the correct endian, flip if necessary).
14. Start Shadowrun Hong Kong. You should now be hearing a completely new music track.

The full set of values for tracks are available in the source code for the Windows[pastebin.com] and Mac[pastebin.com] music replacers. You can manually update all these values, but that’s what programs are for! And if you can write this program, then all other Linux users will be able to use it instead of going through this process themselves. I have no idea what the common way of doing stuff like this is these days, but ideally use whatever tools would be most familiar / comfortable to other Linux gamers.

It's the least I could do. After all, your mods have kept the Shadowrun games alive and I really hope someone will be able to pick up the torch in the future. Or maybe even you will feel the urge to return for another run?

As for the attribution, the steam username is fine. Feel free to contact me if any bug reports start croping up. I'll also be in touch if there are further developments regarding the script. Until then, safe running hehe...

(Btw the new title theme is awesome)

I've been unable to get the script to find the folder. I get the message even after using the -i command mannually to bring it to the exact folder, I still get the same error.

@Ranic
Apologies for being late, from the sound of the error message it seems the problem is not with the path to the install but with the one to the new music file. Have you downloaded the music provided with the mod and given it to the script with the -n option?

I will check once I get a new machine. I currently have nothing that can run Hong Kong. I did have the shadow tunes file in the home folder I also tried pathing to it, but I don't think I tried doing both at the same time.

Very well. Once you get around to give it another try, put the shadow tunes file and the new music (link is above in step 1 of option 2) in the same folder. Then from a terminal do:

1. cd to the folder where the script and the music file are located.

2. From within the folder, try to invoke the script like this:


"resources.assets.reS" should be the name of the music file downloaded beforehand. If the script then complains that it can't find the Shadowrun installation, you can try and use the -i option like this:


The path will in all likelihood contain whitespaces, so it will have to be surrounded with double or single quotes.

Anyway, keep me posted on further results.

i'm sure it's been said here, but the 'perl shadow_tune..pl ...' script command in the faq has a filename that doesn't match the download filename, so it has to be adjusted accordingly. me, being the pessimist that i am, assumed it was a deeper problem and ran through the help menu before starting at square one. hope this helps.

@dogwaffles: Thanks for the note. What browser are you using? What filename did it download as for you? Did you download by clicking the "download" button on the pastebin site, or via some other method? It is named "shadow_tune.pl" for me when I download from pastebin, but there might be something different in our environments that would cause that to change for you.

Interesting, my current firefox version also downloads it as shadow_tune.pl. I see, though, that differing filenames introduce a potential for confusion and I think for the upcoming version 1.1 I'll have the script find its name "dynamically" for the help text so that the user is also free to rename it as prefered.

@dogwaffles
If you have any other observations to share, I'd be happy to hear them. Any reports will help to improve the script.

@cirion
I'm working on a new version of the script. shadow_tune 1.1 will be capable of replacing the soundtrack for all of Harebrained Scheme's Shadowrun games. This added functionality will come at the cost of a more elaborate commandline, I'm afraid, and currently I'm still evaluating various option "styles" to make it as simple as possible.

I also feel that the script needs a proper readme and faq. At this point, it seems reasonable to open a Github project to provide that sort of documentation. It might take some time and it might also be a bit of a bumpy ride, as I've never hosted any of my stuff on Github before, but I will get it done by the end of this month.

@cirion
Right, I went ahead and opened up a repo:

https://github.com/Van-Ziegelstein/Shadow-Tune/blob/master/README.md

This might be a bit of an overkill but I feel that it will be the best long-term solution. In the future, Github's wiki functionality could also serve as a hub for technical documentation regarding the layout of Shadowrun's sound files.

The latest version of the script is now shadow-tune 2.0.6. It can swap out the music files of all three Shadowrun games but defaults to Shadowrun Returns. The add-ons can be targeted with the new -e commandline switch.

This is the first time I wrote a guide to a script of mine, so I'd be happy if you could look through the README and point out some wacky sections. Also, let me know if you wish to be credited with your Steam name or another one.

Addendum:
I'm no fan of that legalese crap but it seemed to me that a choice had to be made. The MIT licence appeared to be the most permissive and simple one, which is why I ended up rolling with that.

@Klotz Van Ziegelstein: Wow, that's awesome! Great work on this, I think it'll be a huge benefit for the Shadowrun mod community. In particular, I think it'll be extremely useful for folks who want to create new Dragonfall campaigns and take advantage of the greater number of tracks available.

The README looks great, very descriptive and readable. In the Acknowledgements, can you also credit Zetor? Zetor did the initial research into Shadowrun Returns and described how resources.assets and resources.assets.resS were used. I updated that with some more current research on the audio format used for Hong Kong and building a program to replace the tracks.

I'll update the instructions at the top of this thread in a bit to reference the new format.