Steam Community :: Guide :: The Internal File Format of the Workshop TempArchiveXX Files

Store Page

Killing Floor

All Discussions Screenshots Artwork Broadcasts Videos Workshop News Guides Reviews

Killing Floor > Guides > cmicroc's Guides

Not enough ratings

The Internal File Format of the Workshop TempArchiveXX Files

By cmicroc

Since more and more maps and mutators are turning up that are only available via the Steam Workop it is becomming more and more frustrating trying to work out what files they require so that the complete set can be successfuly transfered onto a Dedicated Server.

This guide is the result of me having to reverse engineer the internal structure of the TempArchiveXX files that show up in root of the KillingFloor directory when you subscribe to a Workshop item, and writting a utility that will extract the individual files from them.

To all those authors who who publish a link to an external download of you work for use with a dedicated server - a big thank you!
To everyone else - you drove me to this :)

Award

Created by

cmicroc
Offline

Category: Modding or Configuration, Workshop

Languages: English

Posted

Updated

26 Jul, 2014 @ 12:16pm

29 Jul @ 2:02am

594	Unique Visitors
16	Current Favorites

Guide Index

Overview

Basic structure of the file

Sample C# source code for extraction program

Comments

Basic structure of the file

A header containing the number of files

# Bytes Type Meaning 4 int32 Number of files in this archive

Then one block per file:

# Bytes Type Meaning (variable) FString Relative path of File (1-5) FCompactIndex Byte length of File (count) byte File contents

An FString is a null terminated ANSI string stored as follows:

# Bytes Type Meaning (1-5) FCompactIndex Byte length of the string (including null terminator byte) (count) byte String characters each byte appears to be an ANSI code point.

An FCompactIndex is an int32 that is stored in a bit stuffed format.
I found a reference of how to decode an FCompactIndex here:
Packages (snipers paradise - via WayBack Machine)[web.archive.org]
and here:
Package File Format (beyondunreal wiki - via WayBack Machine)[web.archive.org]

Byte 0 1 2 3 4 Range Bit 76543210 76543210 76543210 76543210 76543210 6 bit s0xxxxxx 13 bit s1xxxxxx 0xxxxxxx 20 bit s1xxxxxx 1xxxxxxx 0xxxxxxx 27 bit s1xxxxxx 1xxxxxxx 1xxxxxxx 0xxxxxxx 32 bit s1xxxxxx 1xxxxxxx 1xxxxxxx 1xxxxxxx 000xxxxx bits 5 - 0 12 - 6 19 - 13 26 - 20 31 - 27

Sample C# source code for extraction program

In Visual Studio create a Solution for a Windows Console Application.
Add the following two classes to the Project:

Program.cs

using System; using System.IO; using System.Text; namespace TempArchiveExtractor { class Program { static void Main(String[] args) { if (0 == args.Length || String.IsNullOrWhiteSpace(args[0])) { Console.Out.WriteLine("Usage: TempArchiveExtractor [TempArchiveFile]"); return; } FileInfo archiveFile = new FileInfo(args[0]); if (!archiveFile.Exists) { Console.Out.WriteLine("No such file"); return; } FileStream archiveStream = null; try { archiveStream = archiveFile.OpenRead(); TempArchiveReader reader = new TempArchiveReader(archiveStream); Int32 fileCount = reader.ReadInt32(); Console.Out.WriteLine("Archive contains " + fileCount + " files"); for (int i=0; i< fileCount; i++) { String filePath = reader.ReadFString(); Int32 fileLen = reader.ReadFCompactIndex(); FileInfo fi = new FileInfo(filePath); String msg = String.Format("{0} {1} Size: 0x{2:X} {2:N0} [Y/N] : " , fi.Exists ? "Overwrite" : "Extract " , filePath, fileLen); Console.Out.Write(msg); String resp = Console.In.ReadLine(); if ("Y".Equals(resp.ToUpper())) { DirectoryInfo di = fi.Directory; if (!di.Exists) di.Create(); using (FileStream fs = fi.OpenWrite()) { reader.ReadIntoStream(fileLen, fs); } } else { reader.Skip(fileLen); } } } catch (Exception ex) { Console.Out.WriteLine("Error reading archive"); Console.Out.WriteLine(ex); } finally { if (null != archiveStream) archiveStream.Close(); } } } }

TempArchiveReader.cs

using System; using System.IO; using System.Text; namespace TempArchiveExtractor { public class TempArchiveReader { static Encoding ANSIEncoding = Encoding.GetEncoding(1252); public TempArchiveReader(FileStream archiveStream) { _archiveStream = archiveStream; } FileStream _archiveStream; public Int32 ReadInt32() { Int32 result = _archiveStream.ReadByte(); result += _archiveStream.ReadByte() << 8; result += _archiveStream.ReadByte() << 16; result += _archiveStream.ReadByte() << 32; return result; } public Int32 ReadFCompactIndex() { Int32 result = 0; int B0 = _archiveStream.ReadByte(); //String msg = String.Format("FCompactIndex: 0x{0:X}", B0); if ((B0 & 0x40) != 0) { int B1 = _archiveStream.ReadByte(); //msg += String.Format("{0:X}", B1); if ((B1 & 0x80) != 0) { int B2 = _archiveStream.ReadByte(); //msg += String.Format("{0:X}", B2); if ((B2 & 0x80) != 0) { int B3 = _archiveStream.ReadByte(); //msg += String.Format("{0:X}", B3); if ((B3 & 0x80) != 0) { int B4 = _archiveStream.ReadByte(); //msg += String.Format("{0:X}", B4); result = B4; } result = (result << 7) + (B3 & 0x7F); } result = (result << 7) + (B2 & 0x7F); } result = (result << 7) + (B1 & 0x7F); } result = (result << 6) + (B0 & 0x3F); if ((B0 & 0x80) != 0) result *= -1; //msg += String.Format(" => 0x{0:X} {0}", result); //Console.Out.WriteLine(msg); return result; } public String ReadFString() { Int32 strLen = ReadFCompactIndex(); Byte[] strBytes = new Byte[strLen]; _archiveStream.Read(strBytes, 0, strLen); //String msg = String.Format("FString: length 0x{0:X} {0} : {1:X}..{2:X}" // , strLen, strBytes[0], strBytes[strLen-1]); //Console.Out.WriteLine(msg); String str = ANSIEncoding.GetString(strBytes, 0, strLen-1); return str; } public void Skip(Int32 count) { _archiveStream.Seek(count, SeekOrigin.Current); } const int BLOCKSIZE = 0x10000; public void ReadIntoStream(Int32 count, Stream outStream) { Byte[] buffer = new Byte[BLOCKSIZE]; while (count > BLOCKSIZE) { _archiveStream.Read(buffer, 0, BLOCKSIZE); outStream.Write(buffer, 0, BLOCKSIZE); count -= BLOCKSIZE; } _archiveStream.Read(buffer, 0, count); outStream.Write(buffer, 0, count); } } }

28 Comments

< >

cmicroc [author] 29 Jul @ 2:06am

@Kytö thanks for the heads up :)
I have replaced both (expired) links with references to the captured pages on the WayBack Machine... thankfully the Internet Archive managed to save this stuff!

Kytö 28 Jul @ 6:53pm

Might wanna fix that first link. It goes to some adult site.

Web archive seems to have a capture of the original article from 2014.

madman 13 Apr @ 9:13am

nvm i got it to work after running it through cmd like you said. unfortunately a lot of workshop stuff still seems to be broken to begin with, not corrupted on extraction.

cmicroc [author] 13 Apr @ 2:06am

@madman I had forgotten that @>M@tEo$< had modified the original code quite so much.

But you still didn't answer the question:
How are you trying to run it?
And more importantly what EXACTLY happens when you do run it?

madman 12 Apr @ 1:35pm

the github says it can be run directly, or by dragging the archive on it.

cmicroc [author] 12 Apr @ 11:38am

@madman How are you trying to run it?

For it to do anything "useful" for you you have to run it from the command line with the name of the TempArchiveXX file you want to extract files from.
NB: It will extract the files into the current working directory.

If you just double click the .exe you wont achieve anything...

madman 12 Apr @ 11:24am

the .exe provided on github does nothing, even when run as admin.

peterpumpkineater69 12 Apr @ 10:20am

holy fuck my nigga the steam workshop is really this chopped

Mythical 8 Sep, 2021 @ 2:57pm

Great info and software thanks!

>M@tEo$< 12 Nov, 2020 @ 11:32am

Good evening,

I guess better late than never, I've fixed that crash and created a new tag with the associated EXE file...

See: https://github.com/Mateos81/KFTempArchiveExtractor/releases/tag/1.2

< >