Steam Community :: Guide :: The Internal File Format of the Workshop TempArchiveXX Files

Store Page

Killing Floor

All Discussions Screenshots Artwork Broadcasts Videos Workshop News Guides Reviews

Killing Floor > Guides > cmicroc's Guides

Not enough ratings

The Internal File Format of the Workshop TempArchiveXX Files

By cmicroc

Since more and more maps and mutators are turning up that are only available via the Steam Workop it is becomming more and more frustrating trying to work out what files they require so that the complete set can be successfuly transfered onto a Dedicated Server.

This guide is the result of me having to reverse engineer the internal structure of the TempArchiveXX files that show up in root of the KillingFloor directory when you subscribe to a Workshop item, and writting a utility that will extract the individual files from them.

To all those authors who who publish a link to an external download of you work for use with a dedicated server - a big thank you!
To everyone else - you drove me to this :)

Award

Created by

cmicroc
Offline

Category: Modding or Configuration, Workshop

Languages: English

Posted

Updated

26 Jul, 2014 @ 12:16pm

16 Nov, 2016 @ 11:12am

592	Unique Visitors
16	Current Favorites

Guide Index

Overview

Basic structure of the file

Sample C# source code for extraction program

Comments

Basic structure of the file

A header containing the number of files

# Bytes Type Meaning 4 int32 Number of files in this archive

Then one block per file:

# Bytes Type Meaning (variable) FString Relative path of File (1-5) FCompactIndex Byte length of File (count) byte File contents

An FString is a null terminated ANSI string stored as follows:

# Bytes Type Meaning (1-5) FCompactIndex Byte length of the string (including null terminator byte) (count) byte String characters each byte appears to be an ANSI code point.

An FCompactIndex is an int32 that is stored in a bit stuffed format.
I found a reference of how to decode an FCompactIndex here:
Packages (snipers paradise)[www.snipersparadise.net]
and here:
Package File Format (beyondunreal wiki)[wiki.beyondunreal.com]

Byte 0 1 2 3 4 Range Bit 76543210 76543210 76543210 76543210 76543210 6 bit s0xxxxxx 13 bit s1xxxxxx 0xxxxxxx 20 bit s1xxxxxx 1xxxxxxx 0xxxxxxx 27 bit s1xxxxxx 1xxxxxxx 1xxxxxxx 0xxxxxxx 32 bit s1xxxxxx 1xxxxxxx 1xxxxxxx 1xxxxxxx 000xxxxx bits 5 - 0 12 - 6 19 - 13 26 - 20 31 - 27

Sample C# source code for extraction program

In Visual Studio create a Solution for a Windows Console Application.
Add the following two classes to the Project:

Program.cs

using System; using System.IO; using System.Text; namespace TempArchiveExtractor { class Program { static void Main(String[] args) { if (0 == args.Length || String.IsNullOrWhiteSpace(args[0])) { Console.Out.WriteLine("Usage: TempArchiveExtractor [TempArchiveFile]"); return; } FileInfo archiveFile = new FileInfo(args[0]); if (!archiveFile.Exists) { Console.Out.WriteLine("No such file"); return; } FileStream archiveStream = null; try { archiveStream = archiveFile.OpenRead(); TempArchiveReader reader = new TempArchiveReader(archiveStream); Int32 fileCount = reader.ReadInt32(); Console.Out.WriteLine("Archive contains " + fileCount + " files"); for (int i=0; i< fileCount; i++) { String filePath = reader.ReadFString(); Int32 fileLen = reader.ReadFCompactIndex(); FileInfo fi = new FileInfo(filePath); String msg = String.Format("{0} {1} Size: 0x{2:X} {2:N0} [Y/N] : " , fi.Exists ? "Overwrite" : "Extract " , filePath, fileLen); Console.Out.Write(msg); String resp = Console.In.ReadLine(); if ("Y".Equals(resp.ToUpper())) { DirectoryInfo di = fi.Directory; if (!di.Exists) di.Create(); using (FileStream fs = fi.OpenWrite()) { reader.ReadIntoStream(fileLen, fs); } } else { reader.Skip(fileLen); } } } catch (Exception ex) { Console.Out.WriteLine("Error reading archive"); Console.Out.WriteLine(ex); } finally { if (null != archiveStream) archiveStream.Close(); } } } }

TempArchiveReader.cs

using System; using System.IO; using System.Text; namespace TempArchiveExtractor { public class TempArchiveReader { static Encoding ANSIEncoding = Encoding.GetEncoding(1252); public TempArchiveReader(FileStream archiveStream) { _archiveStream = archiveStream; } FileStream _archiveStream; public Int32 ReadInt32() { Int32 result = _archiveStream.ReadByte(); result += _archiveStream.ReadByte() << 8; result += _archiveStream.ReadByte() << 16; result += _archiveStream.ReadByte() << 32; return result; } public Int32 ReadFCompactIndex() { Int32 result = 0; int B0 = _archiveStream.ReadByte(); //String msg = String.Format("FCompactIndex: 0x{0:X}", B0); if ((B0 & 0x40) != 0) { int B1 = _archiveStream.ReadByte(); //msg += String.Format("{0:X}", B1); if ((B1 & 0x80) != 0) { int B2 = _archiveStream.ReadByte(); //msg += String.Format("{0:X}", B2); if ((B2 & 0x80) != 0) { int B3 = _archiveStream.ReadByte(); //msg += String.Format("{0:X}", B3); if ((B3 & 0x80) != 0) { int B4 = _archiveStream.ReadByte(); //msg += String.Format("{0:X}", B4); result = B4; } result = (result << 7) + (B3 & 0x7F); } result = (result << 7) + (B2 & 0x7F); } result = (result << 7) + (B1 & 0x7F); } result = (result << 6) + (B0 & 0x3F); if ((B0 & 0x80) != 0) result *= -1; //msg += String.Format(" => 0x{0:X} {0}", result); //Console.Out.WriteLine(msg); return result; } public String ReadFString() { Int32 strLen = ReadFCompactIndex(); Byte[] strBytes = new Byte[strLen]; _archiveStream.Read(strBytes, 0, strLen); //String msg = String.Format("FString: length 0x{0:X} {0} : {1:X}..{2:X}" // , strLen, strBytes[0], strBytes[strLen-1]); //Console.Out.WriteLine(msg); String str = ANSIEncoding.GetString(strBytes, 0, strLen-1); return str; } public void Skip(Int32 count) { _archiveStream.Seek(count, SeekOrigin.Current); } const int BLOCKSIZE = 0x10000; public void ReadIntoStream(Int32 count, Stream outStream) { Byte[] buffer = new Byte[BLOCKSIZE]; while (count > BLOCKSIZE) { _archiveStream.Read(buffer, 0, BLOCKSIZE); outStream.Write(buffer, 0, BLOCKSIZE); count -= BLOCKSIZE; } _archiveStream.Read(buffer, 0, count); outStream.Write(buffer, 0, count); } } }

26 Comments

< >

JkMenttonet 24 Jan, 2015 @ 2:54pm

no i mean cmicroc everytime i start up killing floor it wont download the subscribed items is there a part in this guide will download stuff again?

cmicroc [author] 24 Jan, 2015 @ 2:53pm

Do you mean how do you get the files onto a dedicated server?

If so then you use the above information to extract the individual files from the archive. (I have some C# source code for an application to do that, if anyone is interested I am willing to post it here.) The archive contains the original file names and folder locations. Then you upload them to the appropriate folder on the dedicated server.

Or did I misinterpret the question?

JkMenttonet 24 Jan, 2015 @ 1:16pm

how do you get the downloads for killing floor back up?

BadKarMa 5 Dec, 2014 @ 4:13pm

I don't need the sourcecode right now, but maybe some other time.

Many thanks for the explanation, though! I've quoted you and linked your guide in mine ( just published ). Send a friendrequest, if you want to be added as a contributor!

cmicroc [author] 5 Dec, 2014 @ 2:11pm

Those are basically the same files, when you subscribe to the item steam first downloads it to that ugc folder, and the name of the archive will match one of the files included in it.

Then when you run Killing Floor it will move the file from ugc to the root folder of your KF install and call it TempArchiveXX, where xx is some random number.

Then theoretically KF unpacks the archive and moves the files into the correct sub-directories.
I am not sure why it seems to have quite so much trouble...

If you are interested I have some C# source code for an app that will extract the files from one of these archives. I used to to copy some Maps and mutators onto a dedicated server.

BadKarMa 5 Dec, 2014 @ 12:25pm

Interesting attempt here.
Is Steam still creating these Temparchives? I'm experiencing some trouble with the workshop, and while I was browsing the steam directory, I've recently found the folder C:\Program Files (x86)\Steam\userdata\[[i]yourSteamNumber[/i]]\ugc\referenced\[[i]somenumbers[/i]] where you can find animations and other things from your workshop subscriptions.

< >