[Tool] ftldat r7

[bas]

Hi,

I created a python script to pack and unpack the .dat files of FTL.

You can find it here:

https://github.com/bwesterb/ftldat

Kind regards,

UPDATE I have created a stand-alone Windows installer.

• Run the installer found here http://westerbaan.name/~bas/ftldat/ftldat-latest.exe. (You do not need to install Python anymore.)
• Open up a command prompt (Start -> Run... -> "cmd" -> "OK")
• Navigate to the folder with the data.dat and resource.dat.
• To unpack, run ftldat unpack data.dat
  This wil create a data.dat-unpacked folder.
• WARNING, before packing, make a backup of data.dat. To pack, run ftldat pack data.dat.
• To show the contents and fingerprints of a .dat, run ftldat info --hashes data.dat.

See ftldat -h, ftldat pack -h, etc. for more help.

UPDATE (r6) I have uploaded a new version with some useful new commands.

• add Adds a single file to the datfile. Eg. ftldat add data.dat my-ship.xml data/my_ship.xml
• append Appends a file to an existing file in the datfile. Eg. ftldat append data.dat data/blueprints.xml my-extra-blueprints.txt
• replace Replace a single file in a datfile. Eg. ftldat replace data.dat data/blueprints.xml fully-new-blueprints.xml
• extract Extract a single file to the datfile. Eg. ftldat extract data.dat data/blueprints.xml out.xml
• remove Remove a single file. Eg. ftldat remove data.dat data/events_mantis.xml
• hashes Prints a alphabetic list of filename and md5 hash. Useful for integrity checking. ftldat hashes data.dat
• list Lists the files in the datfile. ftldat list data.dat

These commands are all in-place and thus as fast as they can be. One could use ftldat.exe with these commands to create an installer of a patch. It would be as simple as writing a .bat.

If anyone is interested, I could also add support for patches like XDelta or Uniform Diff. If you script in Python, these commands are fairly easy to use by importing the "FTLPack" class.

UPDATE (r7) Another new command: repack. It will efficiently repack a datfile. Usage: ftldat repack path/to/the/file.dat. This is useful to reduce space overhead after a few add, replace, remove, etc commands.

UPDATE Changed subject.

[Andytizer]

Amazing!

I'm interested in being able to use some of the assets from the game for FTLWiki. Do you recommend an easy way to use this script for a noob running Windows?

[bas]

• Install a version of Python 2 ( http://www.python.org )
• Grab a copy of ftldat ( on https://github.com/bwesterb/ftldat there is a "zip" download button )
• Open up a command prompt
• Navigate to the 'src' folder of ftldat. (

  Code: Select all

  cd c:\path\to\ftldat\src
  )
• Execute it

  Code: Select all

  c:\python27\python.exe main.py unpack c:\path\to\my\data.dat

[bas]

Note that I do not know if you are allowed to use the images of the game on your Wiki: you should ask the developers.

[marinepower]

Hi, I tried this out real quick and just wanted to give you a heads up that the game does not accept a repacked data.dat built by this program. I suspect this has something to do with the resulting file size, as the original packed file is 912 KB, while the unpacked / repacked data.dat file is 904 kb.

The error it gave me was an assertion error. Does something similar happen to you?

Thanks.

[bas]

Repacking worked with me, at least. You should always make a backup, though.

Can you run "ftldat info" on the original and the repacked one?

The difference in file size is to be expected: in the original there is are unused entries in the "index".

[marinepower]

C:\Users\*\Downloads\bwesterb-ftldat-f07b6bb\bwesterb-ftldat-f07b6bb\src>python main.py info data.dat

Code: Select all

...
 108  e3360   data/tooltips.xml                                             2 KiB


  109/2027 entries
  900 KiB


C:\Users\*\Downloads\bwesterb-ftldat-f07b6bb\bwesterb-ftldat-f07b6bb\src>python main.py info data2.dat

Code: Select all

...
108  e1568   data\tooltips.xml                                             2 KiB


  109/109 entries
  900 KiB


Seems to be the same. Like you said, the only difference is the number of entries this program detected. I didn't touch the resource.dat file, and I'm using version 1.01, if that changes anything.

[bas]

Ok, I have a new version ("r2") you can download from github.

It adds

• A --hashes parameter to the info command. It will show the MD5 fingerprint of each entry. Then you can really check if something changed.
• A --indexsize parameter to the pack command. If you want to match the index sizes of (the Mac OS X version of) LTF, you should use --indexsize 2027 for data.dat and --indexsize 1918 for resource.dat.
• A --bytes parameter to info command. This will show the sizes in bytes instead of rounding them to KiB, etc.

Hope this helps to debug the issue.

[bas]

The assert on line 1383 of http://rapidxml.sourceforge.net/rapidxml.hpp seems to prevent an empty file from being loaded. Maybe it is the order of the files in the datfile or there is a bug I did not hit.

[marinepower]

I figured it out. The filenames were being created with a backslash ('\') rather than a forward slash ('/'), so I guess that was causing problems somewhere. I fixed the error by adding

Code: Select all

    def add(self, filename, f, size):
        filename = filename.replace('\\','/')


on line 46.

Padding with the index size made no difference.

Thanks for the help.