Big Changes in FTL 1.6.1+

[fdagpigj]

Hello again, I ported the extract-pkg and build-pkg SIL tools to Python (3.5+). They're extremely megacrummy (everything's hardcoded and such, lacks comments) for the time being but they seem to do the job just fine. I probably spent at least 6 hours wondering why FTL wasn't accepting the repacked ftl.dat while I was working on builder.py, only to figure out I needed the

Code: Select all

hash_ &= 0xFFFFFFFF

line inside the loop in pkg_hash

Here it is:
https://github.com/fdagpigj/newftldat

Next step of course is porting it to Java, the reason I went python first was because I'm way more experienced and comfortable with python than I am with Java, and even if this doesn't make it easier for anyone else to port it to Java, it does for me. I'll try to add comments tomorrow because atm it's hardly more readable than the C code

[Vhati]

I finished writing the Java class to safely create/read/grow/add/remove/repack(defrag)/list the new "ftl.dat" yesterday.

Currently working to incorporate it into SMM...
(implications of having a single "ftl.dat" beside the game's executable vs "data.dat" & "resource.dat" in a subdir)


Edit: Here's my code for anyone curious. PkgPack.java

Like FTLPack, it edits resources in place without an intermediate extract/rebuild step. They both are subclasses of AbstractPack, as is FolderPack - which has the same public methods except backed by the filesystem instead of a dat file. Call pkgPack.getInputStream(innerPath), pass that to folderPack.add(innerPath, inputStream), and the file is extracted.

[Vhati]

I asked Matthew where "ftl.dat" is on 1.6.1.

Reminder, here are the 1.5.13 locations.

Windows
Steam: {%ProgramFiles% or %ProgramFiles(x86)%}/Steam/steamapps/common/FTL Faster Than Light/
GOG: {%ProgramFiles% or %ProgramFiles(x86)%}/GOG.com/Faster Than Light/
HumbleBundle: {%ProgramFiles% or %ProgramFiles(x86)%}/FTL/

Dat: {One of above}/resources/*.dat
Exe: {One of above}/FTLGame.exe

Linux
Steam: {$XDG_DATA_HOME or ~/.local/share}/Steam/steamapps/common/FTL Faster Than Light/
Steam: ~/.steam/steam/steamapps/common/FTL Faster Than Light/
HumbleBundle: {Extract tar.gz somewhere}/FTL/

Dat: {One of above}/data/resources/*.dat
Exe: {One of above}/data/FTL
Executable's a script that, in turn, launches x86 or amd64 binaries.

Or
{$WINEPREFIX or ~/.wine}/{Windows GOG or HumbleBundle path above}/

OSX
Steam: ~/Library/Application Support/Steam/steamapps/common/FTL Faster Than Light/FTL.app/
Standalone: /Applications/FTL.app/

Dat: {One of above}/Contents/Resources/*.dat
Exe: {One of above}
Executable's a bundle directory. (Launch with: "open -a FTL.app")


* And "steamapps" vs "SteamApps" PascalCase variants.


Launching via Steam

Windows: {%ProgramFiles% or %ProgramFiles(x86)%}/Steam/Steam.exe
Linux: /usr/bin/steam
OSX: /Applications/Steam.app ("open -a Steam.app")

{Exe above} -applaunch 212680

Windows: It's now in the same directory as the executable

OSX: It's in the same place as 1.5.13 (FTL.app/ Contents/Resources/ftl.dat)

Linux: It's one directory higher (no more Resources folder), FTL Faster Than Light/data/ftl.dat

[Vhati]

@fdagpigj
I'd already written my parser before seeing yours, but this code comment was interesting.

I heard someone else doesn't even get this png file out of extracting but I get it so idk what to believe anymore.
Either way it exists at the same level as the 4 "topdir" folders.

"exe_icon.png"

FTL 1.5.13 Linux HumbleBundle has "exe_icon.bmp", but beside the dats, not inside them.
And another copy beside the executable script, presumably so folks could create a pretty desktop shortcut.

FTL 1.5.13 Windows HumbleBundle has "exe_icon.bmp" beside the executable. Not beside or inside dats.
Redundant, since Windows binaries embed their own icons.

FTL 1.5.13 OSX HumbleBundle has "FTL.icns" in "FTL.app/Contents/Resources/" beside the dats. Not in them.
That was referenced in "FTL.app/Contents/Info.plist" to set the bundle's icon.

My FTL 1.6.1 Windows Steam (Beta BuildID 2321914, circa 2017-11-29) has no "exe_icon.*" image at all.
Not as a file or inside "ftl.dat".

[Vhati]

I've confirmed...
FTL 1.01-1.5.13 decoded text as windows-1252 (despite some files claiming to be UTF-8).
FTL 1.6.1 really does decode text as UTF-8.

This may open up translation possibilities, fonts being the other limitation.
As usual, Slipstream will ultimately convert modded files to whichever encoding FTL wants (deciding based on data.dat+resource.dat vs ftl.dat).

[Vhati]

FTL 1.01-1.5.13 relied on native srand()/rand() calls. This tied saved games to the OS they were created on (Windows/OSX/Linux), in order to interpret seeds corretly.

Matthew has informed me that, as of FTL 1.6.1, a standard algorithm is hard-coded into the game "to better support cross-platform save sharing".

I have already reversed the new RNG for eventual use in the saved game editor. In the process, I also discovered the FTL beta build includes debugging symbols. So CheatEngine reveals function names now!

[Vhati]

Thread: Arsenal+ English Translation

Font problems.
FTL 1.6 changed the font files from .ttf to .font. currently someone is working on this, [ Font dissection tools for FTL 1.6 by D-side (written in Ruby). ]
[...]
Image problems.
FTL 1.6 does not like a number of images that are perfectly fine in FTL 1.5 for some unknown reason. The FTL.log file has shown which images these are for the most part
[...]
Audio problems.
FTL 1.6 apparently does not like .mp3 files anymore. [...] In any case, this can be solved by converting the .mp3 files to .ogg or .wav, which the game likes.
[...]
Images that are no longer used.
for localization reasons, it seems FTL 1.6 removed the text from images that had them and has made those texts editable in text_misc.xml. this is good, but it also means that [mod images] need to remove the text on them.

Confirmed: mp3 is not supported in FTL 1.6.

Source: SIL's sound/decode.c

Code: Select all

static struct {
    const SoundFormat format;
    SoundDecodeOpenFunc *open;
} decode_handlers[] = {
    {SOUND_FORMAT_WAV, decode_wav_open},
    {SOUND_FORMAT_MP3, NULL},  // No cross-platform MP3 decoder available.
#ifdef SIL_SOUND_INCLUDE_OGG
    {SOUND_FORMAT_OGG, decode_ogg_open},
#else
    {SOUND_FORMAT_OGG, NULL},
#endif
};

[Vhati]

I ran the file command to compare a bad image vs a good vanilla image...

Code: Select all

$ file "Arsenal_1.25.3_rus/shotgun_strip8_plus.png"
PNG image data, 296 x 51, 8-bit/color RGBA, interlaced

$ file "FTL_Unpacked_1.6.1/img/weapons/shotgun_strip8.png"
PNG image data, 296 x 51, 8-bit/color RGBA, non-interlaced

Confirmed: interlaced PNGs are not supported in FTL 1.6.
Source: SIL's "utility/png.c"

Code: Select all

png_read_info(png, info);
const unsigned int width      = png_get_image_width(png, info);
const unsigned int height     = png_get_image_height(png, info);
const unsigned int bit_depth  = png_get_bit_depth(png, info);
const unsigned int color_type = png_get_color_type(png, info);
if (UNLIKELY(png_get_interlace_type(png, info) != PNG_INTERLACE_NONE)) {
	DLOG("Interlaced images not supported");
	goto error;
}

[Vhati]

FTL 1.6 stopped supporting "%req" crew name substitution in event text (as seen in "nameEvents.xml").

It was a latent feature in FTL 1.5.13 that some modders had discovered and used.


Article: The Cutting Room Floor Wiki - FTL Test Events

The entire file 'nameEvents.xml', containing six unique events, is unused, and appears to be an experiment with using the names of crew members in events. The specific syntax used in the file seems to differ in many places from the syntax used in other events, implying that this file is relatively old. More than one of these events refer to the player ship as "the Kestrel" by name, implying that these events were written before the other playable ships were added to the game.

[Vhati]

I generated an exhaustive list of tags that vanilla FTL is resolving "id" attributes for.

Note: It's possible mod developers may discover that FTL looks up other tags, too, by putting the "id" attribute in unprecedented places.


Some tags were grandfathered in with an "id", which does NOT get looked up.

Code: Select all

FTL/achievement
FTL/[...]/event/unlockShip
FTL/ship/deadCrew/unlockShip
FTL/ship/destroyed/unlockShip

This is the rest, all the tags with an "id" attribute, which WILL be looked up. (scroll)

Code: Select all

FTL/achievement/desc
FTL/achievement/name
FTL/achievement/shortname
FTL/augBlueprint/desc
FTL/augBlueprint/title
FTL/crewBlueprint/desc
FTL/crewBlueprint/powerList/power
FTL/crewBlueprint/short
FTL/crewBlueprint/title
FTL/droneBlueprint/desc
FTL/droneBlueprint/short
FTL/droneBlueprint/title
FTL/[...]/event/choice/text
FTL/[...]/event/text
FTL/[...]/event/crewMember
FTL/[...]/event/removeCrew/text
FTL/itemBlueprint/desc
FTL/itemBlueprint/title
FTL/sectorDescription/nameList/name
FTL/ship/deadCrew/text
FTL/ship/destroyed/text
FTL/ship/escape/text
FTL/ship/gotaway/text
FTL/ship/surrender/text
FTL/shipBlueprint/class
FTL/shipBlueprint/desc
FTL/shipBlueprint/name
FTL/shipBlueprint/unlock
FTL/step/default
FTL/step/IPAD
FTL/step/PC
FTL/systemBlueprint/desc
FTL/systemBlueprint/title
FTL/textList/text
FTL/weaponBlueprint/desc
FTL/weaponBlueprint/flavorType
FTL/weaponBlueprint/short
FTL/weaponBlueprint/title
FTL/weaponBlueprint/tooltip

The "crewMember" tag is interesting for being more than just a pointer to text.

Code: Select all

<crewMember amount="1" all_skills="1" id="name_Charlie"/>

Background

I ran the following XMLStarlet command:

Code: Select all

xml sel -t -m "//*[@id]" -m "./ancestor::*" -v "local-name(.)" -o "/" -b -v "local-name(.)" -n "*.xml"

That yields every tag with an "id" attribute, prepending its ancestry.

Add some pipes to filter out the repeated lines...

On Linux/OSX (Or windows if you've downloaded a port of the "uniq" command):

Code: Select all

... | sort | uniq

In PowerShell:

Code: Select all

... | Sort-Object | Get-Unique

Edit: Achievement's "id" is a string. unlockShip's "id" is an integer.