Subspace: my idea for FTL modding

Discuss and distribute tools and methods for modding. Moderator - Grognak
rous
Posts: 3
Joined: Mon Sep 07, 2020 2:47 am

Subspace: my idea for FTL modding

Postby rous » Sat May 07, 2022 6:19 am

Hello everyone, I've been thinking about ways to do more interesting modding for FTL, and I've prototyped an idea that seems promising. I'm not really up to date on the current state of modding, so I'm not sure if this has been already accomplished. Sorry for the upcoming textwall, you can skip most of it.
Disclaimer: Everything here is probably never going to work, and is merely a product of my lunatic ramblings. Also, it's not really in a mature (or clean) enough state for me to share any code or executables. Finally, much of this wasn't thought through, so I'd be happy to hear if you find some obvious easier way to do something, a problem I missed, etc.

My project, which I'm calling Subspace (as a play off Hyperspace, though I bet the name's probably already taken) uses a patch tool to allow for programs, directly written in C++ and automatically patched in, to run alongside FTL, which I understand is also how Hyperspace works. This description is... more than a little technical, so feel free to skip to TL;DR at the end. That being said, I'll try to explain what I got working, so when I inevitably disappear off the face of this forum with this knowledge it'll be somewhere.

The core of this project is a program called e9patch, which is a "powerful static binary rewriter". It's not ostensibly an amazing fit for this task, and was probably intended for more mundane instrumentation. However:
  • When all you have is a hammer, everything starts to look like a nail. I haven't looked at any other patch tools.
  • It can scale well enough to reliably work on binaries the size of FTL, and should be able to work in more general cases.
  • I didn't want to learn how to parse ELF files to write my own program for this, and e9patch handles a lot of things for me.
  • It's not too slow or memory intensive to really matter.
  • It's a really cool piece of software and I think it deserves to be more well known.
e9patch, along with its frontend e9tool, allows one to write code in C or C++ and then introduce hooks that call a function defined in that code whenever specific instructions are executed in some binary; you can specify those instructions as, say, the first instruction in a function with a particular name. To make this project actually practical, I'm using the Linux release, which (for some reason) has debug symbols in the binary - I only have experience on Linux anyway. I have no idea whether this would be easier, harder or impossible on Windows. A few problems arise immediately:
  • How can my code work with classes from the original binary if we don't have the source?
  • How can my code work with methods and functions from the original binary, similarly?
  • How can my code interface with the standard library?

The first point is solved by making a new header file (call it "ftl.h") with all the classes in the game binary; since there are debug symbols, we can get this information with e.g. pahole. This is obviously a lot of work, but for testing can be done partially, with a few classes at a time. In order to be able to pass our "clone" classes into methods in the original binary, they must be byte-by-byte binary identical to the original ones; this is, surprisingly, not a major issue. It just tends to work. Particularly, members of C++ classes are always located in memory in the same order they are declared, and g++ being used for both FTL's compilation (you can actually see that in the binary) and my compilation makes other rules like padding usually work.

The second point is a bit trickier. Obviously, the methods we want are located somewhere in the FTL binary, and we can even find out where (debug symbols!) - almost. ASLR causes there to be a random offset to all of these functions' addresses. No problem - we can install a hook somewhere known (like main), examine the stack to find the address we're looking for and use that as a base offset. As for actually calling these functions, we need it to be that when we call a method of one of our clone classes, it actually runs code from the original binary (we could also make clone method implementations, but that would be a lot of work and reverse engineering) - without being able to do anything cool like link against it. I solved this problem by having each method be a trampoline that computes the address of the function by adding an offset to the base offset from the ASLR hook and jumps there directly. Since it doesn't clobber any registers except %rax, it passes through the arguments unaffected. gcc's extended asm was MVP here.

The third point is where it really gets hairy. First off, none of this standard library stuff is officially supported by e9patch as of now (in particular, it doesn't like relocations, whatever those are), so it's anyone's guess when something is going to give because of that, but as far as I'm concerned it hasn't yet with anything I've done. We can try including a standard library header like <string> and tweaking some compiler flags until it works, but we run into an issue; FTL's old string implementation is brutally incompatible with modern GCC string implementations - for example, sizeof(string) is 8 in FTL and 32 in a modern program compiled with G++ 9. The solution is unsatisfying: just use G++ 4.8.4, which is what was used to compile FTL. We lose a lot of modern G++ features or conveniences (and probably optimizations) by using an 8 year old compiler, but it guarantees maximum compatibility. We also have to statically link libstdc++ with our patch binary (if it tries to put important functions in the PLT, that PLT won't be there when the code is running inside of FTL). It seemed determined to leave a couple functions (memcpy, memmove, strlen, operator new, operator delete, among others) in the PLT, so I added trampolines like above to call the actual versions inside FTL.

Finally, this can all be automated with ease. A python script that I quickly wrote up reads in a config file and a C++ source file for your mod, compiles it, links it and patches it in. The config files, roughly speaking, contain two things: a list of new methods that you declare for classes in FTL and what classes they are part of, and what hooks in your code you want to call for what functions in the base game. The script can also handle some symbol mangling issues.

Here's an idea of what you might be able to do with this. Let's say you want to add an '!' to all crew names as they're added, for some inscrutable reason. Then just declare this function

Code: Select all

void CrewMemberFactory::hook(CrewBlueprint *crew, int room, bool intruder) { // the function signature of CrewMemberFactory::CreateCrewmember
   crew->crewNameLong.data.push_back('!'); // a function from the standard library
}

and then set up your config file to hook this into CrewMemberFactory::CreateCrewmember, and voila! Not only could this work, it actually does work - just like that, no extra code on the modder's part. The tricky bit is actually figuring out those method names and class names, which will ultimately come down to good documentation on how FTL works internally, something that I would imagine the Hyperspace team already has some information on?



TL;DR

Subspace does the following - at least in theory:
  • Lets you write code in C++, almost as if you had the FTL source code with you.
  • Lets you use that code as an FTL mod.
  • Possibly could let you perform relatively complex modding tasks without extensive low-level knowledge
What's the catch?
  • Most of these things have only barely been demonstrated, and the project is overall extremely immature. Also, it would need huge header files to be created. I'm not sure if anyone would want to put the time into that, especially if projects like Hyperspace have already done the same thing.
  • It only runs on Linux, and will do so for the foreseeable future because of these three reasons: (1) debug symbols (2) e9patch has shaky Windows support (3) I know very little about Windows calling conventions and executable formats, etc..
  • There might be glaring flaws that are yet to be discovered.
  • It's possible that it could become an unstable mess, and crash too often to be practical.
Would you use Subspace if it was actually finished, or at least progressed further? Is this just what Hyperspace has done but less fleshed out? Have I missed an obvious pitfall that will come back to bite me later? Do you want to condemn my reckless decisions to throw stability by the wayside? I'm interested to see what everyone thinks.

Who is online

Users browsing this forum: No registered users and 26 guests