Chris Bednarz

Terraria's Lighting to Other Devices

Introduction

It’s been two months since the final update for Terraria was released, and with it a variety of new content and features. Of those, there was one feature which I had worked on quite a bit during my time at Re-Logic which I had been patiently waiting to play with again: Peripheral Lighting. With this, Terraria is able to take control of various “RGB”-enabled devices such as keyboards, mice, and headsets, and update their colors depending on what’s going on in the game. Perhaps the whole thing is a bit useless, but I’ve always been a sucker for RGB lighting nonetheless.

Unfortunately, since working on it I no longer have any supported devices. In fact, none of my various peripherals support any sort of host controlled lighting. But why should I limit myself to peripherals anyway? The system behind it certainly isn’t aware of this constraint, so why can’t I make my own RGB device? In theory, so long as a device has controllable RGB LEDs, there’s a way.

A word of warning:
This is by no means meant to be a comprehensive guide, but rather a starting point. I’ll cover the key steps required to get things up and running, but things like polish and stability will be left for another time. Additionally, some knowledge of C#, and embedded devices is recommended.

Targeting A New Device

To start off simple, we’ll look at what it would take to get a strip of WS2812B addressable LEDs working with Terraria. These are fairly common, cheap, and readily available on Amazon. Because I’m working with what I have laying around, here are the main components I’ll be using:

Additional parts, such as a logic level converter, capacitor, power supply, etc. will be required. Adafruit has a fantastic guide on how to control WS2812B LEDs from a 3.3v microcontroller.

We’ll also need some software:

The “Chroma” System

Terraria’s lighting system is modeled very loosely around a GPU pipeline. Devices define collections of pixels called “Fragments” which are shaded, blended, and presented to some external system.

"RgbDevice Fragment Diagram"

The pipeline itself is unaware of what it’s shading beyond some fragment metadata, so we only need to concern ourselves with creating an RgbDevice for our new target.

This gives us a clear set of goals:
1. Find a way to modify Terraria. (Preferably without decompiling/recompiling)
2. Add a new RgbDevice type that’s capable of sending color data to our a microcontroller.
3. Program our microcontroller to receive and use the color data.

Preparations For Modifying Terraria

Typically it’s much easier to work within a dedicated mod loader. However, at the time of writing, none exist that currently support Terraria 1.4, so we’ll need to forge our own path. Fortunately for us, the systems we need to modify don’t actually require any code changes, only additions, which opens some possibilities. The easiest way to inject our new code into Terraria is to produce a new C# application (the language in which Terraria is written), and use Terraria’s executable as a reference. This way, we can add our code as a callback in Terraria’s engine, and then invoke Terraria’s normal entry point.

Now for the unfun part. Before we can get started writing code, we need to first prepare our workspace a bit. In an attempt to remove some clutter, Terraria embeds most of its dependencies into the main executable, including a library called ReLogic.dll that contains the lighting system. Using dotPeek, we can open Terraria’s executable (found in its install directory) and export these resources. In this case, we need to save Terraria.Libraries.ReLogic.ReLogic.dll as “ReLogic.dll” directly into Terraria’s install folder.

"Export ReLogic.dll"

With the library exported, we can move to Visual Studio. Once again, there are some unique setup steps required to ensure compatibility with Terraria.
1. Create a new “Console App (.NET Framework)” project. Be sure to select “.NET Framework 4”.
2. Once the project is loaded, open the project properties menu and navigate to the “Build” tab.
3. Set “Platform target” to “x86”‘. 4. Set “Output path” to Terraria’s install directory. (This will simplify loading Terraria’s other dependencies) 5. From the Solution Explorer panel, add both Terrara.exe and ReLogic.dll as references to your project.

Creating A Custom RgbDevice

With all of the preparation steps out of the way, we can finally look at the code involved.

As outlined above, we’ll need to create our own RgbDevice subclass in order to add the functionality we need. Let takes a look at some boilerplate code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

class MyRgbDevice : RgbDevice
{
  private static Fragment CreateFragment()
  {
      var gridPositions = new Point[150];
      var canvasPositions = new Vector2[150];

      for (var i = 0; i < 150; i++)
      {
          gridPositions[i].X = i;
          gridPositions[i].Y = 0;

          // Dividing by 50 will make our LED strip considered
          // roughly as long as a keyboard. 
          canvasPositions[i].X = i / 50.0f;
          canvasPositions[i].Y = 0;
      }

      return Fragment.FromCustom(gridPositions, canvasPositions);
  }

  public MyRgbDevice() :
      base(RgbDeviceVendor.Unknown,
           RgbDeviceType.Generic,
           CreateFragment(),
           new DeviceColorProfile())
  {
      PreferredLevelOfDetail = EffectDetailLevel.Low;
  }

  public override void Present()
  {
      for (int i = 0; i < LedCount; i++)
      {
          var color = GetProcessedLedColor(i);
          // Do something with color
      }
  }
}

There’s a few things going on here, but the one to start with is the PreferredLevelOfDetail.
Because peripherals come in all shapes and sizes, there isn’t really a “one size fits all” for most effects. While a keyboard has enough lights to show off a giant eyeball blinking at you, that effect would not translate well to a pair of headphones with two lights. As such, there are two versions of almost every effect in the game. One for high detail devices, such as keyboards, and another for low detail devices. Since we’re starting with an LED strip, it’s better to set our preferred detail level to low.

Moving up, we can see there’s a bit going on with the parent class’s constructor. In fact, RgbDevice’s constructor takes four arguments:

1
2
3
4
5

RgbDevice(
  RgbDeviceVendor vendor,
    RgbDeviceType type,
    Fragment fragment,
    DeviceColorProfile colorProfile);

vendor and type are used for categorization. These are mostly irrelevant and we can generally pick Unknown/Generic. There is, however, an exception to this rule. A few effects not only require the device prefer a high level of detail, but they must also be a keyboard. This is typically for things that are meant to be centered on your lighting setup. If you’re looking to display these without registering as a keyboard, you can set your vendor and type to Virtual.

fragment is where we define what LEDs we have and how they’re positioned. Each pixel in a fragment is given two values: A canvas position and a grid position.
Grid Position is an pair of X/Y integer values. It’s typically used for effects that require a whole LED be lit up as opposed to smoothly transitioning to the neighboring LEDs. A good example of this is the rain effect, which has streaks of blue falling down columns of keys.
Canvas Position, on the other hand, is a floating point based X/Y coordinate. Its position is normalized such that one unit is roughly equal to the height of a standard keyboard. This coordinate space is how effects are able to compensate for staggered layout of a keyboard, since keys do not fit neatly into a grid.

Finally we have colorProfile, which acts as a global color multiplier. This was added to combat specific keyboard brands having strong blue tints to their lighting. For our use case, we can simply use a default constructed one.

Down near the bottom of our code snippet, we have the Present function we’ve overridden. This is what gets invoked after all effects have been precessed.

Communicating With A Remote Device

Now that the base of our new RgbDevice is set up, we can have it send color data to our microcontroller. Since I’m using an ESP32, it seems only fitting to take advantage of the wireless functionality. If our microcontroller were to listen for UDP packets, the RgbDevice could simply stream all of the color data via the local network, and we wouldn’t need to worry about serial ports or other more complicated connections. Easy enough, C# has a UdpClient class for this exact purpose.

By adding a member variable:

1

private UdpClient _udpClient;

And a bit of connection code to our constructor:

1
2
3
4
5
6
7
8
9
10
11
12
13

public MyRgbDevice() :
  base(RgbDeviceVendor.Unknown,
          RgbDeviceType.Generic,
          CreateFragment(),
          new DeviceColorProfile())
{
  PreferredLevelOfDetail = EffectDetailLevel.Low;

  _udpClient = new UdpClient();

  // Update this to the local IP of the ESP32.
  _udpClient.Connect(IPAddress.Parse("10.0.0.173"), 8585);
}

We’re now able to connect to our ESP32 on port 8585. From there, sending to our remote device is easy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

public override void Present()
{
  byte[] output = new byte[LedCount * 3];

  for (int i = 0; i < LedCount; i++)
  {
      var color = GetProcessedLedColor(i);
      output[i * 3 + 0] = (byte)(color.X * 255.0f);
      output[i * 3 + 1] = (byte)(color.Y * 255.0f);
      output[i * 3 + 2] = (byte)(color.Z * 255.0f);
  }

  _udpClient.Send(output, output.Length);
}

A complete version of our new RgbDevice class can be found here.
It’s worth noting that you will need to add XNA as a project reference. Visual Studio should pick up on this and prompt you when hovering over items with the missing reference. Otherwise you may need to dig into the XNA dll install location and add them manually.

Hooking Up The New RgbDevice

So we’ve got our brand new RgbDevice ready, but we need to get it registered into Terraria somehow. Well, for better or worse, most of Terraria’s initialization is done through static functions with global states. All we need to do is use reflection to find the ChromaEngine object in the ChromaInitializer static class and register a new device.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

class Program
{
  private static MyRgbDevice device = new MyRgbDevice();

  private static void RegisterOurDevice()
  {
      var engine = (ChromaEngine)typeof(ChromaInitializer).GetField("_engine", BindingFlags.NonPublic | BindingFlags.Static).GetValue(null);
      engine.AddDeviceGroup("LedStrip", new VirtualRgbDeviceGroup(device));
      engine.EnableDeviceGroup("LedStrip");
  }

  public static void Main(string[] args)
  {
      // Invoke Terraria's Main for Windows-based systems.
      typeof(WindowsLaunch).GetMethod("Main", BindingFlags.Static | BindingFlags.NonPublic).Invoke(null, new object[] { args });
  }
}

A word on device groups: Device groups are used for managing multiple devices with a shared backend API. Since we’re only dealing with a single LED strip, we can use the provided virtual group, which will do nothing more than act as a container.

You may notice that we haven’t actually invoked RegisterOurDevice() anywhere. This is because doing so would cause our application to immediately crash. As mentioned earlier, Terraria embeds many of its dependencies into the main executable. In order to reference these during execution, it’s got a bit of extra code which helps with assembly resolution. Unfortunately for us, there’s no easy way to register a callback before this extra code has run. We know it’s somewhere after we invoke WindowsLaunch.Main, but every callback after that point is in a class that requires the assembly resolution code before you can reference it safely.

While we could likely decompile Terraria further and attempt to replicate this functionality in our own main, the easier way is to simply add our hook to the one point we know we’re safe to: After an embedded assembly is loaded.

1
2
3
4
5

AppDomain.CurrentDomain.AssemblyLoad += (sender, sargs) =>
{
  if (sargs.LoadedAssembly.GetName().Name == "Newtonsoft.Json")
      Terraria.Main.OnEngineLoad += RegisterOurDevice;
};

Newtonsoft.Json is a JSON library used for configuration files within Terraria, and the dll file for it is embedded within Terraria.exe. All we have to do is wait for that load to occur. Once it has, we can safely hook into a more reasonable spot in Terraria’s engine.

Again, a complete version of Program.cs can be found here.

Connecting To Terraria

With everything on the client side in place, we can move to setting up our microcontroller. It would likely be far more performant to work with the esp-idf, for the sake of keeping this short, we’ll use the Arduino IDE with the ESP32 board package and both the WiFi and FastLED packages installed.

Unlike the Terraria side, things are fairly straight-forward here. We simply need to connect to the wifi, and listen for UDP packets. Whenever we have enough data to fill all the lights, we read it all off and update our leds buffer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

#include <FastLED.h>
#include <WiFi.h>

#define PIN 21
#define NUM_LEDS 150
CRGB leds[NUM_LEDS];

char ssid[] = "<NETWORK NAME>";
char password[] = "<NETWORK PASSWORD>";

WiFiUDP udp;
int port = 8585;

void setup() {
  Serial.begin(115200);

  int status = WL_IDLE_STATUS;
  while (status != WL_CONNECTED) {
    status = WiFi.begin(ssid, password);
    delay(1000);
  }
  WiFi.setSleep(false);

  Serial.println("Connected to Wifi");

  udp.begin(port);

  FastLED.addLeds<NEOPIXEL, PIN>(leds, NUM_LEDS);
}

void loop() {
  int packetSize = udp.parsePacket();
  if (packetSize >= NUM_LEDS * 3) {
    for (int i = 0; i < NUM_LEDS; i++) {
      leds[i].r  = udp.read();
      leds[i].g  = udp.read();
      leds[i].b  = udp.read();
    }
    FastLED.show();
  }
}

Once the code has been uploaded to our ESP32 board, we simply need to run our mod, and then press reset on the ESP32.

Showcase

The only light strip I had was already attached under my bed (for the sake of not stepping on cats in the middle of the night), but I’d say it worked quite well:

Taking things a step further, you can even display to an LED panel, but perhaps that’s a post for another time.