I am working on an isometric game inspired from Gnomoria, RimWorld, Dwarf Fortress, etc. It uses my own simple engine (with rust and wgpu-rs). Whenever I started my game, my headphones were buzzing. I could play Fortnite, Overwatch or any other game and that doesn’t cause my headphones to buzz. It’s only my game.

And it’s really annoying, as you might imagine.

Why can I play Overwatch and Fortnite fine, while my isometric game makes my headset buzz? I had a fairly decent CPU, a 3090RTX card, 32GB RAM and USB audio through a MODI 2 DAC. Nothing out of this world, but nothing too bad. One important detail here is that the power to the MODI device comes from an USB port in my computer. This was the first clue, I tried other ports with no change in results (headphones still buzzed).

Initially, I started to think it’s some sort of power-use related issue, because maybe my PSU was getting old, or had daemons in it. However, I still couldn’t explain why my tiny game was causing more chaos than say big games that send significantly more work at my PC.

I noticed is that when it didn’t render anything, nothing buzzed (I run tests with rendering disabled). So that eliminated any sort of CPU work causing it. Let’s take a look at what the GPU does.

The pipeline

The game has a simple graphics pipeline. I use WebGPU (more precisely wgpu-rs) and do some compute work to select visible entities, then use draw indirect to draw those entities. In the end, my render pipeline also outputs two things: the buffer that ends up on screen and a “picking texture”.

A picking texture is a very simple idea. As the name says, it’s used to handle picking in the game, when you click somewhere on the screen (e.g. to select an unit), I use this texture to know what you clicked on. Instead of colors, every object instance writes their EntityID to this texture. Then, when you click the mouse, you check what id is in the pixel under the mouse position.

At the end of a frame, I copy that picking texture back to RAM (from GPU memory), to check it against mouse positions in case of a click.

This isn’t ideal as transfers from GPU->CPU memory take time, but it works and is way simpler to implement and debug than casting a ray through the scene:

So why does rendering make my headphones buzz?

Now that we have a picture of how the rendering in my game works, time to debug it. We know it’s something to do with the GPU work, but what can possibly cause this? As the trace above shows, my GPU is not under heavy load.

As I was stuck and had no idea on what can be a likely issue, I proceeded to then disable parts of my rendering pipeline (first the compute, then the rendering, then transferring the picking texture). When I skipped downloading the picking texture the buzzing was fully gone. What was confusing in this process is that disabling parts of the pipeline, somehow made the buzzing a lower volume and less noticeable.

To be sure it was the picking texture download, I also issued the download every 250ms and noticed the noise is almost gone. Increasing the frequency on how often we download it to RAM, increased the buzzing.

So at this point I had a likely source, but no idea why things would interfere in ways to what I assumed was the power to my MODI device. Through a bunch of discussion with other graphics engineers, someone suggested it may be due to the fact that I full on hit the GPU with tons of work, then pause the GPU to wait for that picking texture to transfer, then turn it back on 100% for the next frame. That explanation seems plausible to me.

Now that we know this, all was left is to fix it. In hindsight, the solution is obvious. There’s no need to download the whole texture each frame, just the part of the picking texture that’s under the mouse. So I implemented that and it worked and buzzing is gone. As a bonus, now it’s also not visible at all on the GPU trace.

Intro

We need to first understand how DALL-E works. More precisely how we use it, not how it actually works. Nobody really knows how machine learning works besides multiplying lots of matrices.

Today there’s no official API for DALL-E 2. You get a prompt bar where you can put the input text and you get out 4 images. You can also input an image and get variations on it. Image sizes are fixed and there’s a report button in case something goes horribly wrong. Easy!

You get some free tokens every month and generating a series of 4 images consumes one token. I’ve spent my initial ones on genrating images of cats doing things and Formula 1 cars. While images with cats are always cool and interesting, I can honestly say that I couldn’t generate interesting images of Formula 1 cars. I wanted to generate one for a friend but it’s just not possible. Please save your tokens and don’t even try. Even “Formula 1 car made from jellybeans” had dissapointing results. We can only hope DALL-E 3 will tackle the problem of interesting looking (by my standards) Formula 1 cars.

The limited free tokens are the least of my problems. You can always buy more, and the prices are relatively reasonable for the amount of fun provided. As I said before, my biggest problem is that I have zero creativity. DALL-E is successful in automating the creation of images from a given text, but this solves half of the problem – that I can’t draw. We need to go further. We also need to have a good title for our image. A good title is 90% of any creative work, just like a good variable name.

Generating titles

This means that we need to automate the generation of image titles (prompts). OpenAI, the creators of DALL-E already have our backs here, as they also have GPT3. This one is pretty good at doing things with text.

How would we go about generating descriptions of interesting images? This is easy, we don’t even need to fine-tune a GPT3 model. We need to inject some examples in the prompts for GPT3. The prompts that I used look like this:

Generate a Title for a painting
Title: A cat reading a book, watercolour painting
Title: A cat drawing a picture of another cat, pencil drawing
Title: vacuum cleaner in a museum
Title: anger as abstract art

Then we make a request to GPT3’s completion API with the prompt above. For the model I’ve used the most beefy one davinci-02. I set the temperature to 0.9 so it produces more funky stuff. I get a few responses back (set them to 5), clean them up a bit, keep the interesting ones and put them in the title pool. Over time that title pool increases so everything that follows Title: is chosen as random from the list of previous titles I liked.

Here are some images that DALL-E made from GPT3-generated titles:

the first step, watercolor painting

an old man and his cat, oil painting

A surrealist garden party

Colorful abstract art

a cat made of clouds in a garden party vaporwave

From wiskers to purrs, geometric art

my grandfather’s study

a cat’s life, acrylic painting

Conclusion

I’m like an art curator, these two collaborate and generate things for me, and I just keep the ones I like. I’m sure there are better ways to go about this and further improve the title generation, but I’ve found quite a lot of enjoyment in discovering what they create.

There are some questions people may be interested in and that make up for good Medium articles such as:
Can AIs be creative?
Do humans have any unique traits left that a machine can’t simulate?
Will we all live in pods surviving on disgusting soup?

I don’t know and I don’t care.
In the meantime, I’ll sit back and enjoy watching these “Cats in conversation”:

Debugging

Debugging is one of the rough edges of WebAssembly. To understand why this is a rough edge, we must first have a high-level understanding on how a WebAssembly VM works. We can split them into two categories: WASM VMs with JIT and WASM VMs without JIT. Right now, debugging is possible only for JIT-enabled VM e.g.: Wasmtime.

JIT is the key functionality that they use to enable a seamless debugging experience between the host program (the one that uses the VM) and the WebAssembly program. We usually use GDB or LLDB to debug the host programs. For example, to make this work, Wasmtime generates the JIT code, then it patches the debug info for the rust WebAssembly binary and calls a magical function named: __jit_debug_register_code. After this function is intercepted by GDB/LLDB and the JITed WebAssembly code can be debugged in the same session as the host program. It tells LLDB that this JIT-generated code has that patched debug information. Pretty neat!

In order for us to debug WebAssembly, there are a few key steps that we have to take:

1. Use Visual Studio Code as our IDE (makes step 2 and 3 possible).
2. Install CodeLLDB plugin for VSCode. Other GDB/LLDB plugins are fine but this good on Linux/OSX/Windows too.
3. Tell Wasmtime to emit debug information. We use the configuration object (full program can be found in part 1 of this series): let engine = Engine::new(Config::new().debug_info(true));
4. Set up Visual Studio Code launch.json as if you’d be debugging the host program. For example:

    "configurations": [
        {
            "name": "Debug WASM",
            "type": "lldb",
            "request": "launch",
            "program": "${workspaceFolder}/target/debug/wasm_host.exe",
            "cwd": "${workspaceFolder}",
            "args": [],
        }
    ]
   

Here is a screenshot of me debugging a WebAssembly program (on the right) and the host VM (on the left). As you can observe, the local variables are listed as expected and call stacks work just fine.

If you’re deploying software to a place where a JIT-only VM can’t go, you’re stuck maintaining two VMs in your host program: one you’re using for debug purposes and one that actually ships to the platform you’re interested in. I expect that with time more VMs that include debug protocols and more debuggers will appear. Maybe they will use simpler protocols that don’t require gdb server to be present on the target platform in order to remotely debug and inspect some code, and just require the VM to be built with a debugger-enabled compile option.

Binary size

One of the neat tricks you can do with WebAssembly is update your programs without requiring any native code to be re-deployed on the devices you’re using this on. Other uses involve some sort of compute-at-edge scenarios like Fastly, Cloudflare and others are doing. This is better explained by this video.

In all of the cases above, or places where disk space is a concern, binary size is one dimension we will need to care about when using WebAssembly. This is rough edge number 2.

I do not want this section to turn into a “Here are 10 tips and tricks to optimize software” kind of blog post. The most important thing you can do in order to keep this under control is to monitor the size of your WebAssembly binaries from the start of your project. In doing so, you can find regressions as they happen (e.g. in a CI step) and then proceed to further investigate looking for a fix once your binary size goes above a certain limit.

One advice that I will mention, as it’s generally applicable, is configuring your project to optimize for size and enable LTO. This is done by editing Cargo.toml to include:

[profile.release]
lto = true
opt-level = 's' # or 'z', but may cost performance

Beyond that, here are some tools that will help you push size optimizations further if needed:

1. wasm-opt can provide further optimizations (beyond the opt-level='s' flag). You almost always want to use this.
2. wasm-snip can be used to remove certain functionality that may be unused. I don’t find it particularly useful as the improvements are not that great (this is it’s a very blunt tool that replaces some methods with unreachable!). It can be helpful for some use-cases.
3. twiggy is an excellent tool for finding the top functions that make up for the size of your executable.
4. Most articles on WebAssembly size optimization will suggest replacing the default rust allocator with wee_alloc.

Articles that may be helpful:

1. https://rustwasm.github.io/book/reference/code-size.html#optimizing-builds-for-code-size
2. http://cliffle.com/blog/bare-metal-wasm/

What is WebAssembly?

Web people are on a roll of giving bad names to things (web-gpu is another example).
WebAssembly is neither web or assembly, but a bytecode that can be targeted from languages like C++, C#, Rust and others. This means you can write some Rust code, compile it into WebAssembly and run that code in a WebAssembly virtual machine.

This is powerful because you won’t have to deal with garbage collected scripted languages anymore, and essentially use Rust or C++ as your scripting language. WebAssembly enables predictable and stable performance because it doesn’t require garbage collection like the usual options (LUA/JavaScript).

It’s a relatively new product and there are a lot of rough edges, especially for out-of-browser scenarios. One of the roughest ones in my experience has been documentation for out-of-browser scenarios and this is the reason for my blog posts, to document my findings and hopefully help some people that may be interested in this subject.

Why would we want to run WebAssembly outside of a browser?

For out of browser scenarios, one of its main advantage is that it provides system level access without compromising on security. This is done through WASI, the Web Assembly System Interface. WASI is a collection of C-like functions that provide access to functionality such as fd_read, rand, fd_write, threads (WIP), in a safe way.

Here are a few scenarios where you would be able to use web-assembly outside of a browser:

• A scripting language for a video game.
• To run some code with minimal overhead as Fastly/Cloudflare are doing with their compute-at-edge scenarios.
• To run some easy to update code on IoT devices safely and with minimal runtime overhead.
• Extreamly fast programs in environments where you can’t JIT for reasons.

Prerequisites

For the best experience in this adventure, I suggest using Visual Studio Code as your IDE and install the following extensions:

• rust-analyzer: for autocomplete and other great features.
• Code-LLDB: For debugging with LLDB (even works on Windows)
• WebAssembly by the WebAssembly foundation: Allows you to disassemble and inspect .wasm binaries.

Choosing a Virtual Machine

First you need a Virtual Machine (VM) that can run your WebAssembly program. This VM needs to be embeddable, so you can add it in your game engine, or what we will call from now on host program. There are a few to pick from: WASM3, Wasmtime, WAMR, and many others. They have various characteristics, such as supporting JIT, using as little memory as possible and so on and you have to choose one one that fits your target platform and scenario.

It doesn’t matter too much what VM you’re choosing besides runtime properties, with the exception of debugging. The only VM that allows for a seamless debugging experience that I’ve found is Wasmtime (this is another one of those rough edges). So even if you don’t plan on deploying that anywhere due to other constraints, I suggest using it as the debug VM. Whenever you’d want to debug some WASM code you can launch it with Wasmtime.

Writing our first WebAssembly program

First, we need to create a new lib project:

cargo new --lib wasm_example

In Cargo.toml add the following:

[lib]
crate-type = ["cdylib"]

Now we can edit lib.rs and export the following C FFI compatible function from it:

#[no_mangle]
extern "C" fn sum(a: i32, b: i32) -> i32 {
    let s = a + b;
    println!("From WASM: Sum is: {:?}", s);
    s
}

This a function that takes two numbers, adds them, then prints the result before returning their sum.
WebAssembly doesn’t define a default function that’s executed after a module is loaded, so in the host program you need to get a function by it’s signature, and run it (quite similar to how dlopen/dlsym works).

We expose this sum function (and any other functions we want to call from the host VM) as a function that’s callable from C, using [#no_mangle] and pub extern "C". If you’re coming here from some WASM for the browser tutorials, you may notice we don’t need to use wasm-bindgen at all.

How do we compile it?

Rust supports two targets for WebAssembly: wasm32-unknown-unknown and wasm32-wasi. The first one is bare-bones WebAssembly. Think of it like the [#no-std] of WebAssembly. It’s the kind you’d use for the browser that doesn’t assume any system functions are available.

At the other end, wasm32-wasi assumes that the VM exposes the WASI functionality, allowing a different implementation of the standard library to be used (the implementation that depends on the WASI functions to be available).

You can take a look at the available implementations for the Rust’s stdlib here: https://github.com/rust-lang/rust/tree/master/library/std/src/sys
This is the implementation that assumes WASI functions are available to the rust program when running in a WebAssembly VM: https://github.com/rust-lang/rust/tree/master/library/std/src/sys/wasi.

To comile for wasm32-wasi run:

# Run this just once
rustup target add wasm32-wasi

# Compile for the wasm32-wasi target.
cargo build --target wasm32-wasi

But how does println!() work?

You may have noticed that we’re calling println!() and expecting the program to work and print to the console, but how does a WebAssembly program knows how to do that?

This is why we’re using wasm32-wasi. This target selects for the rust stdlib the version that assumes some functionality to be there (the WASI functions). Printing to the console means just writing to a special file descriptor. Most VMs allow that by default so we don’t need to do any special settings, besides compiling the correct wasm32-wasi target.

If you have installed the required extensions for vscode, you can now right click on target/wasm32-wasi/debug/wasm_example.wasm and select Show WebAssembly and you should have a new file open in vscode that looks like this:

(module
  ....
  (type $t15 (func (param i64 i32 i32) (result i32)))
  (import "wasi_snapshot_preview1" "fd_write" (func $_ZN4wasi13lib_generated22wasi_snapshot_preview18fd_write17h6ec13d25aa9fb6acE (type $t8)))
  (import "wasi_snapshot_preview1" "proc_exit" (func $__wasi_proc_exit (type $t0)))
  (import "wasi_snapshot_preview1" "environ_sizes_get" (func $__wasi_environ_sizes_get (type $t2)))
  (import "wasi_snapshot_preview1" "environ_get" (func $__wasi_environ_get (type $t2)))
  (func $_ZN4core3fmt9Arguments6new_v117hb11611244be67330E (type $t9) (param $p0 i32) (param $p1 i32) (param $p2 i32) (param $p3 i32) (param $p4 i32)
    (local $l5 i32) (local $l6 i32) (local $l7 i32) (local $l8 i32) (local $l9 i32) (local $l10 i32)
    global.get $g0
    local.set $l5
  ...

This is a wat file. wat stands for WebAssembly text format. It’s kind of like looking at x64/ARM ASM instructions when disassembling a binary, just uglier and harder to understand. I have read that this was because the creators of WebAssembly couldn’t decide on a text format so they just left it in this ugly s-expression form.

The import statements here tell us that the WASM program needs the following functions proc_exit, fd_write, environ_get, environ_sizes_get to exist in the wasi_snapshot_preview1 namespace.
All imported or exported functions from a WebAssembly module require a namespace. wasi_snapshot_preview1 is the WASI namespace so you can think of it as a reserved namespace for these functions. println! needs wasi_snapshot_preview1::fd_write to write to stdout.

The host program

You can pick any VM that has WASI available. I will use Wasmtime because later on I want to show you how to debug WebAssembly and this VM is the only one where debugging works at the moment.

The program loads the wasm binary file from the path: examples/wasm_example.wasm.
This is the file you have previously compiled that you can find in wasm_example/target/wasm32-wasi/debug/wasm_example.wasm. Make sure you move it in the right place before running the host program.

Here is the full listing of the host VM rust program that initializes the Wasmtime VM, loads the module, links against WASI and loads and executes the exported sum function from the WASM module:

use std::error::Error;
use wasmtime::*;
use wasmtime_wasi::{Wasi, WasiCtx};

fn main() -> Result<(), Box<dyn Error>> {
    // A `Store` is a sort of "global object" in a sense, but for now it suffices
    // to say that it's generally passed to most constructors.
    // let store = Store::default();
    let engine = Engine::new(Config::new().debug_info(true));
    let store = Store::new(&engine);

    // We start off by creating a `Module` which represents a compiled form
    // of our input wasm module. In this case it'll be JIT-compiled after
    // we parse the text format.
    let module = Module::from_file(&engine, "examples/wasm_example.wasm")?;

    // Link the WASI module to our VM. Wasmtime allows us to decide if WASI is present.
    // So we need to load it here, as our module rquires certain functions to be present from the
    // wasi_snapshot_preview1 namespace as seen above.
    // This makes println!() from our WASM program to work. (it uses fd_write).
    let wasi = Wasi::new(&store, WasiCtx::new(std::env::args())?);
    let mut imports = Vec::new();
    for import in module.imports() {
        if import.module() == "wasi_snapshot_preview1" {
            if let Some(export) = wasi.get_export(import.name()) {
                imports.push(Extern::from(export.clone()));
                continue;
            }
        }
        panic!(
            "couldn't find import for `{}::{}`",
            import.module(),
            import.name()
        );
    }
    // After we have a compiled `Module` we can then instantiate it, creating
    // an `Instance` which we can actually poke at functions on.
    let instance = Instance::new(&store, &module, &imports)?;

    // The `Instance` gives us access to various exported functions and items,
    // which we access here to pull out our `answer` exported function and
    // run it.
    let main = instance.get_func("sum")
        .expect("`main` was not an exported function");

    // There's a few ways we can call the `main` `Func` value. The easiest
    // is to statically assert its signature with `get2` (in this case asserting
    // it takes 2 i32 arguments and returns one i32) and then call it.
    let main = main.get2::<i32, i32, i32>()?;

    // And finally we can call our function! Note that the error propagation
    // with `?` is done to handle the case where the wasm function traps.
    let result = main(5, 4)?;
    println!("From host: Answer returned to the host VM: {:?}", result);
    Ok(())
}

The Cargo.toml of this project needs to have the following dependencies:

[dependencies]
wasmtime = "0.19"
wasmtime-wasi = "0.19"
anyhow = "1.0.28"

Running this with cargo run will print the following output:

Compiling wasm_host v0.1.0 (wasm_host)
 Finished dev [unoptimized + debuginfo] target(s) in 35.38s
  Running `target\debug\wasm_host.exe`
From WASM: Sum is: 9
From host: Answer returned to the host VM: 9

We can observe that the println! from the wasm module has correctly printed to the console and that the returned answer is as expected 9.

Conclusion

In this post of my WebAssembly Outside the Browser series we’ve learned how to compile a program for WebAssembly, set-up a host program to load and run a your WASM binary, execute a function exported by the WASM program and put all that together we ended up adding two numbers and printing their result (from both WebAssembly and the host program).
In the next parts we will touch areas such as debugging, optimizing program size, exposing functions from the host vm to the WASM program and sharing memory between the two VMs.

Bonus study materials

Here is the full WASM specification. For me that it’s one of the hardest spec that I’ve ever had to read.
I would much rather have this spec similar to a CPU user manual (e.g. VR4300), rather than it’s current form that is forced into some math-y language that, while correct, brings no extra clarity or insight to the reader.
I strongly think that the concepts described there could have been very well expressed in an easier to understand and parse language and I don’t buy the usual excuse that “Well actually, it’s targeted at VM writers, not normal people”. We should just accept that it’s not accessible at all, and we could do way better.

Some more materials:
Videos:
Kevin Hoffman: Building a Containerless Future with WebAssembly
Peter Salomonsen: WebAssembly Music

Reading material:
WebAssembly.org
Standardizing WASI: A system interface to run WebAssembly outside the web
Cliff L. Biffle: Making really tiny WebAssembly graphics demos
An overview of WebAssembly’s historical context

I wish today’s games would have accurate temperature simulation…

I hear that all the time.
We already have physically-based rendering, so why did we stop there?!

But… Why?

Because it’s my game and I do whatever I want.
I am working on a sim game where you have to take care of a colony of dwarves and I want seasons to play a big part in this.
I want my dwarves to be cold in the winter, get warm in the summer, lose a limb due to frostbites. Many games have already experimented with such systems to some degree (e.g. Don’t Starve).

But can’t that be solved with a simple radius check and a decay formula for the temperature transferred?
Yes you can and you should do that, but for me that wasn’t enough for a few reasons:
1) I wanted to have cave temperatures be slowly influenced by the outside temperatures. E.g. you can grow mushrooms in your cave but not near the cave entrance as they might need a cooler and more stable temperature.
2) I want certain plants and foods to preserve better at lower temperatures.
3) Another thing that complicated my life are walls. If you have a fire in one room near a wall you don’t heat the air outside of that room at the same rate as you heat the room.

The solution

In addition to the reasons mentioned above it is really hard to tune random formulas that have nothing to do with reality. It’s way easier to start from a realistic system and formulas and tune that to make it fun.

Let’s go to the classic formula of calculating the heat transferred when a system goes from one temperature to another.

Q = Heat, measured in Joules - J
m = mass of the system grams - g
c = specific heat capacity of a substance. That is the energy required (heat) such that a unit of mass (g) of that substance will raise its temperature by one unit (Kelvins - K).

So, as far as units of measurement are, we have an equation of the form:

So it all checks out and we are left with Q being measured in Joules.

But just that formula is not enough as it doesn’t tell us how two adjacent terrain cells at different temperatures should transfer heat between each other.
When two bodies have different temperatures, their molecules have different average kinetic energies (they bounce around at different speeds). When these two bodies are in contact, collisions between moving molecules on the surface of contact will transfer energy from the high-temperature body to the low-temperature one.
As that transfer happens, the higher temperature body slightly cools down and the lower temperature body warms up.
We intuitively already know that different materials conduct energy at different rates. That’s why you have air in between two sheets of glass in your windows. Air is a material that has a high thermal resistivity.

If you research this online you find these material constants defined as either thermal resistivity or thermal conductivity. They are the basically expressing the same thing, but you divide by one and multiply by the other.

For thermal conduction we have this formula using conductivity:

Where:
t = time
Q = heat
A = surface area
T = temperature
d = distance to the point we want to measure the temperature at.
k = thermal conductivity

If we use thermal resistivity it looks like this:

I went with the thermal resistivity version as engineering books seem to favor it and they provide constants for a variety of building materials I have in my game. If you find conductivity values, you can easily switch from one to the other. Just make sure your units of measurement work out correctly. If you express everything in SI units you will be fine.

Putting it all together

The terrain in my game is a 3-D grid of cells (think minecraft-like). Some cells are made of ground, some are air, stone, etc.
That being said, update cycle that happens for temperatures in my game is made up of the following steps.

0 - Heat propagation from the “sun”

All terrain cells that see the sky will get their temperatures updated based on some ambient temperature based on the time of day and season. All the other terrain cells will tend to get towards a different ambient temperature based on how deep they are, etc. This step is here to simulate a fake convection and radiation. Without it all the map will eventually heat up from a fireplace (as there would be just energy added, and none lost).
It’s also very easy to tune seasons temperatures like this so that’s why nothing complex is going on this step.

1 - Heat propagation for environment (terrain cells)

For this we apply this formula as we need to know Q, the heat transferred between two adjacent terrain cells.

Using the formula above we compute the average energy a terrain cell is transferring with its neighbors. I consider a cell’s temperature as the temperature measured at the center of that cell. This energy is used to compute a cell’s new temperature by plugging Q in this equation:

We have the final temperature of our terrain cell.
You can see how this process is super easy to parallelize, as each cell updates it’s temperature from the previous temperature of the neighbor cells. You just need to have two terrain copies (one with the old temperatures, and one that will get the new temperatures based on the old values).

2 - Heat propagation to game entities

For this we use the same formulas as above, with a small exception. We just update the game entity’s temperature and we don’t change the terrain cell temperature based on that game entity. This is for receiving game units (dwarves, objects, cows, etc.). These entities have a MaterialComponent attached to them. That component tracks the temperature and other useful material properties to compute that temperature. Applying the proper formula where the dwarf also affects the game cell temperature doesn’t yeld nice results due to the unrealistic size ratio between a terrain cell and a dwarf (see bottom of the article).
Emitting game units like campfires propagate the temperature in the other direction, from them to the terrain.

This is the definition of the MaterialComponent:

#[derive(Debug, Serialize, Deserialize)]
pub struct MaterialComponent {
    volume: f32,
    area: f32,
    temperature: f32,
    radius: f32,
    mass: f32,
    material: Material,
}

Let’s remember the properties needed to transfer temperature between two entities:

• Surface area
• Initial_temperature
• Distance (that the energy travels)
• Specific specific heat capacity
• Thermal resistivity
• Time
• Mass

From that list, just a few are entity-specific: surface area, distance, mass.
Besides time, the rest depend on the material so we can keep them in a separate material_properties.json table.
That is a json to make it easy to add and tune materials.

But how do we know the mass, surface area and distance that the heat travels for a dwarf body part?

Configuring the distance energy travels, mass and surface area for each body part component is an interesting exercise, but I am lazy and here I went with an approximation with the goal of keeping things intuitive and easy to set-up.

The only parameters actually needed to initialize the MaterialComponent for an entity (head, hand, leg, etc.) are:

{
    "MaterialComponent": {
        "mass": 6000,
        "material": "Water"
    }
}

Besides the temperature-related material_properties.json entry, I also input the density in that table. For example the entry for "Water" is:

"Water": {
    "density": 997000.0,
    "thermal_resistivity": 1.6,
    "specific_heat_capacity": 4.2
}

You probably know where this is going.
For the purpose of temperature transfer, I am approximating all entities to simple shapes (spheres).

That’s where the density comes into play. With it and the mass we can compute the volume of our entity:

Having the Volume, using a simple formula we can compute the radius of the sphere and from that the area.
We now have all the data needed, so we do the same calculations using the same formulas we used for the terrain temperature.
All MaterialComponents can be updated in parallel.

3. Needs system

The needs system reads the temperature values from the MaterialComponent and issues Actions based on the needs of a creature(dwarf, cow, etc.). E.g. If they are cold they go near the fire. I will write more about the needs system and how AI works in a different blog post.

Next steps

The next steps is to take armor and clothes into account when calculating the heat transferred. That just slightly changes the formula we have as we just need to add the R-values of all materials involved in the heat transfer:

Using spheres will make it a bit tricky to handle clothes as the clothes would need to be hollow and I will probably consider all entities with MaterialComponents cube-shaped instead.
I plan on adding more temperature related effects.
Conduction is just one way heat is transferred between things. Another way to exchange heat is convection and radiation. I am not sure if I will add them as the effects in this game world will probably be so small it’s not worth it.

Because you reached the end of this, I leave you with this cozy picture of a dwarf and a cow warming up near a fire.

Besides supporting all three platforms I was interested in, Github also offers a good range of pay as you go prices with a cost per minute of: 0.008$ for Linux, 0.016$ for Windows and 0.08$ for MacOS as well as 2000 minutes of a free tier.
This is great news for hobby projects like mine that happen to be on a private github repository.

Below you can find the action that I’m currently using.
It installs SDL2 on Linux and MacOS and assumes the DLLs for Windows are in the git repo.
If you think that pushing some DLLs in a git repo is not a clean solution, there’s the alternative of using something like this PowerShell script to get the SDL 2 DLLs.

It comes with the stable Rust 1.37 version on Linux and the Windows distribution. I had to manually install it on MacOS.
Using rustup you can also get nightly/beta versions if needed.

More information on how to set up github actions CI for your own projects:

• Virtual environments available
• Workflow yaml syntax

The action

name: Rust

on: [push]

jobs:
  test_Ubuntu:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v1
    - name: install_dependencies
      run: |
        sudo add-apt-repository -y "deb http://archive.ubuntu.com/ubuntu `lsb_release -sc` main universe restricted multiverse"
        sudo apt-get update -y -qq
        sudo apt-get install libsdl2-dev
    - name: Build
      run: |
        rustc --version
        cargo build
    - name: Test
      run: cargo test 
      
  test_MacOS:

    runs-on: macOS-latest
    
    steps:
    - uses: actions/checkout@v1
    - name: install_dependencies
      run: | 
        brew install SDL2
        brew install rustup
        rustup-init -y --default-toolchain stable        
    - name: Build
      run: |
        export PATH="$HOME/.cargo/bin:$PATH"
        cargo build
    - name: Test
      run: |
        export PATH="$HOME/.cargo/bin:$PATH"
        cargo test
      
  test_Windows:
    runs-on: windows-2016
    
    steps:
    - uses: actions/checkout@v1
    - name: Build
      run: cargo build
    - name: Test
      run: cargo test

How it looks like

Some details

On ubuntu-latest I had to do some tricks to install libsdl2-dev.
Just doing sudo apt-get install libsdl2-dev doesn’t work right now as it has a missing package.

On Windows I use windows-2016 it has Visual Studio 2017. For now, that’s the newest version supported by rust-hawktracer.
After I will update my build script for rust-hawktracer to handle Visual Studio 2019, windows-latest will work too.

You can also set it up for PRs like this:

on:
  pull_request:
    branches:
    - master

• Hire armies of people to do it for you.
• Early access.
• Hope all is fine.

We all know how good the support in rust is for writing tests and I would like to show you my improved testing setup for the game I’m working on.

Testing

I just test the high-level behavior as things evolve quite fast in my game and having super detailed unit tests for all individual parts is not really worth the effort. At the end of the day all I care about is that if I tell a dwarf to dig a whole at a certain position, he digs a whole at that position.
Time is also a key part of this strategy. I just don’t have enough time to have a lot of detailed unit tests so these high-level tests have to do.

In my previous post I said:
“Almost all tests are instantiating worlds and various entities and scenarios. This means that if a test breaks, I just copy the world initialization code to the main game and I can visualize that test scenario and debug it really easy, pausing the simulation, inspecting entities with the debug UI, etc.” - me

Copy-pasting things got annoying after some big refactors where I had to check why I was failing a bunch of test cases.
That prompted me to change my testing setup so here’s my improved approach.

Today my tests look like this:

#[test]
fn dwarf_eventually_dies_of_hunger() {
    let mut world = World::new();
    world.spawn_from_entity_type("Dwarf", Vec3::new(0, 0, 0));
    
    let mut game = Game::new(world, false);
    game.update(Some(3000));
    
    //RIP
    let world = game.get_world();
    assert!(world.get_entity_by_type("Dwarf").is_none());
}

At a first glance this looks exactly as the old tests that I presented in the other blog post.
However, there’s an important difference, it is using the new Game struct.
I can enable the full experience: graphics, window, events, UI just by changing Game::new(world, false) to Game::new(world, true).
This means that I can click on objects, inspect, even change tests as they happen by issuing build orders.

And here’s the above test running:

I even tried hard to have the graphics always on for all tests but, due to a limitation of IMGUI, I can’t spawn more than one instance of it on multiple threads so this is out of the question.

I consider this new setup a big step forward for my productivity in debugging tests and in adding writing new tests as right now it’s super easy to check that a test checks the right thing.

Intro

There are simple solutions to this problem, one of the most popular ones, that I actually ended up implementing is HPA* or as the original paper calls it, Near-Optimal Hierarchical Pathfinding. But in reality it’s just Hirerarchical A*.
This is best explained in the paper here.

However, I did some changes in order to adapt that paper to my use case – a minecraft-like world, with multiple levels. That paper only solves it for the 2D case, but it’s easily extended to a 3D-block world.

HPA*

So how does it work?

The main idea of hierarchical A* is to (you guessed it), create a hierarchy first. So we divide our world into bigger cells, and then compute the path and connection points in between these big cells.

As you can see in the image above, the world is split into 10x10 big cells (the yellow cells). The nice thing about this is that you don’t need to tune anything (except maybe the high-level cell sizes). For now let’s keep them at 10x10 and we have this world divided into high-level cells.

Once we did this, we proceed and find the connection points in between high level cells. These are the points on the edges. There can be quite a lot of them in the case of an open field and here the paper goes one step further. It makes a sort of doors between two adjacent high-level cells. This way it can only keep 2 connection points for each door over a certain size. I didn’t do that but it can be a good optimization.

So right now we have a series of connection points for the high level cells. You can see them with the blue in the picture below. We also added some obstacles (black cells).

Now we add connection points in between these cells. First we start with the external connections. These external connections are between two adjacent high-level cells. All blue cells have an external connection between themselves and the blue cell they are adjacent to that sits on another high-level yellow cell. After we’ve found and cached all external connections, we need to handle the internal connections. These are done by just finding a path with HPA* and checking if all points are in a cell.

In the image below we can see the internal connections for the red cell. Green and blue connections are the same thing, I used a different color to make things a bit more visible.

Getting a path

Now that we have our high-level cells and they have connections between them and internal connections we just need to explore two things when we are searching for a path.

1. From the starting point find all possible high-level connections that we can start from
2. Traverse the high-level graph until we reach the high-level cell containing the destination.
3. Once we reached that end high-level cell, do a low-level A* to find if from the entry point we have a path to the destination.

As our entity moves through the world it will encounter a cell that’s connected to another far away cell. This is for the case where we need to traverse a high-level cell. In that case we have to call A* pathfinding again for that high-level cell.

It is also possible to cache the path and just querry it as it saves us a A* search for a 10x10 cell. This is what I do and if you have spare memory to cache these paths I highly recommend doing so.

For example a path from the start point S to the destination D will look like this:

• The orange cells are part of low-level paths.
• The red cells are connected in the high-level pathfinding grid. For them we either have low-level paths cached or we compute them as needed. When we reach the first cell touched by that red arrow.

Depth & digging

Handling depth is simple. For each cell (not only the edges) we check if there is a cell below or above that we are connected to. These cells are kept in case we need to descend to a lower level as we search for a path to our destination.
Handling terrain destruction is simple as we only need to re-compute the high-level connections in the cell where the digging happened.

The end

So this is our short journey into HPA* for now. It’s a simple technique to speed up pathfinding for your games, especially if you’re using A* as this comes at an easy integration with your existing pathfinding code. You have to change the core code just slightly and most of the work is done in the initial high-level cell generation. After implementing it the worst gase went from 7ms to 1ms-1.5ms. There are still ways of improving this, like using the door system that they describe in the paper, but for now I am pleased with the results. Other ways to speed up pathfinding is using alternative data structures and things like nav meshes.

It’s also important to remember that this can go to more than 1 level of hierarchy above the low-level cells. I didn’t need to generate another level above this one, but I think for certain games it may be useful to keep in mind that you can extend this.

I also explained this on my stream a while back so if you like this post in a video format you can watch that explanation here.

What is this about?

So this is how the game looks on the current build.

General architecture

The game has an Entity-Component-System (ECS) architecture. There are many places where this is explained better so I’m going to be lazy and just link them here:

• Overwatch Gameplay architecture
• RustConf 2018 closing keynote as video or text.

My quick TLDR explanation is as follows. This architecture is formed out of three parts:

• Entities - IDs that refer to different components.
• Components - Data without behavior.
• Systems - Behavior that has no state. Each system acts on a collection of components.

I didn’t use specs or any of the other rust libraries because I started this project as a learning experiment that slowly transformed into a game. If I would start now I would take a serious look at specs before rolling my own implementation. There are some advantages of existing libraries over what I have most of them around boilerplate code.

To create an entity right now I just edit some mega json file called: recipe_table.json. For example the Baguette entity recipe looks like this:

    "Baguette": {
      "can_craft": true,
      "items_list": [
        //List of items needed to craft this entity. 
        {
          "entity_type": "Grains",
          "count": 1,
          "processed": false
        }
      ],
      "components": [
        //Data to initialize various components  
        {
          "Item": {
            //Almost all entities have this.
            "can_carry": true,
            "entity_type": "Baguette",
            "blocks_pathfinding": false
          }
        },
        {
          "Renderable": {
            //This is quite clear  
            "texture_type": "Baguette",
            "width": 64,
            "height": 64,
            "layer": 1
          }
        },
        {
          "Consumable": {
            //Consumable entities can be used by dwarves
            //to trigger various effects  
            "modifier": {
              "effect": "Food",
              "effect_value": 60,
              "effect_type": "Instant",
              "ticks_since_last_applied": 0,
              "should_stack": true
            }
          }
        }
      ],
      "workbench_type": "CookingTable"
    }

As primitive and ugly it is compared with modern engines UIs, I am really happy with this system. It allows me to quickly create and modify entities. Above all it just works and it allows me to focus on different aspects of the game.

The component properties that you see here aren’t deserialized directly into a component, but a ComponentDescriptor. A RenderableComponent has many other members, but the data in his corresponding RenderableComponentDescriptor is enough to instantiate a RenderableComponent.

I use Serde for any serialization and deserialization job and it is an amazing library that is a joy to use and almost invisible. If by any chance you don’t know about it, I recommend you to take a look at some of the example code to get an idea on how it’s used.

I am keeping this as open and configurable as possible in the idea that I want to allow people to mod the game. Potentially someone (not me) can even build some fancy UI instead of working directly with this JSON in the future.

There are some limitations to modding and I will keep them moving forward. New components or systems can’t be added to the game but the existing components can be combined to create new interesting entities (like a plant that attacks dwarves when they pass near it).

How much from scratch?

Since people might be interested, here is my cargo.toml file:

[package]
name = "dwarf_game"
version = "0.1.0"
authors = ["Alexandru Ene <[email protected]>"]
edition = "2018"

[dependencies]
sdl2 = "0.31.0"
imgui = "0.0.18"
gl = "0.6.0"
memoffset = "0.1"
png = "0.11.0"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
derive-new = "0.5"
fnv = "1.0.6"
rayon = "1.0"
rand = "0.5.0"
noise = "0.5.1"
lazy_static = "1.2.0"
log = "0.4"
pretty_env_logger = "0.3"

[dependencies.rust_hawktracer]
version = "0.3.0"
#features=["profiling_enabled"]

[profile.release]
debug = true

[profile.dev]
opt-level=0

Rendering

Rendering is done using OpenGL. I have a small wrapper that provides me with simple abstractions like Texture, ShaderProgram, etc. over gl-rs. Simplicity is key here and I don’t have anything more than’s strictly need.

I don’t even use texture atlases yet and I just have a bunch of textures dumped into an assets/images/ folder.

UI

Currently the UI is done using imgui-rs. Right now it looks too much like some debug UI so I am unsure if this will be used for the actual game UI as well. I am evaluating the follwing options:

• Configuring imgui-rs more (Most likely to be the chosen option)
• Making my own (I really want to avoid this)
• Something else?

The best feature of this UI right now is that I can just click on an entity, select it and view all it’s components state so it is an invaluable tool for debugging.

I had to write my own renderer for gl-rs for imgui-rs. Nothing complicated, I basically just copied the C++ Opengl example from the main imgui project did and translated it to rust.

Terrain

Probably the most invisible big piece of work right now is the terrain.
The terrain is actually 3D and generated from a random seed (think minecraft-like).
Pathfinding works in 3D (so dwarves know when two terrain levels are connected and can go between them), but due to the fact that the current camera is top down and I am such a noob at art, this is impossible to notice unless I explain it.

Systems

There are a few systems right now at different levels of completion:

• Combat
• Farming
• Health
• Movement
• Pathfinding
• Task assignment
• Task processing
• Needs - hunger and thirst
• and few smaller ones

Some of them act on a few components, others act on a ton of things. For example the TaskAssignment and TaskProcessing system need to have access to almost all components.

The beauty of this kind of solution is that that complexity and dependency is explicit and contained in one system. It’s clear from just taking a look at the system data that this is complicated, and not hidden away behind other things.

Systems act on collections of components. For example the combat system has the following view:

#[derive(new)]
pub struct CombatData<'a> {
    dwarf_components: &'a mut DwarfComponentContainer,
    combat_log: &'a mut CombatLog,
    item_components: &'a mut ItemComponentContainer,
    terrain: &'a mut Terrain,
    body_part_components: &'a mut BodyPartComponentContainer,
    armor_components: &'a ArmorComponentContainer,
    transform_components: &'a TransformComponentContainer,
}

It also contains the update_combat function:

pub fn update_combat(combat_data: CombatData) -> Vec<Action> {
    ...
}

In world.update() we just do the following:

    systems::combat::update_combat(systems::combat::CombatData::new(
        &mut self.dwarf_components,
        &mut self.combat_log,
        &mut self.item_components,
        &mut self.terrain,
        &mut self.body_part_components,
        &self.armor_components,
        &self.transform_components,
    ));

This makes it clear what the combat system is modifying (terrain, dwarf components, body parts, etc.). Armor components are read-only since armor doesn’t get damaged in combat yet.

This is way more primitive compared to the way things work in specs or other similar projects where you get an iterator with one dwarf component, one item_component, etc. I have a bit more boilerplate to write in order to get to the same result, but I don’t mind that.

Components

There are a bunch of components such as:

• Renderable
• Armor
• Area
• Farm
• Workbench
• Dwarf
• Consumable
• BodyPart

As I said before all components contain data and no functions (except simple getters/setters).

Tests

Rust makes so easy to add tests that it’s silly not to write them. I usually tests for bugs I find. Most of them are like integration tests that test how various systems interact. Each time I find a bug I usually try and add a test for it.

For example, this is one bug I had to solve:
When dwarves got hungry as they were crafting something, as they went to eat a baguette they left that task in a limbo state and it couldn’t be finished.

After I fixed it, it was simple to add a test that checks tasks gets handled properly in that case:

#[test]
fn test_hungry_dwarf_eats_and_finishes_task() {
    let mut world = World::new();
    world.spawn_from_entity_type("WorkbenchWoodsmith", Vec3::new(0, 0, 0));
    world.spawn_from_entity_type("WorkbenchAtDestination", Vec3::new(0, 0, 0));

    let dwarf_id = world.spawn_from_entity_type("Dwarf", Vec3::new(0, 1, 0));
    world.spawn_from_entity_type("Baguette", Vec3::new(0, 5, 0));

    world.spawn_from_entity_type("Mattress", Vec3::new(4, 4, 0));
    for _ in 0..5 {
        world.spawn_from_entity_type("Plank", Vec3::new(3, 3, 0));
    }

    let dwarf_component = find_component(world.get_dwaf_components(), &dwarf_id).unwrap();

    //Make the dwarf hungry
    loop {
        world.update(1);
        let dwarf_component = find_component(world.get_dwaf_components(), &dwarf_id).unwrap();
        let stats = dwarf_component.get_stats();
        if stats.hunger <= stats.hunger_limit + 1 {
            break;
        }
    }

    //Create a task for him
    let task = PlaceItemTask::new(world.generate_entity_id(), Vec3::new(6, 6, 0), "Bed");
    world.add_task(Box::new(task));

    for _ in 0..250 {
        world.update(1);
    }

    //Make sure the task ends and the baguette is eaten.
    assert!(world.get_entity_by_type("Baguette").is_none());
    assert!(world.get_entity_by_type("Bed").is_some());
}

Almost all tests are instantiating worlds and various entities and scenarios. This means that if a test breaks, I just copy the world initialization code to the main game and I can visualize that test scenario and debug it really easy, pausing the simulaltion, inspecting entities with the debug UI, etc.

One year

I started this one year ago and I commited changes to the project constantly.
How did I manage to stay motivated for this long?

The secret to staying motivated for so long is that there is no secret and I wasn’t motivated and hyped to work on it all the time. It’s ok to take breaks. I’ve had periods where I had a lot of activity and pushed a lot of changes followed by weeks I didn’t write a single line of code. Last summer I did almost no work on it for almost two months for example.

When I’m dealing with with big pieces of work that look scary, the only thing that works 100% for me is to just sit down and start writing.

I mainly design by experimenting. I also have a bit of advantage here since I’ve worked in software and gaming for a while so I kind of know how to avoid the most common pitfals. But really, the most important thing is to just sit down and start writing.

Streaming on twitch also helps, even if I don’t have a regular schedule. It brings some sort of order and mini-deadlines. It also brings down distractions (even if sometimes there’s some chatting going on). I never plan what I stream so I have to sit and work through whatever I said I was going to work on, instead of getting distracted by other things.

Other than that, I have no good answers or advice on this topic.
It’s a first for me too since I usually abandon side projects that take more than 2 months. This one kind of stuck with me.
Maybe it’s also the fact that I really enjoy writing rust?

What’s next?

Things that I want to work on next are:

• Picking a name for it. Really I’ve been postponing this for way too long.
• Digging and manipulating terrain
• Refine terrain generation and add natural resources
• UI
• More systems (weather)
• AI director for other factions
• Wild animals
• Professions & military
• Temperature
• Water
• Adding more items and recipes
• Many many other things

But what is game development anyway?

Games are made of complex systems where a lot of things usually need to happen in a short amount of time. In game development you have to do your work fast. In games you your work 16 milliseconds fast. If you’re lucky you get about 32 milliseconds.

Even if we ignore rendering, you need to do: physics, animation, updating various gameplay systems, AI, pathfinding and it usually doesn’t stop here. That’s a lot of things that need to happen and that’s why C++ is usually language of choice for game engine development.

The problem space

I will break down the problem into two. After doing this step, we have two problems, but trust me they are a bit easier.

1. Big engines

These are represented by the big companies that build their own, equally big engine (Ubisoft, DICE, Epic, Unity, Lumberyard, etc.).

There are a few restrictions here, most of them are written in C++ under the hood (even Unity). Most didn’t care about ABI compatibility since it’s all compiled at once so this makes communication to a new Rust module less than ideal. I am not saying we need to solve C++ to Rust bindings, but we need to consider how do we solve fitting Rust systems in an existing engine.

Portability to closed systems. This applies to both categories, but it is really important for this one. While Rust is available on many platforms and architectures, that doesn’t mean it just works on console X or console Y and it’s supported out of the box if you write a hello world program. Console game development is a strange, NDA-filled space, unknown to many, and while things are moving in the right direction, there are still problems that need to be solved.

These mammoth engines also care about performance. Sure, you might say consoles are powerful today, but as we said in the intro, you only get 16ms and players expect a lot of things to happen in today’s AAA game worlds. To achieve this kind of performance a lot of optimization work is put into data structure layouts, SIMD, custom memory allocators, etc. Some of these are already doing quite well in Rust, but others not so well. For example custom memory allocators for the standard containers has a great RFC, but it’s not done yet.

2. Small engines / games

Here we have the small-medium sized companies that don’t use an off-the-shelf engine, like Chucklefish, Killhouse, and many others (even lone-wolf gamedevs like me, I do a game in my spare time in Rust).

I’ve put engines / games in the title place since we have cases where the engine is a custom-built thing for one game. Yes it still happens even today and that’s fine.

I believe that Rust as a language is ready and has enough maturity and features for this to be possible. As I said in the intro, it’s not only me as a mad, game developer who thinks this, others have jumped on board way before me and announced that they will develop their next game fully or by using rust as much as possible.

Why did mention this category if I think we’re there already for most cases?

Because there are unsolved issues and nuisances here too. One problem that I’ve found is that you kind of have to be an expert to make a game and if you start with the wrong path you get into somewhat frustrating situations. Rust punishes you for being wrong more than other languages, and it does at compile time so you have to do things right in order for them to work.

There’s no it works by the power of luck here and sometimes that feels bad.

People have explained it way better than me here and there are resources available, but it’s something to keep in mind - for more info and solutions see Catherine West’s excellent Rustconf 2018 keynote

In this space, there are also some Rust game engines but compared to Unity tutorials they have a higher barrier of entry. For example, for the Amethyst engine, a simple game of Pong starts out with the following code:

impl<'a, 'b> SimpleState<'a, 'b> for Pong {
}

What is that? You might cry, but have no fear, I kind of had the same reaction when I saw that a state needed two lifetime annotations. 'a and 'b. There are good reasons for having them, but for someone who wants to write pong it’s a bit scary. It’s scary for me too and I’ve worked in game development for more than 8 years and I write Rust for more than an year quite intensively.

Do I need that even for pong? I would bet that you can rewrite something like Doorkickers or Stardew Valley or any other 2D game in rust without having to annotate many lifetimes.

Amethyst is shaping to be a nice engine but if all you want to do is a 2D game with simple rules, you could get away with simpler abstractions.

Possibly my point is that there is enough space for more engines to appear and address various targets.

The solution

Now that we’ve seen a bit of the space Rust game development, let’s look at what I think would be a solution.

I propose starting a Game Development focused Working Group.

So we made the working group, now what?

What should this Rust Game Development Working Group do?

Besides the usual WG tasks, the role of this working group is to find and tackle systemic problems that game developers face as they write their games in Rust.

Gathering these pain points sorting and distributing them them to the teams that handle different parts of the ecosystem is one role.

Communicating and teaching through tutorials, a status of what the problems encountered are and general info of what’s happening in this space.

This isn’t going to touch a single area. It impacts multiple parts of the ecosystem and we need to identify and collaborate in solving any pain points found.

Now for the practical steps I will split my solution into two parts. – this is an ongoing theme with me splitting things in two parts

The short term

For the short term I’d see a focus on the second category. We are almost there, but there are things still missing or confusing.

We need more resources focused on problems game developers face daily. Some of these I’m sure have been solved a few times already. For example, if you decided to serialize things with Serde, what’s the best way of serializing / deserializing a Vec<Box<SomeTrait>> object? I’ve personally spent probably 4-5 evenings on this problem. I’m certainly not the brightest tool in the shed, but it would be nice to have a bit more posts and information shared on how you could solve certain things that people usually hit in programming with Rust in this space.

Tooling is another subject. RLS is really good but unfortunately it competes with years of effort put into IDEs like Visual Studio.

I know that Windows is a platform that usually doesn’t get much love, but these days it’s the usual development platform used for games. For example, the rust compiler doesn’t even compile on windows with debug enabled due to linking problems (tries to link too many objects).

C++ / C# patterns don’t directly translate to rust. You can switch from C++ to C# really easy since both accept the same kind of patterns with ease. Rust doesn’t like a lot of these (I’m not going to say bad, but let’s say risky) patterns. Unclear hierarchies of things, shared mutable ownership, etc. These is a space where the rust core team focuses on anyway and provides great solutions and advice, but it’s worth mentioning them as a potential problem that game developers will face.

Custom allocator support are a must and almost everyone mentions them so I’d advocate that that needs to have a higher priority and that’s why I include it in this section.

The Long term

This is a bit more painful to get to, not because of technical reasons but because the world is complex. Changing things in big organizations or systems is hard but not impossible.

I hope for a future when you can just do cargo build and get a binary that runs on a game console of your choice. I think it’s a better future for everyone: players will experience less crashes from common avoidable causes and developers are enabled by a modern language.
Much of the hard work has been done and I don’t know of any language features that are needed in order to make this rust game development initiative a success. If there are, we should start discussing them and drafting a RFC.

As a general note, I don’t think what’s in the long term category has to start after we finish the short term category, but I just feel that it may take longer to move and change incredibly big systems with a lot of moving parts.

Things that we should think about

It has been suggested to have a sort of consensus around Amethyst or another game engine or libraries like specs as the go-to engine/frameworks for game development in Rust.

They are amazing projects and there are advantages in having a one-engine / library focused ecosystem (from the point of discoverability and community), but I think that diversity is important, not only at your workplace and life, but also in the tooling and framework space. Not all games have the same requirements and not all games need engines, so it’s important to be open because it’s quite challenging to create a one-size fits all solution.

That’s not to say that we shouldn’t promote these as options and acknowledge progress, but we should discuss if the focus of this working group should be more towards enabling such frameworks and engines to exist rather than having an agreement on a library/engine.

Conclusion

Game development is a field that’s already full of many unknowns and risks. The ultimate goal of this Game Development Working Group is to take away as many risks as possible by making Rust for game development a viable and I would hope default option.

I am really excited for the core team to announce more structured processes for spinning up working groups in 2019 so that we can move this group forward!