Doom Bots in TensorFlow

ViZDoom is an AI research platform that allows you to train bots to play Doom, the classic first-person shooter originally released by id Software in 1993. ViZDoom interacts with the game using ZDoom, an open source Doom engine.

ViZDoom comes packaged with a whole host of example bots written in C++, Java, Lua, and Python. In many cases the models themselves rely on a variety of underlying Deep Learning libraries such as TensorFlow and Theano.

With ViZDoom, bots are trained against scenarios. ViZDoom includes several scenarios in its source distribution. Scenarios define, among other things, a Doom map, controls available to the bot (such as turn left, attack, etc..), player mode and skill level.

In this blog post I'll walk through setting up ViZDoom and TensorFlow and train bots to take on demons in Doom.

ViZDoom Up and Running

The following was run on a fresh install of Ubuntu Desktop 14.04.4. Normally I'd used the server distribution but I want to see the bot play the game after it's finished training.

I'll be using TensorFlow, Google's Deep Learning Framework, for bot training. For the sake of accessibility I'll describe how to run everything using a CPU but if you want to train using a GPU then please see my TensorFlow on a GTX 1080 blog post for further, GPU-centric installation instructions. In many circumstances, TensorFlow will train one to two orders of magnitude quicker on a GPU compared to a regular desktop CPU.

The machine I'm working on has an Intel Core i5 4670K clocked at 3.4 GHz, 32 GB of DDR3 RAM, a SanDisk SDSSDHII960G 960 GB SSD drive and an Nvidia GeForce GTX 1080 graphics card.

First, I'll install various packaged dependences.

$ sudo apt update
$ sudo apt install \
      build-essential \
      cmake \
      gfortran \
      git \
      libatlas-base-dev \
      libblas-dev \
      libboost-all-dev \
      libbz2-dev \
      libfluidsynth-dev \
      libgme-dev \
      libgtk2.0-dev \
      libjpeg-dev \
      liblapack-dev \
      liblua5.1-dev \
      libopenal-dev \
      libsdl2-dev \
      libwildmidi-dev  \
      nasm \
      openjdk-7-jdk \
      python-dev \
      python-pip \
      python-virtualenv \
      tar \
      timidity \
      zlib1g-dev

I'll then set Java's home folder.

$ export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64

ViZDoom is written in Python so I will create a virtual environment and activate it.

$ virtualenv vz
$ source vz/bin/activate

I'll then install five Python-based dependences. Among these is the wheel containing the CPU-driven distribution of TensorFlow. There is a separate distribution if you want to use the CUDA-backed, GPU-accelerated version.

$ pip install \
      cython \
      numpy
$ pip install \
      scikit-image \
      https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.0.0-cp27-none-linux_x86_64.whl \
      tqdm

I'll then clone the ViZDoom git repository, build the Python module and install it.

$ git clone https://github.com/mwydmuch/ViZDoom.git
$ cd ViZDoom

$ cmake -DCMAKE_BUILD_TYPE=Release \
        -DBUILD_PYTHON=ON \
        -DBUILD_JAVA=ON \
        -DBUILD_LUA=ON
$ make

$ pip install .

TensorFlow-based models are generally agnostic to CPUs and GPUs but if you do train with the CPU-driven TensorFlow distribution you may see warnings that the wheel used was not compiled with all optimisations for your CPU. If you're running a 4th-generation or greater Intel Core i5 CPU this will include a lack of support for SSE3, SSE4.1, SSE4.2, AVX and FMA instructions.

Training a Doom Bot

There is an example model "learning_tensorflow.py" located in the examples/python folder that will load a scenario where the bot is in an empty room with one enemy at the other end of the room. The bot has the ability to move left and right and attack. The bot is scored based on how quickly it can kill its opponent before the match runs out of time.

Here is the configuration file for this scenario:

$ cat scenarios/simpler_basic.cfg

doom_scenario_path = simpler_basic.wad

# Rewards
living_reward = -1

# Rendering options
screen_resolution = RES_640X480
screen_format = GRAY8

render_hud = true
render_crosshair = false
render_weapon = true
render_decals = false
render_particles = false

# make episodes start after 20 tics (after unholstering the gun)
episode_start_time = 14

# make episodes finish after 300 actions (tics)
episode_timeout = 300

# Available buttons
available_buttons =
    {
        MOVE_LEFT
        MOVE_RIGHT
        ATTACK
    }

The following took 14 minutes to train the bot.

$ cd examples/python
$ python learning_tensorflow.py

Once training was complete ZDoom, the underlying open-source Doom engine, launched and I could see the bot take on the enemy across 10 different games.

The bot is given 50 rounds on ammunition for each round played and is given 300 game actions to kill the single demon. The demon isn't fighting back in this scenario nor does it move very much. Unfortunately, even with such a great setup it's rare for the bot to kill the daemon in more than 20% of the games played. Below are the scores from ten games played.

Total score:  -404.0
Total score:  -404.0
Total score:  -404.0
Total score:  -404.0
Total score:  -404.0
Total score:  -404.0
Total score:  56.0
Total score:  -404.0
Total score:  -1.0
Total score:  -404.0

Defend the Center

Among the other scenarios that ship with ViZDoom is a "Defend the Center" scenario in which a bot is stationary in the middle of an arena and only allowed to turn left and right and attack. There are a series of enemies that will steadily approach the bot giving it time to kill them before they strike. Michał Kempka, one of the principle developers of ViZDoom, uploaded this video of this scenario to YouTube.

I modified examples/python/learning_tensorflow.py with the following five lines of code before training the bot using the GPU-accelerated distribution of TensorFlow on my GTX 1080. The training took 41 minutes to complete.

learning_rate = 0.002
epochs = 5
learning_steps_per_epoch = 4000
test_episodes_per_epoch = 1000
config_file_path = "../../scenarios/defend_the_center.cfg"

The bot gets a point for every enemy it kills and it looses a point if it dies. It's given 26 rounds of ammunition for each run. I've noticed the bot starts out very trigger-happy and works its way through its magazine a bit too quickly missing a number of demons in the process. It could be worth giving points for shots on target.

After ten runs the bot managed to kill an average of 5.9 demons per round before being killed.

Total score:  5.0
Total score:  3.0
Total score:  5.0
Total score:  5.0
Total score:  4.0
Total score:  5.0
Total score:  6.0
Total score:  5.0
Total score:  7.0
Total score:  4.0

Thank you for taking the time to read this post. I offer both consulting and hands-on development services to clients in North America and Europe. If you'd like to discuss how my offerings can help your business please contact me via LinkedIn.