Unity ML Agents | Visual Chameleons

chameleons_thumbnail.jpg

Start Here First!

In the video, I skip over the setup (which is common to all ML Agents projects) to get right into the content specific to our Visual Chameleon Agents. Unity has excellent documentation on installation and setup and they will always have the most up-to-date instructions.

⚠IMPORTANT!⚠

👉The ENABLE_TENSORFLOW and Adding TensorFlowSharp are no longer necessary.

  • Unity updated ML-Agents to version 0.7.0 on February 26th, 2019 and there’s a small change that affects this tutorial. It now uses Unity Inference Engine. You should be able to skip the ENABLE_TENSORFLOW step.

  • As of August 2nd, 2019, Unity has updated ML-Agents again. If you are using version 0.9, you may need to update trainer_config.yaml to the latest format. (More details here)

  • After training, you will now have a .nn file instead of a .bytes file. This works the same way, you just drag it into your brain, uncheck “Control” in the Academy, and press play to run inference.

Note: If your .nn file doesn’t work, try allowing it to fully complete training (don’t exit early with Ctrl + C). I had some .nn files fail with unhelpful errors when I didn’t let it go all the way.

Installation

Follow the ML-Agents Installation Instructions first. I’m using Unity version 2018.3 in my video. If you use an older version you may have trouble following along in the prefab section because the prefab workflow has changed recently.

I’m using Anaconda on Windows for training. There are specific Windows Installation Instructions for this.

Common Project Setup

Follow the Basic Guide to make sure your ML Agents project is set up. I would recommend attempting to start training the 3D Ball example just to make sure you don’t get any error messages.

Training

We use Curriculum Learning to train our chameleons. If you haven’t done this before, click the link for a helpful documentation page on this.

Chameleon Project Setup

Once you understand how to run an example ML-Agent training session with curriculum, do this:

  1. Download the Code Files and 3d Mesh Files linked below

  2. Import the Scripts folder into your Unity project, it should include ChameleonAcademy.cs, ChameleonArea.cs, and ChameleonAgent.cs

  3. Import the two .fbx 3d models into a new Meshes folder in your Unity project

  4. Add the “ChameleonLearning” section of the trainer_config.yaml to your config/trainer_config.yaml

  5. Add the ChameleonLearning.json curriculum files to config/curricula/chameleons

3d Mesh Assets and Code

The free assets you will need for this project are here:

https://gumroad.com/l/dbYSC 👈 $0+ on Gumroad means you can get them *completely for free* by typing in a 0.

(however, if you want to give extra support, you’ll make me feel warm and fuzzy inside. 😃)

Helpful Additions

Training Command

The command I used to train (in my Anaconda Prompt) with curriculum was:

mlagents-learn config/trainer_config.yaml --curriculum=config/curricula/chameleon/ --run-id=chameleon01 --train

Contact

You can reach out to me at @aktwelve on Twitter or connect with me on LinkedIn.

For future updates

Deep Learning, UnityAdam KellyUnity ML Agents, AIComment