This originally started as a devlog series on the YouTube channel. So many people requested the project for download that we decided to make it available to our patrons.

Check out the devlog videos here:

Collaborative ML-Agents Devlog Series #1 - YouTube

Collaborative ML-Agents Devlog Series #2 - YouTube

Collaborative ML-Agents Devlog Series #3 - YouTube

Collaborative ML-Agents Devlog Series #4 - YouTube

Collaborative ML-Agents Devlog Series #5 - YouTube

About the Project

I wanted to experiment to see if I could get two agents to collaborate with each other. This is an interesting problem because they both have the exact same "brain" (neural network + weights). It would be problematic if both agents had the same exact inputs at the same time, but because they are in different physical positions, they are able to act differently.

At first look, this seems like a fairly simple task that the agents need to solve.

Two agents start at the back of the store
There is a button on the floor in front of the doors that will open them
There is a second button on the floor outside the doors
The doors will only stay open while one or both of the buttons is pressed
The agents are forced to cooperate to get themselves through the door
Maximum award is given and the episode ends when both agents exit the parking lot

overhead view of collaborative ml-agents unity scene

Really, when you think about it, it's kinda tricky. The agents need to learn to move through the store without getting stuck on obstacles. Then they need to understand that only one of the agents needs to stand on the button while the other exits through the open doors. Then rather than just run for the finish line (at the end of the parking lot) the outdoor agent needs to wait on the button until the second agent is outside. The only easy part is both agents making it to the end of the parking lot.

Training works, but it's not very stable or consistent. I got the best end results by letting it train until both agents appear to be completing the challenge, then stopping it early when they're doing particularly well and before they get confused again. This wasn’t a project I was able to just set and walk away, I had to check on it fairly frequently. I'm not really sure how to stabilize it more but if you find a way to do so, I’d love to hear it.

How It Works

The agents use a combination of Ray Perception Sensors (raycasts) that allow them to “see” the obstacles around them as well as the other agent. They also are given positions of themselves, the buttons, the door, and the final destinaton.

The agents are controlled by the neural networks, but the actual movement calculations are done with a basic character controller which includes some extra stuff that’s not used (like jumping).

What’s Included?

If you become a patron on the Immersive Limit Patreon, you can download the complete Unity project directory with all source code and assets. The project uses ML-Agents v2.0.1 (release 17) and includes a readme with basic instructions for how to train. At this time, it’s just a project download and does NOT include a detailed tutorial. Apologies in advance for any extra code or assets that are left in the project, I cleaned it up a fair bit, but may have missed some things. 🙂

Patreon post with downloads: Collaborative ML-Agent Pears [Downloadable Project] | Immersive Limit on Patreon