No webcam found. To use this demo, use a device with a webcam.
Loading mobilenet...

Camera Setup

Highscore 0
NOT TRAINED!

Welcome to...

×

Webcam Snake

Play the snake game, using your webcam and neural networks for image recognition! This program takes use of Googles Tensorflow port for Javascript (tfjs). It is based on the Pacman example from the tutorial section.

How to play:

1. Take one or more images for each direction, by clicking on the areas on the right.
2. Press Train to train the neural net on your images.
3. Press Start to play!

How does it work?

Each frame, the current webcam view goes into to a pretrained neural network model called mobilenet. Mobilenet was trained on millions of images. The output of mobilenet is then given into our neural net (which just trained on your example images). For the final output, our network makes a prediction about which category of your example images, it thinks, it is currently seeing. The snake then moves in the direction linked to the predicted category.

an illustration of a snake receiving inputs and returning outputs

Controls

Framerate:

Take multiple, slightly different pictures of the same camera view to improve the recognition accuracy!