Doom Air

But can it play Doom?

Well, yes, obviously. Between a Jetson Nano and a MacBook Pro, that shouldn’t be a problem. But how do you play it?

I’ve built a system where Doom runs on a laptop, which is connected to a projector. The laptop also runs a REST API that simulates physical key presses on the keyboard when each endpoint receives a remote request.

A Jetson Nano has a CSI camera pointed at the subject. It is running a convolutional neural network (CNN) model in real time as images are captured by the camera. When a gesture (of the set of gestures the model has been trained on) is detected, an API request is sent to the laptop. This simulates a keypress, and controls the action in Doom. So, for example, point your toy gun at the enemy projected on the wall, and they get blasted.