I made a robot arm throw the perfect game of cornhole, thanks to a Xavier
A webcam is attached to the end of the arm. It uses OpenCV to detect a ring around the hole in the board and calculates the angular position of the hole relative to the camera. Since I know the position and angle of the camera, it can calculate the hole position in space (roughly) and the throw the arm needs to do. It tells the arm over the network (ethernetKRL), and the arm makes the throw! This is a KR20 from Kuka Robotics.
Code is available on Github
3 Likes
It is rather difficult to get a computer arm to work so well with nothing more than a camera. Was the camera stereo, or monocular? Looked monocular. What I thought was interesting (aside from the motorcycle I saw around 4:22) was the use as a CNC machine when cutting the wooden game board. I have a suggestion if you are working on something new which evolves from this: A stereo camera, and an ability to scan an object (perhaps by spinning a cage the object is in), while milling a replica of the object in real time using only the photographic input. For example, a wooden yard ornament like a swan which the camera can view from whatever angle the computer wants, or perhaps a chess piece (or that motorcycle!).
1 Like
@linuxdev It was just a monocular webcam. Something cheap I found on amazon. The lens didn’t distort the view too badly, so I could get a rough horizontal and vertical angle off of centre from the camera. Not ultra accurate, but very simple. A stereo camera would definitely have improved it.
As far as the live object milling, i really like that idea. Definitely going on the list for a future project
Love your video! Looks like a super fun project. Any next plans for your Kuka KR20?
Did the Jetson do the trajectory planning/calculation, in addition to the OpenCV processing?
@dusty_nv Somewhat. The arm always makes the same throw motion. The throw power/velocity and planar angle change with each throw (Jetson calculated). It would be awesome to train a NN to make the throws. Getting training data would take forever though