Robotic Vision Course

Skills Used: OpenCv, Python, Deep Learning, YOLO, Autonomous Systems, Depth Camera, Camera Calibration, Stereo Cameras

This was a class that I took at BYU with many different projects and homeworks that are extremely relevant to areas in perception, control, and autonomous systems.

Fish and Oyster Classification

This homework required classification of fish and oyster images. I used YOLO to perform whole image classification for each image. For the fish images, the network classified based on the species of fish. The confusion matrix is on the left, and the network gets 100% detection accuracy. The oyster images are classified (in order of class below) by broken, good, banana, and irregular. The network struggles to classify the shells as well as the fish (the confusion matrix is on the right). Personally, I had a hard time differentiating between the classes of the shells and I think that made it difficult for the network.

Image Registration

This homework required taking a scantron image and reading out the answers selected by the student. My approach was to find outer features in the scantron to determine the orientation. I would then orient the image to be able to lay a grid over it (center image). Finally, I would threshold so only the pencil marks were visible in the image (last image). From there whichever column had the most pixels for a given answer was used as the final answer. I achieved 100% accuracy for this method on 30 scantrons.

Visual Inspection

This project involves an automated conveyor belt running popcorn. I used YOLOv8 to differentiate between good, bad, and acceptable classes. The good class is characterized by big, puffy popcorn. The bad class are unpopped kernels. The acceptable class are half-popped kernels or burnt popcorn. We had to use a low-resolution camera which required significant data collection and post-processing to achieve good results.

Autonomous Racer

For this project, we had an autonomous RC car with a Nvidia Jetson TX2 and an Intel Real-Sense camera. We used the depth information to determine the best direction to steer the car. We competed with the rest of the class in a race around the BYU engineering building and took 2nd place out of 10 other teams. We required no interventions to fix the car. For turns we thresholded on the blue channel to find the pool noodles and determine how sharply to turn. In the video the car is moving at it's max speed.