Robotic Vision Course
Skills Used: OpenCv, Python, Deep Learning, YOLO, Autonomous Systems, Depth Camera, Camera Calibration, Stereo Cameras
This was a class that I took at BYU with many different projects and homeworks that are extremely relevant to areas in perception, control, and autonomous systems.
Homography
For this project, I performed a homography using SIFT for feature detection and FLANN for feature matching. I find features on the base image (on the left) and then find those same features in the target image (on the right). We can use this to add in a different image, like in the video where I replace my monitor screen.
The algorithm runs in real-time to replace the monitor screen with a video.
Facial Recognition GUI
For this project I used the DeepFace repository for facial recognition. This library works by using a base image recognition network, like VGG-128 and then creates an embedding vector based on the face in the image. I then use that embedding vector to determine whether a new face matches the database of embedding vectors or not. I packaged this functionality into a GUI that allows a user to login or register as a new user.
Fish and Oyster Classification
This homework required classification of fish and oyster images. I used YOLO to perform whole image classification for each image. For the fish images, the network classified based on the species of fish. The confusion matrix is on the left, and the network gets 100% detection accuracy. The oyster images are classified (in order of class below) by broken, good, banana, and irregular. The network struggles to classify the shells as well as the fish (the confusion matrix is on the right). Personally, I had a hard time differentiating between the classes of the shells and I think that made it difficult for the network.
Image Registration
This homework required taking a scantron image and reading out the answers selected by the student. My approach was to find outer features in the scantron to determine the orientation. I would then orient the image to be able to lay a grid over it (center image). Finally, I would threshold so only the pencil marks were visible in the image (last image). From there whichever column had the most pixels for a given answer was used as the final answer. I achieved 100% accuracy for this method on 30 scantrons.
Visual Inspection
This project involves an automated conveyor belt running popcorn. I used YOLOv8 to differentiate between good, bad, and acceptable classes. The good class is characterized by big, puffy popcorn. The bad class are unpopped kernels. The acceptable class are half-popped kernels or burnt popcorn. We had to use a low-resolution camera which required significant data collection and post-processing to achieve good results.
popcorn_mixed2.avipopcorn_simple.aviAutonomous Racer
For this project, we had an autonomous RC car with a Nvidia Jetson TX2 and an Intel Real-Sense camera. We used the depth information to determine the best direction to steer the car. We competed with the rest of the class in a race around the BYU engineering building and took 2nd place out of 10 other teams. We required no interventions to fix the car. For turns we thresholded on the blue channel to find the pool noodles and determine how sharply to turn. In the video the car is moving at it's max speed.
autonomous_lap_raw.mp4