PyCon_Limerick_2020
March 3, 2020
PyCon Limerick 2020 ” Adding Computer Vision to Robot Operating System (ROS)” This article is about the workshop “Adding Computer Vision to Robot Operating System”
PyCon 2019 Dublin
Image Classification Workshop
Preview!
In this workshop we will explore key technologies to help us tackle some of the worlds big problems right now. When you think about it, we have electric motors, robot arms, batteries, wireless networks etc so why do we not have robots doing a lot more useful things? Especially things we are struggling with such as food production and waste recycling.
The answer is partially due to the fact that it is very difficult for a computer to look through a camera and figure out what it is seeing. Recent developments such as GPU’s and image classifiers have at last given us the break through to allow robots to recognize what they are looking at and finally start doing some useful things outside of the lab.
I will present processing techniques from the self driving car industry and apply them to 2 different scenarios. The general technique will be to use a 2 stage pipeline that will apply object detection with pretrained classifiers and then a second stage where image classification with a trainable classifier is used to make some additional determinations and return annotated video.
The first scenario I call RoboFarm. I will demonstrate a pipeline that can find apples in a stream of video, extract and pass the apples into a second classifier and then determine which apples are ripe and annotate the video.
The second scenario is called RoboRecycling. This will be a classifier to make decisions about which items can be recycled. I will write a blog about this soon.
I will walk you through all the code in a Jupyter Notebook. There are many points in the code where you can make adjustments to see how it effect performance. These points will be annotated and I will focus on these points not the boiler plate code that you can’t change. The code should provide you with more than enough to base yoru own work on.
More specifically I will cover finding and selecting pretrained classifiers. How to use object detection categories and extract what you need form images. I will also cover how to create and train image classifiers including data preparation and augmentation to improve you results. Trouble shooting and how best to filter spurious results and deal with over and under sensitivity will be covered.
You will see how to prepare images to be used for training and here are 2 images that have been cropped and reduced in size specifically to be fed into a classifier for training. These images were taken by me and are hundreds of pictures of apples, ripe and unripe, branches from apples trees and leaves from apple trees. These images can teach a classifier to determine apples from branches and leaves. You can use the same images with your own classifier to compare results.
The plan is to introduce the ideas for several minutes while USB sticks etc get passed around. Then I will walk though the classifiers explaining and showing where you can experiment with your own ideas. About half way through I will stop presenting and switch to workshop mode. You then can either experiment with an existing classifier as presented or make a new one based on skeleton code and provided datasets. You should finish with a good idea of how to make a pipeline, at least one pipeline working and enough blocks of code to make your own in future.
This workshop will use Python 3, Jupyter Notebooks, Keras on top of Tensorflow. You will need your own laptop if you have a GPU you can manage video if not use single images on a CPU.
Please be advised, this is an advanced workshop. I will not cover basic concepts. This could be considered postgraduate level material.
Recent Posts
Capstone-Project
August 25, 2019
Capstone Project Programming A Real Self Driving Car The Capstone project is where after 9 months of online lessons and projects we finally get to
Functional Safety
May 2, 2019
Functional Safety Functional Safety is one of the electives in the course. It is the only project that does not require programming. The Nano-Degree has
Path Planning
April 22, 2019
Path Planning Project In this project there was a lot to accomplish, so I had to make a check list. A year ago this would