Computer Vision for Robots in Production Line
One of the main challenges in the factory digitization is collecting different types of data from existing, aged and heterogeneous machines in the factory floor. Gadgeon’s flagship product provides a way out to this.
Results / Outcomes
- Deep learning was applied to the image and text recognition problem which gave very high accuracy than the traditional methods.
- Custom dataset based trained neural network for text recognition.
- A real time machine vision solution that can process live video feeds and provide directions to the robot in a manufacturing line.
- An image recognition module that can be easily trained with less sample images and less processing power.
- Custom image processing methods for user defined tasks.
Technology used :
- TensorFlow, ROS, OpenCV, Robotic Arm, Convolutional Neural Networks, OCR
- Fast and accurate image recognition and localization feature for robot in testing infotainment systems.
- Text recognition that supports graphical backgrounds, dynamic text, varying fonts, warped text in real time.
- Detecting on-screen movements and changes on the display of DUT.
- Fine tuning the software to the particular user model for high accuracy.
- Providing positional feedback, warnings and device status to the robotic system.
Text Recognition Module
- Data preparation, training, neural network customization
- Convolutional neural networks (CNN), Long Short-Term Memory (LSTM), attention-based text recognition using tools such as OpenCV, Tensorflow
Image Recognition Module
- Built a real time classification program by using off the shelf neural network architecture
- MobileNet, InceptionV3 architecture based CNN models, Tensorflow, OpenCV
Custom Image Processing
- Used for tracking movements, text detection.
- Python, OpenCV, image processing algorithms
Python Based Tools
- Tools were developed for image extraction, dataset creation, RoI plotting and image augmentation
- Python, OpenCV, Ubuntu 16.04, Logitech C920 HD webcam
Integration of Robot Framework For Automation Tasks
- Robot framework, Ubuntu 16.04
Text Recognition and Text Searching
- Detecting text region in an image
- Discards region with straight lines and noise from region of interest
- Accuracy is improved as the words are tightly cropped
- Screen detected as “menu” screen in both background images
- Position of icons does not matter for this screen
- Achieved by adding required image samples in training image set
- Icon recognition and localization
- Icon tracking in dynamic video
- Automatically loads region of interests from screen detection