The Need
The Customer, a leading Premium car manufacturer was looking for automating the testing of their infotainment screens, as system testing and regression testing cycles were long as well expensive with sub-optimal level of test coverage and outcome.
Impacts Delivered
- Automated robotic testing of infotainment systems.
- Machine vision in manufacturing.
- Improved Quality control of touch screen-based products.
- Achieved greater than 95% test coverage of all test cases that are scheduled for automation.
- The pre-trained convolutional neural network model has given high accuracy results and hence quality of test outcome.
- Real-time Image recognition from a live video stream.
Our Solution
- Designed computer vision software for Enabling Robot based testing for infotainment systems.
- The vision software combines real-time image recognition and text recognition which enables the robot to identify the on-screen elements and navigate through the menu
Vision based solution
- Industrial grade camera for image acquisition - 5MP, CMOS, 15.0 fps, 2592 x 1944, 1/4", Rolling Shutter
- Neural network-based image recognition for identifying icons and screens
- Custom algorithms for measurements and position detection using OpenCV
- Hand eye coordination of robotic arm with machine vision was implemented
- Trained a custom neural network OCR on Google Cloud for high accuracy text detection for special fonts.
- Automated tool for creating dataset for training OCR.
AI Engine
- The pre-trained convolutional neural network model for high accuracy results for screen/icon recognition than traditional methods.
- Recognition and localization of elements on the screen.
- Image recognition can be done in real-time from a live video stream.
- Text localization and recognition.
- LSTM and attention-based neural network for text detection
Robot
- 4 axis DOF robot with customized gripper.
- Payload 500g
- Max. Reach 320mm
- Position Repeatability(Control) 0.2 mm
- Communication USB / WIFI / Bluetooth
Tools & Technologies
- 4 axis DOF robot with customized gripper.
- Pre-trained convolutional neural network model for image recognition
- LSTM and attention-based neural network for text detection.
- Real-time on-screen slider movement tracking using image processing.
