Test Automation with Robotic Arm and Computer Vision for a Car Infotainment System

The Need

The Customer, a leading Premium car manufacturer was looking for automating the testing of their infotainment screens, as system testing and regression testing cycles were long as well expensive with sub-optimal level of test coverage and outcome. 

Impacts Delivered
  • Automated robotic testing of infotainment systems.
  • Machine vision in manufacturing.
  • Improved Quality control of touch screen-based products.
  • Achieved greater than 95% test coverage of all test cases that are scheduled for automation.
  • The pre-trained convolutional neural network model has given high accuracy results and hence quality of test outcome. 
  • Real-time Image recognition from a live video stream.
Our Solution
  • Designed computer vision software for Enabling Robot based testing for infotainment systems. 
  • The vision software combines real-time image recognition and text recognition which enables the robot to identify the on-screen elements and navigate through the menu

Vision based solution

  • Industrial grade camera for image acquisition - 5MP, CMOS, 15.0 fps, 2592 x 1944, 1/4", Rolling Shutter
  • Neural network-based image recognition for identifying icons and screens
  • Custom algorithms for measurements and position detection using OpenCV
  • Hand eye coordination of robotic arm with machine vision was implemented
  • Trained a custom neural network OCR on Google Cloud for high accuracy text detection for special fonts.
  • Automated tool for creating dataset for training OCR.

AI Engine

  • The pre-trained convolutional neural network model for high accuracy results for screen/icon recognition than traditional methods.
  • Recognition and localization of elements on the screen.
  • Image recognition can be done in real-time from a live video stream.
  • Text localization and recognition.
  • LSTM and attention-based neural network for text detection

Robot

  • 4 axis DOF robot with customized gripper.
  • Payload 500g 
  • Max. Reach 320mm 
  • Position Repeatability(Control) 0.2 mm 
  • Communication USB / WIFI / Bluetooth
Tools & Technologies
  • 4 axis DOF robot with customized gripper.
  • Pre-trained convolutional neural network model for image recognition
  • LSTM and attention-based neural network for text detection.
  • Real-time on-screen slider movement tracking using image processing.

Robotic Arm and CV


Explore More
Success Stories

Contact
Us

By submitting this form, you consent to be contacted about your request and confirm your agreement to our Privacy Policy.