The Customer was looking for automating the testing of their infotainment screens, as system testing and regression testing cycles were long as well expensive with sub-optimal level of test coverage.
- Designed an automation test suit using technologies such as Robotic arm with camera, computer vision software that recognizes icons & texts for performing the test and then comparing the outcome against the stored expected screens.
- The vision software combines real-time image recognition and text recognition models which enables the test automation system:
- The system consists of Test automation suit running in a PC interfaced with a robotic arm instrumented with the camera, Infotainment unit (OAT) and the visual AI model.
Outcome and Benefits Delivered
- The pre-trained convolutional neural network model has given high accuracy results for screen/icon recognition than traditional methods.
- Recognition and localization of elements on the result screens.
- Image recognition is performed in real-time from a live video stream.
- Text localization and recognition.
- LSTM and attention-based neural network for text detection
- Real-time on-screen slider movement tracking using image processing.
- Achieved high test coverage, high level of regression runs, along with accurate test results.
The Problem Statement
The Customer was looking for automating the testing of their infotainment screens, as system testing and regression testing cycles were long as well expensive with sub-optimal level of test coverage and test results.
The Infotainment screens are configurable according to localization and customer preference. In order to automate the testing of screens to ensure adequate test coverage along with accurate comparison of test results, the customer was looking for:
- An Visual AI driven test automation suit that executes the test cases on different screens, and then compare and produce test results
- Ensure targeted test coverage and regression test cycles
The Solution / System Description
Robotic Arm with Industrial grade camera
- 4 axis DOF robot with customized gripper.
Vision AI based Solution
- Industrial grade camera for image acquisition - 5MP, CMOS, 15.0 fps, 2592 x 1944, 1/4", Rolling Shutter
- Neural network based image recognition for identifying icons and screens
- Custom algorithms for measurements and position detection using OpenCV
- Hand eye coordination of robotic arm with machine vision was implemented
- Trained a custom neural network OCR on Google Cloud for high accuracy text detection for special fonts.
- Automated tool for creating dataset for training OCR.
System / Architecture Description
The System Description