Data analytics along with deep learning model for dent detection

The Business Context and Problem Description

The challenge was to identify small dents caused by hailstorm on vehicles for an Insurance application.

All small dents had to be detected whereas an actual gap like between the doors or curves as part of panel design should not be classified wrongly.

The body panel on which the dents are there also should also be identified

Accurate and automatic detection of dents without manual intervention enables improved and high-speed insurance settlements without any frauds during hailstorm season

Design / Architecture Considerations for Model Selection

Customized YOLOv3 network to increase accuracy

YOLOv3 with improved FPN (feature pyramid network) was selected as its feature extraction capability doesnot let small features vanish as we go to deeper layers and still can improve speed and accuracy for small dent detection on low resolution images

Approaches like Faster-RCNN was avoided as it depends on semantic detection. The semantic properties of pixel in dents can change due to lighting conditions which the training data maynot capture properly. Also high-resolution images were not available, which also was one of the rule

Various Other techniques were used to improve small feature detection like

Change the anchor size – By explicitly providing the network with information about the size of objects by using relevant anchor size helps in improving small object detection
Splitting image into tiles – As we reduce the size of image to fit the network say 448*448 of Yolo many small features will be lost . To avoid this, we split the images to preserve small objects.
Tap the network early – by detecting till where small objects were going, earlier tap points were added by reducing the layers

With Such techniques and customization of FPN etc the accuracy of detection was improved compared to using normal networks. Apart from this many pre/post processing techniques were applied to the data to suit the problem

Data Preparation

Data pre-processing and Augmentation ideas used

Data was obtained from previously available hailstorm dent images and a custom labeling tool was used to increase speed of labelling using many automation techniques with manual correction facilities

Data Pre-processing

Data pre-processing was kept to minimum so that the features of dent was kept as natural as possible and enhanced where possible
Denoising - Noise is generally considered to be a random variable with zero mean. Consider a noisy pixel, p= p0+n where p0 is the true value of pixel and n is the noise in that pixel. You can take a large number of the same pixels (say N) from different images and compute their average. Ideally, you should get p=p0 since mean of noise is zero. This property was used to remove the noise from the image.
Intensity Correction - Intensity of the image had to be corrected based on the lightning conditions. This was done using Point operators (pixel transforms) and Neighbourhood (area-based) operators

Data Preparation

Data pre-processing and Augmentation ideas used

Data Augmentation

Random Crop - Random cropping prevents a CNN from overfitting to specific features by changing the apparent features in an image. Random crop helps to increase the variety of training dent images and it hselp the system to learn more variety dent features

Horizontal flipping - Horizontal flipping doubles the variation in an image with specific orientations. It helps in detecting dents in both orientations.

Rotation (Random or Fixed) - Rotation helps us to get the view of the object from different dimensions using rotation matrix. Some car images in real world will differ in orientation due to the parking faults or camera calibration issues. This problem reduces as we apply random rotation in the training set.

Random Scaling - Random scaling produces the object images across all scales and this augmentation helps us to detect the dents across various scales.

Perspective projection - Perspective projection helps us to retrieve bird eye view of the image. This will help us to get a more accurate view from the top of the dents.

Shearing - Shearing, is another bounding box transformation, done with the help of the transformation matrix. Some Dents may not be properly visible due to the position of the car or camera. This will produce lot of negatives and this can be solved using shearing

Neural Network Architecture - Workflow

Download this Success Story

Success Stories