Combining Image, AI & Analytics For Safety and Cost Reasons

The steel teeth on mining excavation equipment like rope shovels and front end loaders are wear items that must be replaced as part of regular maintenance. During normal operation, the connection that affixes a tooth to the shovel or loader bucket occasionally fails, causing tooth detachment. A detached tooth presents a serious hazard if it enters the haulage cycle and makes its way into a crushing unit, where it may become stuck and require the dangerous task of manual removal. Furthermore, wayward teeth cause substantial lost time and production due to jammed crushers and damage to downstream processing equipment. Therefore, it is critical to detect when a shovel tooth goes missing as soon as possible so that preventative action may be taken.

The Problem:

Current methods use equipment mounted cameras and computer vision techniques to generate real-time automated alerts in the event of a missing tooth. While these methods can identify missing teeth with good sensitivity, they produce an unacceptable number of false alarms, which causes equipment operators to ignore the alerts entirely. In some cases, a false positive rate (FPR) of 25% has been observed. Due to the relative infrequency of broken shovel teeth, the false discovery rate (FPR) may be greater than 99%.

There are several challenges associated with the real-time detection of broken shovel teeth. For example, the quality of captured images is compromised by a variety of factors. Dusty operating conditions and variations in lighting, location, and orientation of the shovel bucket, and background composition can make the shovel teeth difficult to distinguish from the material behind it. Furthermore, the biting edge of the shovel bucket is often partially or completely obscured by mined material during operation, which can cause a failure detection algorithm to produce undesirable results. In addition to image quality challenges, the problem itself does not fall neatly into the paradigm of traditional object detection because the target object is an anomalous nuance of the image subject.

The Solution:

A 2-stage approach was selected to address these challenges: 1) row-of-teeth detection and 2) equipment status classification. The location and orientation of the shovel teeth within the images captured by shovel-mounted cameras are highly variable. The purpose of stage 1 in the approach is to isolate relevant information from the image and disregard the rest. This step both normalizes and reduces the size of the images for downstream processing. This technique has been used extensively in real-time facial detection. Stage 2 of the approach performs a binary classification on the detected region from a stage 1. An optimization procedure was applied that aligns sequences by warping them in temporal space such that the distance between signals is minimized. This alignment enables matching of time series based on underlying patterns, irrespective of non-linear temporal variations.

The Result:

This methodology could be used to improve current industry methods, which produce false alarms for 25% of image captures. There are, however, some important points of consideration in the direction of developing a robust, deployable implementation of this methodology.

Net; Net:

New combinations of machine learning methods, even borrowed from different domains, can be leveraged for impressive results. This is not likely without cross-pollination of data science techniques that are likely to come from outside your organization. 

This case study was provided by World Wide Technology (WWT); a leading provider of system integration and global supply chain solutions  https://www.wwt.com/