TinyDefectNet: Highly Compact Deep Neural Network Architecture for High-Throughput Manufacturing Visual Quality Inspection

The promises of the new era for machine learning have motivated different industries to augment and enhance the capabilities and efficiency of their existing workforce with artificial intelligence (AI) applications that automate highly-repetitive and time-consuming processes. In particular, the advances in deep learning have led to promising results in different applications ranging from computer vision tasks [1, 2] such as image classification [1] and video object segmentation [2] to natural language processing tasks [3] such as language translation [4] and question-answering [5]. However, much of success in adopting deep learning in real-world applications have been in newer technology sectors such as e-commerce and social media where the tasks are inherently automation-friendly without human intervention under a controlled environment (e.g., product/content recommendation), with very limited adoption in traditional industrial sectors such as manufacturing where the tasks are currently done manually by human agents in unconstrained environments.

As an example, the visual quality inspection of manufactured components to identify defects and flaws is a critically important task in manufacturing that is very laborious and time-consuming (See Figure 1 for example of a surface defect visual inspection system). As such, automating this process would result in significant cost savings and significantly improved production efficiencies. However, there are a number of key challenges that make it very difficult to adopt deep learning for visual quality inspection in a real-world production setting, including: i) small data problem: the availability of annotated data is limited and thus makes building highly accurate deep learning models a challenge, and ii) highly constrained operational requirements: designing high-performance deep learning models that satisfy very constrained operational requirements with regards to inference speed and size is very challenging especially in high-throughput manufacturing scenarios.

A common practice to the development of custom deep learning models for new industrial applications is to take advantage of off-the-shelf generic models published in research literature such as ResNet [1] and MobileNet [6] architectures and apply transfer learning [7] to learn a new task given the available training data samples. While this approach enables rapid prototyping of models to understand feasibility of the task at hand, such generic off-the-shelf models are not tailored for specific industrial tasks and are often unable to meet operational requirements related to speed and size for real-world industrial deployment. As such, it makes it very challenging to adopt off-the-shelf generic models for tackling visual quality inspection applications under real-world manufacturing scenarios.

A very promising strategy for the creation of highly-customized deep learning models tailored for manufacturing visual quality inspection applications is machine-driven design exploration, where the goal is to automatically identify deep neural network architecture designs based on operational requirements. One path towards machine-driven design exploration is neural architecture search (NAS) [8, 9, 10], where the problem is formulated as a large-scale, high-dimensional search problem and solved using algorithms such as reinforcement learning and evolutionary algorithms. While they have shown promising results in designing new deep neural network architectures [10], they are very computationally intensive and require very large-scale computing resources over long search times [10]. More recently, another path towards machine-driven design exploration is the concept of generative synthesis [11], where the problem is formulated as a constrained optimization problem and the approximate solution is found in an iterative fashion using a generator-inquisitor pair. This approach has been demonstrated to successfully generate deep neural network architectures tailored for different types of tasks across different fields and applications [12, 13, 14].

In this study, we explore the efficacy of machine-driven design exploration for the design of deep neural network architectures tailored for high-throughput manufacturing visual quality inspection. Leveraging this strategy and a set of operational requirements, we introduce TinyDefectNet, a highly compact deep convolutional network architecture design automatically tailored around visual surface quality inspection. Furthermore, we evaluate the capability of the ZenDNN accelerator library for AMD processors in further reducing the run-time latency of the generated TinyDefectNet for high-throughput inspection.

The macro- and micro-architectures of the proposed TinyDefectNet are designed by leveraging the concept of generative synthesis for the purpose of machine-driven design exploration. The concept of generative synthesis revolves around the formulation of the design exploration problem as a constrained optimization problem. More specifically, we wish to find an optimal generator $G^{\star}(\cdot)$ given a set of seeds $S$ which can generate networks architectures $\{\mathcal{N}_{s}|s\in S\}$ that maximize a universal performance function $U$, with constraints defined by a predefined set of operational requirements formulated via an indicator function $\mathbb{1}_{g}(\cdot)$,

Finding $G^{\star}(\cdot)$ in a direct manner is computationally infeasible. As such, we find the approximate solution to $G^{\star}(\cdot)$ through an iterative optimization process, where in each step the previous generator solution $\bar{G}(\cdot)$ is evaluated by an inquisitor $I$ via its newly generated architectures $\mathcal{N}_{s}$, and this evaluation is used to produce a new generator solution. The initiation of this iterative optimization process is conducted based on a prototype, $U$, and $\mathbb{1}_{g}(\cdot)$.

In this study, we define a residual design prototype $\phi$ based on the principles proposed in [1], define $U$ based on [15], and define the indicator function $\mathbb{1}_{g}(\cdot)$ with the following operational constraint: number of floating-point operations (FLOPs) is within 5% of 100M FLOPs to account for high-throughput manufacturing visual inspection scenarios.

The network architecture of TinyDefectNet is demonstrated in Figure 2, and there are two key observations worth highlighting in more detail. First, it can be observed that the micro- and macro-architecture designs of the proposed TinyDefectNet possess heterogeneous, lightweight characteristics that strike a strong balance between representational capacity and model efficiency. Second, it can also be observed that TinyDefectNet has a shallower macro-architecture design to facilitate for low latency, making it well-suited for high-throughput inspection scenarios. These architectural traits highlight the efficacy of leveraging a machine-driven design exploration strategy in the creation of high-performance customized deep neural network architectures tailored around task and operational requirements needed for a given application.

The generated TinyDefectNet architecture is evaluated using the NEU-Det benchmark dataset [16] for surface defect detection, with the performance of the off-the-shelf ResNet-50 architecture [1] also evaluated for comparison purposes. Both architectures were implemented in Keras with a Tensorflow-backend.

The NEU-Det [16] benchmark dataset used in this study is a metallic surface defect dataset comprising of 6 different defects including rolled-in scale (RS), patches (Pa), crazing (Cr), pitted surface (PS), inclusion (In) and scratches (Sc) (See Figure 3 for examples of surface defects). The dataset contains 1,800 grayscale images with equal number of samples from each classes; from 300 images of each class, 240 images are assigned for training and 60 images for testing. The input image size to the evaluated deep learning models is $200\times 200$.

In this study we introduced TinyDefectNet, a highly effective and efficient deep neural network architecture for the high-throughput manufacturing visual quality inspection. We take advantage of generative synthesis for machine-driven design exploration to design micro- and macro-architectures of the proposed TinyDefectNet architecture in an automated manner. Experimental results show that the proposed model is highly efficient which can perform 9$\times$ faster when it runs with ZenDNN accelerator on an AMD CPU compared to an off-the-shelf ResNet-50 architecture in a native Tensorflow environment. This is very desirable especially since the proposed TinyDefectNet performs with 98% accuracy at the same level as ResNet-50 architecture. Explainability-driven performance validation strategy was conducted in this study to ensure correct decision-making behaviour was exhibited by TinyDefectNet to improve trust in its deployment and usage. Future work involves leveraging this machine-driven design exploration strategy for producing high-performing, highly efficient deep neural network architectures for other critical manufacturing tasks such as component localization and defect segmentation.