Since the beginning of the industrial revolution, accurate pipeline system maintenance has been crucial to keeping machine operations sustainable, profitable, and stable. Even though all machine parts and control units evolved from occupying rooms to fitting in our packets, pipeline system maintenance is still one of the most important aspects of keeping machines healthy while running automated manufacturing operations. From cooling processors with water on motherboards to supplying liquefied metal alloy or plastic for injection molding processes, a faulty pipeline system can engender various manufacturing problems while running machine operations, especially for small businesses with limited budgets not enough to cover expensive overhauling costs.
Therefore, establishing an efficient and accurate pipeline diagnostics mechanism conforming to general maintenance regulations can assist technicians in keeping machines durable far more than anticipated and prevent companies from squandering their resources on replacing or repairing high-value machine components due to the omission of proper pipeline diagnostics.
Pipe cracks are one of the most common defects while transferring liquids, especially with differing thermal conditions. During machine operations, mechanical and thermal stress cause minute defects in pipelines due to fatigue. When these small defects accumulate, the outcome mostly results in a varying inside turbulent pressure, which leads to slight form (shape) disfigurations, resulting in gradual deficiency over time due to tension. Furthermore, depending on operation processes and environment, there are lots of possible pipeline defects in addition to cracks, such as corrosion, abrasion, clogged joints due to chemical residue, leaking connection points due to high gas emissions, etc.
Although there are different external pipeline inspection devices utilizing computer vision (camera), magnetic field measurements, and acoustic detection (microphone)[1], these methods cannot be applied interchangeably to different pipeline systems. For instance, a device utilizing object detection with a thermal camera may not be able to detect internal crystals due to high gas permeability in a pipeline system transporting antifreeze to cool components.
Nonetheless, some groundbreaking new methods aim to detect potential pipeline system failures by examining changes in the vibration characteristics. Since accumulating stress due to pipeline defects affects material integrity and structure gradually, these failures can be detected by inspecting fluctuating vibrations as a non-destructive testing and evaluation (NDT&E) mechanism. For example, in recent examinations, researchers applied ground penetrating radar (GPR) to detect cracks in a buried pipe[2] and microwave-based synthetic aperture radar (SAR) to inspect pipeline defects[3].
After perusing recent research papers on pipeline diagnostics based on vibrations, I noticed there are nearly no appliances focusing on collecting data from a mmWave radar module to extract data parameters, detecting potential pipeline defects, and providing real-time detection results with captured images of the deformed pipes for further examination. Therefore, I decided to build a budget-friendly and compact mechanism to diagnose pipeline defects with machine learning and inform the user of the model detection results with captured images of the deformed pipes simultaneously, in the hope of assisting businesses in keeping machines durable and stable by eliminating basic pipeline defects.
To diagnose different pipeline defects, I needed to collect accurate vibration measurements from a pipeline system so as to train my neural network model with notable validity. Therefore, I decided to build a simple pipeline system by utilizing pipes and fittings (adapters) with mediocre thermal conductivity, demonstrating three different pipeline defects in each primary section — color-coded. Since Seeed Studio provides mmWave radar modules with built-in algorithms to detect minute vibration changes to evaluate respiratory rate, heart rate, and sleep status, I decided to utilize a 60GHz mmWave module to extract my data parameters via the mentioned algorithms. Since Arduino Nicla Vision is a ready-to-use and compact edge device with a 2MP color camera and integrated WiFi/BLE connectivity, I decided to use Nicla Vision so as to run my neural network model, capture images of the deformed pipes, and inform the user of the model detection results with the captured pipe images. Due to architecture and library incompatibilities, I connected the mmWave module to Arduino Nano in order to extract and transmit radar data parameters to Nicla Vision via serial communication. Then, I connected four control buttons to Arduino Nano to send commands with the collected mmWave data parameters to Nicla Vision. Also, I added an ILI9341 TFT LCD screen to display the interface menu, including a custom radar indicator.
Since I focused on building a full-fledged AIoT device diagnosing pipeline system defects, I decided to develop a web application from scratch providing various features to the user. Firstly, I employed the web application to obtain the collected mmWave data parameters with the selected label from Nicla Vision via an HTTP GET request, save the received information to a MySQL database table, and display the stored data records on its interface in descending order. Via a single HTML button on the interface, the web application can also generate a pre-formatted CSV file from the stored data records in the database without requiring any additional procedures.
After completing my data set by collecting data from the custom pipeline system I assembled, I built my artificial neural network model (ANN) with Edge Impulse to make predictions on pipeline system defects (classes). Since Edge Impulse is nearly compatible with all microcontrollers and development boards, I had not encountered any issues while uploading and running my model on Nicla Vision. As labels, I utilized the three basic pipeline defects manifested by each main line (color-coded on the system):
Clogged
Cracked
Leakage
After training and testing my neural network model, I deployed and uploaded the model on Nicla Vision as an Arduino library. Therefore, the device is capable of diagnosing pipeline system defects by running the model independently without any additional procedures or latency.
Then, I utilized the web application to obtain the model detection results with captured images of the deformed pipes from Nicla Vision via HTTP POST requests, save the received information to a particular MySQL database table, and display the stored model results with the assigned detection images on the application interface in descending order simultaneously.
Due to the fact that Nicla Vision can only generate raw image buffer (RGB565), this complementing web application executes a Python script to convert the received raw image buffer to a JPG file automatically before saving it to the server. After saving the converted image successfully, the web application adds it to the HTML table on the interface consecutively, allowing the user to inspect all previous model detection results and the assigned deformed pipe images in descending order.
Considering harsh operating conditions, I decided to design a unique PCB after completing the wiring on a breadboard for the prototype and testing my code and neural network model. Since I wanted my PCB design to emanate a unique and powerful water-damage sensation, I decided to design a Dragonite-inspired PCB since it was the first scary water-related Pokémon for me from the anime, despite being a Dragon/Flying type Pokémon. Thanks to the unique orange solder mask and blue silkscreen combination, only provided by PCBWay, this PCB turned out to be my coolest design yet :)
Since I decided to host my web application on LattePanda 3 Delta, I wanted to build a mobile and compact apparatus to display the web application in the field without requiring an additional procedure. To improve the user experience, I utilized a high-quality 8.8" IPS monitor from Elecrow. As explained in the following steps, I designed a two-part case (3D printable) in which I placed the Elecrow IPS monitor.
Lastly, to make the device as sturdy and compact as possible, I designed an emphasizing liquid-themed case with a sliding front cover and a modular camera holder providing a circular snap-fit joint (3D printable) for Nicla Vision and the 60GHz mmWave radar module.
So, this is my project in a nutshell 😃
Click here to inspect code files, STL files, Gerber files, and instructions.
