Nhan Van Nguyen defended his PhD degree at the University of Tromsø (UiT), Department of Physics and Technology, on December 3rd 2019.

At eSmart Systems, Nhan Van Nguyen holds the position of Senior Data Scientist.
The title of his PhD thesis is: «Advancing Deep Learning for Automatic Autonomous Vision-based Power Line Inspection». After having successfully defended his PhD, Nhan was congratulated by the faculty, family and representatives from eSmart Systems.

Background

Nhan Van Nguyen has a B.Eng. degree in Computer Science and Engineering from the Ho Chi Minh City University of Technology, Vietnam, and a M.S. degree in Computer Science from Østfold University College, Norway. Nhan specializes in deep learning for computer vision. 

Assessment Committee

The faculty appointed the following assessment committee:

• Associate Professor Atsuto Maki, Division of Robotics, Perception and Learning, KTH, Royal Institute of Technology, Sweden 
• Professor Kjersti Engan, Department of Electrical Engineering and Computer Science, University of Stavanger, Norway
• Associate Professor Stian Normann Anfinsen, Department of Physics and Technology, UiT, Norway

The candidate’s main supervisor has been Professor Robert Jenssen (UiT).  Chair of the defence was Professor Rune Graversen (UiT). 

The trial lecture took place on December 3rd, 2019 at 10:15, and the public defence of the thesis took place later the same day, at 12:15.

Abstract 

“Electricity is fundamental to the ability to function of almost all modern-day societies. To maintain the reliability, availability, and sustainability of electricity supply, electric utilities are usually required to perform visual inspections on their electrical grids regularly. These inspections have been typically carried out using a combination of airborne surveys via low- ying helicopters and eld surveys via foot patrol and tower climb. The primary purpose of these visual inspections is to plan for necessary repair or replacement works before any major damage that may lead to a power outage. These traditional inspection methods are not only slow and expensive but also potentially dangerous. In the past few years, numerous eorts have been made to automate these visual inspections. However, due to the high accuracy requirements of the task and its unique challenges, automatic vision-based inspection has not yet been widely adopted in this eld.

In this dissertation, we exploit recent advances in Deep Learning (DL), especially deep Convolutional Neural Networks (CNNs), and Unmanned Aerial Vehicle (UAV) technologies for facilitating automatic autonomous vision-based power line inspection. We propose a novel automatic autonomous vision-based power line inspection concept that uses UAV inspection as the main inspection method, optical images as the primary data source, and deep learning as the backbone of data analysis.

Next, we conduct an extensive literature review on automatic vision-based power line inspection. Based on that, we identify the possibilities and six main challenges of DL vision-based UAV inspection: (i) the lack of training data; (ii) class imbalance; (iii) the detection of small power line components and defects; (iv) the detection of power lines in cluttered backgrounds; (v) the detection of previously unseen power line components and defects; and (vi) the lack of metrics for evaluating inspection performance. We address the first three challenges by creating four medium-sized datasets for training component detection and classication models, by applying a series of eective data augmentation techniques to balance out the imbalanced classes, and by utilizing multistage component detection and classication based on Single Shot multibox Detector (SSD) and deep Residual Networks (ResNets) to detect small power line components and defects.

Then, we address the fourth challenge of DL vision-based UAV inspection, which is to detect power lines in cluttered backgrounds, by proposing LS-Net, a fast single-shot line-segment detector, for then to apply it to power line detection. The LS-Net is by design fully convolutional and consists of three modules: (i) a fully convolutional feature extractor; (ii) a classier; and (iii) a line segment regressor. With a customized version of the VGG-16 network as the backbone, the proposed LS-Net outperforms the existing state-of-the-art DL-based power line detection approaches by a considerable margin and can detect power lines in near real-time.

Finally, we propose few-shot learning as a potential solution to the fth challenge of DL vision-based UAV inspection, which is to detect previously unseen power line components and defects. To pave the way for addressing the challenge, we propose an innovative approach for advancing the state of the art of few-shot learning. Specically, we propose a novel dissimilarity measure in terms of the Squared root of the Euclidean distance and the Norm distance (SEN) combined to address the existing issues of the traditional Euclidean distance in high dimensional spaces. We extend the powerful Prototypical Network (PN) by replacing the Euclidean distance by our proposed SEN dissimilarity measure, which we refer to as SEN PN. With minimal modications, the SEN PN outperforms the original PN by a considerable margin and demonstrates good performance on the miniImageNet dataset with no additional parameters as well as almost no additional computational overhead. The sixth challenge, which is to address the lack of metrics for evaluating inspection performance, is left for future work.

The contribution of this dissertation is threefold:
First, it proposes a novel automatic autonomous vision-based power line inspection concept that uses UAV inspection as the main inspection method, optical images as the primary data source, and deep learning as the backbone of data analysis.
Second, it provides an overview of the possibilities and challenges of deep learning in automatic autonomous vision-based power line inspection.
Third, it proposes approaches for addressing the identied challenges, for advancing deep learning, and for paving the way for realizing fully automatic autonomous vision-based power line inspection.”