Autonomous Vehicles for Infrastructure Inspection Applications

Power Line Inspection

8.1.1 Introduction

The inspection of power lines is critical for the safe operation of power transmission grids (Miralles, Pouliot, & Montambault, 2014). The monitoring of power lines incorporates two aspects: power line components and their surrounding objects, especially vegetation. The conditions of the components require regular checking to detect faults that are caused, for example, by corrosion and mechanical damage. There is also a need for the regular inspection of vegetation both inside and near the power line corridor. Trees or tree branches that are too close to power lines should be trimmed; vegetation is conductive, and this could lead to electric arcs. To ensure the safe operation of power lines, it is necessary to have a range of empty space without conductive objects around an extra high-voltage (EHV) power line. However, vegetation within the power line corridor will naturally grow after a power transmission grid becomes operational. A discharge may be generated when the distance between the vegetation and the power line is less than the safety threshold, thereby endangering the safe operation of the power transmission grid (Ahmad, Malik, Abdullah, Kamel, & Xia, 2015). Therefore, much research has been focusing on finding a highly efficient method of detecting obstacles along a power line corridor across a large area.

Seven types of data can be used for the inspection of power line corridors (Matikainen et al., 2016):

• 1. Synthetic aperture radar (SAR) images
• 2. Optical satellite images
• 3. Optical aerial images
• 4. Thermal images
• 5. Airborne laser scanner (ALS) data
• 6. Land-based mobile mapping data
• 7. Unmanned aerial vehicle (UAV) images.

SAR image pixels include the radar backscattering intensity, the phase of the backscattering signal, and the range from the sensor to the target. This information can be easily used to map power lines and towers and to monitor disasters that can harm power lines, such as earthquakes and typhoons.

Optical satellite images can be used to monitor vegetation in power line corridors, although the detected results are rather coarse because of the lower spatial and temporal resolution of satellite images compared to aerial visible wavelength images. The advantages of aerial images are their high resolution and availability; thus, they are regularly used for the reconstruction of power line corridors. Thermal images are sometimes used to inspect electrical faults in high-voltage electric utility transmission and distribution lines. Even at short distances, accurate temperature measurements of electrical faults are impossible to quantify. ALS is an active remote sensing technique that can be applied to power line mapping, vegetation mapping, and power line monitoring. ALS-based inspection methods use a laser scanner mounted on an aircraft to scan the power line corridor, obtaining a 3D point cloud to conduct a 3D reconstruction of the power lines and the ground in order to detect obstacles within the corridor.

At present, this method is applied only within a certain range, and for these applications, the point cloud density is the key factor in the 3D reconstruction of power lines. However, the ALS technique has not been popular because of the high costs associated with laser scanning equipment possessing large scanning ranges and high scanning frequencies. Because of its large size and heavy weight, such equipment is usually mounted on a manned helicopter; thus, the cost is high, making it more difficult to arrange inspection work. Although small laser scanner equipment can be mounted on UAVs, the small scanning range and slow scanning frequency of this equipment make it difficult to efficiently obtain sufficiently dense 3D point clouds. With the development of ALS equipment in the direction of miniaturization, a number of new types of ALS have appeared in the market.

A land-based mobile mapping technology is based on the integration of various positioning, navigation, and imaging data collection sensors that constitute a mobile mapping system (MMS), mounted on a kinematic platform, such as a car or a human. Unfortunately, many EHV transmission lines are located in areas that lack a transportation network.

A UAV equipped with a digital camera is a new and convenient method for power line inspection. UAVs have significantly lower operating costs than helicopters; thus, power line corridors across large areas can be inspected by installing a digital camera on a UAV. Different UAV platforms have been applied. Generally, fixed-wing UAVs can fly higher and faster, and they are more suitable for vegetation monitoring and the rough inspection of long power lines, while helicopter and multi-rotor UAVs can be used to acquire detailed pictures by hovering in the air close to the objects (Zhang, Yuan, Li, & Chen, 2017). The automatic measurement of power lines using stereo images is similar to automatic stereo mapping for a specific target in a particular scene. When performing an inspection, a UAV equipped with a lightweight digital camera flies along the power line while acquiring stereo images of the corridor according to a certain overlap. A 3D reconstruction of the power lines and the ground can be performed using photogrammetric methods, and the obstacles can be automatically identified and located by calculating the spatial distance between the power line and the ground. Zhang, Yuan, Fang, and Chen (2017) proposed a dense matching algorithm for the 3D reconstruction of the ground in power line corridors, known as SPMEC; the proposed method has accomplished the automatic 3D reconstruction of objects such as the crowns of trees and roofs. For a 3D reconstruction of power lines, the researchers adopted the traditional manual stereo measurement method, but it restricts the level of automation of inspection by using UAV images and needs further improvement (Zhang, Yuan, Li. & Chen, 2017).

8.1.2 Inspection with a UAV

Inspection with a UAV is an upgrade of automated helicopter inspection, so both concepts have common problems. An evaluation of using a UAV for power line inspection (Jones & Earp, 1996) found this inspection method could be faster than foot patrol and would yield the same or better accuracy than costly helicopter inspection. The system was concluded to be feasible from a technical point of view. The concept was further investigated by Jones (2005). This study proposed a small electrically driven rotorcraft which could pick up energy from power lines. This vehicle would be equipped with gyro-stabilized cameras, navigation and position regulation, a computer for image and other sensor data processing, a communication link, and a system for electric power pickup. Power would be obtained from the power line using a pantograph mechanism. Other research has considered the development of a vision system for power line tracking and image quality assurance. Good power line tracking is important for visual position control and navigation, while image quality is of utmost importance for inspection (Katrasnik, Pernus, & Likar, 2008b).

8.1.2.1 Position Control

Since power lines have to be inspected from a small distance but must under no circumstances get damaged even in strong wind, position control of the UAV is very important but difficult. Because conductors have to be in the field of view of the camera almost all the time, determining the position of the helicopter visually from the images of the conductors seems a possibility (Campoy et al., 2001;

Golightly & Jones, 2005; Jones, Golightly, Roberts, & Usher, 2006). Position control is thus closely related to the automatic tracking of power lines. The helicopter is a very complex, unstable, and nonlinear system with cross couplings. Campoy et al. (2001) chose a linear quadratic Gaussian (LQG) controller for roll and pitch control and a proportional-integral-derivative (PID) controller for yaw control. The controllers were implemented on the basis of the measured dynamic characteristics of the helicopter. Because only the position of the helicopter could be measured, all other required variables were estimated by the Kalman filter. The robustness was tested when the helicopter was in hover by pulling it with a cable. The regulation worked well in the presence of such external disturbances (Katrasnik, Pernus, & Likar, 2008b).

A rotorcraft model and a position control system for a power line inspection robot were presented by Jones, Golightly, Roberts, and Usher (2006). They derived a mathematical model of a ducted-fan rotorcraft with the center of gravity above the aircraft center and used it to develop a control system. The control is achieved by moving a mass, positioned above the center of gravity, left or right. When the mass is moved, the craft tilts in the same direction and accelerates in that direction. The control system is closely linked with the visual tracking of power lines and controls the height and lateral position of the craft to the lines. Lateral position and height are both measured with image analysis (Katrasnik, Pernus, & Likar, 2008b).

8.1.2.2 Automatic Power Line Tracking

The visual tracking of power lines with a UAV is similar to visual tracking with a helicopter. The only major difference is that the UAV can get closer to the lines. The tracking methods are therefore a little different. Jones, Golightly, Roberts, and Usher (2006) developed a simple tracking algorithm that can track a power line with three lines based on the Hough transform. The main purpose of this tracking algorithm is to provide height and lateral displacement of the vehicle to the control system. The method was tested on a scaled model and was proven to be successful even when the background was cluttered. Another study on visual power line tracking (Campoy et al., 2001) utilized a vector-gradient Hough transform for line detection. Only one line was tracked and simultaneously inspected. The position of the helicopter in relation to the line was determined with stereovision (Katrasnik, Pernus, & Likar, 2008b).

8.1.2.3 Obstacle Avoidance

A problem related to robot mobility is obstacle avoidance and path planning. The space around power lines is usually obstacle free; nevertheless, the robot must be able to avoid obstacles in its way, when it is not controlled by a human operator. A computer vision solution to this problem was proposed by Williams, Jones, and Earp (2001). The researchers determined the positions of the obstacles using optical flow. The obtained positions were used in a path planning algorithm based on the distance transform. The algorithms were tested in a laboratory environment using a test rig with a scaled version of a power line. It was established that the principles used were correct, but the method was sensitive to the variations in background, lighting, and perspective. Another important problem was the computing power because image analysis demands were high and rapid obstacle detection was required (Katrasnik, Pernus, & Likar, 2008b).

8.1.2.4 Power Supply

An important characteristic of an inspection vehicle is the duration of its power supply. The longer the craft can stay operational, the more lines can be inspected. Current battery technology does not permit long flights for small electrically driven helicopters. Power lines are an abundant source of energy, but obtaining that power is not trivial. A concept of a power line power pickup device was proposed by Jones (2005). The power would be acquired by touching two lines of different phases with a special

FIGURE 8.1 Proposed pantograph power pickup mechanism (Katrasnik, Pernus, & Likar, 2008b).

pantograph mechanism (see Figure 8.1). For this concept to work, however, line tracking and position control algorithms have to be highly reliable (Katrasnik, Pernus, & Likar, 2008b).

8.1.2.5 Other Problems

A problem that has not been researched thoroughly is automatic power line fault detection. It would be convenient if a robot could automatically detect faults on-site, because it could then inspect them more thoroughly. Automatic fault detection could be done in the ground station after the inspection; this would be easier to implement but would not provide detailed information about the defects. A big difficulty with fault detection is the quality of images taken from the UAV. Because of the distance from the line and constant movement of the craft, the quality of images is usually poor, making automatic fault detection especially demanding.

With UAV inspections, almost every system on the robot (position control, obstacle avoidance, fault detection, and power pickup) depends on the visual tracking of power lines, and this is not very reliable. Although visual power line tracking is successful in the laboratory, the real environment is much more demanding. Contrast between the lines and the background is usually very low. Lighting varies a great deal and depends on unpredictable weather conditions. The UAV is in constant motion and vibrates, so the images acquired are poor quality, and the faults are difficult to detect, even for a human. Unintentional detection of other straight lines on the image, such as other power lines or railroad tracks, could also pose a serious problem (Katrasnik, Pernus, & Likar, 2008a,b).

8.1.3 Climbing Robots

An alternative approach to power line inspection is to use a climbing robot. The robot must be able to climb on the conductor and overcome all the various obstacles on the power lines. The main advantage of this concept is the inspection accuracy. Namely, close proximity to the line and low vibrations increase the quality of image acquisition. Unfortunately, the development of a robot mechanism for overcoming obstacles on the line is extremely difficult. The main research problem with climbing robots is the development of a robot mechanism and a control system for obstacle crossing. The proximity of the conductor also brings problems related to electromagnetic shielding; sensitive electronics and sensors have to be protected from the electric and magnetic fields of the conductor (Katrasnik, Pernus, & Likar, 2008a).

8.1.3.1 Robot Mechanisms and Obstacle Traversing

One of the first operational robot mechanisms for power line inspection was devised by Sawada, Kusumoto, Maikawa, Munakata, and Ishikawa (1991). The robot consisted of a drive, an arc-shaped rail, a guide rail manipulator, and a balancer with a controller. It could travel on slopes of up to 30°. When the robot came across an obstacle, it would unpack its rail and mount it on the conductor on both sides of the obstacle. Then the drive mechanism would release the conductor and travel on the rail to the other side. The robot was able to negotiate towers and other equipment on overhead ground wires. As it did not have proper shielding and mechanisms for overcoming obstacles, it could not travel on phase conductors (Katrasnik, Pernus, & Likar, 2008a).

A more complex robot mechanism presented by Tang, Wang, and Fang (2004) had two arms (front arm and rear arm) and a body. Each arm had 4° of freedom and a gripper with a running wheel. The body also had a running wheel with a gripper. When overcoming obstacles, the robot would release the conductor with the front arm, elongate it over the obstacle, and grasp the conductor on the other side. Then the body would release, and the two arms would move it across the obstacle, where it would grip the conductor again. Finally, the rear arm would move across the obstacle. This robot could overcome all standard obstacles on phase conductors of overhead power lines. However, it could not travel on bundled conductors (Katrasnik, Pernus, & Likar, 2008a).

The robot configuration in Xinglong, Hongguang, Lijin, Mingyang, and Jiping (2006) had two arms and a special gripper, combined with a driving wheel. The specialty of this mechanism was that the gripper could always grasp the conductor when it was in contact with the running wheel. The gripper pressed on the conductor from the left and right side of the wheel. The main disadvantage was that the gripper could not handle large torque, which can easily occur when crossing obstacles. For that reason, a special obstacle crossing strategy that also simplified the design of the robot was designed (see Figure 8.2). When the robot detected an obstacle ahead, it would stop, grasp the conductor with the front arm, and move its body under the front arm to minimize the torque when crossing the obstacle (see Figure 8.2a). Next, the rear arm would lift the running wheel up, and the front arm would rotate the robot around its own axis. Finally, the rear arm would lower the wheel on the conductor (see Figure 8.2b). The same process would then be repeated with the arms’ roles changed. Because of this obstacle traversing strategy, the robot arms needed only two degrees of freedom; in addition, the torques in the joints and on the conductor were small and, consequently, the motors did not need to be as powerful and heavy (Katrasnik, Pernus, & Likar, 2008a).

FIGURE 8.2 (a and b) Obstacle traversing strategy (Xinglong. Hongguang, Lijin, Mingyang, & Jiping, 2006).

8.1.3.2 Robot Control System

The main purpose of the robot control system is to navigate the robot over obstacles on the line. One of the first robot control algorithms for power line inspection was the one described above, created by Sawada, Kusumoto, Maikawa, Munakata, and Ishikawa (1991). A more complex control system using a distributed expert system divided between the robot and the ground station, also mentioned previously, was designed by Tang, Wang, and Fang (2004). The latter robot control system ran on an embedded PC/104 based computer, connected to the ground station with a wireless data link and a separate image transmission channel. The robot expert system consisted of an inference engine, knowledge base, static database, external information input module, and decision-making module. The inference engine would decide what commands to execute on the basis of sensor information and information in the static database. Sensors provided information about the current position of the robot and the obstacles around it, while the static database contained data on towers and other obstacles on the line. The robot expert system would plan the path of the robot arms so that the robot would overcome obstacles successfully. The ground station was used for monitoring and guiding the robot, as well as for detecting faults on the power line from the images sent by the robot. Similar distributed expert system designs were proposed by Ludan, Hongguang, Lijin, and Mingyang (2006) and are described above.

8.1.3.3 Obstacle Detection and Recognition

Obstacle detection is usually done with a proximity sensor; the method is simple yet effective, but the detection of the obstacle is usually not enough to overcome it. In most cases, the type of the obstacle has to be known before it is overcome. Zhang et al. (2006) presented a computer vision method for obstacle recognition and distance measurement. The method determines the obstacle types from the shapes on the image. An ellipse represents a suspension insulator string with two circles left and to the right of the conductor a strain insulator string. After the obstacle is recognized, its position is located with stereovision. The method was tested on a real power line, and an accuracy of 7% or better was reported.

Another important problem associated with visual obstacle detection and recognition is the elimination of motion blur from the captured images (Fu et ah, 2006). Although a climbing robot is fixed on the conductor, it swings in wind and when traveling along the line (Katrasnik, Pernus, & Likar, 2008a,b).

8.1.3.4 Power Supply

Power lines could provide the inspection robot with energy for its operation. Energy could be extracted from the magnetic field of the line. This concept was proposed by Peungsungwal, Pungsiri, Chamnongthai, and Okuda (2001). A magnetic iron core was placed around the conductor. Current induced in the secondary coil around the core was measured at different numbers of secondary windings. The researchers found the current reached its maximum value at a certain number of secondary windings, and the power transferred to the secondary coil increased with the current of the power line (Katrasnik, Pernus, & Likar, 2008a).

8.1.4 Climbing-Flying Robot

The advantageous features of flying and climbing robots can be combined (see Figure 8.3). The proposed robot would combine a helicopter for flying over the obstacles and a special drive mechanism for traveling on the conductor. During inspection, the robot would travel on the conductor up to an obstacle. Then it would fly off the conductor over the obstacle, land on the other side, and continue traveling along the conductor. Traveling on the conductor would be automated, while flying over the obstacles would likely have to be done manually. Some of the problems to be solved are similar to those described in previous sections, for instance, the power pickup system, obstacle detection and recognition, and drive mechanism for traveling on the conductor (Katrasnik, Pernus, & Likar, 2008b).

FIGURE 8.3 Proposed robot: (a) Illustration of the proposed climbing-flying robot and its components; (b) Sketch of the robot from the front, showing the rails for easier landing on conductors and equipment placement for stability (Katrasnik, Pernus, & Likar, 2008b).

8.1.4.1 Robot Design

Designing a climbing-flying robot is much more difficult than designing a flying robot, although not as difficult as a climbing robot. A design for a climbing-flying robot must consider the weight limitations of the helicopter, which are much stricter than for the flying robot. The addition of the drive mechanism and the electromagnetic shielding significantly increases the weight of the robot.

Another major problem is the weight distribution in the robot. To achieve a good degree of stability on the power line, the center of gravity of the robot must be below the conductor. This conflicts with the design of the helicopter, where the majority of the weight is placed directly below the rotor to achieve good maneuverability. The parts of the robot must therefore be carefully positioned to achieve the optimal position of the center of gravity. A rough distribution of robot parts inside the robot is proposed in Figure 8.3.

Setting the weight limitation and distribution problems aside, the most important problem of the climbing-flying robot is the design of individual systems, i.e., the helicopter, drive mechanism, visual inspection system, power pickup device, and communication system. The design of these systems is discussed in the following subsections (Katrasnik, Pernus, & Likar, 2008a).

8.1.4.2 Helicopter

When choosing the rotor configuration of the helicopter for the climbing-flying robot, we have three choices: Sikorsky configuration, tandem rotor configuration, and coaxial configuration.

The most common is the Sikorsky configuration. Ninety percent of all the helicopters in the world are made in this configuration. It is simple to produce and has good maneuverability and sufficient lift.

The tandem rotor configuration has worse maneuverability but produces more lift, as no power is needed to balance the main rotor torque. This configuration is also more difficult to make and maintain, as it has more moving parts and a more complex design.

The coaxial rotor configuration is more expensive to build and maintain, but it requires less space, while producing the same amount of lift as the other two configurations. This results in a smaller and more maneuverable helicopter for the same payload limitations. The coaxial configuration has better maneuverability than the Sikorsky configuration but is more expensive and has more frequent maintenance. For the climbing-flying robot, the coaxial configuration is, therefore, the best choice (Katrasnik, Pernus, & Likar, 2008a).

8.1.4.3 Drive Mechanism

The drive mechanism consists of the front and the rear drive mechanism. Each of the two drive mechanisms has two wheels (Figure 8.4). The upper wheel is the drive wheel, while the lower wheel provides

FIGURE 8.4 Part of the proposed drive mechanism. After the robot lands on the conductor, the lower wheels grasp the conductor from the sides (Katrasnik, Pernus, & Likar, 2008a).

stability for the robot on the power line. The drive wheel is connected to an electrical motor with a drive chain, whereas the lower support wheel runs freely. The wheels are made of aluminum, and the conductor contact surfaces of the wheels are covered with conductive rubber to increase traction, damp vibrations, and keep the robot on the same electric potential as the conductor. The support wheel will grasp the conductor. At landing, the robot will sit down on the drive wheels with the help of special rails (Figures 8.3b and 8.4). After the robot is positioned on the drive wheels, the support wheels are moved into position with servomotors. The contact force with the conductor is applied with springs. Before takeoff, the support wheels are retracted, and the robot is free to lift off the conductor (Katrasnik, Pernus, & Likar, 2008a).

8.1.4.4 Power Pickup Device

The power pickup device consists of two parts, the toroidal core and the clasping mechanism. The toroidal core (Figure 8.5) is made from a ferromagnetic iron core and is split into two halves. On each half is a winding that transforms the energy of the magnetic field in the iron core to electrical energy, which is then further treated with a special converter circuit to obtain a useable voltage to power the systems on board the robot. The converter must be capable of handling a large range of input voltages, as the voltage in the winding changes linearly with the power line current. The clasping mechanism takes care of

FIGURE 8.5 Power pickup device (Katrasnik, Pernus, & Likar, 2008a).

FIGURE 8.6 Power pickup device with clasping mechanism: (a) Open power pickup device; (b) Closed power pickup device (Katrasnik, Pernus, & Likar, 2008a).

the closing and opening of the toroidal core after landing and before takeoff (Figure 8.6). It is extremely important that the clasping mechanism closes the two halves of the toroidal core as closely together as possible, as even a small slit between the two halves significantly affects the efficiency of the power pickup device. Its precision is, therefore, of great importance.

A very important parameter of the power pickup device is its power-to-weight ratio. It is crucial that a power pickup device is as light as possible, as weight is limited on the robot. The power produced by the power pickup device depends on the power line current and on the geometry of the toroidal core. A preliminary analysis showed that such a power pickup device is feasible, as a power-to-weight ratio of more than 250 W/kg can be achieved for a relatively small 400 A power line current (Katrasnik, 2007; Katrasnik, Pernus, & Likar, 2008a).

8.1.4.5 Communication

To guide the robot over the obstacles effectively, the operator needs real-time visual feedback on the robot’s surroundings, while the guiding data have to be sent with minimal latency. For these reasons, a reliable high-bandwidth wireless data link with very low latency is required. Another requirement, which conflicts with the high-bandwidth requirement, is long communication distance that should reach at least 5 km for an efficient operation (Katrasnik. Pernus, & Likar, 2008b).

8.1.4.6 Visual Inspection System

Visual inspection of the conductor consists of visual inspection of the power line using infrared (IR) and ultraviolet cameras at the front of the robot and visual inspection using three-line scan cameras.

Visual inspection of the power line includes looking for defects on the conductor, insulator, supporting tower, and other equipment. Visual detection of defects on all these systems is not very reliable; the conductor is more accurately inspected with the second visual inspection system, while defects on other equipment are detected with IR and ultraviolet cameras, a part of the front visual inspection system. IR cameras can easily detect overheating of any part of the power line equipment. Ultraviolet cameras detect coronas, usually a sign of a defect, fairly straightforwardly.

The second visual inspection system performs a more accurate visual inspection of the conductor. This visual inspection system consists of three-line scan cameras placed around the conductor 120° apart (Figure 8.7b). The conductor is illuminated with two LED-based lights for each camera (Figure 8.7a). The lights are placed on both sides of the cameras. This lighting configuration provides diffuse illumination of the conductor to enable efficient visual defect detection. An incremental encoder on one of the wheels of the drive mechanism is used to trigger the line scan cameras (Katrasnik, Pernus, & Likar, 2008a).

FIGURE 8.7 Conductor visual inspection system: (a) Illumination, camera, and conductor configuration for one camera; (b) Camera configuration around the conductor (Katrasnik, Pernus, & Likar, 2008a).