Graph-based motor primitive generation framework

Abstract

Unmanned aerial vehicles (UAVs) have many potential applications, such as delivery, leisure, and surveillance. To enable these applications, making the UAVs fly autonomously is the key issue, and requires defining UAV motor primitives. Diverse attempts have been made to automatically generate motor primitives for UAVs and robots. However, given that UAVs usually do not fly as expected because of external environmental factors, a novel approach for UAVs needs to be designed. This paper proposes a demonstration-based method that generates a graph-based motor primitive. In the experiment, an AR.Drone 2.0 was utilized. By controlling the AR.Drone 2.0, four motor primitives are generated and combined as a graph-based motor primitive. The generated motor primitives can be performed by a planner or a learner, such as a hierarchical task network or Q-learning. By defining the executable conditions of the motor primitives based on measured properties, the movements of the graph-based motor primitive can be chosen depending on changes in the indoor environment.

Background

Nowadays, unmanned aerial vehicles (UAVs) have diverse types of goals, as their performance and techniques have improved. Because making UAVs fly autonomously requires motor primitives, defining and generating motor primitives is the key approach to autonomous UAVs. Especially, the framework design of UAVs are further researched [1, 2].

Usually, motor primitives are defined and generated linearly [3, 4]. Therefore, while executing motor primitives, changes in the environment are not considered, which can prevent a UAV from achieving its goal. For example, motor primitives are defined by dividing the movements learned through demonstration-based learning, and are then executed by a planner [4].

One motor-primitive generation approach is based on a graph [5–8]. Graph-based motor primitives have flexibility in dynamic environments because they consider the state of the environment while executing the motor primitives. A method that generates UAV motor primitives using demonstration-based learning [9] is suggested. The process of generating motor primitives is divided into three stages: Operation collection stage, time adjustment stage, and motor-primitive generation stage. The motor primitives, based on measured yaw, pitches and rolls, are divided into sub-motor primitives considering pinpoints, where pinpoints are predefined spots where the UAV should arrive.

When UAVs fly, they are affected by diverse types of external factors, such as wind, building structures, etc. Therefore, motor primitives are not usually performed as well as expected, which makes it difficult to execute multiple motor primitives consecutively. For example, two motor primitives are planned to be executed, one right after the other. If the first executed motor primitive is not performed as defined, the next motor primitive cannot be executed, because the preconditions have changed. Therefore, devising a novel approach to improve graph-based motor primitives is required.

This paper proposes a method that generates graph-based UAV motor primitives using demonstration-based learning. By connecting multiple movements and executing one of the connected movements, UAVs can cope with changing environments. The proposed method can be applied not only to UAVs, but also to robots and virtual characters.

This paper proceeds as follows. “Related work” introduces the approaches used to make UAVs fly autonomously. “Graph-based motor primitive generation process” proposes a graph-based motor-primitive generation framework. “Experiments” shows the performance of the generated graph-based motor primitives. “Conclusion” concludes the paper.

Related work

In the beginning, UAVs were designed for military, but recently low-cost UAVs have been developed and sold to consumers. For example, even though the AR.Drone 2.0 contains two cameras and an infrared sensor, it is sold at low cost [10]. In addition, communication protocols are also defined for developers to control the AR.Drone 2.0 by programming. The AR.Drone 2.0 provides algorithms for inertial sensor correction and altitude estimation to maintain an accurate and robust state. An algorithm to display captured images is also contained. Finally, the AR.Drone 2.0 is connected and controlled through Wi-Fi. Diverse types of control modes based on finite-state machines are utilized.

A variety of studies has been done on controlling a UAV automatically. For example, markers attached to walls are recognized by cameras on the UAV, and then the UAV is controlled autonomously by utilizing the recognized markers [11]. For autonomous flight, direction control and altitude control are required. By recognizing the markers by UAV cameras, the distances between markers and UAVs are measured. When the UAV comes within range of one of the markers, it responds by performing the signal corresponding to the marker.

Other research solves UAV path-finding problems: one conducts useless lines when vanishing points are revealed, and the other recognizes uncrossed lines as crossed lines [12]. A vanishing point is defined as a point where the path-propagation direction is and where the outer lines are crossed. An algorithm to improve the difference when vanishing points are calculated was proposed. UAVs fly autonomously based on the algorithm.

In detail, all captured images are converted to black/white images to detect the border values of the captured images. By utilizing Canny Edge Detection, the border values are obtained and, by utilizing a Probabilistic Hough Transform, straight lines are detected. Vanishing points are found by excluding the errors of the uncrossed straight lines when calculating vanishing points. The improved algorithm was applied to the AR.Drone in an indoor environment and allowed the AR.Drone to fly autonomously.

In addition, research on controlling multiple UAVs for indoor-based group flight has been conducted [13]. Controlling multiple UAVs concurrently not only requires a ground station that controls the UAVs, but also a motion-capture data server that obtains the indoor locations of the UAVs. The ground station controls the UAVs by utilizing a qualitative control algorithm. Motion-capture devices take pictures of the markers attached to the UAVs and then the motion-capture data server recognizes their locations based on the taken pictures.

The shortest path is set to make UAVs fly autonomously in an indoor environment [14]. Given that GPS (global positioning system) signals cannot be received in an indoor environment, an algorithm to generate 3D maps utilizing a depth camera is proposed. The UAV’s current location is recognized based on the 2D images captured by UAV cameras and the generated 3D map of the UAV. An algorithm to reduce the complexity of 3D maps is proposed to calculate the UAV path for avoiding obstacles.

UAV locations are estimated based on crowd-sourced signals [15]. A local Wi-Fi fingerprint is collected and automatically updated through multiple users’ smart phones. The UAV locations are calculated utilizing the strength of the measured UAV Wi-Fi and the updated Wi-Fi fingerprint. In addition, the UAV’s path is estimated by utilizing the sensor values of accelerators and directions.

Traditional control research of autonomous UAVs focuses on recognizing the current location as described above. However, a novel approach is required, not to control UAVs according to a predefined path, but to make UAVs fly autonomously, depending on changes in the dynamic environment. This paper presents a method that applies and improves a motor-primitive generation technique for humanoid robots.

Graph-based motor primitive generation process

UAV motor-primitive expression

In this paper, motor primitives are defined as follows. In Eq. (1), the ith observed and measured motor primitive m _i is defined in terms of its consecutive movements with execution time t _i '. We assume that all motor primitives have the same number of movements. Therefore, p is the total number of measured motor primitives, and q is the number of movements in a motor primitive.

$$m_{i} = \, \{ m_{i,1} , \, m_{i,2} , \ldots , \, m_{i,j} , \ldots , \, m_{i,q} ,\,\,t_{i}^{\prime } \} ,$$

(1)

where the jth movement of the ith motor primitive is m _i,j . The movement m _i,j is defined as shown in Eq. (2). Each movement is expressed by x properties and y commands. p _i,j,x and c _i,j,y is the xth property and yth command. A property can be not only UAV values, but also any external values. The motor primitive m _i,j is executed by considering the properties and then sending commands at execution time t' _i .

$$m_{i,j} = \, \left[ {p_{i,j,1} , \, p_{i,j,2} , \ldots , \, p_{i,j,x} , \, c_{i,j,1} , \, c_{i,j,2} , \ldots , \, c_{i,j,y} } \right]$$

(2)

The movement properties are utilized as the conditions required to send movement commands. When the difference between the movement properties and the UAV properties is smaller than α, the command is sent by the UAV. A graph-based motor primitive G is defined as shown in Eq. (3).

$$G \, = \, \left\{ {J, \, M} \right\},$$

(3)

where J is the joint-ordered set and M is the movement-ordered set. j _i is the joint of the ith movements and an element of joint-ordered set J. The movement-ordered set M contains the sets of movements. Each set of movements contains movements that occur at the same time.

UAV motor-primitive generation

The proposed method generates motor primitives in three stages: Operation-collection stage, time-adjustment stage, and motor-primitive generation stage. In the operation-collection stage, an operator uses a controller to make the UAV fly along a predefined path. Given that motor primitives are generated through demonstration-based learning, the UAV should fly the same path repeatedly. Therefore, the path should be defined in advance.

The operator usually guides the UAV on the same path three to five times. Flying fewer than three times cannot make motor primitives with the flexibility needed for a dynamic environment; more than five times results in complicated motor primitives. While the UAV is controlled, it measures and records its properties (such as pitch, roll, and yaw) and its command signals. Figure 1 shows the measured motor primitives. At this time, the properties of the UAV are measured and the commands of the UAV are obtained. The time t _i,j is the measured time of m _i,j .

Fig. 1

Measured linear motor primitives

Then, the movements and motor primitives of the UAV are defined, based on the properties and the commands. When a UAV faces an intermediate pinpoint to adjust its location, a new movement is defined and added to the motor primitive. The order of the visited pinpoints of motor primitives should be the same during the demonstration-based learning. A single motor primitive is generated based on a single flight.

During the time-adjustment stage, the movement times of all the motor primitives are adjusted. In the proposed method, the executed movements need not depend on a certain motor primitive. For example, movement m _2,2 in motor primitive m ₂ can be executed after executing m _1,1 in m ₁, if movement m _2,2 is more appropriate than movement m _1,1, considering the current properties of the UAV. To execute the movements as above, the ordered movements should have the same time, as shown in Fig. 2.

Fig. 2

Adjusted linear motor primitive structure

t ₁′ is assigned by t ₁, which is 0. From t ₂′, the time of movements is calculated by Eq. (4).

$${\it {\text{t}}_{\text{j}}^{\prime } = {\text{ t}}_{{{\text{j}} - 1}}^{\prime } + {\text{ MAX }}\left( {{\text{d}}_{{ 1,{\text{j}}}} ,{\text{ d}}_{{ 2,{\text{j}}}} , \ldots ,{\text{ d}}_{{{\text{p}},{\text{j}}}} } \right),}$$

(4)

where d _i,j is the duration of m _i,j .

The motor-primitive generation stage merges all motor primitives into a single graph-based motor primitive by connecting the movements in order, as shown in Fig. 3. The graph-based motor primitive of Fig. 3 is defined as {{j ₁, j ₂, …, j _j }, {{m _1,1, m _2,1,…, m _p,i }, {m _1,2, m _2,2,…, m _p,2 },…, {m _1,j , m _2,j ,…, m _p,j }}}.

Fig. 3

Generated graph-based motor-primitive structure

UAV motor-primitive execution

If a graph-based motor primitive G is executed, all joins in the joint-ordered set J are selected in order. If the ith join j _i is selected, then one of the corresponding linked movements is selected, by considering the movement properties, and executed. Given that the UAV selects its movements based on the changed properties, it can consider changes in the environment.

Experiments

In the experiments, AR.Drone 2.0 was utilized as a UAV, as shown in Fig. 4. In the indoor environment, the AR.Drone 2.0 flew four times according to the predefined path—the black rectangle in Fig. 4—and collected the control signals and properties. The AR.Drone adjusts its position when it faces the vehicle number plates, which are defined as pinpoints.

Fig. 4

Screen shot of the indoor environment

The motor primitives of AR.Drone 2.0 were measured based on the control signals of the subject. A single movement command consists of a yaw, pitch, or roll, based on the motor power. The properties of a movement are yaw, pitch, roll, and altitude. Therefore, commands are sent by considering the yaw, pitch, roll, and altitude of the AR.Drone 2.0. When the AR.Drone 2.0 reaches each pinpoint, one movement is defined and added to the motor primitive. Figure 5 shows the yaw, pitch, and roll of the first measured motor primitive.

Fig. 5

First measured motor primitive. (a) Yaw command, (b) Pitch command, (c) Roll command, (d) Measured yaw property, (e) Measured pitch property, (f) Measured roll property, (g) Measured altitude property

During the time-adjustment stage, the movement times are adjusted. Table 1 shows all the times of the measured movements.

Table 1 All measured movement times

Therefore, the times from t ₁ ′ to t ₅ ′ were set to 0, 22,048, 43,333, 70,726, and 89,064 during the time-adjustment stage. After adjusting the time of the first measured motor primitive, Fig. 6 shows the first adjusted motor primitive.

Fig. 6

First adjusted motor primitive. (a) Yaw command, (b) Pitch command, (c) Roll command, (d) Revised yaw property, (e) Adjusted pitch property, (f) Adjusted roll property, (g) Adjusted altitude property

Figure 7 shows the result of the graph-based motor primitive created by the proposed method, based on four motor primitives where each motor primitive has five movements.

Fig. 7

Generated graph-based motor primitive

Conclusion

This paper proposed a novel motor-primitive generation framework for UAVs. The measured movements were integrated into a single graph-based motor primitive, which makes it possible to execute the movements of the graph-based motor primitive consecutively, considering changes in the environment.

Motor primitives are very important, given that the UAVs fly by selecting and performing motor primitives repeatedly. Motor primitives decide the quality of the UAVs. Therefore, the way to define motor primitives is very important. Traditionally, motor primitives are defined by setting the values of motor primitives linearly, which means that UAVs only fly according to the predefined path. However, because of dynamic environments, it is very hard to perform selected motor primitives completely. The motor primitives that reflect dynamic environments are demanded. In this paper, motor primitives are expressed by graph, which means that graph-based motor primitives are performed completely more than linear motor primitives. Therefore, graph-based motor primitives are proper to dynamic environments. In the future, the way to perform graph-based motor primitives needs to be handled.