differential game based air combat maneuver generation using...
TRANSCRIPT
Copyright ⓒ The Korean Society for Aeronautical & Space SciencesReceived: June 18, 2015 Revised: April 28, 2016 Accepted: June 19, 2016
204 http://ijass.org pISSN: 2093-274x eISSN: 2093-2480
PaperInt’l J. of Aeronautical & Space Sci. 17(2), 204–213 (2016)DOI: http://dx.doi.org/10.5139/IJASS.2016.17.2.204
Differential Game Based Air Combat Maneuver Generation Using Scoring Function Matrix
Hyunju Park*, Byung-Yoon Lee** and Min-Jea Tahk*** Department of Aerospace Engineering, KAIST, Daejeon 48235, Republic of Korea
Dong-Wan Yoo**** Agency for Defense Development, Daejeon 34060, Republic of Korea
Abstract
A differential game theory based approach is used to develop an automated maneuver generation algorithm for Within Visual
Range (WVR) air-to-air combat of unmanned combat aerial vehicles (UCAVs). The algorithm follows hierarchical decision-
making structure and performs scoring function matrix calculation based on differential game theory to find the optimal
maneuvers against dynamic and challenging combat situation. The score, implying how much air superiority the UCAV has,
is computed from the predicted relative geometry, relative distance and velocity of two aircrafts. Security strategy is applied
at the decision-making step. Additionally, a barrier function is implemented to keep the airplanes above the altitude lower
bound. To shorten the simulation time to make the algorithm more real-time, a moving horizon method is implemented. An
F-16 pseudo 6-DOF model is used for realistic simulation. The combat maneuver generation algorithm is verified through
three dimensional simulations.
Key words: Air-to-Air Combat, Differential Game, Scoring Function Matrix, Pursuit-Evasion Game
1. Introduction
On account of the recent development in guidance, control
and navigation technology, the Unmanned Combat Aerial
Vehicles (UCAVs) have begun to replace manned aircraft in
not only for reconnaissance missions [1]. Instead of well-
trained human pilots, UCAV must be implemented with an
algorithm to successfully carry out air-to-air combat. To deal
with the dynamic and challenging nature of air combat, the
pilot algorithm should be able to predict the maneuver of the
enemy and decide its own optimal maneuver.
Previous studies have treated the air-to-air combat in
several ways. Rule-based method has been studied by Burgin
and Sidor in [2]. The rule-based adaptive maneuvering logic
combat program was successful in combat against human
pilot; yet requires hardcoding and frequent validation to
apply experiences of skilled pilots. As a variation of the rule-
based method, Jang [3] uses an artificial neural network as a
pilot maneuver decision method. Fast computation time and
re-learning are the prime advantages of this method. Existing
combat simulation programs such as BRAWLER [4] uses a
heuristic value-driven system suggested in [5] to avoid time
and effort consuming characteristic of rule-based method.
The paper combines value-driven system with hierarchical
decision-making structure to follow natural straight-forward
decision-making paradigm of human pilots.
As an optimal control theory approach, [6] and [7]
introduced a nonlinear model predictive tracking controller
for a fixed wing UAV that performs three dimensional evasive
maneuvers. The control problem is formulated as a cost
optimization problem and solved using a gradient-descent
method.
Dynamic programming method has been studied for a long
time to solve the general pursuit-evade game. The optimal
This is an Open Access article distributed under the terms of the Creative Com-mons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduc-tion in any medium, provided the original work is properly cited.
* MS Student ** Ph. D Student *** Professor, Corresponding author: [email protected] **** Research Engineer
(204~213)15-100.indd 204 2016-07-05 오후 7:54:35
205
Hyunju Park Differential Game Based Air Combat Maneuver Generation Using Scoring Function Matrix
http://ijass.org
solution of the game is first defined in [8]. To overcome
the complexity of calculation and difficulty in real-time
simulation, variations of dynamic programming have been
suggested by researchers. McGrew et al. [9-10] have suggested
an approximate dynamic programming method to find the
air combat strategy. Approximate dynamic programming
is different from classical dynamic programming in that it
formulates a continuous function to approximately represent
the future reward. Approximate dynamic programming does
not have to perform the future reward calculation for every
discrete state thus results in much shorter computing time.
Austin et al. [11-12] have additionally combined a game
theoretic approach to discrete dynamic game. The authors
performed recursive decision-making procedures with a
fixed planning horizon by establishing scoring function
matrix.
In this paper, a three-dimensional pseudo 6-DOF model
is used to construct the air combat model for two airplanes.
The proposed combat maneuver generation algorithm
mainly utilizes a differential game theory based scoring
function matrix which is built by grafting the game matrix on
a value-driven based decision model. The decision-making
procedure follows the hierarchical structure and adopts
security strategy.
The organization of this paper is as follows: in section 2,
a three dimensional air combat simulation model used in
this study is described. In section 3, the combat generation
algorithm is addressed. In section 4, several cases of air-to-
air combat are investigated by computer simulation to verify
the combat model. Conclusions are given in section 5.
2. 3-Dimensional Air Combat Modeling
2.1 F-16 Pseudo 6DOF Model
A three-dimensional F-16 pseudo 6-DOF model is used
for a realistic simulation in this paper. Two fighters are
simulated simultaneously in this model. The equation
of motions and the kinematics for model is derived by
assuming each aircraft as a point mass. The 3-DOF equations
of motion of this model are as (1), where Vt implies the total
velocity at body axis and Tt is thrust, D is drag, and γt means
the elevation angle where ψt means the azimuth angle.
3
reward calculation for every discrete state thus results in much shorter computing time. Austin et al.
[11-12] have additionally combined a game theoretic approach to discrete dynamic game. The authors
performed recursive decision-making procedures with a fixed planning horizon by establishing
scoring function matrix.
In this paper, a three-dimensional pseudo 6-DOF model is used to construct the air combat model
for two airplanes. The proposed combat maneuver generation algorithm mainly utilizes a differential
game theory based scoring function matrix which is built by grafting the game matrix on a value-
driven based decision model. The decision-making procedure follows the hierarchical structure and
adopts security strategy.
The organization of this paper is as follows: in section 2, a three dimensional air combat simulation
model used in this study is described. In section 3, the combat generation algorithm is addressed. In
section 4, several cases of air-to-air combat are investigated by computer simulation to verify the
combat model. Conclusions are given in section 5.
2. 3-Dimensional Air Combat Modeling
2.1 F-16 Pseudo 6DOF Model
A three-dimensional F-16 pseudo 6-DOF model is used for a realistic simulation in this paper. Two
fighters are simulated simultaneously in this model. The equation of motions and the kinematics for
model is derived by assuming each aircraft as a point mass. The 3-DOF equations of motion of this
model are as (1), where tV implies the total velocity at body axis and tT is thrust, D is drag, and
t means the elevation angle where t means the azimuth angle.
( ) / sin( sin ) cos cos
( sin )sincos
t t t
t tt
t t
tt
t t
V T D m gL T W
mVL T
mV
, (1)
The kinematic equations are as in (2) where X, Y and h means position at inertia axis. There are
three input commands in this model as real fighters: thrust, roll rate, and load factor. All of the input
(1)
The kinematic equations are as in (2) where X, Y and
h means position at inertia axis. There are three input
commands in this model as real fighters: thrust, roll rate,
and load factor. All of the input commands include lag
compensators.
4
commands include lag compensators.
cos cos
cos sin
sint
t t t
t t t
t
X V
Y V
h V
, (2)
To avoid a singularity which occurs when pitch angle is 90 , this model utilized direction cosine
matrix updated method instead of Euler angle update method. More detailed figure and explanation of
the pseudo 6-DOF model of F-16 fighter is explained in Ref. [16].
2.2 Terminal Condition
A terminal condition and gun score are defined in this combat model. The situation when red
aircraft is in blue aircraft’s gun-range is defined as shown in Fig.1: blue’s bearing angle is smaller
than 60 and red’s aspect angle is smaller than 30 and also the distance between two aircrafts is
smaller than 500m. If the above condition is held for more than 5 seconds, algorithm regards the red is
shot down by the blue. Even though the terminal condition has not been satisfied, the algorithm
calculates how long each aircraft has been in the shooting mode and plot the cumulated time as “gun
score” of each after the algorithm finished.
30BA
60AA
Fig. 1. Definition of Gun Range
(2)
To avoid a singularity which occurs when pitch angle is
±90o, this model utilized direction cosine matrix updated
method instead of Euler angle update method. More detailed
figure and explanation of the pseudo 6-DOF model of F-16
fighter is explained in Ref. [16].
2.2 Terminal Condition
A terminal condition and gun score are defined in this
combat model. The situation when red aircraft is in blue
aircraft’s gun-range is defined as shown in Fig.1: blue’s
bearing angle is smaller than 60o and red’s aspect angle is
smaller than 30o and also the distance between two aircrafts
is smaller than 500m. If the above condition is held for more
than 5 seconds, algorithm regards the red is shot down by
the blue. Even though the terminal condition has not been
satisfied, the algorithm calculates how long each aircraft has
been in the shooting mode and plot the cumulated time as
“gun score” of each after the algorithm finished.
3. Combat Maneuver Generation Algorithm
3.1 Combat Algorithm Outline
The overall combat maneuver decision process is
decomposed into a hierarchical structure as shown in Fig.2.
The most influencing decision is made in the top level, pilot
4
commands include lag compensators.
cos cos
cos sin
sint
t t t
t t t
t
X V
Y V
h V
, (2)
To avoid a singularity which occurs when pitch angle is 90 , this model utilized direction cosine
matrix updated method instead of Euler angle update method. More detailed figure and explanation of
the pseudo 6-DOF model of F-16 fighter is explained in Ref. [16].
2.2 Terminal Condition
A terminal condition and gun score are defined in this combat model. The situation when red
aircraft is in blue aircraft’s gun-range is defined as shown in Fig.1: blue’s bearing angle is smaller
than 60 and red’s aspect angle is smaller than 30 and also the distance between two aircrafts is
smaller than 500m. If the above condition is held for more than 5 seconds, algorithm regards the red is
shot down by the blue. Even though the terminal condition has not been satisfied, the algorithm
calculates how long each aircraft has been in the shooting mode and plot the cumulated time as “gun
score” of each after the algorithm finished.
30BA
60AA
Fig. 1. Definition of Gun Range Fig. 1. Definition of Gun Range
(204~213)15-100.indd 205 2016-07-05 오후 7:54:36
DOI: http://dx.doi.org/10.5139/IJASS.2016.17.2.204 206
Int’l J. of Aeronautical & Space Sci. 17(2), 204–213 (2016)
behavior decision. In this block, the algorithm analyzes
current combat situation to decide whether to pursuit or
evade. In this paper, different strategies for each behavior are
defined. In the lower decision level, the algorithm generates
maneuver according to selected pilot behavior and finally
determines whether to fire or not based on the gun range
mentioned before.
In this paper, the pilot algorithm generates the maneuvers
based only on the combat situations of current state and
predicted next few seconds to grasp the constant changes in
the air combat. The generated maneuver is therefore a short-
term solution that is not available until the end of the game
according to the nature of differential-game theory. However
this method can effectively reduce the simulation time and
also increase the adaptability of this algorithm.
3.2 Scoring Function Matrix
To determine which maneuver is the “optimal maneuver”
that guarantees the advantageous position after few seconds,
every possible consequence of performing various maneuver
choices should be evaluated. The number of maneuver
choices of each aircraft is limited to 7. As shown in Fig.3,
the 7 choices are steady level flight, pull up, push down, left
pull up, left push down and right pull up and push down,
respectively. These maneuvers are selected to resemble the
real motion of fighters in air combats as much as possible.
Since blue and red UCAVs have 7 maneuver choices each,
there are total 49 possible maneuver combinations for both
blue and red as shown in Fig.4. The scoring function matrix
is used to systematically conduct the comparisons between
each end states. The end states are calculated by numerically
integrating the equations of motion with application of the
corresponding maneuver command. This is the “prediction”
step. The prediction is done for 1~2 seconds. The end states
are then scored by the scoring function, which will be
addressed in the next section. The scores imply how much
advantage corresponding aircraft has.
The “decision-making” procedure is carried out based
on the predicted scores. Security strategy based on the
game theorem is applied in this step to decide the optimal
maneuver commands. First, select the worst score of each
column: to figure out the magnitude of severity caused by
the enemy’s performance. Second, select the row including
the best case among the selected worst. By using this min-
max algorithm, the airplane is able to avoid the worst case
no matter which maneuver the enemy has chosen. In this
paper, both blue and red airplanes are assigned with the
same strategy.
3.3 Scoring Function
3.3.1 Orientation Score
The evaluation criterion of the end states mentioned in
previous section is the scoring function. The scoring function
is derived from the meaning of gaining air superiority.
During WVR air-to-air combat, the necessary conditions for
blue’s air superiority are as follows: (1) Blue has to be located
at the rear of the red, (2) Blue should be heading for the red.
Above conditions can be expressed using the relative
geometry of two aircrafts (see Fig.5). By definition, the sum
of blue’s aspect angle (AAB) and red’s bearing angle (BAR)
5
3. Combat Maneuver Generation Algorithm
3.1 Combat Algorithm Outline
The overall combat maneuver decision process is decomposed into a hierarchical structure as
shown in Fig.2. The most influencing decision is made in the top level, pilot behavior decision. In this
block, the algorithm analyzes current combat situation to decide whether to pursuit or evade. In this
paper, different strategies for each behavior are defined. In the lower decision level, the algorithm
generates maneuver according to selected pilot behavior and finally determines whether to fire or not
based on the gun range mentioned before.
Fig. 2 Hierarchical Structure of Decision-Making
In this paper, the pilot algorithm generates the maneuvers based only on the combat situations of
current state and predicted next few seconds to grasp the constant changes in the air combat. The
generated maneuver is therefore a short-term solution that is not available until the end of the game
according to the nature of differential-game theory. However this method can effectively reduce the
simulation time and also increase the adaptability of this algorithm.
3.2 Scoring Function Matrix
To determine which maneuver is the “optimal maneuver” that guarantees the advantageous position
after few seconds, every possible consequence of performing various maneuver choices should be
evaluated. The number of maneuver choices of each aircraft is limited to 7. As shown in Fig.3, the 7
choices are steady level flight, pull up, push down, left pull up, left push down and right pull up and
push down, respectively. These maneuvers are selected to resemble the real motion of fighters in air
Fig. 2. Hierarchical Structure of Decision-Making
6
combats as much as possible.
① Left Up② Left Down③ Pull Up④ Level FlightManeuvers are operated in Body axis
⑤ Push Down⑥ Right Up⑦ Right Down
Fig. 3 Maneuver Choices
Since blue and red UCAVs have 7 maneuver choices each, there are total 49 possible maneuver
combinations for both blue and red as shown in Fig.4. The scoring function matrix is used to
systematically conduct the comparisons between each end states. The end states are calculated by
numerically integrating the equations of motion with application of the corresponding maneuver
command. This is the “prediction” step. The prediction is done for 1~2 seconds. The end states are
then scored by the scoring function, which will be addressed in the next section. The scores imply
how much advantage corresponding aircraft has.
Fig. 3. Maneuver Choices
7
RED MNVR Choices
1 2 3 4 5 6 7
BLUE MNVRChoices
1 0.50096 0.4345 0.49813
2 0.56668 0.50075 0.56437
3 0.50221 0.4355 …
4 0.55787 0.47789 …
5 0.49687 0.44113 …
6 0.53704 0.47971 …
7 0.51065 0.42849 …
BLUE Maneuver Choices
1 2 3 4 5 6 7
REDMNVRChoices
1 0.49904 0.43332 0.49779 0.44213 0.50303 0.46296 0.48935
2 0.5655 0.49925 0.5645 0.52211 0.55887
3 0.50187 …
4 0.55744 …
5 0.49616 …
6 0.53647 …
7 0.50906 …
Fig. 4 Scoring Function Matrix The “decision-making” procedure is carried out based on the predicted scores. Security strategy
based on the game theorem is applied in this step to decide the optimal maneuver commands. First,
select the worst score of each column: to figure out the magnitude of severity caused by the enemy’s
performance. Second, select the row including the best case among the selected worst. By using this
min-max algorithm, the airplane is able to avoid the worst case no matter which maneuver the enemy
has chosen. In this paper, both blue and red airplanes are assigned with the same strategy.
3.3 Scoring Function
3.3.1 Orientation Score
The evaluation criterion of the end states mentioned in previous section is the scoring function. The
scoring function is derived from the meaning of gaining air superiority. During WVR air-to-air
combat, the necessary conditions for blue’s air superiority are as follows: (1) Blue has to be located at
the rear of the red, (2) Blue should be heading for the red.
Fig. 4. Scoring Function Matrix
(204~213)15-100.indd 206 2016-07-05 오후 7:54:36
207
Hyunju Park Differential Game Based Air Combat Maneuver Generation Using Scoring Function Matrix
http://ijass.org
equals to π and vice versa.
8
rBA
bAA
rV
bV
bBA
rAA
Fig. 5 Combat Geometry and Relative Angles between Two Fighters
Above conditions can be expressed using the relative geometry of two aircrafts (see Fig.5). By
definition, the sum of blue’s aspect angle ( BAA ) and red’s bearing angle ( RBA ) equals to and
vice versa.
R B B RAA BA AA BA , (3)
Using this relationship, the scoring function can be derived as follows:
,
,
2 ( )
2 ( )
B BB Ori
R RR Ori
BA AAS
BA AAS
, (4)
Consequently, the scores would have values between 0 and 2. Blue’s score will be 2 when it is on
red’s tail; and 0 when red is on blue’s tail.
3.3.2 Range-Error Score
To maintain the gun range while keeping the safety distance between red and blue, a scoring
function is derived based on the range error. The range error is E safetyR R R , where R means the
distance between blue and red and safetyR represents the safety distance. Following equation is used
to calculate the range error score and the shape is shown in Fig.6.
2 23
2( ) /1 0
1
1 E dR a aRE
E
a aS eR a
(5)
(3)
Using this relationship, the scoring function can be
derived as follows:
8
rBA
bAA
rV
bV
bBA
rAA
Fig. 5 Combat Geometry and Relative Angles between Two Fighters
Above conditions can be expressed using the relative geometry of two aircrafts (see Fig.5). By
definition, the sum of blue’s aspect angle ( BAA ) and red’s bearing angle ( RBA ) equals to and
vice versa.
R B B RAA BA AA BA , (3)
Using this relationship, the scoring function can be derived as follows:
,
,
2 ( )
2 ( )
B BB Ori
R RR Ori
BA AAS
BA AAS
, (4)
Consequently, the scores would have values between 0 and 2. Blue’s score will be 2 when it is on
red’s tail; and 0 when red is on blue’s tail.
3.3.2 Range-Error Score
To maintain the gun range while keeping the safety distance between red and blue, a scoring
function is derived based on the range error. The range error is E safetyR R R , where R means the
distance between blue and red and safetyR represents the safety distance. Following equation is used
to calculate the range error score and the shape is shown in Fig.6.
2 23
2( ) /1 0
1
1 E dR a aRE
E
a aS eR a
(5)
(4)
Consequently, the scores would have values between
0 and 2. Blue’s score will be 2 when it is on red’s tail; and 0
when red is on blue’s tail.
3.3.2 Range-Error Score
To maintain the gun range while keeping the safety
distance between red and blue, a scoring function is derived
based on the range error. The range error is RE=|R|-Rsafety,
where R means the distance between blue and red and Rsafety
represents the safety distance. Following equation is used
to calculate the range error score and the shape is shown in
Fig.6.
8
rBA
bAA
rV
bV
bBA
rAA
Fig. 5 Combat Geometry and Relative Angles between Two Fighters
Above conditions can be expressed using the relative geometry of two aircrafts (see Fig.5). By
definition, the sum of blue’s aspect angle ( BAA ) and red’s bearing angle ( RBA ) equals to and
vice versa.
R B B RAA BA AA BA , (3)
Using this relationship, the scoring function can be derived as follows:
,
,
2 ( )
2 ( )
B BB Ori
R RR Ori
BA AAS
BA AAS
, (4)
Consequently, the scores would have values between 0 and 2. Blue’s score will be 2 when it is on
red’s tail; and 0 when red is on blue’s tail.
3.3.2 Range-Error Score
To maintain the gun range while keeping the safety distance between red and blue, a scoring
function is derived based on the range error. The range error is E safetyR R R , where R means the
distance between blue and red and safetyR represents the safety distance. Following equation is used
to calculate the range error score and the shape is shown in Fig.6.
2 23
2( ) /1 0
1
1 E dR a aRE
E
a aS eR a
(5) (5)
In the above equation, ad is designed as the most desirable
value of RE where a1 is the vertical asymptote and a0 is the
intercept with x-axis and a3 controls the decay for RE>ad.
The range error score has value between 0 and 1, and 1 is
regarded as the best.
3.3.3 Velocity-Error Score
The desired velocity of aircrafts is maintained at the corner
speed to achieve the maximum turn-rate at every moment.
Using the aerodynamic data of given aircraft model, a V-n
diagram of F-16 pseudo 6-DOF model has plotted (see Fig.7)
in the following condition: altitude of 5000 meters and 10G
of load factor limitation. Since the F-16 model used in this
study has aerodynamic data for only positive angle of attack,
only the upper half of V-n diagram is plotted. The accelerated
stall line in Fig.7 represents the maximum load factor that
an airplane can produce based on airspeed; the desirable
velocity to maintain the maximum turn-rate is derived from
this accelerated stall line.
A velocity scoring function is derived to follow this corner
speed during the combat.
10
Fig. 7. V-n Diagram for F-16 Pseudo 6DOF Model
A velocity scoring function is derived to follow this corner speed during the combat.
des refV
ref
V VS
V
(6)
In above equation, desV represents the desired corner speed at corresponding altitude and refV is
the current velocity. That is, the difference between desired velocity and current velocity becomes the
score. Therefore, unlike other scores, smaller value of VS is more favorable.
The implementation of velocity-error score enables energy conserving maneuvers such as
high-yo-yo or low-yo-yo. Following Fig.8 and Fig.9 show the difference before and after application
of the velocity-error score. Both figures are the resultant trajectory of two simulation cases with same
initial condition and are applied with different scoring functions. In both simulation cases, the initial
velocity of blue is 270 /m s and the initial altitude is 7000m while the calculated corner speed at
7000m is 250 /m s . Two aircrafts are slanted head-on and the red performs level flight. It is obvious
that to achieve air superiority over red, blue should turn around at first.
When only orientation score and range-error score are applied, blue makes turn in lower altitude as
0 50 100 150 200 250 3000
2
4
6
8
10
12
14
16
18V-n Diagram for F-16 pseudo 6-DOF dynamics
Velocity (m/s)
Load
fact
or (G
)
Accelerated Stall LineLimit LoadAOA 15(deg)AOA 20(deg)AOA 30(deg)
(6)
In above equation, Vdes represents the desired corner
speed at corresponding altitude and Vref is the current
velocity. That is, the difference between desired velocity and
current velocity becomes the score. Therefore, unlike other
scores, smaller value of SV is more favorable.
The implementation of velocity-error score enables
energy conserving maneuvers such as high-yo-yo or low-
yo-yo. Following Fig.8 and Fig.9 show the difference before
8
rBA
bAA
rV
bV
bBA
rAA
Fig. 5 Combat Geometry and Relative Angles between Two Fighters
Above conditions can be expressed using the relative geometry of two aircrafts (see Fig.5). By
definition, the sum of blue’s aspect angle ( BAA ) and red’s bearing angle ( RBA ) equals to and
vice versa.
R B B RAA BA AA BA , (3)
Using this relationship, the scoring function can be derived as follows:
,
,
2 ( )
2 ( )
B BB Ori
R RR Ori
BA AAS
BA AAS
, (4)
Consequently, the scores would have values between 0 and 2. Blue’s score will be 2 when it is on
red’s tail; and 0 when red is on blue’s tail.
3.3.2 Range-Error Score
To maintain the gun range while keeping the safety distance between red and blue, a scoring
function is derived based on the range error. The range error is E safetyR R R , where R means the
distance between blue and red and safetyR represents the safety distance. Following equation is used
to calculate the range error score and the shape is shown in Fig.6.
2 23
2( ) /1 0
1
1 E dR a aRE
E
a aS eR a
(5)
Fig. 5. Combat Geometry and Relative Angles between Two Fighters
9
In the above equation, da is designed as the most desirable value of ER where 1a is the
vertical asymptote and 0a is the intercept with x-axis and 3a controls the decay for E dR a . The
range error score has value between 0 and 1, and 1 is regarded as the best.
Fig. 6. Range Error Scoring Function 3.3.3 Velocity-Error Score
The desired velocity of aircrafts is maintained at the corner speed to achieve the maximum turn-rate
at every moment. Using the aerodynamic data of given aircraft model, a V-n diagram of F-16 pseudo
6-DOF model has plotted (see Fig.7) in the following condition: altitude of 5000 meters and 10G of
load factor limitation. Since the F-16 model used in this study has aerodynamic data for only positive
angle of attack, only the upper half of V-n diagram is plotted. The accelerated stall line in Fig.7
represents the maximum load factor that an airplane can produce based on airspeed; the desirable
velocity to maintain the maximum turn-rate is derived from this accelerated stall line.
Fig. 6. Range Error Scoring Function
10
Fig. 7. V-n Diagram for F-16 Pseudo 6DOF Model
A velocity scoring function is derived to follow this corner speed during the combat.
des refV
ref
V VS
V
(6)
In above equation, desV represents the desired corner speed at corresponding altitude and refV is
the current velocity. That is, the difference between desired velocity and current velocity becomes the
score. Therefore, unlike other scores, smaller value of VS is more favorable.
The implementation of velocity-error score enables energy conserving maneuvers such as
high-yo-yo or low-yo-yo. Following Fig.8 and Fig.9 show the difference before and after application
of the velocity-error score. Both figures are the resultant trajectory of two simulation cases with same
initial condition and are applied with different scoring functions. In both simulation cases, the initial
velocity of blue is 270 /m s and the initial altitude is 7000m while the calculated corner speed at
7000m is 250 /m s . Two aircrafts are slanted head-on and the red performs level flight. It is obvious
that to achieve air superiority over red, blue should turn around at first.
When only orientation score and range-error score are applied, blue makes turn in lower altitude as
0 50 100 150 200 250 3000
2
4
6
8
10
12
14
16
18V-n Diagram for F-16 pseudo 6-DOF dynamics
Velocity (m/s)
Load
fact
or (G
)
Accelerated Stall LineLimit LoadAOA 15(deg)AOA 20(deg)AOA 30(deg)
Fig. 7. V-n Diagram for F-16 Pseudo 6DOF Model
(204~213)15-100.indd 207 2016-07-05 오후 7:54:37
DOI: http://dx.doi.org/10.5139/IJASS.2016.17.2.204 208
Int’l J. of Aeronautical & Space Sci. 17(2), 204–213 (2016)
and after application of the velocity-error score. Both figures
are the resultant trajectory of two simulation cases with
same initial condition and are applied with different scoring
functions. In both simulation cases, the initial velocity of
blue is 270m/s and the initial altitude is 7000m while the
calculated corner speed at 7000m is 250m/s . Two aircrafts
are slanted head-on and the red performs level flight. It is
obvious that to achieve air superiority over red, blue should
turn around at first.
When only orientation score and range-error score
are applied, blue makes turn in lower altitude as in Fig.8.
However, to conserve its energy as fighters do in real air
combat, blue ought to turn with higher altitude as shown in
Fig.9.
3.4 Pilot Behavior
In this paper, the combat situation is classified into four
cases by the combat geometry: Offensive, neutral, defensive
and head-on (see Fig.10). The pilot behavior block makes its
decision based on this combat situation category.
The strategic difference according to pilot behaviors is
revealed when constructing the final scoring functions.
When the behavior is determined as neutral, defensive or
head-on, the scoring function is derived as follows.
12
0 90 180
180
90
Fig. 10. Pilot Behavior Classification
The strategic difference according to pilot behaviors is revealed when constructing the final scoring
functions. When the behavior is determined as neutral, defensive or head-on, the scoring function is
derived as follows.
total Ori VS S S (7)
Since the purpose of range error score is to avoid crashes with opponent or pursuit it, it does not
included in the total score. Velocity score is considered to maintain optimal turn rate.
Otherwise, when the aircraft puts itself in the state of offense, the total score is derived as follows.
total Ori RES S S (8)
RES would operate to shorten the distance when the adversary is too far away and keep gun range
when the distance is too close. Additionally, in the offensive maneuver, the aircraft would follow the
enemy velocity instead of its corner speed to avoid overtaking. A P-D controller is implemented to
generate corresponding thrust commands. This velocity control is named “Tracking mode” in this
study. Fig.11 shows the different gun score cumulative plots for two cases: with and without Tracking
mode. In Fig.11 (Left), the cumulative gun score does not have any constant section after once it goes
into the enemy’s gun range.
(7)
Since the purpose of range error score is to avoid crashes
with opponent or pursuit it, it does not included in the total
score. Velocity score is considered to maintain optimal turn
rate.
Otherwise, when the aircraft puts itself in the state of
offense, the total score is derived as follows.
12
0 90 180
180
90
Fig. 10. Pilot Behavior Classification
The strategic difference according to pilot behaviors is revealed when constructing the final scoring
functions. When the behavior is determined as neutral, defensive or head-on, the scoring function is
derived as follows.
total Ori VS S S (7)
Since the purpose of range error score is to avoid crashes with opponent or pursuit it, it does not
included in the total score. Velocity score is considered to maintain optimal turn rate.
Otherwise, when the aircraft puts itself in the state of offense, the total score is derived as follows.
total Ori RES S S (8)
RES would operate to shorten the distance when the adversary is too far away and keep gun range
when the distance is too close. Additionally, in the offensive maneuver, the aircraft would follow the
enemy velocity instead of its corner speed to avoid overtaking. A P-D controller is implemented to
generate corresponding thrust commands. This velocity control is named “Tracking mode” in this
study. Fig.11 shows the different gun score cumulative plots for two cases: with and without Tracking
mode. In Fig.11 (Left), the cumulative gun score does not have any constant section after once it goes
into the enemy’s gun range.
(8)
SRE would operate to shorten the distance when the
adversary is too far away and keep gun range when the
distance is too close. Additionally, in the offensive maneuver,
the aircraft would follow the enemy velocity instead of
its corner speed to avoid overtaking. A P-D controller is
implemented to generate corresponding thrust commands.
This velocity control is named “Tracking mode” in this study.
Fig.11 shows the different gun score cumulative plots for two
cases: with and without Tracking mode. In Fig.11 (Left), the
cumulative gun score does not have any constant section
after once it goes into the enemy’s gun range.
3.5 Altitude Barrier Function
To improve the validity of simulation, a barrier function to
11
in Fig.8. However, to conserve its energy as fighters do in real air combat, blue ought to turn with
higher altitude as shown in Fig.9.
Fig. 8. Before implement the Velocity-Error Score
Fig. 9. After implement the Velocity-Error Score
3.4 Pilot Behavior
In this paper, the combat situation is classified into four cases by the combat geometry: Offensive,
neutral, defensive and head-on (see Fig.10). The pilot behavior block makes its decision based on this
combat situation category.
Foe
Friend
Foe
Friend
Fig. 8. Before implement the Velocity-Error Score
11
in Fig.8. However, to conserve its energy as fighters do in real air combat, blue ought to turn with
higher altitude as shown in Fig.9.
Fig. 8. Before implement the Velocity-Error Score
Fig. 9. After implement the Velocity-Error Score
3.4 Pilot Behavior
In this paper, the combat situation is classified into four cases by the combat geometry: Offensive,
neutral, defensive and head-on (see Fig.10). The pilot behavior block makes its decision based on this
combat situation category.
Foe
Friend
Foe
Friend
Fig. 9. After implement the Velocity-Error Score
12
0 90 180
180
90
Fig. 10. Pilot Behavior Classification
The strategic difference according to pilot behaviors is revealed when constructing the final scoring
functions. When the behavior is determined as neutral, defensive or head-on, the scoring function is
derived as follows.
total Ori VS S S (7)
Since the purpose of range error score is to avoid crashes with opponent or pursuit it, it does not
included in the total score. Velocity score is considered to maintain optimal turn rate.
Otherwise, when the aircraft puts itself in the state of offense, the total score is derived as follows.
total Ori RES S S (8)
RES would operate to shorten the distance when the adversary is too far away and keep gun range
when the distance is too close. Additionally, in the offensive maneuver, the aircraft would follow the
enemy velocity instead of its corner speed to avoid overtaking. A P-D controller is implemented to
generate corresponding thrust commands. This velocity control is named “Tracking mode” in this
study. Fig.11 shows the different gun score cumulative plots for two cases: with and without Tracking
mode. In Fig.11 (Left), the cumulative gun score does not have any constant section after once it goes
into the enemy’s gun range.
Fig. 10. Pilot Behavior Classification
13
Fig. 11. (Left) Tracking Mode Implemented, (Right) Without Tracking Mode
3.5 Altitude Barrier Function
To improve the validity of simulation, a barrier function to protect the aircrafts from ground hitting
is implemented. This algorithm is dependent on two variables, the z-axis velocity ( zV ), and the
altitude of the aircraft ( Z ). The feasible clearance of aircrafts is defined as above an altitude of
500m~1000m and the barrier function is designed to prevent the aircrafts going down lower than the
feasible region. For detailed adjustment of the barrier function, adjustment of the barrier function, two
bounds are defined: named caution bound and the critical bound, respectively. The former is located
on 2000 meters up, and the latter is on around 1000~1200 meters. The operation principle of the
barrier function is as follow: when the aircraft goes down below the caution bound, velocity penalty
vP occurs and the value is defined as the following equation.
/ 1, , 00 ,
z v c zv
V K for Z B VP
otherwise
(9)
In (9), cautionB refers to the caution bound where vK is an arbitrary constant. The value of vK
depends on the maneuverability of aircraft: better maneuverability requires smaller vK . The penalty
vP is valid when not only the caution bound is violated but also the direction of ZV is negative.
That is, the penalty is paid only when the aircraft is heading toward the ground.
If the aircraft continues going down and finally invade the critical bound, another penalty term HP ,
Foe
Friend
Foe
Friend
Fig. 11. (Left) Tracking Mode Implemented, (Right) Without Tracking Mode
(204~213)15-100.indd 208 2016-07-05 오후 7:54:37
209
Hyunju Park Differential Game Based Air Combat Maneuver Generation Using Scoring Function Matrix
http://ijass.org
protect the aircrafts from ground hitting is implemented. This
algorithm is dependent on two variables, the z-axis velocity
(Vz), and the altitude of the aircraft (Z). The feasible clearance
of aircrafts is defined as above an altitude of 500m~1000m
and the barrier function is designed to prevent the aircrafts
going down lower than the feasible region. For detailed
adjustment of the barrier function, adjustment of the barrier
function, two bounds are defined: named caution bound
and the critical bound, respectively. The former is located
on 2000 meters up, and the latter is on around 1000~1200
meters. The operation principle of the barrier function is
as follow: when the aircraft goes down below the caution
bound, velocity penalty occurs and the value is defined as
the following equation.
13
Fig. 11. (Left) Tracking Mode Implemented, (Right) Without Tracking Mode
3.5 Altitude Barrier Function
To improve the validity of simulation, a barrier function to protect the aircrafts from ground hitting
is implemented. This algorithm is dependent on two variables, the z-axis velocity ( zV ), and the
altitude of the aircraft ( Z ). The feasible clearance of aircrafts is defined as above an altitude of
500m~1000m and the barrier function is designed to prevent the aircrafts going down lower than the
feasible region. For detailed adjustment of the barrier function, adjustment of the barrier function, two
bounds are defined: named caution bound and the critical bound, respectively. The former is located
on 2000 meters up, and the latter is on around 1000~1200 meters. The operation principle of the
barrier function is as follow: when the aircraft goes down below the caution bound, velocity penalty
vP occurs and the value is defined as the following equation.
/ 1, , 00 ,
z v c zv
V K for Z B VP
otherwise
(9)
In (9), cautionB refers to the caution bound where vK is an arbitrary constant. The value of vK
depends on the maneuverability of aircraft: better maneuverability requires smaller vK . The penalty
vP is valid when not only the caution bound is violated but also the direction of ZV is negative.
That is, the penalty is paid only when the aircraft is heading toward the ground.
If the aircraft continues going down and finally invade the critical bound, another penalty term HP ,
Foe
Friend
Foe
Friend
(9)
In (9), Bcaution refers to the caution bound where Kv is
an arbitrary constant. The value of Kv depends on the
maneuverability of aircraft: better maneuverability requires
smaller Kv. The penalty Pv is valid when not only the caution
bound is violated but also the direction of Vz is negative.
That is, the penalty is paid only when the aircraft is heading
toward the ground.
If the aircraft continues going down and finally invade the
critical bound, another penalty term PH, which is dependent
of the height of the aircraft, occurs and defined as follows:
14
which is dependent of the height of the aircraft, occurs and defined as follows:
/ / ,0 ,
H L H LH
Z K B K for Z BP
otherwise
(10)
Here, HK is also a constant and CriticalB means the lower bound. These penalty terms are applied
to the scoring function exponentially right after the relative orientation of two aircrafts is evaluated.
The final scores of the aircrafts are derived as follows:
( )10 v hP Pfinal totalS S (11)
The “decision-making” procedure is conducted using above final scores.
Fig. 9 Application of Altitude Barrier Function
Figure 12 shows how the altitude barrier function works. Only blue aircraft is implemented with the
altitude barrier function in the above figure. Blue and red aircrafts have maneuvered scissors at the
first time; but after they go down below 1500m, the blue aircraft maneuvers to recover the height.
3.6 Moving Horizon Method
Red
Blue
(10)
Here, KH is also a constant and BCritical means the lower
bound. These penalty terms are applied to the scoring
function exponentially right after the relative orientation of
two aircrafts is evaluated. The final scores of the aircrafts are
derived as follows:
14
which is dependent of the height of the aircraft, occurs and defined as follows:
/ / ,0 ,
H L H LH
Z K B K for Z BP
otherwise
(10)
Here, HK is also a constant and CriticalB means the lower bound. These penalty terms are applied
to the scoring function exponentially right after the relative orientation of two aircrafts is evaluated.
The final scores of the aircrafts are derived as follows:
( )10 v hP Pfinal totalS S (11)
The “decision-making” procedure is conducted using above final scores.
Fig. 9 Application of Altitude Barrier Function
Figure 12 shows how the altitude barrier function works. Only blue aircraft is implemented with the
altitude barrier function in the above figure. Blue and red aircrafts have maneuvered scissors at the
first time; but after they go down below 1500m, the blue aircraft maneuvers to recover the height.
3.6 Moving Horizon Method
Red
Blue
(11)
The “decision-making” procedure is conducted using
above final scores.
Figure 12 shows how the altitude barrier function works.
Only blue aircraft is implemented with the altitude barrier
function in the above figure. Blue and red aircrafts have
maneuvered scissors at the first time; but after they go down
below 1500m, the blue aircraft maneuvers to recover the
height.
3.6 Moving Horizon Method
This method is created based on sliding window
method in order to reduce the simulation time and prevent
unreasonable maneuvering. Since the maneuver choices
defined in Fig.13 include both left turn and right turn, the
algorithm might select radical roll command. Extreme roll
rate command exceeds maneuvering limitation of aircraft
should be rejected for more realistic simulation.
The moving horizon method limits the possible maneuver
choice with regard to the current attitude of the aircraft. The
former maneuver choice history is saved and applied at the
sizing of the window. In this paper, if the aircraft has chosen
left of right direction of maneuver at the very last moment,
the window excludes the opposite side of maneuvers from
choices. Therefore, the size of scoring function matrix is
reduced to 5×7 matrix from 7×7 matrix as circumstance
requires.
The implement of the moving window method also
decreases the calculation time of the algorithm about 71.6%
as predicted.
14
which is dependent of the height of the aircraft, occurs and defined as follows:
/ / ,0 ,
H L H LH
Z K B K for Z BP
otherwise
(10)
Here, HK is also a constant and CriticalB means the lower bound. These penalty terms are applied
to the scoring function exponentially right after the relative orientation of two aircrafts is evaluated.
The final scores of the aircrafts are derived as follows:
( )10 v hP Pfinal totalS S (11)
The “decision-making” procedure is conducted using above final scores.
Fig. 9 Application of Altitude Barrier Function
Figure 12 shows how the altitude barrier function works. Only blue aircraft is implemented with the
altitude barrier function in the above figure. Blue and red aircrafts have maneuvered scissors at the
first time; but after they go down below 1500m, the blue aircraft maneuvers to recover the height.
3.6 Moving Horizon Method
Red
Blue
Fig. 12. Application of Altitude Barrier Function
15
This method is created based on sliding window method in order to reduce the simulation time and
prevent unreasonable maneuvering. Since the maneuver choices defined in Fig.13 include both left
turn and right turn, the algorithm might select radical roll command. Extreme roll rate command
exceeds maneuvering limitation of aircraft should be rejected for more realistic simulation.
The moving horizon method limits the possible maneuver choice with regard to the current attitude
of the aircraft. The former maneuver choice history is saved and applied at the sizing of the window.
In this paper, if the aircraft has chosen left of right direction of maneuver at the very last moment, the
window excludes the opposite side of maneuvers from choices. Therefore, the size of scoring function
matrix is reduced to 5 7 matrix from 7 7 matrix as circumstance requires.
The implement of the moving window method also decreases the calculation time of the algorithm
about 71.6% as predicted.
RED MNVR Choices
1 2 3 4 5 6 7
BLUE MNVRChoices
1 …
2 …
3 ...
4 ...
5 ...
6 ...
7 ...
RED MNVR Choices
1 2 3 4 5 6 7
BLUE MNVRChoices
1 …
2 …
3 ...
4 ...
5 ...
6 ...
7 ...
Fig. 10 Moving Horizon Method
4. Simulations
Fig. 13. Moving Horizon Method
(204~213)15-100.indd 209 2016-07-05 오후 7:54:38
DOI: http://dx.doi.org/10.5139/IJASS.2016.17.2.204 210
Int’l J. of Aeronautical & Space Sci. 17(2), 204–213 (2016)
4. Simulations
4.1 Simulation Parameters
In order to verify the algorithm suggested in this paper,
several simulations are conducted. Simulation parameters
that are used in simulations are listed in Table.1. Detailed
control commands are also listed in Table.4.1. Two different
initial combat conditions are designed in this paper. The
specific conditions are listed in Table.2.
4.2 Simulation Results
First simulation case is the head-on case. Two fighters are
flying in opposite direction and they originally located far
enough to past each other if they do not start chasing. When
the aircrafts approach to each other within the maneuvering
distance, they start chasing each other as shown in Fig.14.
The two star-marks are the start point of each aircraft. The
result is draw as predicted. Two aircrafts are maneuvering
scissors. Fig.15 shows the relative angle history of blue and
red aircrafts. It is shown that two aircrafts has started from
Head-on position and stay in neutral and head on behaviors.
As it is mentioned in the Table.2, the second case is the
tail-chase where blue is advantageous at the beginning. Blue
is not only on the rear of the red aircraft and heading for the
red but also located at the higher altitude than red. In general
WVR air combat, the initial advantage usually has large
influence on the result of combat. Fig.17 represents that the
relative angle history of blue is mostly positioned in offensive
domain while angle history of red is cornered at defensive
domain as predicted.
4.3 Simulation on Different Prediction Time
Prediction time is also a parameter influencing the
performance of this algorithm. The originally set prediction
time is 1 second as written in Table.1. To show the influence
of prediction time on combat result, three more simulations
Table 1. Simulation Parameters and Maneuver Option Sets
16
4.1 Simulation Parameters
In order to verify the algorithm suggested in this paper, several simulations are conducted.
Simulation parameters that are used in simulations are listed in Table.1. Detailed control commands
are also listed in Table.4.1. Two different initial combat conditions are designed in this paper. The
specific conditions are listed in Table.2.
Table 1. Simulation Parameters and Maneuver Option Sets Simulation Parameters Maneuver Option Sets #1
Simulation Time 80 sec Max Pull-up Load Factor (G) 10
Prediction Time 1 sec Max Push-down Load Factor (G) -3
Integration Method RK 4th Max Allowed Roll Rate (deg/s) 150
Table 2. Initial Combat Conditions Scenario #1 Scenario #2
Situation Head-on (Fair Game) Tail-Chase (Blue Adv)
NED Pos (m) RED [1000, 50, -7000] [500, 50, -7000]
BLUE [-1000, -50, -7000] [-500, -50, -8000]
Heading (deg) RED 180 0
BLUE 0 0
Velocity (m/s) RED 250 250
BLUE 250 270
4.2 Simulation Results
First simulation case is the head-on case. Two fighters are flying in opposite direction and they
originally located far enough to past each other if they do not start chasing. When the aircrafts
approach to each other within the maneuvering distance, they start chasing each other as shown in
Fig.14. The two star-marks are the start point of each aircraft. The result is draw as predicted. Two
aircrafts are maneuvering scissors. Fig.15 shows the relative angle history of blue and red aircrafts. It
Table 2. Initial Combat Conditions
16
4.1 Simulation Parameters
In order to verify the algorithm suggested in this paper, several simulations are conducted.
Simulation parameters that are used in simulations are listed in Table.1. Detailed control commands
are also listed in Table.4.1. Two different initial combat conditions are designed in this paper. The
specific conditions are listed in Table.2.
Table 1. Simulation Parameters and Maneuver Option Sets Simulation Parameters Maneuver Option Sets #1
Simulation Time 80 sec Max Pull-up Load Factor (G) 10
Prediction Time 1 sec Max Push-down Load Factor (G) -3
Integration Method RK 4th Max Allowed Roll Rate (deg/s) 150
Table 2. Initial Combat Conditions Scenario #1 Scenario #2
Situation Head-on (Fair Game) Tail-Chase (Blue Adv)
NED Pos (m) RED [1000, 50, -7000] [500, 50, -7000]
BLUE [-1000, -50, -7000] [-500, -50, -8000]
Heading (deg) RED 180 0
BLUE 0 0
Velocity (m/s) RED 250 250
BLUE 250 270
4.2 Simulation Results
First simulation case is the head-on case. Two fighters are flying in opposite direction and they
originally located far enough to past each other if they do not start chasing. When the aircrafts
approach to each other within the maneuvering distance, they start chasing each other as shown in
Fig.14. The two star-marks are the start point of each aircraft. The result is draw as predicted. Two
aircrafts are maneuvering scissors. Fig.15 shows the relative angle history of blue and red aircrafts. It
17
is shown that two aircrafts has started from Head-on position and stay in neutral and head on
behaviors.
Fig. 14. Trajectory: Head-on (Blue ‘-’, Red ‘o’)
Fig. 15. Relative Angle History: Head-on
As it is mentioned in the Table.2, the second case is the tail-chase where blue is advantageous at the
-1500 -1000-500 0 500
1000
-1000-500
0500
10001500
5000
5500
6000
6500
7000
7500
8000
East Y (m)North X (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 14. Trajectory: Head-on (Blue ‘-’, Red ‘o’)
17
is shown that two aircrafts has started from Head-on position and stay in neutral and head on
behaviors.
Fig. 14. Trajectory: Head-on (Blue ‘-’, Red ‘o’)
Fig. 15. Relative Angle History: Head-on
As it is mentioned in the Table.2, the second case is the tail-chase where blue is advantageous at the
-1500 -1000-500 0 500
1000
-1000-500
0500
10001500
5000
5500
6000
6500
7000
7500
8000
East Y (m)North X (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 15. Relative Angle History: Head-on
(204~213)15-100.indd 210 2016-07-05 오후 7:54:40
211
Hyunju Park Differential Game Based Air Combat Maneuver Generation Using Scoring Function Matrix
http://ijass.org
are conducted: 0.5 seconds, 1.5 seconds and 2 seconds of
prediction time for red fighter. The results are shown in Fig.
18 ~ 23. All the simulations are conducted under the Head-
on initial condition.
Not only when the prediction time is shorter(Fig. 18, 19)
but when is longer(Fig. 20 ~ 23), red seems disadvantageous
against blue. From the results, it seems that a certain best
prediction time exists, and it is 1 second here. This occurs
because each fighter holds the decided maneuver for “hold
time”, 0.5 seconds in this paper. The relationship between
hold time and prediction time would be studied in future
research.
5. Conclusion
In this paper, a security strategy algorithm using the scoring
function matrix is developed for automated 3-dimensional
air-to-air combat. Using F-16 pseudo 6-DOF model, scoring
function and scoring matrix are defined and applied to
18
beginning. Blue is not only on the rear of the red aircraft and heading for the red but also located at
the higher altitude than red. In general WVR air combat, the initial advantage usually has large
influence on the result of combat. Fig.17 represents that the relative angle history of blue is mostly
positioned in offensive domain while angle history of red is cornered at defensive domain as predicted.
Fig. 16. Trajectory: Tail-Chase (Blue ‘-’, Red ‘o’)
Fig. 117. Relative Angle History: Tail-Chase 4.3 Simulation on Different Prediction Time
-3000
-2500
-2000
-1500
-1000
-500
0
500
1000-1000 -500 0 500 1000 1500 2000 2500
7000
7500
8000
8500
9000
9500
10000
10500
North X (m)
East Y (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 16. Trajectory: Tail-Chase (Blue ‘-’, Red ‘o’)
18
beginning. Blue is not only on the rear of the red aircraft and heading for the red but also located at
the higher altitude than red. In general WVR air combat, the initial advantage usually has large
influence on the result of combat. Fig.17 represents that the relative angle history of blue is mostly
positioned in offensive domain while angle history of red is cornered at defensive domain as predicted.
Fig. 16. Trajectory: Tail-Chase (Blue ‘-’, Red ‘o’)
Fig. 117. Relative Angle History: Tail-Chase 4.3 Simulation on Different Prediction Time
-3000
-2500
-2000
-1500
-1000
-500
0
500
1000-1000 -500 0 500 1000 1500 2000 2500
7000
7500
8000
8500
9000
9500
10000
10500
North X (m)
East Y (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 17. Relative Angle History: Tail-Chase
19
Prediction time is also a parameter influencing the performance of this algorithm. The originally set
prediction time is 1 second as written in Table.1. To show the influence of prediction time on combat
result, three more simulations are conducted: 0.5 seconds, 1.5 seconds and 2 seconds of prediction
time for red fighter. The results are shown in Fig. 18 ~ 23. All the simulations are conducted under the
Head-on initial condition.
Fig. 18. Trajectory: Prediction time 0.5 sec (Blue ‘-’, Red ‘o’)
Fig. 19. Relative Angle History: Prediction time 0.5 sec
-10000
10002000
30004000
5000
-1000
0
1000
5000
6000
7000
8000
9000
10000
East Y (m)North X (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 19. Relative Angle History: Prediction time 0.5 sec
19
Prediction time is also a parameter influencing the performance of this algorithm. The originally set
prediction time is 1 second as written in Table.1. To show the influence of prediction time on combat
result, three more simulations are conducted: 0.5 seconds, 1.5 seconds and 2 seconds of prediction
time for red fighter. The results are shown in Fig. 18 ~ 23. All the simulations are conducted under the
Head-on initial condition.
Fig. 18. Trajectory: Prediction time 0.5 sec (Blue ‘-’, Red ‘o’)
Fig. 19. Relative Angle History: Prediction time 0.5 sec
-10000
10002000
30004000
5000
-1000
0
1000
5000
6000
7000
8000
9000
10000
East Y (m)North X (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 18. Trajectory: Prediction time 0.5 sec (Blue ‘-’, Red ‘o’)
(204~213)15-100.indd 211 2016-07-05 오후 7:54:43
DOI: http://dx.doi.org/10.5139/IJASS.2016.17.2.204 212
Int’l J. of Aeronautical & Space Sci. 17(2), 204–213 (2016)
generate the maneuver command to conduct WVR combat
successfully. Altitude barrier function and tracking mode is
implemented to improve the validity and performance of the
combat model. Moving horizon method is also introduced to
refine the algorithm.
Two engagement scenarios of head-on, tail-chasing
of blue are investigated to check the performance of the
developed model and proposed algorithms. The verification
of combat model has done under the controlled computation
environment and the combat results and maneuvers are
similar to those of real air combat.
Studies on air-to-air combat between different maneuver
algorithms would be helpful for algorithm improvement.
Acknowledgement
This research was supported by Agency for Defense
Development as part of the research project ‘A study on UAV
Maneuver Generation Algorithm for Air-to-Air Combat’
20
Fig. 20. Trajectory: Prediction time 1.5 sec (Blue ‘-’, Red ‘o’)
Fig. 21. Relative Angle History: Prediction time 1.5 sec
-2000
-1500
-1000
-500
0
-1000
-500
0
500
1000
6000
6500
7000
7500
8000
East Y (m)North X (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 21. Relative Angle History: Prediction time 1.5 sec
21
Fig. 22. Trajectory: Prediction time 2 sec (Blue ‘-’, Red ‘o’)
Fig. 23. Relative Angle History: Prediction time 2 sec
Not only when the prediction time is shorter(Fig. 18, 19) but when is longer(Fig. 20 ~ 23), red
seems disadvantageous against blue. From the results, it seems that a certain best prediction time
exists, and it is 1 second here. This occurs because each fighter holds the decided maneuver for “hold
time”, 0.5 seconds in this paper. The relationship between hold time and prediction time would be
studied in future research.
-1500
-1000
-500
0
-1000
-500
0
500
1000
1500
5600
5800
6000
6200
6400
6600
6800
7000
7200
East Y (m)
North X (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 22. Trajectory: Prediction time 2 sec (Blue ‘-’, Red ‘o’)
21
Fig. 22. Trajectory: Prediction time 2 sec (Blue ‘-’, Red ‘o’)
Fig. 23. Relative Angle History: Prediction time 2 sec
Not only when the prediction time is shorter(Fig. 18, 19) but when is longer(Fig. 20 ~ 23), red
seems disadvantageous against blue. From the results, it seems that a certain best prediction time
exists, and it is 1 second here. This occurs because each fighter holds the decided maneuver for “hold
time”, 0.5 seconds in this paper. The relationship between hold time and prediction time would be
studied in future research.
-1500
-1000
-500
0
-1000
-500
0
500
1000
1500
5600
5800
6000
6200
6400
6600
6800
7000
7200
East Y (m)
North X (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 23. Relative Angle History: Prediction time 2 sec
20
Fig. 20. Trajectory: Prediction time 1.5 sec (Blue ‘-’, Red ‘o’)
Fig. 21. Relative Angle History: Prediction time 1.5 sec
-2000
-1500
-1000
-500
0
-1000
-500
0
500
1000
6000
6500
7000
7500
8000
East Y (m)North X (m)
Alti
tude
-Z (m
)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Blue)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
0 20 40 60 80 100 120 140 160 1800
20
40
60
80
100
120
140
160
180
Neutral Defensive
Offensive
Offensive
Head-on
Relative Angle History (Red)
Aspect Angle (Deg)
Bea
ring
Ang
le (D
eg)
Red
Blue
Fig. 20. Trajectory: Prediction time 1.5 sec (Blue ‘-’, Red ‘o’)
(204~213)15-100.indd 212 2016-07-05 오후 7:54:46
213
Hyunju Park Differential Game Based Air Combat Maneuver Generation Using Scoring Function Matrix
http://ijass.org
under contract UD130041JD.
References
[1] Austin, R., Unmanned Air Systems UAV Design,
Development and Deployment, John Wiley & Sons Ltd, 2010.
[2] Burgin, G. and Sidor, L., “Rule-Based Air Combat
Simulation”, NASA CR 4160, 1988.
[3] Jang, S., “Study of Intelligent Pilot Model Based on
Basic Fighter Maneuvering for Air Combat Simulation”,
KDMT1201222126 , Department of AE, Inha Univ., 2012.
[4] https://www.dsiac.org/resources/models_and_tools/
brawler
[5] Lazarus, E., “The Application of Value-Driven Decision-
Making in Air Combat Simulation”, Proceedings of the IEEE
International Conference on Systems, Man, and Cybernetics,
Piscataway, NJ, Vol. 3, 1997, pp. 2302-2307.
[6] Sprinkle, J., Eklund, M., Kim, H. and Sastry, S., “Encoding
Aerial Pursuit/Evasion Games with Fixed Wing Aircraft into a
Nonlinear Model Predictive Tracking Controller”, 43rd IEEE
Conference on Decision and Control, Bahamas, 2004.
[7] Sprinkle, J., Eklund, M. and Sastry, S., “Implementing
and Testing a Nonlinear Model Predictive Tracking
Controller for Aerial Pursuit/Evasion Games on a Fixed Wing
Aircraft”, 2005 American Control Conference, Portland, OR,
2005.
[8] Isaacs, R., “Games of Pursuit”, Rand Corp. TR P-257,
Santa Monica, CA, Nov. 1951.
[9] McGrew, J. S., “Real-Time Maneuvering Decisions
for Autonomous Air Combat”, Massachusetts Institute of
Technology, 2008.
[10] McGrew, J. S., How, J. P., Williams, B. and Roy,
N., “Air-Combat Strategy Using Approximate Dynamic
Programming”, Journal of Guidance, Control, and Dynamics,
Vol. 33, 2010, pp. 1641-1654.
[11] Austin, F., Carbone, G., Falco, M., Hinz, H. and Lewis,
M., “Automated Maneuvering During Air-to-Air Combat”,
Tech.rep., Grumman Corporate Research Center, Bethpage,
NY, CRC Rept. RE-742, Nov 1987.
[12] Austin, F., Carbone, G., Falco, M., Hinz, H. and Lewis,
M., “Game Theory for Automated Maneuvering During Air-
to-Air Combat”, Journal of Guidance, Control, and Dynamics,
Vol. 13, No. 6, 1990, pp. 1143-1149.
[13] Hwang, Y., Yoo, D., Won, D. and Tahk, M., “Study on
Advanced BVR Combat Maneuvers for the Improvement of
Kill/Survival Probabilities”, 2012 Asia-Pacific International
Symposium on Aerospace Technology, Nov. 13-15, Jeju,
Korea, 2012.
[14] Park, H., Yoo, D. and Tahk, M., “Generation of
the Optimal Air-to-Air Combat Maneuver using Scoring
Function Matrix”, 2014 KSAS Spring Conference, Wonju,
Korea, 2014.
[15] Park, H., Yoo, D. and Tahk, M., “Optimal WVR Air-
to-Air Combat Maneuver Generation Algorithm using
Scoring Function based on Security Strategy”, 2014 Asia-
Pacific International Symposium on Aerospace Technology,
Shanghai, China, 2014.
[16] Park, H., Lee, B., Yoo, D. and Tahk, M., “Differential
Game based 3-Dimensional Air Combat Maneuver
Generation Using Scoring Function Matrix”, 2015 KSAS
Spring Conference, Jeongseon, Korea, 2015.
(204~213)15-100.indd 213 2016-07-05 오후 7:54:46