1. Introduction and related work
Nowadays, target tracking and prediction technologies have been extensively studied and applied to various kinds of robots, such as the intelligent mobile robots, sport robots and Unmaned Aerial Vehicle. The tracking and prediction technologies of a moving object are very important. A regressor-free Fourier series-based tracking controller [Reference Hewawasam, Ibrahim, Kahandawa and Choudhury1] was designed to improve the controlling accuracy when the dynamics model is uncertain. For mobile robots, many tracking methods have been developed, such as the agoraphilic navigation algorithm which was proposed to solve navigation problem in dynamic environment [Reference Villalobos-Chin and SantibÁÑez2], the laser beam tracking mechanism [Reference Ling, Wu, Zhou, Wang and Wu3], the formation tracking control method based on fast terminal sliding mode controller [Reference Yousuf, Khan and Noor4], the obstacle-avoidance method based on improved artificial potential field and PID adaptive tracking control algorithm [Reference Pan, Li, Yang and Deng5]. These methods are mainly used in the field of navigation and obstacle avoidance. The laser radar is commonly used for map building.
Different from some tracking methods in the field of navigation, the target tracking and prediction approaches can also be applied to sport robots. These robots have been concerning to many researchers in recent years, such as table tennis robots and soccer robots, but the studies on badminton robots are less. Generally, the shuttlecock robot system is composed of three core parts, the vision subsystem, the servo subsystem and the machinery subsystem [Reference Man, Hu, Tang and Zhao6]. That vision subsystem is used to obtain the positional signal of the shuttlecock, the servo subsystem and the machinery subsystem are the executive devices with three degree freedom [Reference Man and Zhao7]. The proposed algorithm is used for obtaining the object’s law of motion in the physical world without knowing the precise dynamic model. It is applied to a badminton robot in this paper. Besides, it is also a method to obtain the trained model of different kinds of flying or moving objects.
There are several researches about the dynamics and the prediction methods of the shuttlecock. The air resistance force is proportional to the square of a shuttlecock velocity, and the motion equation of a shuttlecock’s flying trajectory can be constructed by determining the terminal velocity [Reference Chen, Pan and Chen8]. Due to the irregular shape of the shuttlecock and the effect of the air drag force, the pose of a shuttlecock will change in flight, which makes it difficult to forecast its state. In a badminton game, it often travels at high speeds and appears as a blurred object in a frame which leads to a large appearance distortion and makes the detection work more difficult [Reference Huang9]. To solve the motion blur problem in shuttlecock detection work, a trajectory estimation method using the motion blur was proposed in [Reference Shishido, Kitahara, Kameda and Ohta10]. This method includes three parts, a novel tracking method, a probability model and the switching method. The tracking method was developed to evaluate the speed of the shuttlecock from the information derived from the motion blur region. To detect the small moving object with a few pixels in one frame image, the texture information and color information in the image were used to build up the probability model. When a shuttlecock was detected, the observed location and velocity were transmitted into the Kalman filter as the speed was low, while the observed location and the speed were estimated by the shape of the motion blur. The classic Kalman filter was also used in shuttlecock trajectory prediction in [Reference Man and Zhao7]. The study on the shuttlecock’s aerodynamics [Reference Texier, Cohen, Quere and Claneta11] indicates its trajectory is not parabolic [Reference Waghmare, Borkar, Saley and Chinchore12]. In addition, the classic Kalman filter is a dynamic estimation of the target using the minimum mean square error in the linear Gaussian case, which is suitable for linear systems. The kinematic model of shuttlecock is a nonlinear model; therefore, it requires linearization when using the ordinary Kalman filter, which restricts the tracking accuracy of filtering.
In paper [Reference Alam, Chowdhury, Theppadungporn and Subic13], the experimental results indicate that there is notable variation in drag coefficients between the standard feather and synthetic shuttlecocks. Paper [Reference Alam, Nutakom and Chowdhury14] studied the flow characteristics around the shuttlecock and explained the effect of the porosity of the skirt plate on the drag coefficient. Several configuration models were established based on autoregressive exogenous (ARX) algorithm and neural network ARX(NARX) algorithm, and simulation verified that the configuration of ARX model was more stable. [Reference Lubis15] What is more, RMIT industrial wind tunnel was used to measure the aerodynamic characteristics of shuttlecocks, and the aerodynamic characteristics and flight trajectory of badminton were analyzed. [Reference Le Personnic, Alam, Le Gendre, Chowdhury and Subic16] In addition, a method to obtain the flight parameters of the feather shuttlecock and the prototype synthetic shuttlecock was designed. [Reference Lin, Chua and Yeo17] Seigo Kitta presented the measurements of aerodynamic forces at high Reynolds numbers and studied the effect of shuttlecock deformation on aerodynamic properties. [Reference Kitta, Hasegawa, Murakami and Obayashi18] The parameter estimation of time domain models in the form of ARX or NARX was studied in paper [Reference Wang, Lum and Yang19].
Utilizing the motion features and scene information, a novel real-time nonrecursive tracking algorithm has been developed in [Reference Huang9]. This algorithm consists of the model of the correlation between the shuttlecock and the information derived from the scene context, a multifilter model (including the object filter, seed-path filter, state filter and correlation filter) and a model function. Each filter of the multifilter model was designed to eliminate the false candidates generated by the frame difference algorithm. The model function was used to select the true candidate. Besides, the special motion model of the shuttlecock and the player’s motion were taken into account in order to select the optimum object as the true shuttlecock in the current frame. But the tracking accuracy was not high enough for the badminton robot to predict the fall point in real time.
What is more, an analytic flying model to estimate the ball self-rotational velocity was proposed in paper [Reference Zhang, Xu and Tan20, Reference Huang, Xu, Tan and Su21]. In paper [Reference Nakashima, Ogawa, Kobayashi and Hayakawa22–Reference Nonomura, Nakashima and Hayakawa24], a method that used multiple high-speed cameras with 900 frames per second to analyze the effects of the rebounds and aerodynamics was developed. However, these approaches have to rely on high-performance vision devices. All vision detection systems require more computational resources to process the image. To reduce the computational hardware requirement, a method is proposed using a two-dimensional laser scanner to locate the shuttlecock [Reference Waghmare, Borkar, Saley and Chinchore12]. It makes the whole robot system smaller than the systems mentioned earlier, and it has the capability to detect shuttlecocks flying at high speed. Unfortunately, the accuracy of prediction is too large to meet the accuracy requirements of kicking the shuttlecock in real time.
In the present study, we present an aerodynamic analysis of the shuttlecock based on different coordinate systems and propose a novel method to track and predict the shuttlecock trajectory which only requires the speed and the position detected in real time as input and can be used to implement the algorithm of return strategies. This prediction method contains three parts. First, in order to eliminate most of the environmental noise, the two infrared filters and two infrared lights are added in the front of the cameras of the binocular vision system as shown in Fig. 1 and a kind of high reflection thin material is affixed on the surface of the cork base of the shuttlecock. Second, we utilize the unscented Kalman filter (UKF) algorithm instead of the classic Kalman filter to filter the errors of measurement and recognition algorithm. A trajectory data set containing more than 1500 tracks has been collected manually through the infrared-based binocular vision system and the three-frame difference algorithm. This data set serves as the input of the UKF algorithm to obtain three important data; the velocity of the shuttlecock at each sampling moment of one trajectory, the corresponding movement time and the corresponding movement distance. Third, these three kind of data are combined into three special data sets in three directions of the world coordinate system (WCS) as shown in Fig. 5. It is used as the training data of the RBF neural network in X, Y and Z directions, respectively. These RBF networks are designed for predicting the remaining part of the whole track with the input of the shuttlecock’s speed at the moment when a shuttlecock is detected and the position derived from the vision system in real time. The whole robot system is shown in Fig. 1.
Figure 1. The whole badminton robot system. It contains the infrared-based binocular vision subsystem and the robot subsystem which consists of the chassis motion subsystem of four Mecanum wheels, the two-dimensional platform servo subsystem and the ball striking system.
2. General introduction of the trajectory prediction algorithm
In order to get more time for the badminton robot to respond and provide a high precision drop point coordinate, the infrared-based binocular vision system is used to eliminate the background noise when detecting the shuttlecock. The kinetic equations are obtained from the analysis of shuttlecock dynamics. The UKF filter algorithm is designed for tracking the state of the shuttlecock in flight, as shown in Fig. 2. To predict the drop point of the shuttlecock, we split the shuttlecock track along the X, Y and Z axes of the WCS, and three RBF-based prediction models are designed for forecasting the movement distance in three directions, respectively.
Figure 2. Flowchart diagram of the trajectory prediction algorithm. The used velocities and relative distances are based on the WCS.
The Z-direction RBF network is used to predict the relative time when the shuttlecock is falling to the given height with the input of the shuttlecock’s speed in the Z direction. The X-direction and the Y-direction RBF networks are used to predict the movement distance along the X-axis and Y-axis, respectively.
3. Aerodynamic analysis of the shuttlecock
3.1. Definition of coordinate systems
To describe the motion equation of the shuttlecock, the suitable coordinate systems are required that will affect the complexity of the motion equations. The existing coordinate systems for flight target analysis are ground coordinate system (GCS), missile coordinate system, ballistic coordinate system and velocity coordinate system (VCS). In the present study, based on four commonly used coordinate systems in analysing aircraft, we redefined the coordinate to conduct the following aerodynamic analysis of the shuttlecock. The main difference here for the special structure of badminton, in order to facilitate the description and simplify the dynamic analysis of badminton, the VCS and the shuttlecock coordinate system (SCS) are approximately equal. At the same time, the coordinate system of a badminton court is added.
-
Shuttlecock coordinate system.
$O_1X_1Y_1Z_1$
as shown in Fig. 3. The centroid of the shuttlecock is set as the origin.
$O_1X_1$
axis is the vertical axis of the body of the shuttlecock.
$O_1Y_1$
axis is perpendicular to
$O_1X_1$
axis in the longitudinal symmetry plane of the shuttlecock body.
$O_1Z_1$
axis is derived from the right-hand rule. -
Velocity coordinate system.
$O_2X_2Y_2Z_2$
as shown in Fig. 3.
$O_2$
coincides with
$O_1$
.
$O_2X_2$
axis is consistent with the velocity vector of the shuttlecock centroid.
$O_2Y_2$
axis is perpendicular to
$O_2X_2$
axis in the vertical symmetry plane of the shuttlecock body. -
Trajectory coordinate system (TCS).
$O_3X_3Y_3Z_3$
. Its origin is consistent with the one mentioned above. Its X-axis is the same as that in VCS. But its Y-axis is perpendicular to the X-axis in the vertical plane of the velocity vector. -
Ground coordinate system.
$O_4X_4Y_4Z_4$
. The starting point of a trajectory is set as the origin. The intersection between the ballistic plane and the ground is the X-axis, pointing to the fall point which is positive. Its Y-axis is perpendicular to the X-axis in the vertical plane of its X-axis. Z-axis can be derived from the right-hand rule. -
World coordinate system.
$\mathrm{O}_5\mathrm{X}_5\mathrm{Y}_5\mathrm{Z}_5$
. The lower left corner of a standard badminton court is set as the coordinate system origin, the X-axis is in the transverse direction of a standard badminton court, while the Y-axis is in the vertical direction, and the Z-axis can be derived from the right-hand rule.
Figure 3. Coordinate systems for analyzing the aerodynamics of the shuttlecock.
$O_1X_1Y_1Z_1$
indicates the SCS and
$O_2X_2Y_2Z_2$
represent the VCS.
Due to the special physical structure of the shuttlecock, its hemispherical cork’s weight takes most of the whole weight and the center of mass is in the hemispherical cork, which indicates a significant result that the vertical axis of the shuttlecock body can be considered to be the same as the velocity vector of the shuttlecock when tracking it. According to the definition of the coordinate system above, that means the SCS and VCS coincide with each other. Conversions of different coordinate systems are as follows.
\begin{equation} \begin{cases} O_{TCS}=M_1\cdot O_{VCS}\\[4pt] O_{GCS}=M_2\cdot O_{TCS} \end{cases}\end{equation}
$O_{VCS}$
,
$O_{TCS}$
and
$O_{GCS}$
are the coordinates corresponding to the VCS, TCS and the GCS.
$M_1$
and
$M_2$
are the coordinate transformation matrices.
\begin{equation} M_1=\begin{bmatrix} 1 & \quad 0 & \quad 0 \\[4pt] 0 & \quad \cos{\gamma_v} & \quad -\!\sin{\gamma_v}\\[4pt] 0 & \quad \sin{\gamma_v} & \quad \cos{\gamma_v} \end{bmatrix}\end{equation}
\begin{equation} M_2=\begin{bmatrix} \cos{\theta}\cos{\psi_v} & \quad -\!\sin{\theta}\cos{\psi_v} & \quad \sin{\psi_v} \\[4pt] \sin{\theta} & \quad \cos{\theta} & \quad 0 \\[4pt] -\!\cos{\theta}\sin{\psi_v} & \quad \sin{\theta}\sin{\psi_v} & \quad \cos{\psi_v} \end{bmatrix}\end{equation}
$\gamma_v$
is the angle between the
$O_2Y_2$
as shown in Fig. 3 and the vertical plane of the velocity vector,
$\theta$
denotes the angle between the velocity vector and the ground,
$\psi_v$
indicates the angle between the projection of the velocity vector on the ground and the X-axis of the GCS.
3.2. Kinetic equations
Generally, the air dynamic force is decomposed into three components along the axes of the VCS while the air dynamic torque is decomposed along the axes of the SCS. However, because of the special object of this study, these two coordinate systems are considered as one coordinate system.
3.2.1. Aerodynamic force analysis
The aerodynamic force and rolling moment of the shuttlecock are defined as follow.
\begin{equation} R=\begin{cases} C_fqS=F_f \\[4pt] C_sqS=F_s \\[4pt] C_cqS=F_c \\[4pt] q=\frac{1}{2}\rho V^2 \end{cases}\end{equation}
\begin{equation} M=\begin{cases} M_{x1}=m_{x1}qSL\\[4pt] M_{y1}=m_{y1}qSL\\[4pt] M_{z1}=m_{z1}qSL \end{cases}\end{equation}
R represents the whole aerodynamic force of the shuttlecock and is a set of equations of aerodynamics. In Eqs. (6) and (7), the rotation matrixes of the corresponding aerodynamics between
$O_{TCS}$
,
$O_{VCS}$
and
$O_{GCS}$
are described.
$C_f$
,
$C_s$
and
$C_c$
are proportional factors that indicate the resistance force coefficient, the lift force coefficient and the lateral force factor respectively. S denotes the maximum cross-section area of a shuttlecock.
$F_f$
,
$F_s$
and
$F_c$
are fractions of the aerodynamic force along the axes of the VCS, which are called the air resistance, the lift force and the lateral force, respectively.
$\rho$
is the air density and V denotes the speed of the shuttlecock. q is called the dynamic head of the shuttlecock.
$M_{x1}$
,
$M_{y1}$
and
$M_{z1}$
denote the rolling moment, the yawing moment and the pitching moment, respectively, and
$m_{x1}$
,
$m_{y1}$
and
$m_{z1}$
are the corresponding proportional factors. L is the length of the shuttlecock.
From (1) we get
$O_{TCS}=M_1\cdot O_{VCS}$
. According to (1), (2) and (3), the projection of aerodynamic forces in TCS and GCS can be derived as follows.
\begin{equation} R_b=\begin{bmatrix} F_{x3}\\[4pt] F_{y3}\\[4pt] F_{z3} \end{bmatrix}=M_1\cdot \begin{bmatrix} F_f\\ F_s\\[4pt] F_c \end{bmatrix}\end{equation}
\begin{equation} R_d=\begin{bmatrix} F_{x4}\\[4pt] F_{y4}\\[4pt] F_{z4} \end{bmatrix}=M_2\cdot R_b\end{equation}
3.2.2. Dynamic equations of the shuttlecock
To simplify the dynamic model, the dynamic vector equations are projected on the axes of the SCS. We have
\begin{equation} H=J\cdot \omega=\begin{bmatrix} J_{x1} & \quad -J_{x1y1} & \quad -J_{z1x1}\\[4pt] -J_{x1y1} & \quad J_{y1} & \quad -J_{y1z1}\\[4pt] -J_{z1x1} & \quad -J_{y1z1} & \quad J_{z1} \end{bmatrix}\cdot\begin{bmatrix} \omega_{x1}\\[4pt] \omega_{y1}\\[4pt] \omega_{z1} \end{bmatrix}\end{equation}
H is the moment of inertia, J denotes the momentum moment, and
$\omega$
is the angular velocity. Because the shuttlecock is axisymmetric, the dynamic equation of the shuttlecock is simplified as below:
\begin{equation} \begin{bmatrix} M_{x1}\\[6pt] M_{y1}\\[6pt] M_{z1} \end{bmatrix}=\begin{bmatrix} J_{x1}\dfrac{\mathrm{d}\omega_{x1}}{\mathrm{d}t}+\left(J_{z1}-J_{y1}\right)\omega_{z1}\omega_{y1} \\[11pt] J_{y1}\dfrac{\mathrm{d}\omega_{y1}}{\mathrm{d}t}+\left(J_{x1}-J_{z1}\right)\omega_{x1}\omega_{z1} \\[11pt] J_{z1}\dfrac{\mathrm{d}\omega_{z1}}{\mathrm{d}t}+\left(J_{y1}-J_{x1}\right)\omega_{y1}\omega_{x1} \end{bmatrix}\end{equation}
4. Optimization and tracking of shuttlecocks trajectory data by the UKF filter
4.1. Description of the unscented Kalman filter
As we know, the UT transformation can approach the posterior mean and covariance of any nonlinear Gaussian system with at least third-order Taylor accuracy.
4.1.1. Initiallization of the UKF
To better describe the state of the shuttlecock in flight. We define
\begin{equation} X_k=\left[L,V,\Omega_v,\psi_v,\theta\right]^\mathrm{T}\end{equation}
where
$L=[x_5,y_5,z_5]$
denotes the location of the shuttlecock in the WCS,
$V=[v_{x5},v_{y5},v_{z5}]$
indicate the velocities decomposed along three directions of the WCS of the speed vector.
$\Omega_v=[\omega_{x1},\omega_{y1},\omega_{z1}]$
. It indicates the angular velocity around the axes of the SCS.
4.1.2. Construction of sigma points
The sigma points can be defined as follow.
\begin{equation} \begin{cases} X^0_{k-1}=\hat{X}_{k-1}\\[8pt] X^i_{k-1}=\hat{X}_{k-1}+\left(\sqrt{(n+\kappa)P^i_{k-1})}\right) , i=1,\cdots ,n\\[8pt] X^{i+n}_{k-1}=\hat{X}_{k-1}-\left(\sqrt{(n+\kappa)P^i_{k-1}}\right) \end{cases}\end{equation}
where
$\hat{X}_{k-1}$
is the state estimation at time
$t_{k-1}$
,
$P_{k-1}$
is the corresponding covariance matrix of the state vector.
4.1.3. Prediction of the state of shuttlecock
First, according to
$f=ma$
, (7) and (9), we can obtain the recursive function F of the state vector.T represents time.
\begin{equation} F=\begin{cases} L_k=L_{k-1}+VT+A\dfrac{R_d^\mathrm{T}}{2m}T^2\\[8pt] V_k=V_{k-1}+A\dfrac{R_d^\mathrm{T}}{m}T \\[8pt] \Omega_{v|k}^\mathrm{T}=\begin{bmatrix} \omega_{x|k}\\[6pt] \omega_{y|k}\\[6pt] \omega_{z|k} \end{bmatrix}=\begin{bmatrix} \omega_{x|k-1}+\dfrac{\mathrm{d}\omega_{x|k-1}}{\mathrm{d}t}T\\[9pt] \omega_{y|k-1}+\dfrac{\mathrm{d}\omega_{y|k-1}}{\mathrm{d}t}T\\[9pt] \omega_{z|k-1}+\dfrac{\mathrm{d}\omega_{z|k-1}}{\mathrm{d}t}T \end{bmatrix}\\[35pt] \psi_{v|k}=\psi_{v|k-1}+\dfrac{\omega_{y|k}}{\cos{\theta_{k-1}}}T \\[10pt] \theta_{k}=\theta_{k-1}+\omega_{z|k}T \end{cases}\end{equation}
where A is the transformation matrix from the ground coordinates to the world coordinates, we define
$\theta_w$
as the angle between the X-axis of GCS and the X-axis of WCS,
$A=\begin{bmatrix} \cos{\theta_w} & -\sin{\theta_w} & 0 \\[4pt] 0 & 0 & -1 \\[4pt] \sin{\theta_w} & \cos{\theta_w} & 0 \end{bmatrix}$
\begin{equation} \begin{cases} X^i_{k|k-1}=F\!\left(X^i_{k-1}\right)+q_{k-1} , i=0,1,\cdots ,L \\[4pt] \hat{X}_{k|k-1}=\sum\limits_{i=0}^L W^m_iX^i_{k|k-1} \\[8pt] P_{k|k-1}=\sum\limits_{i=0}^L W^c_i XX^\mathrm{T}+Q_{k-1} \end{cases}\end{equation}
where
$X=X^i_{k|k-1}-\hat{X}_{k|k-1}$
, and
$Q_{k-1}$
is the prediction variance.
$W^m_i$
and
$W^c_i$
are the weight coefficients shown as follows, where
$\lambda=\alpha^2(n+\kappa)-n$
. The value range of constant parameter
$\alpha$
is
$[0.0001,1]$
,
$\kappa$
is the redundancy and
$\beta$
is the parameter related to the state vector distribution.
\begin{equation} W^m_i=\begin{cases} \lambda/(n+\lambda) , i=0 \\[4pt] 1/2(n+\lambda) ,i\not=0 \end{cases} W^c_i=\begin{cases} \lambda/(n+\lambda)+1+\beta-\alpha^2 ,i=0 \\[4pt] 1/2(n+\lambda) , i\not=0 \end{cases}\end{equation}
4.1.4. Updating of the prediction with measurement Zk
\begin{equation} \begin{cases} \hat{Z}_{k|k-1}= \sum\limits_{i=0}^L W^m_i Z_{k-1} \\[7pt] P_{z_k}=\sum\limits_{i=0}^L W^c_iZZ^\mathrm{T}+R_k \\[7pt] P_{x_kz_k}=\sum\limits_{i=0}^L W^c_i XZ^\mathrm{T} \end{cases}\end{equation}
where
$Z=Z_{k-1}-\hat{Z}_{k|k-1}$
,
$P_{x_kz_k}$
and
$P_{z_k}$
are the cross-covariance and auto-covariance, respectively.
\begin{equation} \begin{cases} H_k=P_{x_kz_k}P_{z_k}^{-1}\\[4pt] \hat{X}_k=\hat{X}_{k|k-1}+H_k\!\left(Z_k-\hat{Z}_{k|k-1}\right)\\[4pt] P_k=P_{k|k-1}-H_kP_{z_k}H_k^\mathrm{T} \end{cases}\end{equation}
4.2. Experiment results
In order to test the tracking accuracy of the designed UKF algorithm, we randomly selected 609 tracks data for testing. The measured value and the tracking value by the UKF algorithm are decomposed along three axes of the WCS, as shown in Fig. 4(a), (b) and (c). We defined the mean square error of each track,
$MSE=(\frac{1}{N}\sum\nolimits_{j=1}^{N}(\hat{x}_j-x_j)^2)^{\frac{1}{2}}$
. N is the number of sampling points, and
$\hat{x}_j$
and
$x_j$
denote the filtered value and measured value, respectively. The tracking error statistics is shown in Table I. The experiment parameters
$\alpha=0.01$
,
$\kappa=0$
,
$\beta=2$
.
Table I. Tracking error statistics of the UKF algorithm
Figure 4. The measured and filtered value of the shuttlecock’s position in X, Y and Z directions of the WCS.
5. Prediction model of the shuttlecock trajectory based on RBF neural networks
Based on the kinetic analysis of the shuttlecock and the UKF algorithm, we successfully collected the data of the shuttlecock’s speed, movement time and movement distance. They have been used to construct three prediction models as shown below.
\begin{equation*}\qquad\qquad\qquad\qquad\qquad\qquad\qquad\quad\qquad\left\{\begin{array}{c}\Delta D_x=F_x(V_x,\Delta t)\qquad\qquad\qquad\qquad\qquad\qquad\quad\ \, \, \left(17\rm{a}\right)\\[6pt]\Delta D_y=F_y\left(V_y,\Delta t\right)\qquad\qquad\qquad\qquad\qquad\qquad\ \quad(17\rm{b})\\[6pt]\Delta D_z=F_z\left(V_z,\Delta t\right)\qquad\qquad\qquad\qquad\qquad\qquad\ \ \quad(17\rm{c})\end{array}\right.\end{equation*}
$F_x$
,
$F_y$
and
$F_z$
represent three RBF networks(xRBF, yRBF, zRBF).
$\Delta D_x$
,
$\Delta D_y$
and
$\Delta D_z$
are the movement distances corresponding to
$\Delta t$
along the X, Y and Z axes of the WCS respectively. The corresponding training set of these models have been established, all of which contain more than 2 million samples, as shown in Fig. 5(a), (b) and (c).
Three classic RBF neural networks with three levels are designed to fit the data models, all of which have two nodes in the input layer and one node in the output layer, as shown in Fig. 6. The activation function is
\begin{equation} h_j=\exp{\left(-\frac{\left\| \vec{x}-\vec{c}_j \right\|^2 }{2b_j^2}\right)}\end{equation}
where
$\vec{x}=[v,\Delta t]^\mathrm{T}$
, weight matrix is
$W=[w_1,\cdots,w_m]^\mathrm{T}$
, then the output y(t) is obtained as shown below.
\begin{equation} y(t)=W^\mathrm{T}H=w_1h_1+\cdots+w_mh_m\end{equation}
5.1. Experiment results in real-time prediction
We established and trained these models in Python-3.7 under the TensorFlow framework. These models’ parameters are shown in Table II. The values of center vectors
$\vec{c}_j$
and
$b_j$
are parameters which are solved by using the least square method with the data set as shown in Fig. 5. Three RBF models are solved, then we get the prediction models
$F_x$
,
$F_y$
and
$F_z$
.
Table II. Parameters of three RBF network
Figure 5. The training data sets of the RBF-based prediction models, which are derived from 1500 tracks data. The red color and blue color represent the beginning and the end of the shuttlecock tracks, respectively.
Figure 6. Structure of the RBF network.
The proposed tracking and predicting method has been implemented in the vision servo system of the badminton robot system. The robot system only needs to get the state of drop point then it can take a reaction. After the models have been trained, we obtain the model parameters and rebuild the prediction model with C++ language. The experiment verifies that the prediction model can calculate the state of the drop point in 10 ms on average. The test configuration of the vision servo system: Windows 10, CPU:Ryzen5 2600x, 3.6GHz, 6 core 12 threads.
As shown in Fig. 7(a) and (b), the continuous prediction experiment has been conducted on a badminton court. First, after being filtered by the designed UKF algorithm in real time, the filtered values of the shuttlecock’s state is put into these RBF models, then we obtain the trajectory of the prediction in real time, as shown in Fig. 8(a), (b) and (c). Obviously, at the very beginning of the trajectory, the corresponding predicted X-axis, Y-axis and Z-axis coordinates of the fall point are deviated slightly from the filtered position value. As the prediction continues, the errors decrease dramatically. The absolute values of the position errors of the predicted drop point are shown in Fig. 9.
Figure 7. Continuously prediction testing.
Figure 8. Real-time prediction of the shuttlecock trajectory.
Figure 9. Absolute value of the prediction errors of the drop point.
6. Future work
The RBF-based prediction model can be used to implement the return strategies algorithm.
First, a model of the robot’s return strategies needs to be designed to calculate the type of trajectory (such as clear, punch clear and drop) for striking back, which may require the position of the player and target fall point, the features of trajectory (like the maximin height), etc. as input.
Second, the skill of controlling the ball is required. The RBF-based prediction model needs to be associated with an optimization function to obtain a suitably operated striking force and speed.
Finally, an evaluation function is also needed to judge whether the operated hitting force, the speed
$\omega$
and the selected direction of the striking are good enough.
7. Conclusion
In this paper, an infrared-based binocular vision system has been constructed. An aerodynamic analysis of the shuttlecock in different coordinate systems is proposed. Differing from the existing approaches, it is based on the velocity coordinates, the shuttlecock coordinates, the trajectory coordinates and the ground coordinates. Then the UKF algorithm is designed for tracking the measured value and eliminating the noise. Following this, a RBF-based prediction model is proposed to predict the state of the shuttlecock in real time. Finally, the experiment results verify that the proposed method satisfies the accuracy requirements of striking shuttlecocks in real time for a badminton robot. This algorithm is easy to use, efficient and highly accurate. It can be used to implement the algorithm of return strategies in the near future.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0263574721001053.
