2.1 Overall architecture

The flowchart in Figure 1 shows the fishing vessel operation identification process, divided into two parts: basic feature extraction with 1D CNN and time-domain feature extraction with LSTM.

Figure 1. Flowchart of fishing vessel operations identification

2.2 1D CNN model

The CNN model is a deep learning model (Krizhevsky et al., Reference Krizhevsky, Sutskever and Hinton2017), including 1D-CNN, 2D-CNN and 3D-CNN. 1D-CNN is generally used for processing sequential data, and 2D-CNN for image processing. 3D-CNN incorporates input time dimension and is primarily applied in video processing (Lecun and Bottou, Reference Lecun and Bottou1998; Teng et al., Reference Teng, Chen, Liu, Cheng and Sun2021; Mohtavipour et al., Reference Mohtavipour S, Saeidi and Arabsorkhi2022). 1D-CNN can analyse the local features of 1D data. In this study, a 1D-CNN is employed to analyse the acquired fishing vessel trajectory data. Figure 2 illustrates its specific structure.

Figure 2. Convolution process of 1D-CNN

The left side of the figure represents the input time series data, which is 3D. The convolution is performed from top to bottom, and the red and blue areas denote the convolution kernels of size 2 × 3. The resulting feature map from the convolution is denoted as N*x, where N is related to the data dimensions, kernel size and convolution stride, and x represents the dimensions of the input data. The input matrix for fishing vessel trajectory data in this study is four-dimensional, using 64 convolutional kernels.

2.3 LSTM model

LSTM is a type of recurrent neural network (RNN) applied to the time dimension and incorporates memory units within its hidden layers to manage the memory information of time series (Hochreiter and Schmidhuber, Reference Hochreiter and Schmidhuber1997). LSTM possesses long-term memory capabilities. It can transmit and express information in long time series while using useful information, providing a solution to the issue of vanishing and exploding gradients encountered by simple RNNs. This study uses the LSTM model to identify fishing vessel operational behaviours based on vessel trajectory data.

The LSTM model includes three gate structures: the forget gate, input gate and output gate. Figure 3 shows the unit structure of the network. The circular elements represent point-wise operations on vectors, while rectangles signify neural network layers annotated with the activation function used in that layer. Mathematically, LSTM can be defined as in Equations (1)–(6).

(1)\begin{gather}{i_t}\textrm{ = }\sigma ({{W_{ii}}{x_t}\textrm{ + }{b_{ii}}\textrm{ + }{W_{hi}}{h_{t - 1}}\textrm{ + }{b_{hi}}} )\end{gather}

(2)\begin{gather}{f_t}\textrm{ = }\sigma ({{W_{if}}{x_t}\textrm{ + }{b_{if}}\textrm{ + }{W_{hf}}{h_{t - 1}}\textrm{ + }{b_{hf}}} )\end{gather}

(3)\begin{gather}{g_t}\textrm{ = tanh}({{W_{ig}}{x_t}\textrm{ + }{b_{ig}}\textrm{ + }{W_{hg}}{h_{t - 1}}\textrm{ + }{b_{hg}}} )\end{gather}

(4)\begin{gather}{o_t}\textrm{ = }\sigma ({{W_{io}}{x_t}\textrm{ + }{b_{io}}\textrm{ + }{W_{ho}}{h_{t - 1}}\textrm{ + }{b_{ho}}} )\end{gather}

(5)\begin{gather}{c_t}\textrm{ = }f \odot {c_{t - 1}}\textrm{ + }{i_t} \odot {g_t}\end{gather}

(6)\begin{gather}{h_t}\textrm{ = }{o_t} \odot \textrm{tanh}({{c_t}} )\end{gather}

where x is the input series, h is the state of the hidden layer, c is the state of the unit structure, i_t is the coefficient of the input gate, f_t is the coefficient of the forget gate, o_t is the coefficient of the output gate, W is the learning weight and b is the learning bias.

Figure 3. Unit structure of LSTM

2.4 Fishing vessel operational behaviour identification model based on 1D CNN-LSTM

We combined the characteristics of CNN and LSTM to enhance the accuracy of fishing vessel behaviour identification, proposing a hybrid 1D CNN-LSTM model. This proposed hybrid model takes multidimensional time series data as input and predicts the operational behaviour types of fishing vessels as output. Figure 4 illustrates its structure. This model first extracts features with CNN, which consists of two layers of 1D convolutional layers and one maxpooling layer. A flattened layer follows to transform the data into a format suitable for LSTM, integrating spatial feature extraction from CNN with time series representation from LSTM. In addition, to address the issue of overfitting in deep neural networks, a dropout layer is introduced between the flattened and LSTM layers. Dropout (Hinton et al., Reference Hinton, Srivastava, Krizhevsky, Sutskever and Salakhutdinov2012) temporarily drops neural units from the network during training based on a defined probability, simplifying the network structure. In this model, the dropout coefficient is set to 0 ⋅ 5.

Figure 4. Structure of the 1D CNN-LSTM model

Activation functions introduce nonlinear factors into neural networks. They can solve complex problems through nonlinear combinations, enhancing the expressive and learning capabilities of the target network. Commonly used activation functions include the sigmoid function (Kyurkchiev and Markov, Reference Kyurkchiev and Markov2015), the tanh function (Zhang et al., Reference Zhang, Liu, Wei and Zhang2021) and the rectified linear unit (ReLU) function (Li et al., Reference Li, Tang and Yu2020). ReLU somewhat improves the vanishing gradients problem compared to the other two functions, offering faster computation and convergence rates. Its mathematical formula is given by:

(7)\begin{equation}\textrm{ReLU}(x )\textrm{ = }\left\{ {\begin{array}{*{20}{l}} {x,\; }& {x \ge 0}\\ {0,\; }& {x\mathrm{\ < }0} \end{array}} \right.\end{equation}

3.1 Experimental data

The experimental data originates from location data of fishing vessels equipped with BeiDou devices in the Tianchi Lab. It includes dynamic information, such as the ID, speed and direction of each fishing vessel. Table 1 lists partial trajectory information for vessel ID 20042. Complete trajectory point information is obtainable based on the dynamic information from BeiDou and the static information registered by the fishing vessels. Considering this, it is possible to generate a complete trajectory series of a vessel during operational activities. Figure 5 shows the trajectory series, where (a), (b) and (c) represent gill net, trawl and purse seine, respectively. Preliminary observations from the figure indicate that fishing vessel trajectories vary with behaviour.

Table 1. Information of fishing vessels

Figure 5. Vessel trajectories: (a) gill net, (b) trawl and (c) purse seine

The dataset contained certain anomalies. To address this, the study randomly divided the database multiple times into a training set (75%) and a test set (25%) and recorded the misjudgement rates of each trajectory. Trajectories with high misjudgement rates were manually assessed for potential anomalies. A data point was removed from the database if identified as an anomaly. This approach significantly mitigated the impact of anomalous data on algorithm training, enhancing the detection accuracy of the model. Some instances of anomalous data, such as those associated with vessel IDs 20355, 20544 and 26536, appeared due to sensor malfunctions. Vessel ID 28004 was labelled as a purse seine vessel in the database, but its actual trajectory did not bear the characteristics of such a vessel, suggesting the likelihood of anomaly. Vessel ID 20300, marked as a purse seine vessel, maintained speeds consistently between eight and 10 knots, indicating that it belongs to the transport state rather than the purse seine operation state. Vessel ID 28098, identified as a gill net vessel, lacked the characteristics of gill net operations in its trajectory and maintained an average speed exceeding eight knots. Thus, it was deemed as anomalous data, as shown in Figure 6.

Figure 6. Abnormal data of vessel trajectories: (a) ID: 20355, (b) ID: 20544, (c) ID: 26536, (d) ID: 28004, (e) ID: 20300 and (f) ID: 28098

3.2 Evaluation methods

Experiments were conducted using the aforementioned dataset and evaluation standards to validate the effectiveness of the proposed hybrid model. Accuracy (Accuracy) was employed to measure classification precision, which is expressed as:

(8)\begin{equation}\textrm{Accuracy = }\left( {\sum\limits_i^n {\frac{{\textrm{T}{\textrm{P}_i}\textrm{ + T}{\textrm{N}_i}}}{{\textrm{T}{\textrm{P}_i}\textrm{ + F}{\textrm{P}_i}\textrm{ + F}{\textrm{N}_i}\textrm{ + T}{\textrm{N}_i}}}} } \right)\textrm{/}n\end{equation}

where TP_i is the number of correctly predicted positive samples in a class, TN_i is the number of correctly predicted negative samples in a class, FP_i is the number of negative samples in a class that were incorrectly predicted as positive, FN_i is the number of positive samples in a class that were incorrectly predicted as negative, i is the class label and n is the number of classes.

3.3 Experimental results and analysis

The width and size of convolutional kernels are essential parameters for 1D CNN. By adjusting these parameters, the detection performance of the model can be enhanced. This study employed convolutional kernels with a width range of 1–12. The number of convolutional kernels was initialised as 6 and gradually increased to 16. Figure 7 shows the detection results of the model. The precision of the model consistently improves with a kernel width below 5, while the loss function decreases. Regarding the number of kernels, precision gradually improves before k reaches 10, coinciding with a decline in the loss function. Thus, the optimal model parameters were determined as w = 5 for the kernel width and k = 10 for the kernel quantity.

Figure 7. Detection accuracy and loss function curves of convolutional kernels with different widths and quantities: (a) identification accuracy, (b) loss function curve

We utilised the confusion matrix obtained from the test set, as illustrated in Figure 8, to verify the identification performance of the proposed model. Based on the matrix, the identification accuracy for purse seine operations was the lowest, at 91 ⋅ 6%. Out of the 1,296 purse seine vessels in the test set, five were misclassified as gill net vessels, and 103 were misclassified as trawl vessels. The identification accuracy for gill net operations was at 93 ⋅ 5%. Fifty-two instances were misclassified as trawl vessels out of the 8,810 gill net vessels in the test set. For trawl operations, the identification accuracy was 92%. Thirty-one were misclassified as purse seine vessels and 229 were misclassified as gill net vessels out of 3,440 trawl vessels in the test set. Overall, classification errors with each class indicated certain similarities among the three operational methods, such as similar fishing speeds.

Figure 8. Normalised confusion matrix of classification for purse seine, gill net and trawl

This study compared the classification results of SVM, traditional DCNN and LSTM algorithms with the proposed model to further validate the identification accuracy of the proposed model. Table 2 lists the identification results. The classification results show that LSTM outperformed DCNN regarding identification accuracy, achieving 87% and F1 score of 87%. The 1D CNN-LSTM obtained a higher accuracy of 92% and F1 score of 93%. In addition, the experiment added Recall, Precision and F1 Score as the evaluation criteria of the model based on the overall accuracy, to verify the effect of various algorithms on identifying each type of fishing operation behaviour. The results show that the evaluation indexes of trawling and gillnet were better than those of purse Seine, which may be caused by the more flexible way fishing vessels conduct purse seine operations. This signifies that the proposed model, combining spatial feature extraction from DCNN and the time-series representation capability of LSTM, achieves superior effectiveness in identifying fishing vessel operational classes.

Table 2. Identification results of different models