Dual-Stage Attention-Based LSTM Network for Multiple Time Steps Flood Forecasting

Flood forecasting plays a crucial role in supporting decision-making for flood management. In addition to conceptual and physical-based models, the data-driven models have garnered increasing attention in recent years. The proposed model in this study employs LSTM networks, Encoder-Decoder framework, as well as feedback and attention mechanism to effectively utilize diverse observed data and future rainfall as inputs for multiple time steps flood forecasting. The accuracy and reliability of the model have been validated across case studies in multiple watersheds in China. The results demonstrate the high performance of the LSTM-based flood forecasting model. Meanwhile, the efficacy of both the feedback mechanism and attention mechanism has been validated in the domain of flood prediction.

1 Introduction

The artificial neural network (ANN) stands out as the most widely adopted algorithm among data-driven flood forecasting models, showcasing exceptional performance and remarkable stability. After years of development, the simple feedforward network has evolved into sophisticated architectures such as radial basis function (RBF) network and recurrent neural network (RNN) (Xiang et al., 2020; Kao et al., 2020), among which, the RNN with gating mechanism, such as the LSTM network (Hochreiter and Schmidhuber, 1997), has shown great performance in the field of series prediction. The structure of RNN can effectively utilize the information inherent in time series data, making it highly suitable for predicting rainfall and runoff series characterized by significant autocorrelation and cross-correlation.

Based on the CAMELS dataset (Addor et al., 2017; Newman et al., 2015), the LSTM network has been employed to construct runoff prediction models for multiple watersheds in the United States (Kratzert et al., 2018). Thapa et al. (2020) developed an LSTM-based discharge model driven by snowmelt in a Himalayan basin, which utilized MODIS snow images and remotely sensed meteorological data as input variables. The LSTM network and seq2seq architecture have been utilized to construct flood forecasting models for Clear Creek and Upper Wapsipinicon River in the United States, demonstrating the effective application of the LSTM-seq2seq model in short-term flood forecasting (Xiang et al., 2020). The LSTM network and Encoder-Decoder framework have been adopted to achieve multi-step prediction, providing a maximum lead time of 6 h for reservoir inflow prediction (Kao et al., 2020). The LSTM-based flood forecasting model developed by Dazzi et al. (2021) utilized the observed upstream and downstream water levels as inputs to accurately predict the flood stages of critical gauge stations.

In this paper, a flood forecasting model based on the LSTM network, Encoder-Decoder framework, as well as the feedback and attention mechanism are proposed to achieve multiple time steps flood forecasting. Three representative watersheds with distinct climate and underlying surface characteristics are selected to conduct case studies to validate the performance of the proposed model.

2 Methodology

2.1 LSTM-based flood forecasting model

The internal structure of the LSTM cell is illustrated in Fig. 1, encompassing three gates for regulating information: the “forget gate”, “input gate”, and “output gate”.

Figure 1 Internal structure of LSTM cell.

The information processing in “forgetting gate” is shown in Eq. (1), which mainly controls the information to be discarded in the internal state.

$$\begin{array}{}\text{(1)}& {\mathit{z}}_{\mathrm{f}}^{\left(t\right)}=\mathit{\sigma }\left({\mathbf{W}}_{\mathrm{f}}\left[{\mathit{h}}^{\left(t-\mathrm{1}\right)};{\mathit{x}}^{\left(t\right)}\right]+{\mathit{b}}_{\mathrm{f}}\right)\end{array}$$

In which, σ is sigmoid function, W_∗ is weight matrix, b_∗ is bias weight, h^(t−1) is the hidden state of previous time step, x^(t) is the input at time t.

The computation of vectors in the “input gate” is illustrated in Eq. (2), which determines the novel information incorporated into the internal state at each time step. The candidate states are obtained through a feedforward network layer using the “tanh” function as the activation function, as depicted in Eq. (3).

$$\begin{array}{}\text{(2)}& {\displaystyle }{\mathit{z}}_i^{\left(t\right)}=\mathit{\sigma }\left({\mathbf{W}}_i\left[{\mathit{h}}^{\left(t-\mathrm{1}\right)};{\mathit{x}}^{\left(t\right)}\right]+{\mathit{b}}_i\right)\text{(3)}& {\displaystyle }{\stackrel{\mathrm{̃}}{\mathit{c}}}^{\left(t\right)}=\tanh \left({\mathbf{W}}_{\stackrel{\mathrm{̃}}{c}}\left[{\mathit{h}}^{\left(t-\mathrm{1}\right)};{\mathit{x}}^{\left(t\right)}\right]+{\mathit{b}}_{\stackrel{\mathrm{̃}}{c}}\right)\end{array}$$

The “output gate” process, depicted in Eq. (4), governs the transmission of information from the internal state to the hidden state at each time step. Nonlinear transformation is also essential in the output process and can be achieved through activation functions, as illustrated in Eq. (5).

$$\begin{array}{}\text{(4)}& {\displaystyle }{\mathit{z}}_{\mathrm{o}}^{\left(t\right)}=\mathit{\sigma }\left({\mathbf{W}}_{\mathrm{o}}\left[{\mathit{h}}^{\left(t-\mathrm{1}\right)};{\mathit{x}}^{\left(t\right)}\right]+{\mathit{b}}_{\mathrm{o}}\right)\text{(5)}& {\displaystyle }{\mathit{h}}^{\left(t\right)}={\mathit{z}}_{\mathrm{o}}^{\left(t\right)}\odot \tanh \left({\mathit{c}}^{\left(t\right)}\right)\end{array}$$

The proposed forecasting model is referred to as LSTM-DSA, with DSA representing the dual stage attention mechanism. The architectural depiction of LSTM-DSA can be observed in Fig. 2. The dual stage attention operates on both the Encoder and Decoder components individually. The Encoder part utilizes the attention mechanism to enable adaptive selection among multiple driving factors, thereby facilitating the utilization of point rainfall data as input instead of area rainfall data. In the Decoder part, the attention mechanism assigns weight coefficients to the hidden state of the encoder at each time step. Additionally, the model encompasses a feedback mechanism within the Decoder part to utilize future rainfall data as additional input.

Figure 2 Architecture of the LSTM-DSA model.

2.2 Benchmark model

The forecasting model based on Encoder-Decoder framework proposed by Kao et al. (2020) is utilized as the benchmark model, named as LSTM-ED, whose architecture is illustrated in Fig. 3.

Figure 3 Architecture of the benchmark model.

2.3 Evaluation metrics

The model performance is assessed based on the simulated peak value, peak occurrence time, and flood process. The evaluation metrics include the relative error of the peak, error in peak occurrence time, and Nash efficiency coefficient.

The relative error of the peak is calculated as follows:

$$\begin{array}{}\text{(6)}& {\mathit{\delta }}_{\mathrm{RPE}}={\displaystyle \frac{Q_{max}^{\mathrm{sim}}-Q_{max}^{\mathrm{obs}}}{Q_{max}^{\mathrm{obs}}}}\times \mathrm{100}\mathit{\%}\end{array}$$

In which, $Q_{max}^{\mathrm{sim}}$ (m³ s⁻¹) and $Q_{max}^{\mathrm{obs}}$ (m³ s⁻¹) are the peak value of the simulated and the observed sequence, respectively.

The computation of the peak time error is as follows:

$$\begin{array}{}\text{(7)}& {\mathrm{\Delta }}_{\mathrm{PTE}}=T_{\mathrm{peak}}^{\mathrm{sim}}-T_{\mathrm{peak}}^{\mathrm{obs}}\end{array}$$

Where, $T_{\mathrm{peak}}^{\mathrm{sim}}$ (h) is the peak occurrence time of the simulated sequence and $T_{\mathrm{peak}}^{\mathrm{obs}}$ (h) is the peak occurrence time of the observed sequence.

Equation (8) shows the computation of the Nash efficiency coefficient, which quantifies the overall discrepancy between the simulated and observed sequence.

$$\begin{array}{}\text{(8)}& C_{\mathrm{NSE}}=\mathrm{1}-{\displaystyle \frac{{\sum }_{t=\mathrm{1}}^M{\left(Q_t^{\mathrm{obs}}-Q_t^{\mathrm{sim}}\right)}^{\mathrm{2}}}{{\sum }_{t=\mathrm{1}}^M{\left(Q_t^{\mathrm{obs}}-{\overline{Q}}^{\mathrm{obs}}\right)}^{\mathrm{2}}}}\end{array}$$

In which, $Q_t^{\mathrm{obs}}$ and $Q_t^{\mathrm{sim}}$ represent the value of the observed and simulated sequence, respectively. ${\overline{Q}}^{\mathrm{obs}}$ is the mean value of the observed sequence.

3 Study Watersheds and Data

In order to comprehensively assess the performance of the proposed model, we carefully select three watersheds as study areas, each characterized by distinct climatic and geomorphic features. The three watersheds are situated within Shaanxi Province, China, and boast well-maintained rain gauges and hydrological stations, along with an extensive collection of observational data. The basic information the study watersheds is shown in Table 1, and the locations are shown in Fig. 4.

Table 1 Basic information of the study watersheds.

Figure 4 Locations of the study watersheds.

4 Experiments

4.1 Model inputs and outputs

The input steps of the LSTM-based models for the three watersheds are determined as 3, 9 and 9 h by analysing the ACF and PACF of the rainfall and runoff data comprehensively as well as conducting preliminary modelling analysis. The inputs and outputs of each flood forecasting model are illustrated in Table 2.

Table 2 Model inputs and outputs.

4.2 Data pre-processing and sample division

The linear normalization method is employed for pre-processing, as indicated in Eq. (9), to account for the disparities in dimensions and scales between rainfall and runoff data. This approach ensures improved accuracy and convergence speed of the model.

$$\begin{array}{}\text{(9)}& x_{\mathrm{norm}}={\displaystyle \frac{x-x_{min}}{x_{max}-x_{min}}}\end{array}$$

The sample set is constructed using normalized rainfall and runoff data. Firstly, 20 % of all flood events are randomly selected as testing events. Secondly, the remaining 80 % of the events are sliced according to input and output steps to generate samples, from which 10 % are randomly chosen as validation set. Finally, the remaining samples constitute the training set. Table 3 shows the number of samples in each set.

Table 3 Sample set information.

4.3 Settings and training

The forecasting model is constructed based on the Keras in Python, with a single-layer LSTM utilized for both the Encoder and Decoder layer, while a full connection layer serves as output layer. The “MSE”, “Adam” and “ReLU” are selected as loss function, optimizer and activation function, respectively. The other hyperparameters of the model are determined through trial calculations with the loss function as indicators, wherein the number of neurons and batch size are optimized using powers of 2, which are presented in Table 4.

Table 4 Hyperparameters of models.

5 Results and Discussion

The evaluation metric distributions of the prediction results for each model are depicted in Figs. 5–7 respectively. The predicted hydrographs for typical flood events in each watershed are illustrated in Figs. 8–10.

Figure 5 Metrics distribution for models in Xichuan.

Figure 6 Metrics distribution for models in Bahe.

Figure 7 Metrics distribution for models in Yuehe.

Figure 8 Results for the “19910610” event in Xichuan.

Figure 9 Results for the “19860909” event in Bahe.

Figure 10 Results for the “20100718” event in Yuehe.

For Xichuan watershed, the Nash efficiency coefficient and relative error of flood peak significantly deteriorate when the prediction step exceeds 3 h. Overall, the estimations of flood peaks are slightly lower than the observed value. The performance of the LSTM-DSA model, meanwhile, surpasses that of the LSTM-ED model.

For Bahe watershed, the performances of both models decrease with the growing of prediction step. For flood peaks, the predicted values are higher than the observed one. The evaluation metrics show improvement with the inclusion of future rainfall data.

The metrics distributions in Yuehe watershed exhibit a comparable phenomenon. The prediction accuracy of both models decreases as the prediction step extends. The estimations of flood peaks are lower than the observed value. Incorporating feedback and a dual-stage attention mechanism enhances the model performance.

The experiments reveal that the performance of all models has exhibited varying degrees of decline as the prediction step extends. The attenuation of information in the input data is believed to contribute to this portion of the prediction error. This phenomenon has been effectively mitigated through the implementation of the feedback mechanism. Overall, the performance of the LSTM-DSA model is greatly enhanced by incorporating feedback and a dual-stage attention mechanism, surpassing that of the LSTM-ED model. The inclusion of the attention mechanism has significantly enhanced prediction accuracy for the Bahe and Yuehe watersheds, whereas its impact on the Xichuan watershed is less pronounced, suggesting that the attention mechanism proves more effective in larger-scale watersheds.

Although we employ a sophisticated model structure, accurate predicting certain floods remains challenging. We believe this error can be attributed to two factors. Firstly, the input of each model solely relies on rainfall and runoff data without considering other relevant variables. The information contained in the input data is insufficient for precise flood prediction. Secondly, there is still room for improvement in optimizing the model, refining the loss function, adjusting batch size, and fine-tuning the hyperparameters.

6 Conclusions

The paper introduces the LSTM-DSA, a dual-stage attention-based LSTM network for multiple time steps flood forecasting. Three representative watersheds are selected as study areas to access the model performance. The experiment results are evaluated, and the sources of prediction errors are analysed. In summary, the proposed model exhibits a significantly high level of accuracy in predicting floods within distinct prediction steps, demonstrating its exceptional predictive capability. The efficacy of both the feedback mechanism and attention mechanism has also been validated in the domain of flood prediction.