With the intensification of global climate change and the increasing frequency of extreme weather events, the uncertainty of runoff processes continues to rise, posing severe challenges to global water resource security, ecosystem stability, and socio-economic development (Amoussou et al., 2020). However, reliable hydrological models for streamflow simulation are fundamental for water resources management such as; flood risk reduction, reservoir management, irrigation planning, and climate adaptation particularly in data-scarce regions exposed to hydro-climatic variability (Kumanlioglu et al., 2019; Li et al., 2022).

Process-based hydrological models (PBHM) such as, the TOPMODEL, the SWAT model and the GR4J model have been widely used for streamflow simulation (Afféwé et al., 2025; Janjić and Tadić, 2023; Perrin et al., 2003). These models are grounded in physical laws and empirical formulations, describing hydrological processes by means of discretized sets of ordinary differential equations or simplified sets of partial differential equations. In fact, PBHM models provide interpretability and physical consistency in hydrological science, their face challenges related to structural uncertainties, simplifying assumptions to represent complex physical processes, parameter calibration, and computational demands (Li et al., 2022) lead to systematic simulation errors. In parallel, machine learning (ML) methods have demonstrated strong predictive capability in hydrology. Tree-based ensemble techniques such as Random Forest (Breiman, 2001) and Extreme Gradient Boosting (Chen and Guestrin, 2016), as well as deep learning approaches such as Long Short-Term Memory networks (Hochreiter and Schmidhuber, 1997), are able to capture complex nonlinear relationships directly from data and widely used for streamflow simulation (Kumanlioglu et al., 2019; Nevo et al., 2022; Shen and Lawson, 2021; Zohou et al., 2023), but often lack interpretability regarding hydrological elements due to their “black-box” nature.

Whether we use ML model or a PBHM, the issue of accurate and interpretability of simulation arises, especially in regions where observed data are limited. If these challenges remain unaddressed, streamflow simulations may retain systematic biases and reduced reliability during extreme events. In data-scarce tropical basins, this can undermine the credibility of hydrological predictions used for flood preparedness, infrastructure planning, and climate change impact assessments. Improving predictive performance while maintaining physical coherence is therefore a critical challenge.

To overcome these limitations, recent research has explored hybrid modeling, integrating physically based models with ML techniques to benefit from the complementary strengths of physically based and data-driven models. Various integration strategies have been proposed, including model output fusion (Zhang et al., 2022), physics-guided ML where physical constraints are embedded into loss functions (Khandelwal et al., 2020), and feature augmentation using physically based models simulated variables as additional ML inputs (Kumanlioglu et al., 2019). Among these strategies, residual-based one of feature augmentation methods has been widely used for hybridization of PBHM and ML; assumes that a PBHM captures the dominant water balance dynamics, preserving physical consistency, while remaining systematic errors can be learned from data using machine learning model. In the Ouémé catchment, where this study is carried out, most existing work has focused on process-based hydrological models or a standalone machine learning (Afféwé et al., 2025; Biao et al., 2016; Obada et al., 2025; Zohou et al., 2023). To address these gaps, this study investigates a residual-based hybrid modeling framework combining the GR4J hydrological model with machine learning approaches. Specifically, the study addresses two research questions: (i) Does residual-based hybridization significantly improve the predictive performance of GR4J, and (ii) which machine learning family; tree-based models (Random Forest and XGBoost) or deep learning (LSTM) is most suitable for correcting GR4J residual errors and improving streamflow prediction in a tropical data-scarce catchment.

The Table 2 summarizes the performance of the GR4J model and the three hybrid approaches (GR4J–RF, GR4J–XGB, and GR4J–LSTM) over the training (1996–2012) and validation (2013–2020) periods using KGE, NSE, R2, and PBIAS metrics.

The standalone GR4J model provides a reasonable baseline performance, with a KGE of 0.82 during training and 0.75 during validation. The corresponding NSE and R2 values are 0.71 and 0.65 for the validation period, indicating a good ability to reproduce observed streamflow variability. The PBIAS values close to zero (-2.24 % for training and -0.86 % for validation) suggest a limited overall bias in simulated flows. All hybrid models outperform the standalone GR4J model across most performance metrics in both training and validation periods. The GR4J–RF model exhibits the highest performance among the tested approaches, achieving a KGE of 0.88 during training and 0.80 during validation. Similarly, NSE and R2 values increase substantially compared to GR4J alone, reaching 0.85 (training) and 0.73 (validation). The PBIAS remains close to zero, indicating a good balance between overestimation and underestimation of streamflow.

The GR4J–XGB model also demonstrates strong performance, in validation period and achieve KGE, NSE, and R2 values of 0.79, 0.73, and 0.73 respectively. Bias is minimal, with PBIAS values of 0.05 % during training and -0.36 % during validation, suggesting a stable correction of GR4J residuals across periods. The GR4J–LSTM model improves upon the standalone GR4J results but shows comparatively lower performance than the tree-based hybrid approaches. Validation KGE and NSE reach 0.78 and 0.69, respectively. The R2 value of 0.74 during validation indicates a reasonable reproduction of flow variability, while PBIAS values of 3.57 % (training) and 1.47 % (validation) reveal a slight tendency toward overestimation.

Overall, the results indicate that hybridization with machine learning models leads to systematic improvements in model performance relative to the standalone GR4J model, with the most pronounced gains observed for the GR4J–RF and GR4J–XGB approaches. These results align with these of Kumanlioglu et al. (2019), Shen (2018), Li et al. (2022) and Immerzeel et al. (2015), who reported that ML model have the capacity to balance correlation, variability, and bias, making integrated models ideal for data scarce regions watershed, where accurate water management is critical. Then residual-based hybrid models outperform the standalone Process-based hydrological models for streamflow simulation. These improvements suggest that the hybrid modeling approach can provide more reliable streamflow information, which is essential for effective water resources planning, flood forecasting and early warning systems, reservoir operation, irrigation water allocation and hydrological analysis in data-scarce tropical catchments.

The graphical comparison between observed and simulated streamflow highlights clear differences in performance among the tested models (Fig. 3). Although the model generally matched observed outflow, it tended to overestimate during low water periods from November to April and the high-water period, due to the effect of rainfall totals amount in June, July, August, September and October over the study area. These findings are consistent with those Japhet Kodja et al. (2018) and Afféwé et al. (2025), who reported that GR4J model overestimate discharge during the low-water and high-period. The standalone GR4J model reproduces the general seasonal dynamics of the basin but exhibits limitations in capturing peak flows and short-term variability. The hybrid GR4J–RF and GR4J–XGB models substantially improve streamflow simulation, particularly during high-flow events, by effectively correcting systematic residuals while preserving the physically based structure of GR4J. These results are consistent with the findings of Yang et al. (2024), who reported that applying conceptual models with machine learning reduces residual errors in conceptual model simulation and improves the accuracy of runoff predictions, particularly during extreme events. These tree-based hybrid approaches show strong consistency across training and validation periods, indicating robust generalization. In contrast, the GR4J–LSTM hybrid model improves temporal coherence relative to GR4J alone but remains less effective in correcting extreme flows, suggesting sensitivity to data volume and model configuration.

Overall, the figures demonstrate that residual-based hybrid modeling enhances streamflow prediction, with tree-based methods outperforming the deep learning approach in this application. The relatively lower performance of the GR4J–LSTM hybrid, compared to the tree-based methods (RF and XGBoost), can be attributed primarily to the data-intensive nature of deep learning models. LSTM networks, while powerful for capturing complex temporal dependencies, typically require large volumes of training data to generalize effectively and avoid overfitting (Shen, 2018). In data-scarce regions like the Ouémé catchment, the available daily record may be insufficient for the LSTM to fully learn the nuanced error patterns of the GR4J model (Zohou et al., 2023), especially given its substantial number of parameters and the need for careful hyperparameter tuning.

This study developed and evaluated a residual-based hybrid modeling framework that integrated the conceptual GR4J hydrological model with three machine learning techniques; Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Long Short-Term Memory (LSTM) networks for daily streamflow simulation in the data-scarce Ouémé River basin. The key finding is that all hybrid models (GR4J–RF, GR4J–XGB, and GR4J–LSTM) outperformed the standalone GR4J model across multiple performance metrics (KGE, NSE, R2, PBIAS), confirming the value of using machine learning to correct systematic errors in physically based simulations. The proposed hybrid modeling framework provides improved streamflow simulations compared with the standalone GR4J model and offers a promising tool for supporting water resources management. The GR4J–LSTM hybrid also provided improvement over the base model but to a lesser extent, a result largely attributable to the data-intensive nature of deep learning, which poses a significant constraint in regions with limited historical records.

Future research should test the performance of this approach across different sub-basins of the Ouémé River Basin to better understand the spatial variability of hydrological processes and model responses between basins. Further work should also focus on the use of large-scale, publicly available datasets such as Caravan a global hydrometeorological dataset could provide the extensive and diverse data needed to better train and validate deep learning models like LSTM, potentially unlocking their full potential within hybrid frameworks in data-scarce environments.