the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Comparative study from groundbased rain gauges vs. rainfall products at different time steps in the southeast of the Republic of Djibouti
Golab Moussa Omar
JeanEmmanuel Paturel
Christian Salles
Gil Mahé
Mohamed Jalludin
Frédéric Satgé
Mohamed Ismail Nour
Abdillahi Hassan Hersi
The Republic of Djibouti is a small country in the Horn of Africa and as for most regions of Africa, ground rainfall stations are sparse. This study aims to compare at different time steps (annual, monthly, and daily) 15 rainfall estimation products (Pdatasets) to 5 reference groundbased rainfall stations, over the period of 1980–1990. To classify the reliability of these products, several metrics were considered, the Kling Gupta Efficiency (KGE) and the Heidle Skills Scores (HSS at daily time step). For the study area, 5 Pdatasets present high KGE values at annual and monthly time step: EWEMBI, GPCC, JRA55 Adj, MSWEP and WFDEIGPCC; the best 5 performing products at daily time step are CHIRPS, EWEMBI, JRA55 Adj, MSWEP and WFDEIGPCC. To complete this comparative analysis, it will be necessary to perform an analysis at the watershed scale and on a new temporal window.
 Article
(1821 KB)  Fulltext XML
 BibTeX
 EndNote
UPH 16; field observation; precipitationdatasets; Republic of Djibouti
The Republic of Djibouti is a small country (23 200 km^{2}) located in the Horn of Africa and as for most regions of Africa, ground rainfall stations are sparse and unequally distributed.
To palliate the low density of the rainfall observation network in some areas, there are alternative or complementary products we can find on the Web such as rainfall estimate products (PDatasets). Since the 1980s, these PDatasets have blossomed and offer interesting opportunities because, usually the dataset is continuous, and spatially available and, rainfall data are assessed at daily or hourly time steps.
We can define three groups of rainfall estimation products according to the input data and rainfall data retrieval process: those based on spatial information from ground measurement points, those based on reanalysis data derived from physical and dynamic models, and those based on satellite information using passive microwave (PMW) and infrared (IR) information (Le Coz and van de Giesen, 2020; Satgé et al., 2020). Lots of combinations of these three groups exist on the WEB.
The objective of this study is to determine which of these rainfall products is the most reliable compared to groundbased rainfall stations for the southeast of Djibouti. The southeast zone concentrates most of the population and economic exchanges between Djibouti and its neighbouring countries. Therefore, our study focuses on this part of the country This study is the first component of a broader work that aims to improve the knowledge and understanding of the hydrological processes involved in the Ambouli basin. The Ambouli basin is one of the country's largest catchments (794 km^{2}) and is of major importance. The Ambouli aquifer is the main source of drinking water for the city of Djibouti. The Ambouli basin. is of particular importance for flood risk of the city of Djibouti.
2.1 Study area
The whole territory of the Republic of Djibouti presents an arid tropical climate which is characterized by the irregularity and the weakness of precipitations, by high temperatures during all the year, by the absence of perennial water course and by a very intense evaporation. The climate of the country is characterized by two distinct seasons: a cool season from October to April and a hot season from May to September (HoumedGaba, 2009). Due to geology of volcanic origin, the Republic of Djibouti is characterized by a very steep relief which is a succession of massifs, plateaus and plains also represented in the southeastern part (study area).
2.2 Rain gauges
There are 14 rainfall stations in the southeast of the Republic of Djibouti managed by the “Agence Nationale de la Météorologie de Djibouti (ANM)”. Out of 14 stations, only 5 have at least 10 months or more of data available per year. Thus, we use a network of 5 rain gauges: Aerodrome, Serpent, AliSabieh, HolHol and Loyada (Fig. 1). The rainfall data from the network cover the period 1951–1990. The rainfall are at monthly time step for 4 rain gauge stations (Serpent, AliSabieh, HolHol and Loyada) and the Djibouti Aerodrome station is at daily time step (over the period 1981–2020). These rainfall data are regulated by the quality control and quality assurance system of the World Meteorological Organization (WMO).
2.3 PDatasets
Products selection for testing was motivated by their availability over the region of interest and as they are commonly used in operational or research studies focusing on hydrology or agronomy. Table 1 presents temporal and spatial characteristics of the PDatasets.
This study assesses the performance of 15 precipitation estimation products at daily, monthly, and annual time step against the ground rainfall stations of the southeastern zone of the Republic of Djibouti.
The prerequisite for the choice of studied period is a shared time window between PDatasets and rain gauge data. Since the PDatasets are only available from the beginning of the 1980's, the common study period is 1980–1990.
The PDatasets validation is carried out on a spatially aggregated level: pointtopixel. The grid values of the precipitation estimation products containing each rain gauge station are extracted and pairwise compared.
The performances of PDatasets were evaluated using the quantitative metrics (KGE and MBE) and categorical indices (POD, FAR, Accuracy, Error and HSS), at annual, seasonal, monthly and daily temporal scale
3.1 Preprocessing (pretreatment)
As the PDatasets differ in spatial resolution – ranging from 0.0375° for TAMSAT v.3 to 1° for GPCC v.7 – and for easy comparison, the PDatasets were resampled to a spatial resolution of 0.1° by 0.1° using bilinear interpolation (Nikulin et al., 2012; Akinsanola et al., 2016; Satgé et al., 2020). Moreover, to address the uniformity in the temporal span, the PDatasets available at a subdaily time step were aggregated to a daily time step records over the period of study.
3.2 Metrics
The comparative study includes quantitative and categorical metrics. Although wide range of metrics are available to assess data performance (Wang et al., 2003; Segele et al., 2008; Diro et al., 2009; Ayehu et al., 2018), there is no single one that encapsulates all aspects of interest. For this reason, it is useful to consider several metrics and to understand the type of information or insight they might provide (Akinsanola et al., 2016).
3.2.1 Quantitative metrics
The metrics used for the ground to rainfall products statistical comparison are:

Kling Gupta Efficiency (KGE) is the Euclidean distance computed using the coordinates of bias ($\mathit{\beta}=\frac{\stackrel{\mathrm{\u203e}}{y}}{\stackrel{\mathrm{\u203e}}{x}}$), standard deviation ($\mathit{\gamma}=\frac{\mathrm{1}}{\mathit{\beta}}\times \frac{{\mathit{\sigma}}_{y}}{{\mathit{\sigma}}_{x}}$) and correlation coefficient $\left(r=\frac{\sum _{i=\mathrm{1}}^{n}({x}_{i}\stackrel{\mathrm{\u203e}}{x})({y}_{i}\stackrel{\mathrm{\u203e}}{y})}{\sqrt{\sum _{i=\mathrm{1}}^{n}({x}_{i}\stackrel{\mathrm{\u203e}}{x}{)}^{\mathrm{2}}}\surd \sum _{i=\mathrm{1}}^{n}({y}_{i}\stackrel{\mathrm{\u203e}}{y}{)}^{\mathrm{2}}}\right)$ (Gupta et al., 2009; Clark et al., 2021):
$$\begin{array}{}\text{(1)}& \mathrm{KGE}=\mathrm{1}\sqrt{{\left(r\mathrm{1}\right)}^{\mathrm{2}}+{\left(\mathit{\beta}\mathrm{1}\right)}^{\mathrm{2}}+(\mathit{\gamma}\mathrm{1}{)}^{\mathrm{2}}}\end{array}$$ 
Mean Bias Eror (MBE) is used to estimate the average bias in the rainfall product, and it provides a good indication of the mean overestimate of predictions. A positive value of MBE means an overestimation:
$$\begin{array}{}\text{(2)}& \mathrm{MBE}={\displaystyle \frac{\mathrm{1}}{n}}\sum _{i=\mathrm{1}}^{n}{x}_{i}{y}_{i}\end{array}$$where x corresponds to the ground reference rainfall and y corresponds to the PDatasets.
3.2.2 Categorical metrics
The performances of rainfall estimate products are also analyzed using a, b, c and d categorical indices. Five statistical indicators based on a contingency table (Table 2) are computed:

Probability of detection (POD):
$$\begin{array}{}\text{(3)}& {\displaystyle \frac{a}{a+c}}\end{array}$$ 
False Alarm Ratio:
$$\begin{array}{}\text{(4)}& \mathrm{FAR}={\displaystyle \frac{b}{a+b}}\end{array}$$ 
Accuracy:
$$\begin{array}{}\text{(5)}& A={\displaystyle \frac{a+d}{a+b+c+d}}\end{array}$$ 
Error
$$\begin{array}{}\text{(6)}& E={\displaystyle \frac{b+c}{a+b+c+d}}\end{array}$$ 
Heidle Skill Scores
$$\begin{array}{}\text{(7)}& \mathrm{HSS}={\displaystyle \frac{\mathrm{2}\cdot (a\cdot db\cdot c)}{[\left(a+c\right)\cdot \left(c+d\right)+\left(a+b\right)\cdot \left(b+d\right)]}}\end{array}$$
4.1 Annual Comparisons (Fig. 2)
Strong agreement with the rain gauge data is observed for some of the Pdatasets. Five PDatasets have a high KGE – EWEMBI, GPCC, JRA_55_Adj, MSWEP and WFDEI_GPCC. WFDEI_GPCC is the best PDatasets whereas MERRA2 PTC and ERA 5 had the worst performance.
Eight of rainfall estimates products – CPC, ERA5, EWEMBI, GPCC, JRA55_Adj, MERRA2_PT, MERRA2_PTC and WFDEI_GPCC – overestimated rainfall with positive MBE values.
Loyada rain gauge has the lowest KGE except for the rainfall estimation products TAMSAT and WFDEI_CRU. This can be explained either by the location of the station or the digitalization of the data (because it is a manual rain gauge station, when data are collected and then they are digitalized by a technician which can generate an uncertainty.
4.2 Monthly comparisons (Table 3)
Regarding the KGE, all PDatasets systematically present values within −0.16 and 0.70 which is much more efficient than the annual time step. The top five PDatasets at monthly time step are the same as those at annual time step: EWEMBI, GPCC, JRA55_Adj, MSWEP and WFDEI_GPCC. WFDEI_GPCC is always the best among these fives. As for the annual time step, MERRA2 PTC corresponds to the least reliable PDataset.
4.3 Daily comparisons (Table 4)
For the daily time step analyses, we use data from Djibouti Aerodrome station for the period 1981–2005 which is the common period between the station and all the PDatasets. The GPCC product is not present in the analysis of daily time step data because it has a monthly temporal resolution.
The ability of the PDatasets to quantify the amount of daily precipitation is relatively low, with most products having negative KGE values (Table 4). Only the best performing products at the annual and monthly time step EWEMBI, JRA55_Adj, MSWEP v.2.2, and WFDEI_GPCC have positive KGE values. Concerning MBE, values are relatively homogeneous for all products from −0.13 to 0.42 mm. The low value of MBE is due to the low number of rainy days.
Table 4 shows the results of the POD, FAR, Accuracy, Error and HSS indices. Regarding POD, the PDatasets MERRA2 (PT and PTC) and ERA5 (the worst performing products at annual and monthly time step) almost do not detect nonrainy days, which explain the almost perfect POD values for these products. EWEMBI, JRA55 Adj, and WFDEIGPCC (the best performing products at the annual and monthly time step) show POD values <0.7. Regarding FAR, we observe relatively low values for the products (FAR > 0.9 except for CHIRPS which presents a FAR equal to 0.75). For the Accuracy and Error indices, the results are moderately satisfactory for most of the products; however, as for the FAR values, the CHIRPS product presents the best Accuracy and Error values, respectively 0.9 and 0.1.
Satgé et al. (2020), defined a nonrainy day using several threshold values ranging from 0 to 25 mm were applied for the PDatasets. They found that maximum value of HSS was obtained with a threshold value of 1 mm. We consider the same threshold of 1 mm and observe that “HSS > 1 mm” are higher than “HSS” for all the PDatasets. The best performing PDatasets in terms of “HSS > 1 mm” are the same as at the annual and monthly time step (EWEMBI, JRA55 Adj, MSWEP and WFDEIGPCC) with the addition of CHIRPS.
Quantitative and categorical metrics allow us to determine which of the PDatasets is the most reliable to reproduce the rainfall amounts measured by a 5 rain gauge network regulated by the quality control and quality assurance system of the World Meteorological Organization (WMO) over the southeast of Republic of Djibouti. For the annual and monthly time step, the KGE is used for the product rankings. For the daily time step, we add to KGE a categorical index, the HSS performance index, which allows us to assess the ability of these products to detect rainfall events.
For our study area, five PDatasets present high KGE values at annual and monthly time step: EWEMBI, GPCC, JRA55 Adj, MSWEP and WFDEIGPCC. Therefore, the best performing products at the daily time step are CHIRPS, EWEMBI, JRA55 Adj, MSWEP and WFDEIGPCC.
To complete this first study, and to deepen the knowledge concerning the Ambouli basin, it would be necessary to carry out a comparative analysis at the basin scale and use another temporal window to validate the reliability of these products.
The data used are from the databases of the Centre d'Etudes et de Recherches de Djibouti (CERD) and the Agence Météorologique Nationale (ANM) are available on request from these institutions. Concerning the data of the rainfall estimation products, they are available in the official sites of the rainfall products mentioned in Table 1.
GMO did the methodology, data collection, data analysis, writing and editing. JEP did the methodology, data analysis, writing/reviewing and editing. CS did the methodology, data analysis, writing and editing. GM did the methodology, data analysis, writing and editing. MJ did the data analysis, review writing and editing. FS did the methodology and data collection. MIN did the data collection. AHH did the data collection.
The contact author has declared that none of the authors has any competing interests.
Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the special issue “IAHS2022 – Hydrological sciences in the Anthropocene: Variability and change across space, time, extremes, and interfaces”. It is a result of the XIth Scientific Assembly of the International Association of Hydrological Sciences (IAHS 2022), Montpellier, France, 29 May–3 June 2022.
This paper was edited by Christophe Cudennec and reviewed by two anonymous referees.
Akinsanola, A. A., Ogunjobi, K. O., Ajayi, V. O., Adefisan, E. A., Omotosho, J. A., and Sanogo, S.: Comparison of five gridded precipitation products at climatological scales over West Africa Comparison of five gridded precipitation products at climatological scales over West Africa, Meteorol. Atmos. Phys., 129, 669–689, https://doi.org/10.1007/s0070301604936, 2016.
Ayehu, G. T., Tadesse, T., Gessesse, B., and Dinku, T.: Validation of new satellite rainfall products over the Upper Blue Nile Basin, Ethiopia, Atmos. Meas. Tech., 11, 1921–1936, https://doi.org/10.5194/amt1119212018, 2018.
Clark, M. P., Vogel, R. M., Lamontagne, J. R., Mizukami, N., Knoben, W. J. M., Tang, G., Gharari, S., Freer, J. E., Whitfield, P. H., Shook, K. R., and Papalexiou, S. M.: The abuse of popular performance metrics in hydrologic modeling, Water Resour. Res., 57, e2020WR029001, https://doi.org/10.1029/2020WR029001, 2021.
Diro, G. T., Grimes, D. I. F., Black, E., O'Neill, A., and PardoIguzquiza, E.: Evaluation of reanalysis rainfall estimates over Ethiopia, Int. J. Climatol., 29, 67–78, https://doi.org/10.1002/joc.1699, 2009.
Gupta, H. V., Kling, H., Yilmaz, K. K., and Martinez, G. F.: Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling, J. Hydrol., 377, 80–91, https://doi.org/10.1016/j.jhydrol.2009.08.003, 2009.
HoumedGaba, A.: Hydrogéologie des milieux volcaniques sous climat aride. Caractérisation sur site expérimental et modélisation numérique de l'aquifère basaltique de Djibouti (Corne de l'Afrique), PhD thesis, University of Poitiers, Poitiers, France, 220 pp., 2009.
Le Coz, C. and van de Giesen, N.: Comparison of rainfall products over subsaharan africa, J. Hydrometeorol., 21, 553–596, 2020.
Nikulin, G., Jones, C., Giorgi, F., Asrar, G., Buchner, M., CerezoMota, R., Christensen, O. B., Deque, M., Fernandez, J., Hansler, A., van Meijgaard, E., Samuelsson, P., Sylla, M. B., and Sushama, L.: Precipitation climatology in an ensemble of CORDEXAfrica regional climate simulations, J. Climate, 25, 6057–6078, 2012.
Satgé, F., Defrance, D., Sultan, B., Bonnet, M.P., Seyler, F., Rouché, N., Pierron F., and Paturel, J.E.: Evaluation of 23 gridded precipitation datasets across West Africa, J. Hydrol., 581, 124412, https://doi.org/10.1016/j.jhydrol.2019.124412, 2020.
Segele, Z. T., Leslie, L. M., and Lamb, P. J.: Evaluation and adaptation of a regional climate model for the Horn of Africa: rainfall climatology and interannual variability, Int. J. Climatol., 29, 47–65, https://doi.org/10.1002/joc.1681, 2008.
Wang, Y., Sen, O. L., and Wang, B.: A highly resolved regional climate model (IPRCRegCM) and its simulation of the 1998 severe precipitation event over China. Part I: model description and verification of simulation, J. Climate, 16, 1721–1736, 2003.