To investigate the needs and demands of potential users of an envisaged flood early warning system, a quantitative survey was carried out, based on an online questionnaire. The questionnaire comprised 15 questions, with 12 multiple-choice questions, two questions with gradually-scaled answers, and one question for the submission of verbal comments. Strictly speaking, the survey comprised quantitative and qualitative elements. For the sake of brevity, the full questionnaire is not presented herein but can be found in Philipp et al. (2015).

The surveyed sample was selected systematically (i.e., not randomly) and included all legal users (i.e., according to the Saxon Flood Alarm Bylaw; HWMO, 2014) of Flood Center products (n = 578) who were reachable via email to be invited for participating in the online survey (n = 491). The interviewee affiliation spanned administration/authorities at local/district/state level, fire departments and civil protection agencies, as well as the private sector. It has to be stated that the interviewees do not represent lay people since they participate in the official flood management procedures on a legal and regular basis.

The survey results were evaluated using descriptive statistics and subgroup analyses by means of contingency tables. Therefore, given answers were investigated in an user-group specific manner, i.e., more than one variable is considered at a time (multivariate approach). A question to address was whether specific user groups answered differently or not. Such an effect can be induced by strongly differing sizes of sub-samples or can indicate a truly diverse response behavior. The literature suggests χ2-based dependency measures to clarify such questions (Sachs, 1999). For the present study, Cramér's V and χ2-based p values were used.

The verification of meteorological data comprised two Quantitative Precipitation Forecasts (QPFs) which are operationally used by the Saxon Flood Center: the deterministic numerical weather prediction COSMO-DE product (Baldauf et al., 2011) and the probabilistic “Quantile Forecast” (QF) for 16 specific areas in Saxony (cf. Fig. 1), issued by German Met Service's Regional Center in Leipzig. The two QPFs are compared against a Quantitative Precipitation Estimate (QPE), emerging from rain gauge data, which was spatially interpolated (Ordinary Kriging) to derive areal precipitation estimates. Additionally, weather radar data (Met Service's RADOLAN-RW product; Sacher et al., 2011) was employed as another QPE reference. A comprehensive overview of the herein considered QPFs and QPEs is given in Table 1.

The Quantile Forecast represents a probabilistic, qualitative expert estimate of areal precipitation for the next 36 h and consists of three values/quantiles per forecasting time step. Since the forecast is issued for 16 specific areas in Saxony (i.e., river catchments with topographic partitioning according to elevation), verification was based on the comparison of areal rainfall for the mentioned 16 regions, and spanned a period from April 2011 to June 2014.

The comparison of areal rainfall was based on consecutive 6 h sums, starting from 06:00 and 18:00 UTC. 6 h sums were chosen to accommodate the coarsest temporal resolution of the investigated products, given by the Quantile-QPF. The product features areal rainfall totals (for the 16 forecasting regions) with 0.9, 0.5, and 0.1 exceedance probability for two consecutive 6 h and two further 12 h intervals. The forecast is updated twice a day (at 06:00 and 18:00 UTC). However, a main task of the herein presented verification was to evaluate the quality of this product against highly resolved numerical weather prediction output (i.e., COSMO-DE).

A QPF/QPE comparison typically employs a number of tools and methods (Jolliffe and Stephenson, 2012), ranging from simple diagnostic (e.g., time series and totals comparisons, residual and bias analyses, scatter and frequency plots) to integral, quantitative methods. Analyses are often based on threshold-oriented contingency table evaluation and deliver typical verification/skill scores, e.g., False Alarm Rate, Probability Of Detection (FAR, POD) or combined products, e.g., Receiver Operating Characteristic curves (ROC curves; Fawcett, 2006). A prototypical work flow of threshold-oriented skill assessment is shown in Fig. 2. More detailed information on the herein employed QPF/QPE verification methodology (as well as concerning the results) can be obtained from Kerl and Philipp (2015).

Three different hydrological modeling techniques were implemented and applied for three pilot areas in Saxony (cf. Fig. 1): first, a semi-distributed deterministic model (DeHM), second, a data-driven, neural-network model (DaHM) and, third, a simple classification model, based on the scoring of flood-relevant parameters (ScoHM). Subsequently, the modeling concepts and their application (with regard to calibration, data assimilation, etc.) are briefly described. Only snow-free conditions were regarded for model development and application.

Subsequently, the most important results of the user survey (cf. Sect. 2.1) are presented in a concise manner; a more detailed report can be found in Philipp et al. (2015). The response rate was 76 % (n = 373), which is extraordinarily high and is mainly a result of the systematic sampling. For 11 out of 15 questions, user-group specific replies were not distinguishable in a statistical sense. The outcomes of the statistical analysis of the survey data can be summarized as follows:

Information and pathways: (1) The interviewees request selective, event-related information or inform themselves on an event-related basis (rather than on a regular basis). (2) 37 % of all users trust that a more regular and more frequent distribution of warning products will provide increased security for their management decisions, even if the meteorological and hydrological trend remains unchanged. (3) All groups, except the group “private persons”, attach greatest importance to the internet in contrast to other communication channels (e.g., fax, video text, voice mail). The official flood warnings issued by fax or email are also used for information by a majority of users. (4) A high availability of warning services and products is deemed important by a vast majority of users, especially in case of flooding.

Flood warning products: (1) A short-termed, but more precise warning is preferred over a long-term estimation, carrying presumably more uncertainty. (2) The majority of users (> 65 %) are interested in receiving a possibly reliable forecast of the peak water level. 45 % of users would appreciate being informed about the peak timing. (3) Most popular products for fulfilling early warning purposes are forecasted hydrographs with uncertainty bands (about 50 % of all persons interviewed), as well as catchment-oriented classification products (“traffic light”, approximately 40 % of all persons interviewed).

Lead time and miscellaneous: (1) The minimum required lead times amount to ≤ 3 (9 % of users), ≤ 6 (27 %), ≤ 12 (50 %), ≤ 24 (83 %), ≤ 72 h (98 %). (2) A lead time of ≤ 12 h is deemed to be adequate by a slim majority of users in small catchments (< 200 km2). (2) The interviewed user groups vary significantly in terms of the replies given when being asked for the requested updating frequency of flood warnings and their communication via email or fax. (3) Furthermore, the interviewees of various user groups specifically replied to the questions concerning the quality of current products and the quality of the work of the Saxon Flood Center. (4) Moreover, no significant differences in the response behavior of the various user groups could be identified by statistical means.

The investigated QPFs (COSMO-DE and Quantile Forecast) were compared against areal precipitation estimates, based on gridded rain gauge data, and, additionally, a radar-based QPE (RADOLAN-RW product). First, threshold exceedance frequencies were derived from the QPEs and QPFs for threshold values from 10 to 30 mm/6 h (cf. Fig. 3). COSMO-DE delivers exceedance frequencies which are close to the ones obtained from rain gauge data. RADOLAN slightly underestimates the threshold exceedance frequencies from rain gauge data, whereas the chance of underestimation is higher at lower thresholds, and vice versa. Threshold exceedances drawn from the Quantile Forecast's 50th and 10th percentiles are generally more frequent than the observed ones (i.e., from rain gauge data), whereas the 90th percentile underestimates observed frequencies.

Second, for a more in-depth view at the regarded QPFs, the contingency-based measures POD and FAR were evaluated down to thresholds of 0.1 mm/6 h (Figs. 4 and 5). Due to product-specific conventions of the Quantile Forecast (areal precipitation sum < 4.5 mm/6 h is set to zero), the results are constant for thresholds < 4.5 mm. Following Winterrath et al. (2012), a minimum of 10 observed or predicted threshold exceedances should be required for the calculation of skill scores. Therefore, POD and FAR were not always evaluated for higher thresholds. Generally, higher precipitation thresholds are connected with lower POD and lower FAR values, and vice versa. Furthermore, for POD, the skill variance amongst the forecast areas increases with increasing precipitation thresholds. POD = FAR indicates a boundary for which the considered QPF has no predictive benefit anymore. This boundary is not reached for both QPFs, concerning the investigated thresholds. Finally, for the regarded QPFs, COSMO-DE exhibits the highest performance with regard to POD/FAR relations and skill variance amongst the forecast areas.

The three presented models (DeHM, DaHM, and ScoHM) were applied for the three aforementioned pilot areas (cf. Fig. 1). The herein investigated QPEs (gridded rain gauge data and RADOLAN data) and QPFs (COSMO-DE and Quantile Forecast; cf. Sects. 2.2 and 3.2) were used as meteorological drivers (for the current state of work, on the QPF side, ScoHM was charged with the Quantile Forecast only). Validation for the DeHM and DaHM models is straightforward since modeled hydrographs are simply compared against observed ones. Model evaluation is a bit more delicate for the ScoHM results, since the ScoHM output (i.e., dimensionless scores) does only qualitatively correlate with observed flow values. Therefore, a quantile-mapping procedure (Piani et al., 2009) was applied to relate thresholds of Q with corresponding total-score values.

Model performance was evaluated on the basis of threshold-oriented contingency table analyses, i.e., it is checked if modeled output matches/exceeds a certain observed flow level or not. More specifically, the variation of threshold values delivers a set of corresponding skill scores, e.g., POD values with corresponding FARs. These POD/FAR tuples were used to establish catchment-specific Receiver Operating Characteristic curves (Fawcett, 2006). The curves were finally integrated to deliver the Area Under Curve (AUC), with values near unity for a near-perfect model prediction and near 0.5 for no predictive skill (cf. Fig. 2 and Sect. 2.2). For brevity, results are presented and discussed for the Mandau catchment only, featuring four river gauges.

Generally, different combinations of lead times and update cycles (i.e., the time after which a new forecast is processed) were investigated; herein, results for an update cycle length of 12 h are presented. Event-specifically masked, hourly hydrograph data and hourly rainfall observations were used during model validation. Data which were employed in model calibration/training were not used for validation purposes. Calibration/training data originated from the period of 2006 to 2011 (11 events), validation data from 2010 to 2015 (10 events). Figure 6 comprehensively shows the validation results for the Mandau pilot area. For the Quantile Forecast, results for the 50th percentile are exemplarily shown.

For the smallest sub-catchment, Niederoderwitz (29 km2), DeHM performs best; for the three larger sub-catchments, DaHM features the highest Area Under Curve values. However, DeHM and DaHM performance trends to decrease with increasing lead time; ScoHM features a quite constant/robust skill development. The reason for this might be that for shorter lead times (6 h) the auto-correlative Qt-x signal, included directly or indirectly in the DeHM and DaHM model (cf. Sect. 2.3), leads to improved performance. This does not apply for the ScoHM results, since the model is not dependent on observed flow data. Generally, ScoHM exhibits Area Under Curve values around 0.8 which indicates a good overall predictive skill, foremost, when keeping in mind the generality and straightforwardness of the model approach.

It can be further seen from Fig. 6 that QPE data delivers highest predictive skill with a tendency of RADOLAN outperforming the rain gauge data. Predictive skill under QPF data (Quantile Forecast and COSMO-DE) is mostly lower. For different QPFs as drivers, resulting skills do not differ greatly. Apparently, the observed differences in QPF quality (cf. Sect. 3.2) do not systematically impact hydrological model skill. Furthermore, it is important to say that validation was carried out on the basis of hourly values; a more general evaluation, e.g., comparing only the highest values within a specific temporal window (e.g., 6 h), would yield considerably higher skill scores.

Finally, it should be stated that the results for the other investigated pilot areas are consistent with the herein presented findings for the Mandau pilot region when focusing on catchments with areas of up to 200 km2. For larger scales, when wave translation and diffusion impact flood expression, the deterministic and data-driven models outperform the scoring approach since it does not account for such processes.