433: Scores

Various verification scores are calculated for the probabilistic forecasts by pooling all gridpoints / stations / series. By default, the 2AFC scores for the individual categories will not necessarily agree with the ROC areas shown on the ROC graphs because the areas are calculated using binned forecast probabilities, whereas the scores page shows scores for continuous probabilities.

The following scores can be calculated using binned probabilities if the appropriate Verification Settings option is selected:

• Ignorance score (all-category and category-specific)
• Brier score and skill score
• Ranked probability score and skill score
• ROC areas

The ignorance and Brier score decompositions are calculated using binned probabilities, regardless of the setting. In most cases it is probably preferable not to bin the probabilities, but if it is important for the ROC areas to be consistent with those indicated on the ROC diagrams , or to have total scores consistent with their decompositions, for eample, then binning the probabilities may be desirable.

There is an option to calculate bootstrap confidence intervals for the scores from Options ~ Resampling Settings .