553: Verification Settings

The verification settings option also allows you to specify how many probability bins are used to construct the attributes and ROC diagrams , as well as some of the scores, as discussed below. The default number is 11, as recommended by the WMO Standardized Verification System for Long-Range Forecasts (SVSLRF) , which means that each bin is 10% wide (except for the first (0-5%) and last (95-100%) bin), and is suitable for GCM outputs, for example. However, there is an option to use 21 bins, which means that each bin is 5% wide (except for the first (0.0-2.5%) and last (97.5-100.0%) bin). Since the Regional Climate Outlook Forums tend to issue probabilities rounded to the nearest 5% rather than 10%, using 21 bins permit the attributes diagrams to be constructed with a point for each discrete probability (except for 33%, which is binned with 35%).

There is also an option to use the binned probabilities when calculating some of the Verification Scores . The default is to bin the probabilities only for those scores that require binning, namely:

• Ignorance score decompositions
• Brier score decompositions

The resolution slopes are calculated using the actual forecast probabilities (not binned) even though the points on the graph are calculated using binned probabilities. If the option to calculate the scores using binned probabilities is selected then only the following scores are affected, while all others are calculated using the actual probabilities:

• Ignorance score (all-category and category-specific)
• Brier score and skill score
• Ranked probability score and skill score
• ROC areas

In most cases it is probably preferable not to bin the probabilities, but if it is important for the ROC areas to be consistent with those indicated on the ROC diagrams , or to have total scores consistent with their decompositions, for eample, then binning the probabilities may be desirable.