412: Bootstrap

The bootstrap window provides confidence limits and significance tests for a variety of forecast performance scores. The confidence limits are calculated using bootstrap resampling, and provide an indication of the sampling errors in each performance measure. The bootstrap confidence level used is indicated, and can be adjusted using the Options ~ Resampling Settings menu item. The actual sample scores are indicated, and are the same as those provided by Performance Measures .

As well as providing confidence limits, significance levels are also provided. The p-value indicates the probability that the sample score would be bettered by chance. Permutation procedures are used to calculate the p-values. The accuracy of the p-values depends upon the number of permutations, which can be set using the Options ~ Resampling Settings menu item. It is recommended that at least 200 permutations be used, and more if computation time permits.