Given a forecast and set of observations, compute the accuracy (# inside quantile interval / # total) of the forecast. Raw data and/or provided quantiles will be used to compute the quantile interval.
Arguments
- fcst
A forecast object (see output of
create_forecast()
).- obs
An observations data frame.
- summarize
A boolean, defaults to TRUE. If TRUE, a single number will be returned as the score for the forecast. If FALSE, a data frame with columns named
time
,val_obs
, andscore
will be returned, containing the scores for each individual time point. This can be used by plotting functions to colour-code observations, for example.- quant_pairs
(Optional) A list of pairs of numbers between 0 and 100, or a single pair of numbers between 0 and 100. If provided, the score for each corresponding pairs of quantiles will be calculated. If not provided, it will default to every symmetrical pair of quantiles that can be found in
fcst
, ordered from widest to narrowest (e.x. the 25% and 75% quantiles are symmetrical).If
summarize
isFALSE
, an additional column namedpair
will indicate which pair of quantiles each row represents. Ifsummarize
isTRUE
, the output will be a vector with the same length asquant_pairs
, containing the respective score for each pair.
Value
A number from 0 to 1, the rate at which the observations were inside the specified quantile interval
Examples
# forecast with raw data
fc1 <- create_forecast(dplyr::tibble(time=c(1,1,1,2,2,2,3,3,3), val=4:12))
obs1 <- data.frame(time=1:3, val_obs=c(5, 7.4, 11.6))
# calculate quantiles and accuracy from raw data
# returns 2/3
accuracy(fc1, obs1, quant_pairs=list(c(25, 75)))
#> Scoring accuracy using quantile pairs c(25, 75)
#> Used 3 time points to calculate accuracy
#> [1] 0.3333333
# forecast with quantile data
fc2 <- create_forecast(dplyr::tibble(
time=1:3, val_q5=1:3, val_q25=4:6, val_q50=100:102, val_q75=200:202, val_q95=203:205
))
obs2 <- data.frame(time=1:3, val_obs=c(4, 202, 1000))
# infer quantile pairs from forecast data (`c(5,95)` and `c(25, 75)`)
# returns c(2/3, 1/3)
accuracy(fc2, obs2)
#> Scoring accuracy using quantile pairs c(5, 95), c(25, 75)
#> Used 3 time points to calculate accuracy
#> [1] 0.6666667 0.3333333
# return a data frame with a `time`, `pair`, `val_obs`, and `score` columns
accuracy(fc2, obs2, summarize=FALSE)
#> Scoring accuracy using quantile pairs c(5, 95), c(25, 75)
#> time val_obs score pair
#> 1 1 4 TRUE 1
#> 2 2 202 TRUE 1
#> 3 3 1000 FALSE 1
#> 4 1 4 TRUE 2
#> 5 2 202 FALSE 2
#> 6 3 1000 FALSE 2