Aggregation Method: ExtremisationWAgg — ExtremisationWAgg • aggreCAT

Calculate beta-transformed arithmetic means of best estimates.

ExtremisationWAgg(
  expert_judgements,
  type = "BetaArMean",
  name = NULL,
  alpha = 6,
  beta = 6,
  cutoff_lower = NULL,
  cutoff_upper = NULL,
  placeholder = FALSE,
  percent_toggle = FALSE
)

Arguments

expert_judgements: A dataframe in the format of data_ratings.
type: One of "BetaArMean" or "BetaArMean2".
name: Name for aggregation method. Defaults to type unless specified.
alpha: parameter for the 'shape1' argument in the stats::pbeta function (defaults to 6)
beta: parameter for the 'shape2' argument in the stats::pbeta function (defaults to 6)
cutoff_lower: Lower bound of middle region without extremisation in "BetaArMean2" aggregation types.
cutoff_upper: Upper bound of middle region without extremisation in "BetaArMean2" aggregation types.
placeholder: Toggle the output of the aggregation method to impute placeholder data.
percent_toggle: Change the values to probabilities. Default is FALSE.

Value

A tibble of confidence scores cs for each paper_id.

Details

This method takes the average of best estimates and transforms it using the cumulative distribution function of a beta distribution.

type may be one of the following:

BetaArMean: Beta transformation applied across the entire range of calculated confidence scores.

\[\hat{p}_c\left( \text{BetaArMean} \right) = H_{\alpha \beta}\left(\frac{1}{N} \sum_{i=1}^N B_{i,c} \right),\]

where \(H_{\alpha \beta}\) is the cumulative distribution function of the beta distribution with parameters \(\alpha\) and \(\beta\), which default to 6 in the function.

The justification for equal parameters (the 'shape1' and 'shape2' arguments in the stats::pbeta function) are outlined in Satopää et al (2014) and the references therein (note that the method outlined in that paper is called a beta-transformed linear opinion pool). To decide on the default shape value of 6, we explored the data_ratings dataset with random subsets of 5 assessments per claim, which we expect to have for most of the claims assessed by repliCATS.

BetaArMean2: Beta transformation applied only to calculated confidence scores that are outside a specified middle range. The premise being that we don't extremise "fence-sitter" confidence scores.

\[\hat{p}_c\left( \text{BetaArMean2} \right) = \begin{cases} \displaystyle H_{\alpha \beta}\left(\frac{1}{N} \sum_{i=1}^N B_{i,c} \right), \text{ for } \frac{1}{N} \sum_{i=1}^N B_{i,c} < \textit{cutoff\_lower} \cr \displaystyle \frac{1}{N} \sum_{i=1}^N B_{i,c}, \text{ for } \textit{cutoff\_lower} \leq \frac{1}{N} \sum_{i=1}^N B_{i,c} \leq \textit{cutoff\_upper} \cr \displaystyle H_{\alpha \beta}\left(\frac{1}{N} \sum_{i=1}^N B_{i,c} \right), \text{ for } \frac{1}{N} \sum_{i=1}^N B_{i,c} > \textit{cutoff\_upper} \cr \end{cases}\]

Examples

if (FALSE) ExtremisationWAgg(data_ratings)