DistributionWAgg.RdCalculate the arithmetic mean of distributions created with expert judgements. The aggregate is the median of the average distribution fitted on the individual estimates.
DistributionWAgg(
expert_judgements,
type = "DistribArMean",
name = NULL,
placeholder = FALSE,
percent_toggle = FALSE
)A dataframe in the format of data_ratings.
One of "DistribArMean" or "TriDistribArMean".
Name for aggregation method. Defaults to type unless specified.
Toggle the output of the aggregation method to impute placeholder data.
Change the values to probabilities. Default is FALSE.
A tibble of confidence scores cs for each paper_id.
This method assumes that the elicited probabilities and bounds can be considered to represent participants' subjective distributions associated with relative frequencies (rather than unique events). That is to say that we considered that the lower bound of the individual per claim corresponds to the 5th percentile of their subjective distribution on the probability of replication, denoted \(q_{5,i}\), the best estimate corresponds to the median, \(q_{50,i}\), and the upper bound corresponds to the 95th percentile, \(q_{95,i}\). With these three percentiles, we can fit parametric or non-parametric distributions and aggregate them rather than the (point) best estimates.
type may be one of the following:
DistribArMean: Applies a non-parametric distribution evenly across upper, lower and best estimates.
Using the three percentiles we can build the minimally informative non-parametric distribution that spreads the mass uniformly between the three percentiles.
\[F_{i}(x) = \begin{cases} \displaystyle 0, \text{ for } x<0 \cr \displaystyle \frac{0.05}{q_{5,i}}\cdot x, \text{ for } 0 \leq x< q_{5,i}\cr \displaystyle \frac{0.45}{q_{50,i}-q_{5,i}}\cdot(x-q_{5,i})+0.05, \text{ for } q_{5,i}\leq x< q_{50,i}\cr \displaystyle \frac{0.45}{q_{95,i}-q_{50,i}}\cdot(x-q_{50,i})+0.5, \text{ for } q_{50,i}\leq x< q_{95,i}\cr \displaystyle \frac{0.05}{1 - q_{95,i}}\cdot(x-q_{95,i})+0.95, \text{ for } q_{95,i}\leq x< 1\cr \displaystyle 1, \text{ for } x\geq 1. \end{cases}\]
Then take the average of all constructed distributions of participants for each claim:
\[AvDistribution = \frac{1}{N}\sum_{i=1}^N F_i(x),\]
and the aggregation is the median of the average distribution:
\[\hat{p}_c\left( DistribArMean \right) = AvDistribution^{-1}(0.5).\]
TriDistribArMean: Applies a triangular distribution to the upper, lower and best estimates.
A more restrictive fit with different assumptions about the elicited best estimates, upper and lower bounds. We can assume that the lower and upper bounds form the support of the distribution, and the best estimate corresponds to the mode.
\[F_i(x)= \begin{cases} \displaystyle 0, \text{ for } x < L_{i} \cr \displaystyle \frac{\left( x-L_{i}\right)^2}{\left( U_{i}-L_{i}\right)\left( B_{i}-L_{i} \right)}, \text{ for } L_{i} \leq x < B_{i}\cr \displaystyle 1 - \frac{\left( U_{i}-x\right)^2}{\left( U_{i}-L_{i}\right)\left ( U_{i}-B_{i}\right)}, \text{ for } B_{i} < x < U_{i}\cr \displaystyle 1, \text{ for } x \geq U_{i}. \end{cases}\]
Then take the average of all constructed distributions of participants for each claim:
\[ AvDistribution = \frac{1}{N}\sum_{i=1}^N F_i(x),\]
and the aggregation is the median of the average distribution:
\[ \hat{p}_c\left(TriDistribArMean\right) = AvDistribution^{-1}(0.5).\]
if (FALSE) DistributionWAgg(data_ratings)