ShiftingWAgg.RdWeighted by judgements that shift the most after discussion
ShiftingWAgg(
expert_judgements,
type = "ShiftWAgg",
name = NULL,
placeholder = FALSE,
percent_toggle = FALSE
)A dataframe in the format of data_ratings.
One of "ShiftWAgg", "BestShiftWAgg", "IntShiftWAgg", "DistShiftWAgg", or "DistIntShiftWAgg".
Name for aggregation method. Defaults to type unless specified.
Toggle the output of the aggregation method to impute placeholder data.
Change the values to probabilities. Default is FALSE.
A tibble of confidence scores cs for each paper_id.
When judgements are elicited using the IDEA protocol (or any other protocol that allows experts to revisit their original estimates), the second round of estimates may differ from the original first set of estimates an expert provides. Greater changes between rounds will be given greater weight.
type may be one of the following:
ShiftWAgg: Takes into account the shift in all three estimates
Considers shifts across lower, \(L_{i,c}\), and upper, \(U_{i,c}\), confidence limits, and the best estimate, \(B_{i,c}\). More emphasis is placed on changes in the best estimate such that:
\[w\_Shift_{i,c} = |B1_{i,c} - B_{i,c}| + \frac{|L1_{i,c} - L_{i,c}|+|U1_{i,c} - U_{i,c}|}{2},\]
where \(L1_{i,c}, B1_{i,c},U1_{i,c}\) are the first round lower, best and upper estimates (prior to discussion) and \(L_{i,c}, B_{i,c},U1_{i,c}\) are the individual’s revised second round estimates (after discussion).
\[\hat{p}_c(ShiftWAgg) = \sum_{i=1}^N \tilde{w}\_Shift_{i,c}B_{i,c}\]
BestShiftWAgg: Weighted according to shifts in best estimates alone
Taking into account the fact that the scales best estimates are measured on are bounded, we can calculate shifts relative to the largest possible shift.
\[w\_BestShift_{i,c}= \begin{cases} \frac{|B1_{i,c} - B_{i,c}|}{B1_{i,c}}, \begin{aligned} \displaystyle &\ for\ (B1_{i,c} > 0.5\ and\ B_{i,c} \leq 0.5) \cr \displaystyle &\ or\ B_{i,c} < B1_{i,c} \leq 0.5\ or\ B1_{i,c} > B_{i,c} > 0.5 \end{aligned} \cr \frac{|B1_{i,c} - B_{i,c}|}{1- B1_{i,c}}, \begin{aligned} \displaystyle &\ for\ (B1_{i,c} < 0.5\ and\ B_{i,c} \geq 0.5) \cr \displaystyle &\ or\ B1_{i,c} < B_{i,c} < 0.5\ or\ B_{i,c} > B1_{i,c} > 0.5. \end{aligned} \end{cases}\]
\[\hat{p}_c(BestShiftWAgg) = \sum_{i=1}^N \tilde{w}\_BestShift_{i,c}B_{i,c}\]
IntShiftWAgg: Weighted by shifts in interval widths alone.
Individuals whose interval widths narrow between rounds are given more weight.
\[w\_IntShift_{i,c} = \frac{1}{(U_{i,c}-L_{i,c})-(U1_{i,c}-L1_{i,c})+1}\]
\[\hat{p}_c(IntShiftWAgg) = \sum_{i=1}^N \tilde{w}\_IntShift_{i,c}B_{i,c}\]
DistShiftWAgg: Weighted by whether best estimates become more extreme (closer to 0 or 1) between rounds.
\[w\_DistShift_{i,c} = 1 - (\min (B_{i,c}, 1-B_{i,c}) - \min (B1_{i,c}, 1-B1_{i,c}))\]
\[\hat{p}_c(DistShiftWAgg) = \sum_{i=1}^N \tilde{w}\_DistShift_{i,c}B_{i,c}\]
DistIntShiftWAgg: Rewards both narrowing of intervals and shifting towards the certainty limits between rounds.
We simply multiply the weights calculated in the "DistShiftWAgg" and "IntShiftWAgg" methods.
\[w\_DistIntShift_{i,c} = \tilde{w}\_IntShift_{i,c} \cdot \tilde{w}\_DistShift_{i,c}\]
\[\hat{p}_c(DistIntShiftWAgg) = \sum_{i=1}^N \tilde{w}\_DistIntShift_{i,c}B_{i,c}\]
if (FALSE) ShiftingWAgg(data_ratings)