P1_ratings

Anonymized expert judgements of known-outcome claims, assessed at the 2019 SIPS repliCATS workshop

data_ratings

Format

A table with 6880 rows and 7 columns:

round

character string identifying whether the round was 1 (pre-discussion) or 2 (post-discussion)

paper_id

character string of the claim ids (25 unique claims total)

user_name

character string of anonymized IDs for each participant (25 participants included in this dataset)

question

character string for the question type, with four options: direct_replication, involved_binary, belief_binary, or comprehension

element

character string for the type of response coded in the row, with five options: three_point_lower, three_point_best, three_point_upper, binary_question, or likert_binary

value

numeric value for the participant's response

group

character string of group IDs that contained the participants