https://orcid.org/0000-0003-3824-4898
Figures & data
Figure 1. The structure of the performance testing.
Figure 2. An overview of the performance testing.
Table 1. Tasks and responsibilities in the testing of INVITES-IN.
Figure 3. Estimated workload for the participants. *The number of cell culture studies each participant will receive depends on the total number of participants recruited (see Section 2.3.1).
Figure 4. The seven steps to create the pool of cell culture studies for the practical exercise and the gold-standard for each study.
Table 2. Eligibility criteria for participation in the user testing study.
Figure 5. An overview of the practical exercise.
Table 3. Illustrating the distribution of studies to the judge pairs.
Table 4. Sample size estimates for the number of cell culture studies needed to be assessed by the judge pairs to have a MOE ≤ 0.15 given three rating options of a signalling question (Y = 1, 2, 3).
Table 5. Sample size estimates for the number of cell culture studies needed to be assessed by the judge pairs to have a MOE ≤ 0.15 given four rating options of a signalling question (Y = 1, 2, 3, 4).
Table 6. Sample size estimates for the number of cell culture studies needed to be assessed by the judge pairs to have a MOE ≤ 0.15 given five rating options of a signalling question (Y = 1, 2, 3, 4, 5).
Table 7. For all data from the practical exercise; illustrating results indicating no difference between the gold-standard and user-ratings.
Table 8. Data restricted to the first attempt to apply INVITES-IN to a cell culture study; results indicating small difference between gold-standard and user-ratings.
Table 9. Data restricted to the second attempt to apply INVITES-IN to a cell culture study; results indicating no difference between gold-standard and user-ratings.
Table 10. Results indicating a learning effect, assuming 40 judge pairs rating one signalling question for 40 studies with a gold-standard rating Y = 1.