A novel method for observing proportional group awareness and consensus of items arising from list-generating questioning

Figures & data

Table 1. Summary of case studies.

Case study	Open-ended question asked in Round 1 (list-generating round)	Criteria for expert participation	Systematic literature review (search terms)*
1. Setting up a European registry for patients with XLH	With specific reference to your experience in setting up and implementing the XLH Registry what recommendations would you make to other groups wishing to implement a similar European rare-disease registry?	Recruitment was from the Medialis Real World Evidence Asset Management Division team of clinical study managers. Relevant patient registry experience. Participants have a similar level of knowledge regarding rare-disease registries, specifically an XLH registry, and exposure to the same project information.	No
2. Limitations of empirical treatment of XLH in children	What are the limitations of empirical treatment for XLH in children?	Experience in treatment of children with XLH. Management of XLH in a recognized specialist centre. Author of publication in paediatric XLH. Participation in paediatric clinical trials in XLH.	X-linked rickets; XLH; X-linked hypophosphatemia; Familial hypophosphatemia; Familial hypophosphatemic rickets; X-linked hypophosphatemic rickets; Vitamin D-resistant rickets; VDRR; Genetic rickets; Hereditary rickets Phosphate(s) and vitamin D (calcitriol, 25-hydroxycholecalciferol, 1,25-dihydroxycholecalciferol, cholecalciferol, vitamin D3/D2/D)
3. Limitations of empirical treatment for XLH in adults	What are the limitations of empirical treatment for XLH in adults?	Experience in treatment of adults with XLH. Management of XLH in a recognized specialist centre. Author of publication in adult XLH. Participation in adult clinical trials in XLH.	X-linked rickets; XLH; X-linked hypophosphatemia; Familial hypophosphatemia; Familial hypophosphatemic rickets; X-linked hypophosphatemic rickets; Vitamin D-resistant rickets; VDRR; Genetic rickets; Hereditary rickets Phosphate(s) and vitamin D (calcitriol, 25-hydroxycholecalciferol, 1,25-dihydroxycholecalciferol, cholecalciferol, vitamin D3/D2/D)
4. Triggers for treatment of XLH in adults	What are your triggers for treatment of XLH in adults?	Experience in treatment of adults with XLH. Management of XLH in a recognized specialist centre. Author of publication in adult XLH. Participation in adult clinical trials in XLH.	No

*Online databases MEDLINE via PubMed were used to identify search terms. Abbreviations. XLH, X-linked hypophosphatemia.

Figure 1. Overview of awareness and agreement consensus. The awareness and agreement consensus overview is categorized according to the Awareness Index (frequency of each coded item in relation to the overall most frequently occurring coded item in Round 1) and the Consensus Index (calculated using the percentage of participant agreement with each statement during the structured questionnaire in Round 2). Complete awareness of an item has an Awareness Index of 100% and complete agreement consensus has a Consensus Index of 100%. Partial awareness and consensus have indices >50–100%. Low awareness and no agreement consensus have indices 0–≤50%. XLH, X-linked hypophosphatemia.

Table 2. Case study participants.

Case study number	Description of participants	Number of participants who responded to:
		Round 1	Round 2
1. Setting up a European XLH registry	The participants all worked on a European XLH patient registry. In addition, two of the three participants had more than 10 years of experience in study management and all three participants had at least 5 years of experience in national regulatory submission processes. One of the participants had additional experience in local site regulatory approval (3 years), site contract execution (8 years), and engagement of chief (n = 8) and primary investigators (n = 32).	3	3
2. Limitations of empirical treatment of XLH in children	The participants were all physicians with experience of treating children with XLH, in Europe. Participants were selected from sites across Europe, all worked in a national centre recognized for the management of paediatric XLH patients and had authored articles on XLH in the paediatric population	8	8
3. Limitations of empirical treatment for XLH in adults	The participants were all physicians with experience in treating adults with XLH in Europe. Participants were selected from sites across Europe, all worked in a national centre recognized for the management of paediatric XLH patients in their country and had authored articles on XLH in the paediatric population.	10	10
4. Triggers for treatment of XLH in adults	The participants were all physicians with experience in treating adults with XLH in Europe. Participants were selected from across Europe, all worked in a national centre recognized for the management of paediatric XLH patients in their country and had authored articles on XLH in the paediatric population.	10	10

Abbreviations. XLH, X-linked hypophosphatemia.

Figure 2. Awareness (item generation) and prompted agreement (consensus) for case study 1.

Figure 3. Awareness (item generation) and prompted agreement (consensus) for case study 2.

Figure 4. Awareness (item generation) and prompted agreement (consensus) for case study 3.

Figure 5. Awareness (item generation) and prompted agreement (consensus) for case study 4.

Figure 6. Insights into stratifying knowledge awareness and prompted agreement. Interpreting differences between group awareness and consensus of items. Round 1 – Item Generation: Green circles represent group awareness of items identified during a list-generating question; orange circle represents items are possibly require more education about or represent a time-lag between expert awareness and the knowledge spreading to the wider academic community; red represents items that are possibly not as relevant to the study question; blue represents external source of item, for example, an item from a systematic literature review. Round 2 – Consensus: Green circles represent agreement or prompted agreement; orange circle represent prompted agreement but in the minority of participants, possibly because more research is required to provide more evidence; red circle represents agreement that item should be excluded. Arrows between Round 1 and Round 2 represent the level of prompted agreement with the steeper the slope the greater the level of prompted change of opinion. The Index Score represents the amount of prompted agreement between awareness (item-generating) and observed consensus.

Figure 7. Planning future actions based on awareness and consensus categories. Group awareness of items and prompted agreement consensus can be used to plan future actions according to pre-defined categories. Abbreviations. AI, Awareness Index; f, frequency of item reported by participants in their responses to Round 1 question; f_max, the number of times the most frequently mentioned item was mentioned in the responses to the Round 1 question; f_min, the number of times the least frequently mention item was mentioned in the response to the Round 1 question; CI, Consensus Index.

Data availability statement

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.