Accuracy of the Swedish quality registry for caries and periodontal diseases (SKaPa) – evaluation in 6- and 12-year-olds in the region of Värmland, Sweden

Abstract

Objectives

This study evaluates the agreement of data on dental caries between electronic dental records and data retrieved from the national SKaPa-registry (Swedish Quality Registry for caries and periodontal disease), with special reference to e/M in deft/DMFT.

Methods

In a random sample of 500 6- and 12-year-old children having received dental care in 2014 in the county region of Värmland, Sweden, the diagnostic accuracy of data in electronic dental records with corresponding data obtained from the SKaPa-registry was compared by using Cohen’s Kappa and Intraclass correlation coefficient (ICC).

Results

For dft/DFT the Kappa was 0.95, and ICC 0.98 (total population). For deft/DMFT in the total population the Kappa was 0.80 and ICC 0.96. For 6-year-olds (deft) the Kappa was 0.89 and ICC 0.99 and for 12-year-olds (DMFT) the Kappa was 0.70, and ICC 0.83. The corresponding figures for Kappa and ICC when excluding individuals without caries (deft/DMFT = 0) were: Total population 0.63 and 0.94; 6-year-olds 0.79 and 0.99; 12-year-olds 0.42 and 0.68.

Conclusion

Agreement between data in the dental records and SKaPa was very high for dft/DFT confirming that transfer from the dental records to the SKaPa-registry is safe and correct. As the accuracy of deft/DMFT was considerably lower than for dft/DFT we advise against using deft/DMFT data from SKaPa for research purposes at this point.

Keywords:

• Data accuracy
• registries
• child
• dental caries

Introduction

Registries compiling data on diseases, medical and dental diagnoses, treatments, and treatment outcomes are important in public health work. Depending on how the registries are constructed and their degree of coverage of the population, they can provide useful information for monitoring health and enable evaluation of treatments and interventions. The Nordic countries, including Sweden, have a long tradition of official national registries and registry-based research [1,2]. A key element in registry-based research is the possibility to combine information about individuals from different registries. In Sweden, this is possible thanks to a personal identity number (PIN). Everyone who is registered in the country has a unique PIN consisting of twelve digits [3]. The first eight digits show date of birth (year, month and day of birth) and are followed by four check-digits where sex is shown by the second last digit, even for female and odd for males [4]. The PIN is used as a person’s identifier in all communication with government authorities, with banks, tax agency and in all medical and dental records. The PIN is also shown on all identification cards, passports etc.

All dental records in Sweden are electronic, and in addition to using the PIN, they also use a national code system for documentation of diagnoses and treatments linked to the social insurance system regulating dental care and reimbursement for Swedish citizens [5]. This allows development of dental health registries that can be used for studies of dental treatment measures and oral health. Further, by combining dental health registries with e.g. registries for medical conditions, taxation, level of education etc. it is possible to monitor and follow oral and general health in large populations and also to study the impact of possible risk factors and to evaluate interventions [6].

Swedish healthcare is governed by 21 county regions. Their councils have the over-riding responsibility to ensure that dental care is available for people living in the region with a special obligation to ensure comprehensive dental care, including specialist dental care, for children and young adults (up to the age of 23 years). They are also responsible for the Public Dental Service (PDS) [7]. Dental care is free of charge up to the age of 23 and can be carried out by either PDS (responsible for approximately 88% of the dental care for children and young adults in Sweden) or private practitioners (PP). About 95% of Swedish children and adolescents undergo regular dental examinations and comprehensive dental care over a period of one to three years.

The Swedish Quality Registry for caries and periodontal disease (SKaPa) is a national registry started in 2008, comprising data which is transferred completely automatically from dental records held by dental clinics within the PDS and many PP across all of Sweden [8]. More than 8,5 million individuals are recorded in SKaPa and the registry receives data from six different digital dental record systems (Lifecare Dental, Carita, T4, FRENDA, ALMASOFT and OPUS Dental). The registry holds information about dental care organizations and dental professionals individually for each patient. At a patient level, the information covers the PIN, age, sex, dental health status, dental diagnoses, dental treatment, and risk assessments for caries and periodontal disease [9]. SKaPa is the only quality registry in dentistry in Sweden incorporating children, and presently it includes data on about 90% of the Swedish child population (0-19 years of age). The registry produces continuous national, regional and local epidemiological information and provides a resource for dental research and health economic evaluations of dental healthcare [9].

It is important that the quality of data in SKaPa is high. To ensure this it is essential that the clinical registrations of diagnoses and treatment are made correctly, that the transfer of data to SKaPa works without interference, and finally that the data extracted from SKaPa provide accurate information. With respect to dental caries (defined as decayed and filled teeth), this has been evaluated in a previous study which followed the whole chain of data [10]. That study compared dental health registrations from the researcher’s clinical dental examination, dental health status recorded by the child’s customary examiner (i.e. registrations in the electronic dental records) and dental health data retrieved from the SKaPa registry in 170 children. Reliability and accuracy were satisfactory along the whole chain: between the researcher and the customary examiner; between the researcher and SKaPa; and between the customary examiner and SKaPa. At that time, it was not possible to retrieve data for extracted/Missing teeth (et/MT) in the deft/DMFT (number of decayed, extracted (due to caries), or filled primary teeth/number of Decayed, Missing (due to caries), or Filled permanent Teeth) from SKaPa, but for dft/DFT (number of decayed, or filled primary teeth/number of Decayed, or Filled permanent Teeth) both reliability and accuracy were very good throughout the entire data chain. Since patients having teeth extracted because of dental caries may have a more severe degree of the disease the dft/DFT indices overestimate the oral health in these individuals. As the SKaPa registry is used for planning of resources for the dental care and for research, also the deft/DMFT indices, particularly the variable et/MT, as well as data on dental treatments need to be further validated. As the previous study [10] showed very good agreement between the researcher and the customary examiner it was possible to establish that the electronic dental records are valid as reference standard. This is important as it makes it easier to include larger groups of patients when continuing to validate the SKaPa registry. Against this background, the research questions in the present study were: How well do the data on dental caries (including extracted and missing teeth in deft/DMFT) and dental treatments (treatment codes for nine preventive treatments) in the electronic dental records in 6- and 12-year-old children agree with the data retrieved from the SKaPa registry? The following hypothesis was tested: The data in the electronic dental records for 6- and 12-year-old children, with respect to dft/DFT, deft/DMFT and preventive dental treatments differ minimally from data extracted from the SKaPa registry (defined as an Intraclass Correlation Coefficient (ICC) with the lower bound of a 95% confidence interval above 0.90).

Material and methods

Study design

This retrospective study compared data for the year 2014 in electronic dental records in the data system Carita (Swedish Care System AB), with data extracted from the SKaPa-registry for the same time period. Carita is used in four county regions representing almost 22% of Swedish children and adolescents. The study was carried out in one of these regions.

Participants and setting

A random sample of 500 6- and 12-year-old children (250 per age group) having received dental care in 2014 was identified by the PDS (Public Dental Service) administration in Värmland. The children had not been included in the preceding study [10].

The study was carried out in the county region of Värmland, Sweden. Värmland is situated almost in the middle of Sweden and on the border to Norway and has about 283,000 inhabitants. There are both rural and urban areas and the main city Karlstad hosts a university. Approximately 92% of the children in Värmland receive their dental care within the PDS.

The year 2014 was chosen as it at this time was mandatory to carry out annual dental examinations of children in Värmland. In 2015 Sweden received more than 70,000 asylum seekers under the age of 18, mainly from Syria. Many of these children needed emergency dental care and registrations from emergency appointments are likely to be less accurate that could have affected the present study. Excluding asylum seekers would require collation with other registries and this was not possible in this study. Data compilation from dental records started in 2017 and were finalized in 2018. After that SKaPa provided output data and statistical analyses were completed in 2021.

In 2014 there were approximately 2,700 children aged 6 and 12-years, each, in Värmland. Of these, 2623 6-year-olds and 2460 12-year-olds had a dental examination carried out within the PDS. According to PDS statistics, the dental health in this background population was good; 78% of 6-year-olds and 74% of 12-year-olds in the region were reported to be caries free (6-year-olds: dft mean 0.72; sd 1.79; 12-year-olds: DFT mean 0.48; sd 1.06). Based on this information and in consultation with a biostatistician, it was decided that a random sample of 250 from each age group (approximately 10%) would be sufficient to ensure inclusion representative of the background population.

Test methods

Output data from the SKaPa registry was used as index test (the test that is being evaluated against a reference standard) while data from the electronic dental records served as reference standard. Dental caries diagnoses (represented by the indices dft/DFT, et/MT, deft/DMFT) and dental treatments, namely nine different codes for preventive treatments, were compiled independently from dental records and SKaPa. The preventive measures were chosen as they were likely to be common treatments and thereby frequent findings in the dental records. All variables are described in Table 1.

Table 1. Variables collected from dental records and retrieved from the SKaPa registry.

Age	Variable	Explanation
Dental caries
6-year-olds	dft	number of decayed and filled teeth in the primary dentition
	deft	number of decayed, extracted (due to caries) and filled teeth in the primary dentition
	et	number of extracted (due to caries) teeth in the primary dentition
12-year-olds	DFT	number of Decayed and Filled Teeth in the permanent dentition
	DMFT	number of Decayed, Missing due to caries, and Filled Teeth in the permanent teeth
	MT	number of Missing Teeth due to caries in the permanent dentition
Treatment codes
6-and 12-year-olds	Health prevention codes when risk of oral health-related diseases or problems
	code 201	Information or instruction in case of risk of oral health-related diseases or problems
	code 204	Fluoride applicator tray
	code 205	Fluoride treatment, shorter treatment time, includes, where applicable, professional tooth cleaning
	code 206	Fluoride treatment, treatment time 20 min or more, includes, where applicable, professional tooth cleaning
	Preventive treatment codes when oral health-related diseases or problems
	code 311	Information or instruction for oral health-related diseases or problems
	code 312	Follow-up information or instruction in case of oral health-related diseases or problems
	code 313	Behavioral intervention, treatment with behavioral change techniques, 60 min or more
	code 314	Behavioral intervention, treatment with behavioral change techniques, shorter treatment
	code 321	Non-operative treatment of caries; intensive fluoride treatment, antimicrobial treatment or in-depth dietary advice based on a dietary history

Starting with the dental records, information for all variables were gathered and registered in Excel® by two of the authors (TK and CC), Figure 1. Prior to the registrations they underwent training and calibration in data extraction from dental records in the dental record system Carita (Swedish Care System AB). Thereafter output data from SKaPa on the same variables were added for each individual matched by the PIN. Data were pseudonymized throughout the compiling and analyzing process.

Figure 1. Schematic description of data collection.

Statistical analysis

The proportions of registrations with exact agreement (exactly the same measure in both data sources) between index (SKaPa) and reference (electronic dental records) were calculated and are presented as exact agreement in percentage. To provide more information and give a better understanding of the data we also included two other measures of agreement between index and reference. Cohen’s Kappa, which is presented with the test statistic and standard error and, Intraclass Correlation Coefficient (ICC), which is presented with the test statistic and 95% confidence interval. All analyses were performed using Stata 15 SE.

Ethics

The study was registered in Clinical Trials www.clinicaltrials.org; NCT03039010 and approved by the Regional Ethics Review Board in Uppsala, Sweden in 2016 (#2016/051).

Results

Participants

The study population included 250 6-year-olds (46% boys, 54% girls) and 250 12-year-olds (48% boys, 52% girls). Information on dental caries and frequency of preventive measures recorded from the dental records (reference standard) is shown in Table 2.

Table 2. Information on dental caries recorded from the dental records (reference standard).

Age			Outcomes
Dental caries	Variable	N total	Mean	SD	N with score >1	Mean	SD
6-year-olds	dft	250	0.98	2.30	65	3.79	3.15
	deft	250	1.08	2.44	68	3.99	3.24
	et	250	0.10	0.42	18	1.39	0.85
12-year-olds	DFT	250	0.44	1.03	61	1.82	1.35
	DMFT	250	0.63	1.26	76	2.08	1.49
	MT	250	0.19	0.63	27	1.74	0.98

Test results

For the total group of 500 individuals the Kappa value was calculated for agreement in dft/DFT between the electronic dental records and the SKaPa-registry to 0.95 (Table 3), ICC was calculated to 0.98 (Table 4). There was an exact agreement in 490 cases (98%) for dft/DFT between dental records and SKaPa (whereof 373 cases with dft/DFT = 0).

Table 3. Agreement between data on dental caries from patients’ electronic dental records and the SKaPa registry shown as Kappa-values with standard error (SE), and exact agreement in percent. Data on dft, deft and et provided for primary teeth in 6-year-olds, and DFT, DMFT, and MT for permanent teeth in 12-year-olds. For the total group dft/DMFT, deft/DMFT and et/MT are combined. For dft/DFT >1 only, deft/DMFT >1 only and et/MT >1 only the analyses include cases where the value for dft/DFT/deft/DMFT/et/MT exceeds 0 in the dental record.

	Total (6-and 12-year-olds)			6-year-olds			12-year-olds
	N	Kappa (SE)	Exact agreement (%)	N	Kappa (SE)	Exact agreement (%)	N	Kappa (SE)	Exact agreement (%)
										dft/DFT	500	0.95	98.0	250	0.96	98.4	250	0.94	97.6
(0.03)	(0.03)	(0.05)
dft/DFT >1 only	126	0.91	92.9	65	0.93	93.9	61	0.86	91.8
		(0.04)			(0.05)			(0.07)
deft/DMFT	500	0.80	90.8	250	0.89	94.8	250	0.70	86.8
		(0.03)			(0.03)			(0.04)
deft/DMFT >1 only	144	0.63	70.1	68	0.79	82.4	76	0.42	59.2
		(0.03)			(0.05)			(0.05)
et/MT	500	0.38	91.8	250	0.71	96.0	250	0.03	87.6
		(0.03)			(0.05)			(0.04)
et/MT >1 only	45	0.13	26.7	18	0.34	61.1	27	0.02	3.7
		(0.04)			(0.11)			(0.02)

Table 4. Intraclass Correlation Coefficient (ICC) and 95 % confidence interval (CI) of dental caries in 6- and 12-year-old children recorded in electronic patient dental records and SKaPa. Data on dft, deft and et provided for primary teeth in 6-year-olds, and DFT, DMFT, and MT for permanent teeth in 12-year-olds. For the total group dft/DMFT, deft/DMFT and et/MT are combined. For dft/DFT >1 only, deft/DMFT >1 only and et/MT >1 only the analyses include cases where the value for dft/DFT/deft/DMFT/et/MT exceeds 0 in the dental record.

	Total (6- and 12-year-olds)				6-year-olds				12-year-olds
	N	ICC	95% CI		N	ICC	95% CI		N	ICC	95% CI
													dft/DFT	500	0.98	0.98	0.99	250	0.98	0.98	0.99	250	0.98	0.98	0.99
dft/DFT >1 only	126	0.97	0.96	0.98	65	0.97	0.95	0.98	61	0.96	0.94	0.98
deft/DMFT	500	0.96	0.96	0.97	250	0.99	0.99	1.00	250	0.83	0.77	0.87
deft/DMFT >1 only	144	0.94	0.90	0.96	68	0.99	0.98	0.99	76	0.68	0.38	0.82
et/MT	500	0.36	0.29	0.44	250	0.62	0.54	0.69	250	0.00	−0.11	0.12
et/MT >1 only	45	0.05	−0.17	0.30	18	0.30	−0.19	0.67	27	−0.01	−0.09	0.12

The calculated Cohen’s Kappa for deft/DMFT for the total population was 0.80, deft (6-year-olds) 0.89 and DMFT (12-year-olds) 0.70 (Table 3). The Intraclass correlation coefficient (ICC) for deft/DMFT was calculated for the total population: 0.96, deft (6-year-olds) 0.99 and DMFT (12-year-olds) 0.83 (Table 4). There was an exact agreement in 454 cases (91%) for deft/DMFT between dental records and SKaPa (whereof 353 cases with deft/DMFT = 0).

To fully understand how an extracted or missing tooth (et/MT) affects the accuracy of data in SKaPa we calculated Cohen’s Kappa, exact agreement and ICC for et/MT and also for individuals with caries, i.e. for cases where the values of dft/DFT, deft/DMFT and et/MT, exceeded 0 in the dental records. As shown in Tables 3 and 4 this resulted in a clear decrease of outcomes in all comparisons (Kappa, exact agreement and ICC) most pronounced in the 12-year-olds.

Three treatment codes for preventive measures (codes 204, 313 and 314) were not used at all. The remaining six codes were identified in dental records of 284 children (at least one measure). Comparisons with output data from SKaPa for these six codes revealed very high Kappa values and ICC. Cohen’s Kappa varied between 0.99 and 1.00 for the six preventive measures and the ICC varied between 0.99 and 1.00.

Discussion

This study shows that calculated Kappa and ICC values for agreement in dft/DFT between the electronic dental records and the SKaPa-registry were very high, whereas the agreement for deft/DMFT values were considerably lower. The discrepancies are related to extracted or missing teeth (et/MT) and the problem is greater among 12-year-olds compared to 6-year-olds.

As SKaPa is a national quality registry, intended to monitor dental health in the population and for registry-based research, with the potential for linking into other registries, it is important to ascertain that the quality of data in SKaPa is high. Our results confirm that transfer of data for dft, DFT and treatment codes from electronic dental records to the SKaPa-registry is safe and correct. Thus, these variables can be used in registry-based studies incorporating output data from SKaPa. However, the results also clearly show that the accuracy concerning deft and DMFT is too low why it is advised not to use these variables.

By merging official registries, it is possible to analyze and identify differences in general and dental health in specific societal groups. Registry-based research is likely to increase as more registries become available [1, 11]. This kind of research enables inclusion of large populations and data volumes which makes it an attractive way to gain new knowledge. At the same time, there is a risk that researchers fail to see the challenges of registry research, e.g. ethical issues and the importance of the quality of the registries [12–16].

The benefit of the SKaPa-registry is that the data are exported automatically from the dental records to SKaPa [9] and it is expected that this should result in safer data transfer to the registry than manual registration, but this has not been fully evaluated. A previous study which followed the complete chain of data, for dft and DFT in 170 child patients from the researcher’s dental examination and the data registered in the electronic dental record, to the data retrieved from the SKaPa registry, showed very good correlations [10]. That study did not investigate if data on deft/DMFT, et/MT, or treatments are transferred correctly to the SKaPa registry. This was evaluated in the present study based on 500 new dental records, where no records from the previous study were included.

The present evaluation was conducted in the county region of Värmland, where the data system Carita (Swedish Care System AB) is used to register patient data in the dental records. Other dental data systems used in Sweden were not evaluated in this study. However, the same file specifications are used from all providers of dental records, and the structure and transfer method to the registry is the same. As Carita is used in 4 of the 21 regions representing almost 22% of the total child population in Sweden the present study is an important indicator that the data transfer works.

We included several measures of agreement, ICC, Cohen’s Kappa, and exact agreement. We would have preffered ICC only, as it is a more effective method fitting our data, and this is also our main analysis. However, we also included Kappa and exact agreement as we believe that some readers might have interest in that information and as all measures combined could give a better understanding of the data. A large proportion of the included patients were caries free which is good from an oral health perspective, and this also mirrors the relatively good oral health status in children in Värmland. At the same time, this floor effect is likely to impact on both ICC and Kappa values as it might be easier to identify a caries free tooth compared to diagnose caries and define if a lesion is incipient or manifest (cavity). This was also found in the present study where it was addressed and emphasized by adding the stratified analyses that only included those with registered values above 0 in the dental records.

The accuracy of deft/DMFT was considerably lower than for dft/DFT, whereby it is assumed that dft/DFT is more stable than deft/DMFT. As severe dental caries may result in extractions, the level of agreement for e/M in the deft/DMFT was of special interest. When we saw lower correlations for deft and DMFT compared with dft and DFT we decided to analyze the agreement for the part of the population having dental caries (i.e. individuals with dft/DFT/deft/DMFT values of 1 or more) separately. Doing so it became evident that the e/M component is less accurate and inclusion of the variable e/M in deft/DMFT would give a misleading information from SKaPa on dental health status.

The difference was larger in 12-year-olds, possibly a consequence of the mixed dentition stage, where registration of dental health status in the dental records must accommodate both primary and permanent teeth, a complication which increases the risk of errors. There is a risk that a not yet erupted permanent tooth that is missing in the oral cavity can be unintentionally, but wrongly, recorded as extracted. In the same way primary teeth that are missing because of exfoliation can be wrongly registered as extracted. This misclassification is due to the human factor, a tick in the wrong box happens in digital systems. The risk is larger in the mixed dentition and especially if it is a new patient where the dentist starts a new dental record with no history of previous treatment. If a tooth is wrongly marked as extracted in the dental record and this has been undersigned by the dentist, the information about an extracted tooth will be sent to SKaPa. This underlines the importance of the dental team ensuring that accurate registrations are entered into the dental record, so that the status of each tooth is correct. This is particularly important in children with unerupted or extracted teeth, or hypodontia. False registrations affect the variable missing/extracted in deft/DMFT and there is a potential risk of systematic errors. Another problem is that the digital dental record systems don’t provide options that specify the reason for a tooth being missing, it cannot identify if a tooth was extracted due to caries or due something else like trauma, periodontal disease etc.

Based on our results we advise researchers and public health workers to refrain from using output data from SKaPa on the et/MT and deft/DMFT indices. At the same time this is far from optimal as excluding the variable e/M in deft/DMFT can distort the statistics: there is a risk that children with the poor dental health are not identified in the data, that dental health is overestimated. If correlation analyses are made with only dft/DFT and not deft/DMFT, the effect size of dental disease will be overestimated. However, it is worth noting that a similar dilemma with discrepancy in number of teeth between dental records and registry output has been described for another registry of dental health in adults (the Swedish Dental Health Register) [17]. That study reported of a high accuracy when comparing number of intact teeth between dental records and the registry for patients having less than 32 intact teeth (but not edentulous) while the correctness for patients being edentulous or having 32 remaining intact teeth was low [17]. Further, the Swedish National Board of Helth and Welfare has chosen to exclude e/M when compiling and reporting of dental health in children and adolescents [18].

The reported low accuracies for extracted and missing teeth in the registries are problematic as they impose bias to both dental health monitoring and research. It is therefore an urgent need to develop conditions and strategies for a more accurate reporting of extracted and missing teeth in the dental records, including reason for extraction or why a tooth is missing.

As the present study showed high agreements between data in the electronic dental records and the SKaPa-registry regarding treatments it is in the meantime recommended to use treatment codes for extractions if researchers wish to gather output data from SKaPa on missing teeth. It may also be of value to look at older age groups in adolescents in future studies to see if the problems related to e/M is different in the young permanent dentition compared with in the mixed dentition.

Systematic collection of health data is important for modern health evaluation and research [13] and concequently accurate dental health data in registries are central. More and more dental registers are developed in other countries, and it will be possible to link registers from different countries for collaboration. Since SKaPa has a high national coverage and is available for international research collaborations its properties are of interest also outside of Sweden. This study shows that with the SKaPa-registry it is possible to evaluate the prevalence of dental caries in children and to analyse the effects of interventions based on dft and DFT. But, at this moment there is too much uncertainty around extracted and missing teeth why we recommend great caution when using the variables deft/DMFT.

Conclusion

The present study reveals high agreement between data in the electronic dental records and the SKaPa-registry regarding dft/DFT and six preventive treatments. Data from the SKaPa-registry dft and DFT may be regarded as reliable for the purposes of monitoring dental health in the population, for studying different preventive measures and for registry-based research. As the agreements were substantially lower for deft/DMFT, especially in 12-year-olds, we advise against using output data on deft or DMFT from SKaPa for research purposes at this point.

Author contributions

TK, GK, ST and AC conceived ideas and designed the study. TK and CC managed the data collection. TK, ANA and GK performed the statistical analysis. All authors participated in interpretation of the study results, contributed to the manuscript, and read and approved the final draft. All authors read and approved the final manuscript.

Acknowledgements

The authors thank The Swedish Quality Registry for Caries and Periodontal Diseases (SKaPa) for providing access to the registry’s data.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This study was supported by research funds from Centrum for research and education (CKF), Region Värmland, Sweden. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.