Diagnostic accuracy of HIV test kits, Genscreen Ultra and Bioelisa

Abstract

Purpose

Genetic diversities in different countries affect the performance of HIV test kits. Therefore, WHO recommends evaluation of every HIV test kit in countries’ context before its use. Therefore, this study aimed to evaluate the performance of Genscreen ULTRA HIV Ag–Ab and Bioelisa.

Materials and methods

The study had used 400 characterized plasma samples obtained from CDC Atlanta bio-bank derived from Africa, USA, and Thailand.

Results

Diagnostic performance of both test kits under evaluation was assessed at 95% CI. Genscreen ULTRA HIV Ag–Ab had sensitivity and negative predictive value of 99.5% [95% CI, 97.2–99.9] and the specificity and positive predictive value of 98.5% [95% CI, 95.7–99.7]. Bioelisa HIV test kit had exhibited sensitivity and negative predictive value of 99% [95% CI, 96.4–99.7] and specificity and positive predictive value of 98.5% [95% CI, 95.7–99.7]. Both test kits were able to detect almost all samples with HIV-2, dual infections, and seroconversion.

Conclusion

Both the test kits were highly sensitive and specific in detecting HIV. However, there are still few samples containing HIV antibody which were not identified by both kits. Therefore, additional screening measures should be done in using these assays for blood transfusion and organ transplantation. In addition, the study can be used as a reference by other African countries.

Video abstract

Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:

http://youtu.be/6mev50pk2p0

Keywords:

• ELISA
• sensitivity
• positive predictive
• specificity
• validation

Introduction

According to the 2017 HIV/AID Fact Sheet, HIV is one of the major public health problems in the world; more than 35 million people living with HIV/AIDS and 1.0 million people died from HIV-related causes. At the end of 2016, there were approximately 36.7 million people living with HIV, 19.5 million people of these were receiving antiretroviral therapy (ART) globally. Of all people living with HIV, 1.8 million became newly infected in 2016. Currently, a total of 54% of adults and 43% of children living with HIV are receiving lifelong ART. Around 26 million people living with HIV are found in the African region in 2016, accounting for almost two-thirds of the global total new HIV infections. Currently, it is expected that only 70% of people with HIV know their HIV status. To reach the target of 90%, an additional 7.5 million people need to access HIV testing services.¹

In Ethiopia, it is estimated that 613,825 people were living with HIV in 2017, of which 62% were female.² The overall prevalence of HIV has fallen by 82%, from 3.38 in 2000 to 0.92 in 2017. Annual death rate also showed a decline of 81%, from 83,055 annual deaths in 2000 to 15,439 in 2017. Regional prevalence in the country is heterogeneous in its kind ranging from 0.1% in Somali to 4.8% in Gambella.³ Diagnosis of HIV infection is usually based on a multi-test algorithm for detecting antibodies to HIV. Screening tests are used for identification of specimens that contain antibody to HIV. It is reported that enzyme immune absorbent assays (EIAs) are preferred for their high sensitivity of detecting antibodies/antigens of HIV. Additional tests, such as Western blot (WB), can be used to confirm infection in samples that are initially reactive on conventional EIAs or rapid tests. For practical purposes, resource-poor settings depend heavily on EIA and rapid tests for screening and confirmation of HIV infection.⁴

EIAs are the most commonly used tests because it allows testing a large number of specimens. So far, there are four generations of EIAs with relatively different specificities and sensitivities in pathogen detection. The first-generation assays which use purified HIV whole viral lysates have poor sensitivity and specificity. Assays were improved as the generation number increases to reduce window period.⁵

The genetic diversities in different countries affect the performance of HIV test kits; hence, WHO recommends countries to validate new test kits in the context of countries before use. Failure to validate the kits will result in lack of integrity of the facilities, personals as well as the quality of reported test results.⁴ In line with this recommendation, in Ethiopia, there was no study conducted to assess the performance of Genscreen™ ULTRA HIV Ag–Ab (Bio-Rad, France) and Bioelisa HIV-1 +2 Ag/Ab (Biokit S.A., Spain) kits. Therefore, this study assessed the diagnostic accuracy of Genscreen ULTRA HIV Ag–Ab and Bioelisa HIV-1 +2 Ag/Ab HIV test kits using characterized reference plasma samples. This will assist researchers and epidemiologists concerned with HIV epidemic in selecting accurate ELISA-based HIV test kits, particularly for surveys and surveillance that involve large sample size.

Materials and methods

Assay kit and test procedure

Genscreen ULTRA HIV Ag–Ab: The Genscreen ULTRA HIV Ag–Ab is a fourth-generation enzyme immunoassay based on the principle of the sandwich technique for the detection of HIV antigen and of the various antibodies associated with HIV-1 and/or HIV-2 virus in human serum or plasma.⁶

Bioelisa HIV-1 +2 Ag/Ab: This is also a fourth-generation assay for the simultaneous detection of antibodies to HIV-1 and HIV 2 as well as the p24 antigen of HIV-1. The assay is based on recombinant and synthetic peptides for antibody detection and the monoclonal antibody specific to the p24 protein of HIV-1.⁷

The test procedure of both Genscreen ULTRA HIV Ag–Ab and Bioelisa HIV-1 +2 Ag/Ab was strictly followed as recorded on the test kit insert of each respective test kit.

Reference panels

Characterized reference plasma specimens obtained from CDC Atlanta and stored at –80°C in a duplicate of 1.0 mL using a sterile polypropylene tube were used. The tubes were labeled using waterproof permanent markers to help identify the specimens. The specimens included all subtypes of HIV including subtypes C and C’. All specimen and data were collected before the study was started. The refrigerator temperature was monitored and documented daily.

The panel included 38 plasma specimens commercially obtained from HIV-infected individuals at seroconversion period to challenge the test kits. The samples were tested using double ELISA and confirmed by modified electrophoresed WB.

Study population and sample size

The reference panels were obtained from East, West, and South Africa (namely, Ethiopia, Cameroon, Cotd’Ivoire, Kenya, South Africa, and Uganda). It has also included samples from USA and Thailand. These panels were used because Ethiopia is a diplomatic site for the African Union and is an important country in East Africa, like playing the lead role for Intergovernmental Authority on Development (IGAD) member countries. The country has also a strong trade relationship with neighboring countries. Hence, high mobility and interaction of the above-listed countries’ population with Ethiopians are expected and worth to anticipate viral strain exchange during the disease transition. This has enabled the study to report the diagnostic accuracy of the test kits with diversified population groups. Therefore, the test kits to be used in Ethiopia was validated and challenged with all the HIV type that could possibly exist in the country.

A total of 400 well-characterized reference plasma specimens (200 HIV1/2 positives and 200 HIV1/2 negatives) were used to measure each test kit’s performance.

Testing/reference plasma

The reference plasma specimens were obtained from a well-recognized international laboratory, CDC Serology Laboratory at Atlanta. The specimens were tested by double ELISA and confirmed by WB in the laboratory indicating that the results of the panels are reliable.

The integrity and quality of the characterized and stored specimens were verified before proceeding to test according to Ethiopian Public Health Institute’s (EPHI’s) SOP for specimen handling. There was documentation for all pieces of information including specifics on the test kits used including identification of testers. The manufacturers’ instructions for use and the validation protocol were strictly adhered to, and all specimens were tested in a blinded fashion.

Quality assurance

Three days orientation training was provided to all testers who perform the testing to ensure the competency of the testers. General SOP for workflow and assay-specific SOPs for individual assays were prepared/customized, read, signed, and strictly followed by all those who were involved in the process. Senior laboratory professionals with ample experience in HIV testing had conducted intensive supervision on a daily basis. Testers were blinded to the results of the reference panel.

Resolution of discrepant results

Specimens with discordant results (the result of assay under validation different to the reference result) were repeated by a different tester. To assist in the resolution of discrepant results that might not be truly discrepant, investigations were performed through inspection to rule out if there were specimen mix-up or if transcription error occurred and whether the manufacturer’s instructions for use were strictly followed or not. After such further investigations, the specimens with a result different from the reference result were repeated by a different tester who was blinded to the reference and assay result. The specimens that repeatedly produce discrepant results were included in the analysis as a final result of the assay.

Data management and analysis

Tracking information of specimen shipment, storage condition, date, and time of transfer were recorded. An Excel sheet was created to keep the record of the test kits results (nonreactive/reactive/invalid). Data were entered into a password-protected computer and were accessible only to the principal investigator.

Sensitivity, specificity, positive predictive values, negative predictive values, and 95% CIs for each value were calculated and confirmed using an online calculator, STATCALC.

Ethical considerations

In consideration of the less than minimum risk in involvement, anonymity, and unlinked nature of the samples used, the study was approved by the Ethiopian Public Health Institute’s Science and Ethics Review Board.

Results

The findings of the laboratory-based validation for both Genscreen and Bioelisa test kits upon testing a panel of 400 well-characterized reference plasma specimens (200 positive and 200 negative for HIV antibodies) were analyzed using the methods described under “Study population and sample size” section. The reference panels were obtained from East, West, and South Africa. It also includes samples from USA and Thailand. Majority of the samples both negative and positive were drawn from Ethiopian population. The specimens contain a pool from HIV-1, HIV-2, and dual infections (Table 1).

Table 1 Panels used for validation

Country/organization of origin	HIV negative	HIV positive	HIV-1	HIV-2	HIV dual
Ethiopia	80	50	50
Cameroon	7	15	15
Cote d’Ivoire	45	30	16	4	10
Kenya	45	30	30
South Africa	3	15	15
Thailand	20	15	15
Uganda		7	7
SeraCare		32	32
BBI (Boston Biomedical Inc.)		6		6
Total	200	200	180	10	10

Diagnostic performance of Genscreen ULTRA HIV Ag–Ab

Sensitivity, specificity, positive predictive value, and negative predictive value of Genscreen ULTRA HIV Ag–Ab test kit were calculated at 95% CIs. Out of the 200 HIV positive known reference specimens, the test kit has identified 199 of them as positive and missed one HIV positive sample. Among the 200 HIV negative reference samples, it has correctly identified 197 as HIV negative and missed three HIV negative samples. The kit’s sensitivity, specificity, positive predictive value, and negative predictive value were found to be high (Tables 2 and 3).

Table 2 Reference and Genscreen ULTRA HIV Ag–Ab test result

Test kit	Reference standard positive	Reference standard negative
Genscreen ULTRA HIV Ag–Ab Positive	199	3
Genscreen ULTRA HIV Ag–Ab Negative	1	197
	200	200

Table 3 Test accuracy of Genscreen ULTRA HIV Ag–Ab

Indicators	Estimate (%)	95% CI
Sensitivity	99.5	97.2–99.9
Specificity	98.5	95.7–99.7
Positive predictive value	98.5	95.7–99.7
Negative predictive value	99.5	97.2–99.9

Diagnostic performance of Bioelisa HIV-1+2 Ag/Ab

Sensitivity, specificity, positive predictive value, and negative predictive value of Bioelisa HIV-1 +2 Ag/Ab test kit were also calculated at 95% CIs. Out of the 200 HIV positive known reference specimens, the test kit had identified 198 of them as positive and missed two HIV positive samples. Among the 200 HIV negative reference samples, it has correctly identified 197 as HIV negative and missed three samples. The kit’s sensitivity, specificity, positive predictive value, and negative predictive value were found to be high (Tables 4 and 5).

Table 4 Reference and Bioelisa HIV-1 +2 Ag/Ab test results

Test kit	Reference standard positive	Reference standard negative
Bioelisa HIV-1 +2 Ag/Ab positive	198	3
Bioelisa HIV-1 +2 Ag/Ab negative	2	197
	200	200

Table 5 Test accuracy of Bioelisa HIV-1 +2 Ag/Ab

Indicator	Estimate (%)	95% CI
Sensitivity	99	96.4–99.7
Specificity	98.5	95.7–99.7
Positive predictive value	98.5	95.7–99.7
Negative predictive value	99	96.4–99.7

Both the test kits have correctly identified 38 challenge panels drawn from HIV positive clients in window period. Ten of the reference panels had HIV 2 and another 10 panels had both HIV 1 and HIV 2. Both the test kits were able to identify all panels that contain HIV 2 and dual infection as HIV positive correctly.

Discussion

Both Genscreen ULTRA HIV Ag–Ab and Bioelisa HIV-1 +2 Ag/Ab had exhibited a very high sensitivity of 99% and above in this study. This finding is in line with the principle of the test kits that fourth-generation test kits have techniques that enable them to detect HIV antigen and various antibodies associated with HIV.^6.7

A study conducted in India had also shown a significantly higher sensitivity of fourth-generation test kits particularly Genscreen ULTRA HIV Ag–Ab. The study had suggested the use of the fourth-generation test kits for HIV screening especially at the blood bank to ensure safe blood transfusion.⁸ A comparative study of rapid and ELISA test kits was also highlighted that higher sensitivity and ability to test a large number of samples at a time has made ELISA test kits preferable than rapid test kits for the screening test.⁹

Genscreen ULTRA HIV Ag–Ab and Bioelisa HIV-1 +2 Ag/Ab had missed only 1 and 2 true positive samples, respectively. The two test kits were not able to detect a specimen labeled KE-229 and additional one true positive sample was missed by Bioelisa. Both false negative samples on the assays under evaluation were obtained from Kenya, and they were HIV-1 positive on WB showing 160, 24, and 17 bands. After the introduction of HIV into the body, there will be a consistent sequence of antibody response. Presence of such bands, 160, 24, and 17 on WB indicates HIV infection in its earliest time in the sample. These earliest antibodies are directed against gp160, gp120, p24, and p17, followed shortly by antibodies to gp41, p55, p66, and p51. In general, antibodies to p24 and p55 reduce after the beginning of symptoms of AIDS, whereas antibodies to envelope glycoprotein persist.¹⁰ Thus, it is possible that the two assays have an inherent limitation in detecting antibodies against HIV in its earliest infection. Another possible explanation is that the concentration of HIV-specific antibodies in the two sera is very low and could not be detected by both ELISA tests. This shows that the sensitivity of Bioelisa and GenScreen is inferior to that of WB.

In this study, Genscreen ULTRA HIV Ag–Ab and Bioelisa HIV-1 +2 Ag/Ab had three false positive results. Common reasons for false positivity and false negativity like mislabeling of samples and other technical errors were ruled out by repeating the specimens to be tested by other blinded technician for the original result. There are several factors that may result in the production of antibodies which could lead to a false positive result for HIV test. Some of the causes mentioned in the literature were autoimmune diseases like systemic lupus erythematosus, scleroderma, connective tissue disease, and dermatomyositis; flu, hepatitis B,¹¹ and tetanus vaccinations;¹² and many infections not limited to TB, leprosy,¹³ flu,¹⁴ malaria,¹⁵ and hepatitis.¹⁶

Both Genscreen ULTRA HIV Ag–Ab and Bioelisa HIV-1 +2 Ag/Ab are qualitative tests used to identify p24 antigen and antibodies to HIV-1 (groups M and O) or HIV-2 in human body fluids in clinical laboratories and as a first-line screening tests in blood centers.^6,7 This study has shown that both test kits are able to detect the majority of seroconversion panels except those in the earliest period of infections. Therefore, caution or additional screening measures should be done when used for blood transfusion and organ transplantation.

The purpose of this study was not to compare the performance of the two ELISA test kits with each other, and we have conducted the laboratory test separately in a serial testing pattern. However, since the same characterized plasma samples were used to conduct the validation for both the test kits by the same testers in the same laboratory, it is important to mention that Genscreen ULTRA HIV Ag–Ab exhibited slightly higher sensitivity than Bioelisa HIV-1 +2 Ag/Ab. Specificity and positive predictive value of the two test kits were found to be equal.

Conclusion

Genscreen and Bioelisa had high sensitivity. Genscreen ULTRA HIV Ag–Ab assay was found to be highly specific too. They were able to detect all samples with HIV-2 and dual infection obtained from different countries. The test kits could also detect almost all seroconversion panels.

Recommendation

We recommend both Genscreen ULTRA HIV Ag–Ab and Bioelisa HIV-1 +2 Ag/Ab can be used as a screening and confirmatory assay in countries’ algorithm to diagnose HIV infection. Since few samples containing HIV were missed by both test kits, to avoid any risk of HIV transmission, caution or additional screening measures should be done when used for blood transfusion and organ transplantation. In addition, the study can be used as a reference by other African countries that has similar HIV strains with Ethiopians.

Acknowledgments

The study team is grateful to the Ethiopian Pubic Health Institute for supporting the evaluation. We would also like to acknowledge CDC Atlanta, International Laboratory Branch for providing us with the characterized plasma panels used for the study. Particularly our appreciation goes to Dr Bharat S Parekh and Dr Clement Zeh. Finally, we also appreciate Dr Tsigereda Kifle, Mr Tesfaye Tilahun, and Mr Mulusew Getaneh for their valuable contributions to the successful completion of the study.

Disclosure

The authors report no conflicts of interest in this work.