Assessment of completion of early medical abortion using a text questionnaire on mobile phones compared to a self-administered paper questionnaire among women attending four clinics, Cape Town, South Africa

Abstract

In-clinic follow-up to assess completion of medical abortion is no longer a requirement according to World Health Organization guidance, provided adequate counselling is given. However, timely recognition of ongoing pregnancy, complications or incomplete abortion, which require treatment, is important. As part of a larger trial, this study aimed to establish whether women having a medical abortion could self-assess whether their abortion was complete using an automated, interactive questionnaire on their mobile phones. All 469 participants received standard abortion care and all returnees filled in a self-assessment on paper at clinic follow-up 2–3 weeks later. The 234 women allocated to receive the phone messages were also asked to do a mobile phone assessment at home ten days post-misoprostol. Completion of the mobile assessment was tracked by computer and all completed assessments, paper and mobile, were compared to providers’ assessments at clinic follow-up. Of the 226 women able to access the mobile phone assessment, 176 (78%) completed it; 161 of them (93%) reported it was easy to do so. Neither mobile nor paper self-assessments predicted all cases needing additional treatment at follow-up. Prediction of complete procedures was good; 71% of mobile assessments and 91% of paper assessments were accurate. We conclude that an interactive questionnaire assessing completion of medical abortion on mobile phones is feasible in the South African setting; however, it should be done later than day 10 and combined with an appropriate pregnancy test to accurately detect incomplete procedures.

Résumé

Selon les directives de l’Organisation mondiale de la santé, le suivi dans le centre de santé pour s’assurer que l’avortement médicamenteux est complet n’est plus nécessaire, pourvu que des conseils adaptés soient prodigués. Néanmoins, il est important de déceler rapidement une poursuite de la grossesse, des complications ou un avortement incomplet, qui exigent un traitement. Dans le cadre d’un essai plus large, cette étude visait à déterminer si femmes ayant eu un avortement médicamenteux pouvaient évaluer elles-mêmes si leur avortement était complet, à l’aide d’un questionnaire interactif automatisé sur leur téléphone portable. Toutes les 469 participantes ont reçu des soins types liés à l’avortement et toutes les patientes revenues au centre ont rempli une autoévaluation sur papier lors du suivi dans ce centre, deux ou trois semaines plus tard. Les 234 femmes sélectionnées pour recevoir les messages téléphoniques ont aussi été invitées à faire une évaluation sur téléphone portable chez elles dix jours après la prise de misoprostol. L’achèvement de l’évaluation sur le portable a été contrôlé par ordinateur et toutes les évaluations complétées, sur papier et téléphone, ont été comparées à l’évaluation des praticiens lors du suivi dans le centre. Des 226 femmes ayant eu accès à l’évaluation sur portable, 176 (78%) l’ont complétée ; 161 d’entre elles (93%) ont indiqué que cela avait été facile. Les autoévaluations, que ce soit sur portable ou sur papier, n’ont pas prévu tous les cas nécessitant un traitement complémentaire lors du suivi. Les prédictions de la complétude de la procédure étaient bonnes : 71% des évaluations sur portable et 91% sur papier étaient exactes. Nous avons conclu qu’un questionnaire interactif évaluant la complétude de l’avortement médicamenteux sur le téléphone portable est faisable dans le contexte sud-africain ; néanmoins, il devrait être soumis plus tard que le dixième jour et associé à un test de grossesse approprié pour détecter rigoureusement les procédures incomplètes.

Resumen

El seguimiento en la unidad de salud para determinar si la mujer tuvo un aborto con medicamentos completo ya no es un requisito, según la guía de la Organización Mundial de la Salud, siempre y cuando se brinde consejería adecuada. Sin embargo, es importante reconocer con prontitud la continuación del embarazo, complicaciones o aborto incompleto, que requiere tratamiento. Como parte de un ensayo clínico más extenso, este estudio tuvo como objetivo establecer si mujeres en proceso de aborto con medicamentos podrían determinar por sí mismas si tuvieron un aborto completo, utilizando un cuestionario interactivo automatizado en su teléfono móvil. Todas las 469 participantes recibieron servicios estándares de aborto y todas las que regresaron llenaron un formulario de autoevaluación durante el seguimiento en la unidad de salud 2 o 3 semanas después. A las 234 mujeres asignadas para recibir los mensajes por teléfono también se les pidió que hicieran una evaluación domiciliaria por teléfono móvil diez días post-misoprostol. La conclusión de la evaluación móvil fue seguida por computadora y todas las evaluaciones terminadas, en versión impresa y móvil, fueron comparadas con las evaluaciones de los prestadores de servicios durante el seguimiento en la unidad de salud. De las 226 mujeres que tenían acceso a su evaluación por teléfono móvil, 176 (78%) la concluyeron; 161 de ellas (93%) informaron que fue fácil de hacer. Ni la autoevaluación móvil ni la impresa previeron todos los casos que necesitaban tratamiento adicional en el seguimiento. La predicción de procedimientos completos fue buena; el 71% de las evaluaciones móviles y el 91% de las evaluaciones impresas fueron precisas. Concluimos que un cuestionario interactivo que evalúa la finalización del aborto con medicamentos por teléfono móvil es factible en Sudáfrica; sin embargo, debe hacerse después del décimo día y combinarse con una prueba de embarazo apropiada para detectar con precisión los procedimientos incompletos.

Keywords:

• text messaging
• medical abortion
• follow-up
• self-assessment of complete abortion
• mobile phones
• South Africa

Medical abortion in the first trimester of pregnancy using mifepristone and misoprostol was approved by the South African Medicines Control Council in 2001 and has been provided in non-governmental organization (NGO) and private sector clinics in South Africa since 2002. Only in 2010 did provincial guidelines authorize the use of the evidenced-based 200 mg dosage of mifepristone, and approval given to roll-out medical abortion in public health facilities, led by the Western Cape Province and expanding into the rest of the country.¹ Improving access to abortion services by introducing medical abortion at more primary care facilities is expected to address some of the existing barriers to safe abortion care in South Africa, in particular the shortage of providers willing to perform surgical abortions.^2–4

Current standard medical abortion care up to 63 days LMP in both the NGO and public sectors in South Africa involves oral mifepristone (200 mg) followed by misoprostol used 1–2 days later at home (800 mcg = 400 mcg sublingual and 400 mcg oral or buccal), and a follow-up visit at the abortion facility to assess completion of abortion.¹ Earlier studies in South Africa have reported a high level of acceptance of use of misoprostol at home among both women³ and providers.⁴

In the USA, in an effort to simplify medical abortion provision and thus improve acceptability and access, the Planned Parenthood Federation of America recommended in 2005 that the in-clinic follow-up requirement be waived should it create undue hardship for women, if they could manage their follow-up with a serum hCG pregnancy test at a facility closer to them.⁵ More recently, the World Health Organization (WHO) 2012 guidance said that routine, in-clinic follow-up was not needed as long as women received adequate counselling on potential complications and symptoms of ongoing pregnancy.⁶

Barriers to a follow-up visit may include cost, the need for privacy and the need to balance competing demands of home, family, work or school. In addition, studies have shown that as medical abortion with mifepristone and misoprostol becomes better known and is further demedicalized,^* women increasingly don’t return to the clinic if they consider the procedure to have been complete.^7,8 Tracing women by telephone who have not returned for follow-up, in order to reassure providers that treatment has been successful, can be time consuming⁹ and may not be feasible in resource-constrained settings like the South African public sector (Personal communication, regional health manager, 2010).

WHO guidelines also say the use of ultrasound routinely is neither recommended nor required for assessment of completion of abortion. Research into follow-up strategies that avoid the use of ultrasound have included an in-clinic consultation between the provider and the woman either with or without a high sensitivity urine pregnancy test,^10–12 or a set of questions on her symptoms and a clinical examination in combination with a low sensitivity urine pregnancy test.⁸ Alternative out-of-clinic strategies involving providers have included a standardized telephone questionnaire between provider and woman,¹³ or a telephone consultation plus a urine pregnancy test — high sensitivity at 30 days¹⁴ or low sensitivity at 14 days.^15,16 Web-based approaches have used an email consultation or an online form.^17,18 Studies exploring safe self-assessment methods have included a set of questions on the symptoms experienced plus different pregnancy tests.^7,8,19

A 2012 review of the evidence²⁰ pointed towards a symptom checklist combined with a suitable pregnancy test as the most promising method to accurately self-assess ongoing pregnancies, complications or incomplete procedures.

More recent studies which used a standardized set of simple questions^9,13,15 most closely correspond to the design we implemented for mobile phone-based self-assessment in this study. However, in 2011, at the start of our study, it was not clear from the published literature exactly what wording to use for the questions or to what extent clinical judgement was involved when the questions were asked by a provider. Also, before 2013, many of these studies had been conducted in developed countries; only recently has limited evidence been emerging from low- and middle-income settings.^7,10

The purpose of this study was to determine whether women could complete an accurate self-assessment of the outcome of medical abortion on their mobile phones while at home. We aimed to assess the accuracy of the mobile assessment in predicting the provider’s assessment of the need for manual vacuum aspiration or additional misoprostol at follow-up; we also compare these results to results using the same self-assessment done on paper at the women’s follow-up clinic visit.

Methods

This study formed part of a larger randomized, controlled trial investigating the benefits of sending information by text message on mobile phones to strengthen and simplify medical abortion provision. The details of the text messaging programme and the instruments used to measure their effectiveness in providing support to women for managing their abortion symptoms between clinic visits are reported elsewhere.²¹ Informed consent to join the study was in writing in the participant’s language of choice. The study protocol was approved by the World Health Organization Research Ethics Review Committee, and the University of Cape Town’s Faculty of Health Sciences Human Research Ethics Committee.

To design and develop the self-assessment component of the intervention, the principal investigators reviewed the published literature and consulted three key informants with extensive experience in providing first and second trimester abortion procedures. We then interviewed all six nurse providers at the four study sites on the questions they used to determine the success of the procedure. In addition, a convenience sample of 20 consenting women (not included in the 469 study sample) were interviewed by a trained fieldworker at their in-clinic follow-up visit on any challenges they experienced in attending the follow-up clinic visit, their language preference for messages, and mobile phone ownership and usage.

Following piloting and iterative modification to improve the mobile intervention, the study was conducted from October 2011 to May 2012 at two NGO and two public sector abortion facilities. These were all the clinics in the Cape Town metropole where a first trimester medical abortion service had been fully introduced as of May 2011, with the exception of one NGO facility where patient numbers were too low to be considered for the study. Eligibility was confined to women who were over the age of 18, who were eligible for an early medical abortion (gestation up to 63 days by ultrasound assessment and no clinical contraindications), and willing to give informed consent. They also needed to have a working cell phone with them and to consider their phone private. Participant flow is shown in Figure 1. At the clinic visit when women received the mifepristone, 598 women were approached, 469 were enrolled, gave informed consent and were randomised on a 1:1 basis, to a standard-of-care (SOC) group (n = 235) or a standard-of-care plus mobile (SOC+m) group (n = 234). Trained fieldworkers administered a structured questionnaire to all participants recording socio-demographic information, reproductive history, and mobile phone usage. Fieldworkers gave participants in the SOC+m group a short training on how to access and complete the self-assessment on their phone or the fieldworkers’ phone, which took a few minutes.

Figure 1 Participant flow for assessment of medical abortion outcome.

The self-assessment questionnaire was implemented in two ways via mobile phone: through an instant message chat application that some participants already had installed on their phones; or through an unstructured supplementary service data (USSD) system which allowed users to interact with a server using text on their mobile phones. USSD costs 20 cents (South African) per 20 seconds of usage, so for our study purposes we sent women R10 (US$ 1.10) airtime on their scheduled day for self-assessment and an automated text message prompting them to do it. Details on the technical development of the full intervention have been published elsewhere.²²

On day 10 following misoprostol, women needed to dial in to access the questions and could select their language preference (English, Afrikaans or isiXhosa). The self-assessment was an automated, interactive questionnaire routing the user through a maximum of five questions (Figure 2). There were four possible endpoints in the questionnaire that corresponded to ongoing pregnancy, possibly incomplete, probably complete, very probably complete procedure. As the self-assessment design was experimental, when reaching an endpoint the woman was not notified of these specific outcomes, but did receive an appropriate message (Figure 2) and a reminder to attend her follow-up clinic appointment. For all women in the SOC+m group, log files from the USSD system tracked their access to and use of the self-assessment and which endpoint they reached.

Figure 2 Self-assessment routing diagram^a. a Interpretations contained in the Notes were not included in the text to the woman.

All participants (both SOC and SOC+m groups) who attended their in-clinic follow-up visit were interviewed a second time by fieldworkers at the clinic, prior to seeing the provider. This was scheduled at 14 days after the visit for mifepristone for the public sector clinics and at 21 days for the NGOs. Participants first completed a paper version of the mobile self-assessment without assistance from the fieldworker. Thereafter information was collected on participants’ experience with doing the mobile self-assessment for those in the SOC+m group. For all returning participants, details of their clinical assessment were extracted from their clinic file following their consultation with the provider.

Findings

Provider interviews: standard abortion care at study sites

All six providers interviewed were registered nurses, and most were experienced in providing medical abortion (median 2.5 years; range 3 months–5 years). In the public sector clinics, provider assessment involved a clinical history and ultrasound scan. At the NGOs, a high sensitivity urine pregnancy test was routinely done in addition to clinical history and ultrasound scan. According to providers, the scan image followed by the patients’ account of their bleeding and cramping were key in the assessment of ongoing or incomplete procedures. Management at follow-up for ongoing pregnancy, persistent gestational sac and moderate to heavy persistent bleeding was manual vacuum aspiration (MVA). For cases where the scan showed retained products and there was some ongoing bleeding or significant amounts of spotting, women were either managed expectantly or given additional misoprostol and asked to return in a week.

Participant characteristics

A total of 469 consenting participants were enrolled into the study. Mean age at baseline was 26 years (SD 5.5 years). The most common home language spoken was Xhosa (47%, 219/469) followed by English (35%. 165/469), then Afrikaans and other African languages. The majority had completed their high school education (78%, 367/469), had full-time jobs or were full-time students (77%, 364/469) and most lived in formal housing structures (84%, 394/469). Mean gravidity was 1.2 (SD 1.2), gestation age was 4–6 weeks for 55% (257/469) and 7–9 weeks for 45% (212/469) of participants. Sixteen percent (73/469) reported having a prior abortion. There were no differences for these characteristics between the SOC+m (n = 234) and SOC group (n = 235).

Computer logs: use of the mobile self-assessment on day 10 after misoprostol

Eight women in the SOC+m study group did not receive the intervention due to mis-registration, invalid numbers or system failures. Of the 226 receiving the intervention, 204 (90%) tried to do the mobile self-assessment, and 176 of the 204 (86%) completed it, as indicated by the USSD system log files. Forty-seven of the 176 women completed the assessment a few times (27%), sometimes with different results (7%, 12/176); in reporting accuracy, we use the last result of their completed assessments. None of the 176 logs indicated that there had been “no bleeding”. For only 65% (115/176) of completed assessments the endpoints corresponded to complete abortion. The remaining 35% of assessments had endpoints suggesting incomplete abortions or ongoing pregnancies where it would be recommended to consult a provider regarding additional care.

In-clinic follow-up: provider assessment

In-clinic follow-up included 81% (381/469) of all enrolled participants. There were no significant differences in provider assessment of abortion outcome or additional treatment between the two study groups for participants returning for follow-up (Table 1). Of all participants returning, 2% (7/381) did not wait to see the provider after completing the interview with the study fieldworker. Of the 12 women who had vacuum aspiration, two were assessed by a provider as probably having ongoing pregnancies corresponding to a failure rate of 0.5% (2/374). Other indications for vacuum aspiration included a persistent gestational sac or persistent bleeding (10/374). Of all returnees, 25% (95/374) received additional misoprostol at follow-up with instructions to return to the clinic again the following week for evaluation. \

Table 1 Provider assessment of participants returning to the four abortion clinics for follow-up (no./%)^ab.

Mobile self-assessment results

Results available for pairwise comparisons are shown in Figure 1. For participants completing mobile assessments and returning for in-clinic follow-up, 34%, (51/149) self-assessed on mobile to possibly have incomplete abortions or ongoing pregnancy and 66% (98/149) to have complete procedures. Accuracy of the mobile assessment to predict provider assessment of ongoing pregnancy or incomplete abortion requiring vacuum aspiration or extra misoprostol had sensitivity (true positive rate) of 46% (22/48). Of the 6 (SOC+m group) who had only vacuum aspiration at follow-up, 3 had completed the mobile assessment. Of these, only one identified her abortion on mobile as incomplete while the other two women had assessed their procedures to be complete. Specificity (true negative rate) identifying complete abortion with no need for any additional treatment was better at 71% (72/101) (Table 2).

Table 2 Performance characteristics of mobile self-assessment in detecting incomplete abortion according to clinical assessment and treatment of incomplete abortion.^a

For the 15 women interviewed at in-clinic follow-up but who did not try the mobile self-assessment, reasons given were: difficulty with the first set of instructions, followed by not feeling like it and not having phone access at the time. Of those who tried, 93% (161/173; two had missing data) said it was easy or very easy to do. For those who said it was hard, problems were to do with mobile network or system failures. However, of the 24 who tried the assessment but didn’t reach an endpoint (data from computer log), 50% (12/24) said that that they thought they had finished all the questions.

Paper self-assessment results

Agreement between the paper self-assessment done at clinic follow-up (either 14 or 21 days) and the mobile self-assessment done at 10 days after taking misoprostol was 69.5% (Kappa = 0.198) – indicative of only slight agreement. Agreement between the two assessments for a complete procedure was 71% and for an incomplete or ongoing procedure was lower at 60%.

Accuracy of the paper assessment to predict provider assessment for need for vacuum aspiration or extra misoprostol is shown in Table 3 . Accuracy was good for predicting complete abortion, but was again inadequate for predicting ongoing pregnancies or incomplete procedures. Compared to the mobile assessment, the accuracy of the paper self-assessment was better in predicting complete procedures – fewer women (n = 20) assessed their procedure as incomplete at follow-up on paper compared to their earlier assessment by mobile (n = 47).

Table 3 Performance characteristics of paper self-assessment in detecting incomplete abortion according to clinical assessment and treatment of incomplete abortion.^a

We examined in detail the self-assessments, both mobile and paper, of the two cases that providers assessed to have had ongoing pregnancies. One (SOC+m group) was 24 years old, gestational age 4–6 weeks, had tertiary education and was from a francophone African country. She completed her mobile and paper self-assessment identically: She had bleeding less than a normal period, but thought she had passed clots or tissue, suggesting a complete abortion. The other (SOC group) was 23 years of age, gestational age 7–9 weeks, had completed high school, her home language was isiXhosa. Her paper assessment also showed bleeding less than a normal period, but she didn’t still feel pregnant.

Discussion

This study aimed to explore whether women undergoing medical abortion could complete a self-assessment of their procedure on their mobile phones or on paper and whether these assessments would accurately predict the outcome of their medical abortion, thereby providing an alternative to in-clinic follow-up. Our results showed that most, but not all, of our study population could complete an automated interactive questionnaire on their phones, following a short training to familiarize them with the process. While USSD was preferred by participants over instant message chat, and therefore doing the assessment used some airtime, it is likely that technological progress will include other no-cost solutions in future. Future improvements on the design used here should include a feedback mechanism to the user that they have completed the questionnaire.

We think that the unexpectedly low agreement between the mobile and paper self-assessment in this study may be due to timing. The questions and routing for both assessments were identical; however, women completed the mobile assessment 10 days after taking their misoprostol and approximately 2/3 of the SOC+m group had their follow-up in-clinic visit 10 days after this (as per NGO clinic protocol). Since the assessments included a question on persistent bleeding, it could be that the interval between the two assessments accounted for the differences. A recommendation emerging from this is that self-assessments using standardized questions be done later than day 10 to avoid false positive results suggesting incomplete procedures. Other factors contributing to low agreement between the mobile and paper questionnaires may include recall or errors in following the instructions on their mobile phones.

Our results also showed that this self-assessment, either paper or mobile, was inadequate in predicting ongoing pregnancies, or the need for vacuum aspiration or extra misoprostol. This is similar to another study that reported modest success for prediction of ongoing pregnancy or retained gestational sac using a standard set of questions.¹² Recent research suggests that a set of questions or a symptom checklist combined with an appropriate pregnancy test may be the best strategy for out-of-clinic self-assessment of abortion outcome.^7,16,19,20 For out-of-clinic self-assessment the importance of timely identification of ongoing pregnancies is obvious. In the South African public sector, second trimester abortion services are only available in a small number of secondary and tertiary hospitals and the services are overbooked and difficult for women to access.

The high proportion of additional care rendered (25% of all participants received additional misoprostol) in our study may well be indicative of “defensive” management by the providers. This figure is higher than usually reported elsewhere where additional treatment for ongoing pregnancy, persistent gestational sac or unacceptable, persistent bleeding is usually 5—10%.^10,23 For the public sector sites in the study, one provider was new to the service and may have needed more experience with assessing outcome, but this was not so for the other providers. In the interviews conducted in the formative part of this study, providers clearly described their management protocol with respect to assessment and decision for additional MVA at follow-up. However, indications for additional misoprostol versus expectant treatment were less clear, and it is possible that providers were inclined towards caution by issuing additional misoprostol more frequently than was required. Support and clarification for providers, possibly through discussion forums, may be helpful in understanding this issue more clearly.

We designed the mobile assessment to resemble a consultation as closely as possible, and used an interactive design with routing according to the user’s answer to a question. This introduced some difficulties for the decision tree, such as where to insert a question around pregnancy symptoms. Unpublished studies have shown that in many cases women in South Africa are not aware of pregnancy symptoms such as nausea or breast tenderness or do not make the association between pregnancy symptoms and being pregnant at early gestation (up to 63 days). In addition, our interpretation of the literature and our results suggest to us that a standardized self-assessment is a very different modality from a clinical consultation in which a trained provider exercises nuances of judgement at every point. A self-assessment needs to be simple, clear and conservative to avoid missing cases needing treatment, but also should not cause unnecessary distress to users around the need for additional care.

In this study, SOC+m participants were not asked whether they preferred to do the assessment on their mobiles, on paper or with a provider; offering a choice of these options could be a good solution. Providing the questions on mobile may have a number of advantages over a paper version. People are less likely to lose their phones than a piece of paper and phone privacy is not an issue since users can initiate the assessment themselves when they feel it is private enough to do so. However, if a paper assessment is incorporated into a pregnancy kit, then its loss is much less likely, and women with unclear results could text an assessment to the provider at the clinic to get clarification, or they could call.

A mobile assessment can be combined with support and informational text messages sent from a secure server – a package that can be offered to a woman at her first clinic visit. Messages on contraceptive method choice can be incorporated to form a comprehensive mobile package strengthening abortion care. If telemedicine for abortion is further incorporated into service provision, mobile-based self-assessment could be integrated into this approach. Outcomes can be tracked online by clinicians working at distance centres, and followed up with a telephone consultation, when needed.

Conclusion

Women undergoing medical abortion in South Africa are for the most part able to complete a self-assessment on their mobile phone, given a short training session. Self-assessment questionnaires should be done later than day 10 post-misoprostol, and may need to be combined with an appropriate pregnancy test to accurately predict ongoing pregnancies or incomplete abortions. Alternative follow-up strategies using mobile phones are in line with other developments that use technology to simplify and increase access to medical abortion and hold promise for the future.

Acknowledgements

We thank all the women and providers who participated in the study, and the fieldworkers who recruited participants, interviewed and recorded the data. We also acknowledge Sarah Crede and Joanna Romell for their contributions to preliminary analyses. We acknowledge the technical and financial support from HRP (UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction), the University of Cape Town Research Development Fund and the University of Cape Town Harry Crossley Senior Clinical Fellowship. This study was registered retrospectively on the Pan African Clinical Trials Registry (PACTR201302000427144).

Notes

* Demedicalization: A process whereby a life process under medical jurisdiction is reconsidered as no longer a medical problem and no longer requiring medical personnel. For medical abortion up to 63 days gestation, this includes taking misoprostol at home, and self-assessment of completion of the abortion with the option of clinic support if needed.

a Seven missing – did not have provider assessment.

b Percentages do not add up to 27% due to rounding.

a Both clinical assessments were on the same set of participants.

a Both clinical assessments were on the same set of participants.