Vaccine coverage is low in Nigeria. In 2017, only 1 in 3 children received three doses of the pentavalent vaccine.[1] Lack of awareness (42%) has been implicated as the main reason for non-and under-vaccination in Nigeria. To overcome the gap in information and knowledge about the vaccination program, a short messaging service (SMS) vaccination reminder project was designed and implemented in 14 Local Government Areas (LGAs) in Kebbi State, Northwest Nigeria. The Immunization Reminder and Information SMS System (IRISS) used SMS to inform and educate the public about the importance of immunization; and remind caregivers/parents of their child’s immunization schedules, including health facility vaccination schedules in their locality.

Between May 20, 2020, and May 31, 2020, IRISS sent generalized messages to over 190,000 subscribers on the benefits of immunization and advertisements for people to opt-in to IRISS e-registry for more immunization information. IRISS also sent targeted broadcasts on benefits of immunization to 1,010 Community gatekeepers, 3,924 Leads, 406 Health workers, and 22 State Routine Immunization Coordinating Committee members; as well as individualized reminders to 21,902 caregivers who voluntarily registered into the IRISS e-registry for their child’s immunization due date and health facility immunization day. Upon receiving the messages, caregivers were expected to take their children to the specified health facility for immunization while the gatekeepers were expected to disseminate the messages in their community.

To evaluate the receipt of IRISS messages, attitudes, and actions taken by gatekeepers and caregivers after receiving the messages, as well as the awareness and perception of caregivers to the intervention, an evaluation was conducted between May 15 and 23, 2020. We had planned to conduct the evaluation in person, but due to the movement restrictions in Nigeria to mitigate the spread of the COVID-19 pandemic, the evaluation was conducted via phone calls.

At the time of this evaluation, 16,110 caregivers and 1,010 gatekeepers were registered into the IRISS application from the 14 intervention LGAs. We interviewed 131 gatekeepers and 171 caregivers (a total of 308) in the 14 intervention LGAs during the evaluation and they were randomly selected using a stratified random sampling method.

Kebbi is a rural state with low cell phone coverage. The low phone ownership presented some challenges during the evaluation. To enable us to reach our sample size of 308, we called 719 (268 gatekeepers and 451 caregivers) phone numbers due to technological, behavioral, and contextual challenges encountered.

Phone switched off: This was a major challenge faced during the evaluation specifically in rural and poor network areas. Out of the 719 phone numbers called, 28%(200) were switched off.

In instances where a phone number is switched off, we highlighted the number and move to call the next on our list. In most cases, the phone numbers remained switched off despite several attempts and we had to replace them.

Not responsive: 3%(23) respondents did not pick the phone calls while 2%(13) hung up on the interviewer after listening to the introduction and purpose of the call. For respondents who hung up, all efforts to reach them again were unsuccessful.

Proxy registrations: We also had some contextual challenges where RI providers used their phone numbers for proxy registrations instead of the phone number of the subscribers’ relative, friend, or neighbor. At the time of this evaluation, 7%(1,150) of caregivers were registered into IRISS through proxy. Proxy registrations are conducted when a subscriber does not have a phone but has access to a relative, friend, or neighbor’s phone.

Poor network: Another challenge faced was poor network. 16%(115) phone numbers were either not connecting, not available, or not reachable due to poor network in that area.

Inconvenient interview timing: Respondents were sometimes busy at their farms or preparing for prayers and thus, were not willing to respond at the time we called. However, they were polite to give a time which was more favorable for them to respond to the interviews.

Child not eligible and proxy registrations not applicable: 5%(22) respondents do not have young children of vaccine age and could not recall using their phone numbers for proxy registrations. In such a situation, we moved to the next phone number on our list.

Wrong number: 2%(10) respondents informed that they do not recide in Kebbi State and do not know how their phone numbers got registered into the IRISS application.

Privacy and security concerns: Respondents were reluctant to provide information especially to persons they do not know, due to the incidences of kidnap and fraud in the state.

Despite the challenges encountered, the 308 respondents who were successfully interviewed appreciated the intervention and informed that they received messages from IRISS and the messages were clear, precise, timely and needed.

Recommendations

Implementing the IRISS intervention in Kebbi state provided us a first-hand experience and insight into evaluating the feasibility of implementing mobile health interventions in developing countries. We believe our experience will be beneficial for other developing countries that plan to evaluate similar interventions to improve health outcomes. We have therefore outlined our major recommendations for evaluating SMS-based projects using phone interviews. Our recommendations include:

1. Provide prior communication to community members: To mitigate against non-compliance and privacy concerns, respondents should be informed about the interviews beforehand and the time for the interview should be well communicated through the community leaders. The community leaders can instruct the town announcers to inform community members to expect phone calls from a specific intervention such as IRISS at a certain duration of days and times. They should also encourage them to comply.

2. Plan to conduct interviews during the non-festive period: To have respondents’ complete participation, interviews should not be conducted during the festive or fasting period as participants may be busy and easily exhausted during fasting.

3. Develop a short and precise questionnaire: The survey questionnaire should be as short and precise as much as possible while ensuring that all pertinent information will be collected during interviews. We recommend 5 to 10 questions to avoid respondents getting fatigued and not providing quality responses.

4. Select adequate interview time: The best time for phone interviews is in the evenings when respondents are done at their farms, markets, and offices and can respond to questions without distractions.

5. Consider respondents’ gender: Gender is very important especially in the northern part of Nigeria, where women do not have the right to make decisions. They seek their husband’s permission to speak to a man and/or perform certain tasks. Therefore, for quality responses, female interviewers should speak to female respondents and vice versa. The case may not be the same for men but it is advisable to put this into consideration.

Data collection through phone call interviews thought me to be patient with respondents. The process is cost-effective because the interviewers will not be transported to the community for data collection. However, there are some challenges such as poor network coverage, privacy concerns, and phones switched off amongst others. It is therefore pertinent that interviewers be trained to handle phone interviews, possible challenges associated with phone interviews, and what to do when faced with such challenges.

Reference

[1] World Health Organization-United Nations Children’s Fund, 2016. National Immunization Coverage Survey (NICS). Abuja, Nigeria: World Health Organization and United Nations Children’s Fund.