Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore

by Prof. Minah Park
Department of Health Convergence
PURE Research Profile
minah.park@ewha.ac.kr

Chatbots have become an increasingly popular tool in the field of health services and communications. Despite chatbots' significance amid the COVID-19 pandemic, few studies have performed a rigorous evaluation of the effectiveness of chatbots in improving vaccine confidence and acceptance.

In this multi-site randomized controlled trial (RCT), we assessed the effectiveness of COVID-19 chatbots on people who were unvaccinated or had delayed vaccinations until the government vaccine mandates in Thailand, Hong Kong, and Singapore.

From February 11th, 2022 to June 30th, 2022, 2045 participants were enrolled and randomly assigned to the control and intervention groups. After excluding 1,280 participants who were lost to follow-up, responses from 748 participants were included in the final analysis (Fig 1).

Fig. 1: Flow diagram of the randomised controlled trial in Thailand, Hong Kong, and Singapore. Flowchart presenting the number of participants assessed, enrolled, randomized, lost to follow-up, and analysed in each child group and senior group.

After a week of using COVID-19 vaccine chatbots, the differences in vaccine confidence and acceptance were compared between the intervention and control groups. Compared to non-users, fewer chatbot users reported decreased confidence in vaccine effectiveness in the Thailand child group [Intervention: 4.3 % vs. Control: 17%, P = 0.023]. However, more chatbot users reported decreased vaccine acceptance [26% vs. 12%, P = 0.028] in the Hong Kong child group and decreased vaccine confidence in safety [29% vs. 10%, P = 0.041] in the Singapore child group. There was no statistically significant change in vaccine confidence or acceptance in the Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability.

Chatbots were found to be significantly more effective at improving vaccine confidence and acceptance among people who are minorities (i.e., non-Thai in Thailand and non-Chinese in Hong Kong and Singapore) and those who had lower education levels (i.e., below college level). Specifically, in the child groups, being a minority was associated with higher odds of an improved belief in vaccine effectiveness [OR: Thailand = 12.91 (3.56–46.78); Hong Kong = 2.73 (1.11–6.7)], effectiveness regardless of manufacturers [OR: Thailand = 2.99 (1.28–6.97)], and safety [OR: Hong Kong = 5.68 (2.01–16.1); Singapore = 10.46 (1.14–95.79)] (Fig 2 and 4).

Fig. 2: Associations between vaccine confidence and acceptance and sociodemographic factors, misinformation, risk perception, and chatbot use in Thailand child group. A proportional odds logistic regression model adjusted for respondent’s sex, age, and employment status. Reference groups for a: respondent’s ethnicity: Thai (number of participants:125, percentage of the total participants: 93%); respondent’s education level: below college level (66, 49%); respondent is a healthcare worker?: yes (17, 13%); financial situation: low (29, 22%); geographical location: urban (51, 38%); child’s gender: female (64, 48%); chatbot use: no (65, 49%). *p-value < 0.05; **p-value < 0.01; ***p-value < 0.001. b Refers to the association between pre-intervention misinformation awareness and risk perception with primary outcomes, for example, higher misinformation awareness (“COVID-19 vaccines cause death”) in the pre-intervention questionnaire is positively associated with increase in vaccine confidence (“Vaccines are effective against all variants”) (OR = 1.41 (95% CI: 1.07–1.84)).

Respondents who were better at identifying COVID-19 vaccine misinformation in the baseline survey were more likely to have improved vaccine confidence and acceptance. In the Hong Kong child group, parents who correctly identified the misinformation: “COVID-19 vaccines cause infection” in the pre-intervention questionnaire were more likely to have improved beliefs in vaccine importance [OR = 2.24 (1.39–3.62)], safety [OR = 6.09 (3.51–10.55)], effectiveness [OR = 1.67 (1.09–2.56)], and acceptance [OR = 1.77 (1.32–2.36)] (Fig 4).

Fig. 4: Associations between vaccine confidence and acceptance and sociodemographic factors, misinformation, risk perception, and chatbot use in Hong Kong child group. A proportional odds logistic regression model adjusted for respondent’s sex, age, and employment status. Reference groups for a: respondent’s ethnicity: Chinese (number of participants: 87, percentage of the total participants: 44%; respondent’s education level: below college level (99, 50%); respondent is a healthcare worker?: yes (10, 5.0%); family income: under 30 K HKD (127, 64%); child’s gender: female (86, 43%); chatbot use: no (109, 54.8%)). *p-value < 0.05; **p-value < 0.01; ***p-value < 0.001. b Refers to the association between pre-intervention misinformation awareness and risk perception with primary outcomes, for example, higher misinformation awareness (“COVID-19 vaccines cause infection”) in the pre-intervention questionnaire is positively associated with increase in vaccine confidence (“Vaccines are important”) (OR = 2.24 (95% CI: 1.39–3.62)).

This multi-site, parallel RCT study on vaccine chatbots found mixed success in improving vaccine confidence and acceptance among unvaccinated Asian subpopulations. Further studies that link chatbot usage and real-world vaccine uptake are needed to augment evidence for employing vaccine chatbots to advance vaccine confidence and acceptance.

* Related Article
Kristi Yoonsup Lee, Saudamini Vishwanath Dabak, Vivian Hanxiao Kong, Minah Park, Shirley L. L. Kwok, Madison Silzle, Chayapat Rachatan, Alex Cook, Aly Passanante, Ed Pertwee, Zhengdong Wu, Javier A. Elkin, Heidi J. Larson, Eric H. Y. Lau, Kathy Leung, Joseph T. Wu, Leesa Lin, Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore, NPJ Digital Medicine, 6(1):96., 5744-5802, May 2023

London School of Hygiene and Tropical Medicine News (Link)