Effective communication can be as important as medical treatment itself whenever there is a public health crisis. This is especially true during disease outbreaks, several of which Uganda has experienced in the last decade, including the ongoing COVID-19 crisis. Fear is a common denominator during these outbreaks, which sometimes feeds into mis and disinformation. With heightened demand for reliable information, this often adds a burden on health practitioners to convey health-related information in a way that is clear and understandable – requiring time commitment and dedicated human resources. To address this challenge, our team has been developing an AI-supported chatbot with 24/7 automation and real-time response.
Its automation increases the ability to handle a large volume of inquiries simultaneously – a significant benefit compared to one-on-one conversations. In this blog post, we delve into the techniques, mechanics and challenges of deploying AI and Natural Language Processing (NLP) techniques to create such a chatbot. This was done first in English and then in Luganda, an under-resourced language. To the best of our knowledge, this is the best attempt at such an endeavor in Uganda. It is envisioned that with time, it will open up a new frontier in public health communications, as well as in other contexts where AI-powered conversations can be a relief. The journey to creating this chatbot involved multiple stages, each with unique experiences and challenges.
Data collection
We collaborated with the Academy for Health Innovation Uganda, a sister department at the Infectious Diseases Institute(IDI) to collect the data needed for this project. This was done by leveraging their Call for Life (CFL) system which was at the time receiving a high volume of calls from patients inquiring about COVID-19. The Academy doctors typically collected the questions and then provided corresponding responses based on information gathered from research. Sometimes this required gathering details from various sources such as Uganda's Ministry of Health (MOH), the World Health Organization(WHO) and Center for Disease Control (CDC) websites to supplement the CFL data.
Whilst the information was reliable, some of the content was not contextualized to Uganda which made it less than useful. The mere differences in the content on the CDC and MOH websites made it clear that local content was needed to address local problems, as well as develop sustainable solutions that could be adopted smoothly within a local context.
Database creation
The technical team at the African Centers of Excellence and Data Intensive Sciences(ACE) got involved at this stage. After receiving several pairs of question and answer from The Academy doctors, we created a Google sheet to allow the data team to collaboratively work and add information in order to streamline the tasks needed to develop the chatbot. More specifically, this helped with outlining intents and utterance examples: an utterance is what a user says or asks a chatbot whereas utterance examples are the different ways the same question can be asked. On the other hand, the intents help the artificial intelligence chatbot get the gist of the question being asked. Together, these pieces of information provide the chatbot with some basic information from which it can learn and answer questions correctly. Between intents and utterance examples, formulating the intent in a question was easier, while suggesting several ways in which the same question could be asked was more complex. For any chatbot to be fairly accurate, the minimum number of question variations for each question is 10 – the minimum entry requirement for examples. For the team, crowdsourcing the variations was a fun brainstorming activity, albeit challenging.
A visual representation of the flow of intents and how they are interlinked in Botpress, a chatbot development framework
Translations
Communication is effective when a receiver comprehends the information or idea a sender intends to convey. At the onset of the COVID-19 pandemic, members within the community needed to understand key messages concerning many aspects of life including health. This was the motivation for the chatbot. While functional, we found that there were challenges in understanding the messages in English, thus the need to translate messages, concepts and terms to Luganda. This chatbot is the first of its kind! Some words were not previously in Luganda such as “Kkovidi” meaning COVID and “Kkolona” for Corona. We therefore sought the help of linguists and language experts to identify alternate words that were close in meaning. How would you translate “chatbot” in your native tongue? Don’t think much about it, get to the bot and enjoy the conversation. It will really surprise you.
Pipeline Design
One of the most crucial parts of training a bot is designing the brain that will be used to evaluate how to behave like a human. This means that when someone asks a question, it's this model that considers the context, entities of the question and evaluates how to appropriately respond. It is trained on previous human interaction therefore it makes a calculated decision on how a human would respond. In Artificial intelligence terminology, we call the procedure which a machine uses or the set of instructions a machine uses a pipeline. One way to think of it is as a pipe through which a machine looks to get perspective on how to respond like a human.
The design of the bot pipeline consists of knitting together different pieces of code that help the bot pick up human behavior. An interesting part of this is the chitchat (a module in the bot that handles response when someone types something such as “Lol” or uses an emoji; it makes the bot relatable). This handles the in-conversation expressions that may not be contextual sometimes. For example when a person types: Hmmm! How should the bot respond? “I do not understand” as a response would not suffice.
Overfitting
We also needed to find a way to evaluate whether our model is doing well. Think of this as your result sheet after an intense final paper. The bot is also evaluated on certain metrics; it depends on the objective one is trying to achieve. If for example, you are trying to make the model pick up more on context as opposed to the accuracy of spellings, then certain knobs can be moved, like when you decide that the bot should learn a particular part of language more than others then you have to turn some knobs to achieve that.
When the model overfits, essentially this means it has learnt the training data so well that it's unable to answer new questions. Just like we used to use cram-work in high school, an overfitted model is a model that has crammed and fails on the subsequent evaluations. In order to solve this, we updated our learning methods to build a futuristic scenario in which the bot is shown some data and the rest of it is used to test it. That way we can evaluate beforehand how it would respond to new information.
Botpress Team
The chatbot users were primarily the general public and the intention was to use this bot to share information about COVID-19. It has been quite an interesting journey with lots of twists! Right into the development, our first major task was to pick a developer framework or platform as there are a lot on the market, each with its own pros and cons. The team found it very important to pick the best framework in terms of security, customization ability, price/cost, a journey that required immense levels of development and testing of bots and assessing their performance.
Through early reviews and desk reviews, we found that Botpress was one of the best frameworks for building conversational assistants. Setting up a conversational chatbot using the Botpress framework in parallel with RASA enabled us to get the fundamentals of making chatbots easier and more effective. Botpress simplifies the process of creating chatbots with its pre-made components such as a graphical user interfaces for managing all the aspects of chatbot administration i.e. a visual conversation studio that helps in creating conversational flows; content management; natural language processing and understanding; analytics and monitoring; and a dialog engine that handles interactions with users. Additionally chatbots developed using Botpress framework can be easily accessed using various channels that are convenient for users, such as the web, Facebook messenger and others. Configuring and setting up a chatbot using Botpress is relatively easy, but may require prior experience when setting it up on bare servers.
The Botpress framework is quite good based on our experience for quickly setting up the chatbot for simpler use cases. However, with the unique and multiple use cases that the COVID chatbot we are developing requires, it was challenging for the framework to meet all the requirements that we needed. We therefore opted to go for a more customisable framework (RASA), which seemed to offer more customisation options and support for the unique requirements we needed. The RASA framework performed reasonably well when put to test with real users.
The chatbot will be an automated, AI-driven virtual assistant that will expand the potential for enhanced automated health telecommunications within communities. It will also provide individuals with health care assistance through both audio and text, and through enhanced modelling it will be able to produce user-specific responses to a wide variety of questions.
The COAST project is being implemented with grant support from Canada’s International Development Research Centre (IDRC) and the Swedish International Development Cooperation Agency (Sida) as part of the Global South AI4COVID Program. To learn more about our work follow us on Twitter and visit our website.
