Automated surveys are designed to extract data from people and ignore them. Burdened by this, people share poor and messy data in return. This can be done better. Our survey tool enables organisations to hold attentive remote surveys that treat people with dignity, code messages in situ and analyse the results live - all in the same space. This has been used in a project with the Mastercard Foundation and Ghana Enterprises Agency.
What gets lost with automated SMS surveys?
The method most commonly used to handle large scale remote research are surveys. In low income contexts, SMS is often the best channel to reach people. Current research tools allow you to conduct automated SMS surveys which ask an individual a series of questions, automatically sharing the next question upon receiving any reply. By design, this avoids being attentive to the response to move onto the next question.
This can have a few consequences:
- If a person shares a concern or a question in their response, it will go unaddressed and they will be pushed to answer the next question. To be ignored in such a fashion can be jarring, especially for sensitive topics (e.g. ‘please please help, I think I am pregnant’ gets a response of ‘how often do you use contraception?’)
- If a person shares a partial answer or a very interesting one, you miss the opportunity to explore this with them
- It’s not nice being spoken to by a robot! People are not encouraged to share the deeper opinions that lead to quality insights, or to stay engaged. We tested this through an A/B test where a more ‘active listening voice’ led to a statistically significant increase in response rates and substantive comments
This poor quality data then has to be cleaned, coded without any context and analysed in a separate space. Moving data around like this also brings data risks.
Turnline: Katikati’s tool for attentive SMS surveys with real-time insights
Within Katikati, from our research experience at the University of Cambridge, we have added capabilities to solve for these problems. We call this feature Turnline which includes quick and thoughtful human intervention at each step of the survey to enable deeper social research through richer conversations and flexible interprative tagging.
Turnline (on the right hand side) has a list of questions are laid out as a series of steps (intro, Q1, Q2, Q3, done). With an easy click one can send a pre-formatted message and upon receiving an answer, you add a relevant suggested tag 'answer' and ‘Business - access to funds’. This tagging confirms the person is ready to receive the next loaded question, closes ‘Q2 - challenges’ and opens ‘Q3 - sector view’.
Where researchers are handling large volumes of conversations (e.g. 1,000+), we also have an integrated tool called Coda. In Coda, researchers can quickly sweep incoming messages for ‘answer’ or ‘not answer’ and later on give it richer interpretation based on a dropdown of tags from the same coding frame they created for turnline.
What are Katikat’s benefits?
In addition to ensuring researchers can listen to surveyed participants and treat them with greater dignity, this approach to tagging brings a number of benefits:
Better, richer and faster interpretation: The tagging of a message as ‘Business - access to funds’ serves both to confirm it is appropriate to move on as well as to give it an interpretive tag. This interpretation is done in view of the context of the conversation and skips the need to clean and code the survey responses at a later stage.
(Maybe) Higher response rates: From past client projects using Katikati, the response to questions varies from 25-60%, higher than normal SMS surveys. Our A/B test suggests if you send more personal tailored messages, people are more likely to respond.
Creating cohorts dynamically: Tagging in Katikati also enables you to filter for messages with a given tag. This can be used to re-engage a smaller group cohort on something they shared to dig deeper into their experience.
Live trends: Tags update real-time charts in Katikati which display the responses to questions and demographics of the surveyed group. This allows you to analyse how things are developing and adjust your approach. You can also export all of this data into excel, csv, json for further analysis later.
Katikati in Action
Conclusion
Automated SMS surveys are popular for their ease of use but they ignore the counterparts in favour of collecting data at a lower quality. Katikati’s survey tool allows you to hold attentive remote SMS surveys with a large number of people that treat them with dignity, code messages in situ and analyse the results live - all in the same space. This results in higher quality interactions and ultimately richer, more insightful research.
If taking such an approach is interesting to you, reach out to us for a demo of Katikati at arnav@katikati.world.
