Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about treatment effects: one-year follow up of a randomised trial

Introduction Earlier, we designed and evaluated an educational mass media intervention for improving people’s ability to think more critically and to assess the trustworthiness of claims (assertions) about the benefits and harms (effects) of treatments. The overall aims of this follow-up study were to evaluate the impact of our intervention 1 year after it was administered, and to assess retention of learning and behaviour regarding claims about treatments. Methods We randomly allocated consenting parents to listen to either the Informed Health Choices podcast (intervention) or typical public service announcements about health issues (control) over 7–10 weeks. Each intervention episode explained how the trustworthiness of treatment claims can be assessed by using relevant key concepts of evidence-informed decision-making. Participants listened to two episodes per week, delivered by research assistants. We evaluated outcomes immediately, and a year after the intervention. Primary outcomes were mean score and the proportion with a score indicating a basic ability to apply the key concepts (> 11 out of 18 correct answers) on a tool measuring people’s ability to critically appraise the trustworthiness of treatment claims. Skills decay/retention was estimated by calculating the relative difference between the follow-up and initial results in the intervention group, adjusting for chance. Statistical analyses were performed using R (R Core Team, Vienna, Austria; version 3.4.3). Results After 1 year, the mean score for parents in the intervention group was 58.9% correct answers, compared to 52.6% in the control (adjusted mean difference of 6.7% (95% CI 3.3% to 10.1%)). In the intervention group, 47.2% of 267 parents had a score indicating a basic ability to assess treatment claims compared to 39.5% of 256 parents in the control (adjusted difference of 9.8% more parents (95% CI 0.9% to 18.9%). These represent relative reductions of 29% in the mean scores and 33% in the proportion of parents with a score indicating a basic ability to assess the trustworthiness of claims about treatment effects. Conclusions Although listening to the Informed Health Choices podcast initially led to a large improvement in the ability of parents to assess claims about the effects of treatments, our findings show that these skills decreased substantially over 1 year. More active practice could address the substantial skills decay observed over 1 year. Trial registration Pan African Clinical Trial Registry (www.pactr.org), PACTR201606001676150. Registered on 12 June 2016.


Part 2. Questions about claims
Instructions: Read the text above each question and then answer the question using one of the provided answers. For each question, choose what you think is the best answer and write the letter for that answer in the box provided.

Example
A teacher says that the children in his school run faster than the children going to school in another village.

Question: How can the teacher be sure about this?
Options: A doctor did a research study to find out if drinking tea keeps people from getting sick. He tossed a coin to decide who should get the tea and who should not. People who got tea went to the doctor's office every day to drink their tea. At the end of the study, people who got the tea were less likely to be sick than those who got no tea.
Based on the text above, please answer the following questions: 2.1 Who went to the doctor's office every day?

Options:
A) People who did not get tea B) People who got tea

D) People who got sick
Answer: 2.2 How did the doctor decide who should get tea?

Options:
A) By tossing a coin

B)
By asking people what they would like C) He gave tea to those who were more likely to be sick D) He asked people who came to his office Answer:

3.
A doctor did a research study to find out if drinking tea keeps people from getting sick. He tossed a coin to decide who should get the tea and who should not. People who got tea went to the doctor's office every day to drink their tea. At the end of the study, people who got the tea were less likely to be sick than those who got no tea.
Based on the text above, please answer the following questions: 3.1 What was the treatment? Options:

4.
Annette sees an advert on TV for a new soap which the makers say protects people from getting skin rashes. Annette thinks that this soap must be better than other soaps for protecting her skin.
Question: Is Annette right?

Options:
A) No, the soap may be newer, but that does not mean that it is better than other soaps B) Yes, the new soap is probably better than most other soaps because it is newer C) Yes, the new soap is probably better than most other soaps because a well-known company makes it Answer:  5. Regina has an illness that makes it difficult for her to breathe. She hears on the radio about a medicine that has helped many people for their breathing problems.
Question: How sure can Regina be that the medicine does not have any harms?

Options:
A) It is not possible to say. However, medicines are rarely harmful B) Not very sure, because all medicines may harm people as well as help them C) Very sure, since the medicine has helped many people, it is unlikely that it also harms people Answer:  6. John has a skin rash on his leg. A shop sells several creams to treat skin rashes. John chooses a cream from a well-known company, even though it is more expensive than the other creams. John thinks the cream is more likely to heal his rash than the other creams because it is more expensive.
Question: Is John right?

Options:
A) No, just because the cream is expensive does not mean that it will work better than other creams B) It is not possible to say. However, expensive creams are likely to be better because the companies spend more time making them C) No, the cream is probably not as good as the other creams. Well-known companies are usually better at advertising D) Yes, the company is well-known for a reason, so it is more likely to be better than creams sold by lesser-known companies Answer:  7. Two companies make two different medicines for treating stomach pain. Each of them says that their medicine is the better one.
Question: How can you know which of the two medicines is better for stomach pain?

Options:
A) It is not possible to say. The companies may just say their medicine is best because they want to make money B) I would rely on the best known company; it is more likely to have the best medicine C) I cannot trust either of the companies. They are probably both wrong 10. Sarah has an illness. There is a medicine for it, but she is unsure if she should try it. A research study comparing the medicine with no medicine found that the medicine was helpful but also that it could be harmful. Three of Sarah's friends are giving her advice about what to do.
Question: Which advice below given to her by her friends is the best advice? Options: A) She should only take the medicine if many people have tried the medicine before B) She should only take the medicine if she thinks it will help her more than it will harm her C) If Sarah has enough money to buy the medicine, it could not hurt to try it Answer:  11. Dr. Acheng is an expert on treating headaches. A news reporter interviews Dr. Acheng about a new medicine. Dr. Acheng says that, in her personal experience, the new medicine is good for treating headaches.
Question: How sure can we be that Dr. Acheng is right?

Options:
A) It is not possible to say. It depends on how long Dr. Acheng has been an expert on treating headaches B) Not very sure. Even though Dr. Acheng is an expert, the new medicine still needs to be compared in studies with other treatments C) Very sure. Dr. Acheng is an expert, so she knows if the new medicine is good or not based on her experience D) Very sure. Dr. Acheng would not be interviewed by a news reporter if her advice was not good Answer:  12. Edith has a stomach pain. Edith's mother says that fruit juice is a good treatment for stomach pain. She learnt about this treatment from Edith's grandmother. Over many years, other families she knows have also used fruit juice to treat stomach pain.
Question: Based on this, how sure can we be that fruit juice is a good treatment for stomach pain?

Options:
A) Not very sure. Even though people have used fruit juice over many years, that does not mean that it helps stomach pain B) Very sure. If it has worked for Edith's mother and other people who have tried it, it will probably work for her too C) Not very sure. Edith should ask more families if they use fruit juice to treat stomach pain Answer:  13. At David's school, some students have poor parents. The students with poor parents drink less fruit juice than the children of other parents. The students with poor parents are also more often sick. Based on this link, David thinks that people who drink fruit juice, are less likely to get sick.
Question: Is David correct?

Options:
A) It is not possible to say, it depends on whether or not David has poor parents B) Yes, students with poor parents do not drink fruit juice and are more often sick C) Yes, the juice is the only possible reason why the students with the poor parents are more often sick D) It is not possible to say. There could be other reasons why students with poor parents are more often sick Question: Based on this link between using cream and smooth skin, is Judith correct?

Options:
A) It is not possible to say. It depends on how many younger and older girls there are B) It is not possible to say. There might be other differences between the younger and older girls C) Yes, because the younger girls use cream on their skin and they have smoother skin

D) No, Judith should try using the cream herself to see if it works for her
Answer:  20. Dr. Wasswa has done a research study giving a new medicine to people who were vomiting. Some of the people stopped vomiting after they got the new medicine. Dr. Wasswa says that this means that the medicine works.

Options:
A) No. The people who used the medicine were not compared with similar people who did not use the medicine B) Yes, some of the people stopped vomiting

C) No, since not all of the people stopped vomiting
Answer: 

Instructions:
Read the text at the top of the box. Then read the text in each row and choose what you think is the best answer by making a tick in one of the two boxes. There should be only one tick in each row.

21.
When you are sick, sometimes people say that somethinga treatment -is good for you. Below you will find different things people say about such treatments.

Do you agree or disagree with each of the following things being said?
For each thing being said below, use a tick to mark whether you "agree" or "disagree".

Things being said:
I agree I disagree 21.1 Peter says that if a treatment works for one person, the treatment will help others too 21.2 Alice says that if some people try the treatment and feel better, this means that the treatment helps 21.3 Habibah says that, just because many people are using the treatment, this does not mean that it helps 21.4 Julie says that companies sometimes say that the treatment they make is best just to make money

22.
A doctor wanted to know if a new medicine for treating headaches is better than an older medicine. The doctor did a research study, comparing the two medicines.
Would the actions below make you more sure or less sure about the results of the study?
For each action below, use a tick to mark whether you think the action would help you become "more sure" or "less sure".

23.
To know if a treatment helps you, the treatment should be compared in research studies to other treatments (fair comparisons). Below you will find different things people say about such studies.

Do you agree or disagree with each of the following things being said?
For each thing being said below, use a tick to mark whether you "agree" or "disagree". Instructions: For the following questions, choose what you think is the best answer and write the letter for that answer in the box provided.
27. It is common for people to say that something will help improve your health or that it will not help. Some may say that it will be bad for your health. What people say about treatments may be correct but sometimes it may be wrong. We call these treatment claims.
How often do you hear treatment claims?

Options:
A) One treatment claim or more on most days  28.2 A treatment is anything done to care for yourself, so you stay well or, if you are sick or injured, so you get better and not worse What was the treatment in the claim you last heard about?

Options:
A) Using a medicine (for example, taking a tablet or a syrup)

B)
Getting an operation (for example, removing a bad tooth) C) Using something to feel better or to heal more quickly (for example, using a bandage, or glasses) 29.3 What was the treatment for which you or someone else made the decision?

Options:
A) Using a medicine (for example, taking a tablet or a syrup)

B)
Getting an operation (for example, removing a bad tooth) C) Using something to feel better or to heal more quickly (for example, using a bandage, or glasses) D) Eating food or drinking something to feel better (for example, herbs or fruit)

E)
Avoiding doing something to feel better (for example, not drinking milk)

F) Something else
Answer:  29.4 It is common for people to say that something will help improve your health or that it will not help. Some may say that it will be bad for your health. What people say may be correct but sometimes it may be wrong. We call these treatment claims.
What was the claim about the treatment for which you made the decision? (What did they say the treatment would change or not change about your health?) ______________________________________________________________________________