Community Monitoring in Health Resources for the Practitioner |
Conducting the Enquiry: Generating Evidence from Experience
Due to time constraints and because many of the groups already have research experience, this will be filled later, drawing from standard research methods manuals. It will include:
Who could be on the enquiry team – need for training
How the different methods could be applied
Collation and Analysis: Making the Report Card
This is the last and the step one needs to be most careful about. Here one is going to represent the community experience as a numeric or traffic light rating so a reasoned judgment or explanation has to be there for providing the traffic light colour. The process of collation can be seen as a two step process – the first of combining the answers from different sources to create a score and the second is to convert that score into an interpretive traffic light. The process is explained below:
Who could be on the enquiry team – need for training
How the different methods could be applied
- Group Discussion
- Interviews
- Observation
- FOI applications
Collation and Analysis: Making the Report Card
This is the last and the step one needs to be most careful about. Here one is going to represent the community experience as a numeric or traffic light rating so a reasoned judgment or explanation has to be there for providing the traffic light colour. The process of collation can be seen as a two step process – the first of combining the answers from different sources to create a score and the second is to convert that score into an interpretive traffic light. The process is explained below:
In this case, information about quality of care is obtained from 4 different sources, and there are a different number of questions for each source and different number of interviews from each source, thus it will be one set of observations and one interview with the doctor and two focus groups and may be ten interviews. What is explained below is just one way of dealing with the collation process. The practitioner may use a different way – as long as there is a logical explanation for using that process and it is consistent. For the purposes of this collation the following justifications are being provided
Quality of Services – Scoring the Responses from Different Sources During the Enquiry
In the example given above information about quality of services is being received from four different sources. The total number of sources is varied according to the source – thus there are 2 Group discussions with 10 questions each from parents and one interview with a doctor with five questions and may be 10 parent interviews with ten questions each. In such a situation one needs to find a logical way of balancing the information received from different sources. So for example if the Maximum score obtained from all the four sources equals to 100, then the four shares could be Parent’s FGD 30, Parent’s Interview 30; Doctors Interview 20 and Observation 20.
Max Score 100 = FGD (30) + P Int (30) + Prov Int ( 20) + Observe (20).
How this may be done is described below. It must the remembered that the scores and proportional values are given at the discretion of the facilitating organization using a logical explanation, there are no hard and fast rules.
A. Scoring Parents Group Discussion
Both of the Group discussions can have a scoring/documentation sheet which looks as follows:
- The community experience is going to be provided more value or weight compared to doctor’s opinion. Group discussions are going to be provided the same value as collation of all individual experience. Direct observation and doctors’ opinion are going to be provided similar value.
Quality of Services – Scoring the Responses from Different Sources During the Enquiry
In the example given above information about quality of services is being received from four different sources. The total number of sources is varied according to the source – thus there are 2 Group discussions with 10 questions each from parents and one interview with a doctor with five questions and may be 10 parent interviews with ten questions each. In such a situation one needs to find a logical way of balancing the information received from different sources. So for example if the Maximum score obtained from all the four sources equals to 100, then the four shares could be Parent’s FGD 30, Parent’s Interview 30; Doctors Interview 20 and Observation 20.
Max Score 100 = FGD (30) + P Int (30) + Prov Int ( 20) + Observe (20).
How this may be done is described below. It must the remembered that the scores and proportional values are given at the discretion of the facilitating organization using a logical explanation, there are no hard and fast rules.
A. Scoring Parents Group Discussion
Both of the Group discussions can have a scoring/documentation sheet which looks as follows:
If the total number of questions relating to Quality of Care from Parent’s group discussion is 10 then each group discussion can yield a maximum of 5 times 10 or 50 marks. For the two group discussions the maximum marks would be 100. However for every group discussion the actual score would be less.
B. Scoring of Individual Interviews
B. Scoring of Individual Interviews
Each parents response to Quality of Care questions can provide a maximum of 2 times 10 or twenty points. IF there are ten interviews with parents the maximum marks obtained can be 200.
C. Doctors Interview
The doctor’s interview can also be scored in the same way. However since there is one doctor and the doctor was asked 5 question the maximum marks that can be obtained from the doctor is 5.
D. Observation Checklist
The observation checklist can also be made in the same manner. One can choose to give 2, 1 and 0 as scores for each parameter or 1 and 0 for each parameter. If the score is 2, 1, 0 and there are 15 parameters then the maximum score that can be obtained for quality of care from the observation checklist is 30.
C. Doctors Interview
The doctor’s interview can also be scored in the same way. However since there is one doctor and the doctor was asked 5 question the maximum marks that can be obtained from the doctor is 5.
D. Observation Checklist
The observation checklist can also be made in the same manner. One can choose to give 2, 1 and 0 as scores for each parameter or 1 and 0 for each parameter. If the score is 2, 1, 0 and there are 15 parameters then the maximum score that can be obtained for quality of care from the observation checklist is 30.
Note: For each method of enquiry the total score that can be obtained will be much more because in this example only those questions related to quality of care are being discussed. This method can be replicated for all the other issues/domains/categories on the report card.
Quality of Care - Collating the scores & Balancing so they are Comparable
Suppose the scores obtained are as follows
If one adds the scores then the score from the parents’ individual interviews and group discussion will influence the score heavily and the doctors opinion will not be very relevant. In order to provide balance we can decide that the final score will be 100 with the distribution like Individual Interview and Group discussion = 30; Observation and Doctors opinion = 20.
Quality of Care - Collating the scores & Balancing so they are Comparable
Suppose the scores obtained are as follows
- Parents group discussion = two discussion scores total to 60 out of 100 max
- Individual interview = 10 interviews total score is 128 out of 200 max
- Doctors interview = 4
- Observation checklist score is 20 out of 30.
If one adds the scores then the score from the parents’ individual interviews and group discussion will influence the score heavily and the doctors opinion will not be very relevant. In order to provide balance we can decide that the final score will be 100 with the distribution like Individual Interview and Group discussion = 30; Observation and Doctors opinion = 20.
If a decision was made that all scores above 80 will be scored with a green traffic light, between 60 percent and 80 percent with yellow and below 60 percent with red light – this particular value will be a YELLOW. However the decision of cut off scores is not fixed and depends on what the community thinks is a non-negotiable level of good and unacceptable services. What is below the unacceptable should be red and what is above and acceptable level indicated as green. If for example the scores for outreach services and quality of care are both scored at 75 and given YELLOW, the community may choose to disagree and say that we understand that the outreach services are hampered by lack of vehicles and so feel that 75% is a good performance but there should be no reason why quality of services should not be better. This discussion could lead to the decision where the cut off marks for outreach services are revised to 75% and 50% while those for quality of care are kept at 80% and 60%. However as explained there must be a process which allows the community to provide its own logic for the decision and this is also explained in the public sharing process.
When the report card is presented the first sheet will include the different issues and the colour of the traffic light as indicated through this analysis. However this is not the complete report card. The second part of the report card will include a narrative justification of why the score or traffic light was obtained. This will include a summary of the different scores obtained from different sources, any particular stories which justify a point of view, any substantial differences in the opinions of the doctors and the observations and the community experiences and so on.
This report card should first be presented to the community representatives and endorsed / verified that it does represent their collective experience. Once the report card is endorsed then it can be publicly shared with municipal and health and other authorities.
Adverse Outcomes – Issues for documentation
When the report card is presented the first sheet will include the different issues and the colour of the traffic light as indicated through this analysis. However this is not the complete report card. The second part of the report card will include a narrative justification of why the score or traffic light was obtained. This will include a summary of the different scores obtained from different sources, any particular stories which justify a point of view, any substantial differences in the opinions of the doctors and the observations and the community experiences and so on.
This report card should first be presented to the community representatives and endorsed / verified that it does represent their collective experience. Once the report card is endorsed then it can be publicly shared with municipal and health and other authorities.
Adverse Outcomes – Issues for documentation
- What was the experience- Description
- What were the specific problems faced ? Why were these problems important to the client or her family?
- How did the problem evolve?
- What kind of medical help was requested? – Government/Private? If not Government why not?
- Time gap between problem and help seeking?
- How many times were service providers approached ? What was their response?
- What was the advice given by health providers if any? Was it possible to comply with this advice? Investigations? Hospitalisation?
- Any refusal to help? What happened when there was a refusal if any?
- Total costs involved? What was the dominant feeling when going through the experience?What is the dominant feeling about the problem now?
- Any documents?