Assignment 2

The program that is going to be reviewed is entitled Description and evaluation of a prenatal exercise program for urban Aboriginal women. This program seeks to reduce the incidence of gestational diabetes mellitus (GDM) and Type 2 diabetes in Aboriginal women and their offspring through exercise. I have chosen to use a mixture of models for evaluating this program. The three models that I think would be appropriate to evaluate this program are Scriven’s model, Stake’s countenance model, and Stufflebeam’s CIPP model.

In terms of Scriven’s model, since this program has already been implemented and has come to a completion, this program evaluation will be a summative one. In this sense, the program evaluation will look at the program’s success of meeting its goal of reducing the rate of GDM and Type 2 diabetes in Aboriginal women and their offspring through exercise. Since this program evaluation is summative, there will be no opportunity to modify the program as it is implemented. The summative nature of this program evaluation however, will allow for the continuance, modification, or cessation of the program in the future.

Stake’s countenance model will also play a role in this program evaluation in two main ways. First, in order to receive a clear picture of the impact of this program on Aboriginal women, their lives, and future generations, both anecdotal and descriptive data will be utilized. In doing this, this program evaluation will use both qualitative and quantitative data to ensure an accurate portrayal of the program is achieved. Anecdotal questions that will be asked include:

• Have you noticed a difference in your health since taking part in this program?
• Were you generally satisfied or dissatisfied with the program?
• Is there anything that you would like to see changed if the program were to run again?

Descriptive data would include:

• Medical history of GDM and Type 2 diabetes
• Age of the participant
• The usual diet of the participant
• Correlation between amount of sessions attended and propensity of GDM or Type 2 diabetes in offspring
• Rate of GDM and/or Type 2 diabetes in the offspring of the women involved in the program

Another component of the countenance model that is applicable to this program evaluation is the issue of contingency: what is the relationship between exercise and the reduction of GDM and Type 2 diabetes in the offspring of Aboriginal women? This question may prove hard to answer given the abundance of possible interacting and confounding variables in this study. Given prior research however, if there is a reduction in incidence of GDM and Type 2 diabetes in the offspring of the women who participated in this program, one could make a logical connection between prenatal exercise and lower incidence of GDM and Type 2 diabetes in offspring however, one should not accept these results blindly. Questions that need to be asked to determine contingency include:

• Could the nutritious snacks after the exercise have had an influence of GDM or Type 2 diabetes?
• Could the free educational materials that were given out have caused the participants to start leading healthier lives which may contribute to the reduction of GDM and Type 2 diabetes?
• Were the women using the pool or doing other exercise at times other than in the program?
• Are the women on any type of medication that may interact with GDM or Type 2 diabetes?
• Could the act of forming relationships with the other women in the program promote a healthier lifestyle and a support network causing the participants to be less stressed and less prone to GDM or Type 2 diabetes?
• Or, is there a combination effect where exercise along with these factors contributes to decreased GDM or Type 2 diabetes?

Lastly, Stufflebeam’s CIPP (context, input, process, and product) model, more specifically a process and product evaluation, will be used to evaluate this program. In terms of a process evaluation, some of the questions that need to be asked have been described in this post previously. These questions will help to clarify and interpret the outcomes. To recap however, these questions would include (as mentioned above):

·   Could the nutritious snacks after the exercise have had an influence of GDM or Type 2 diabetes?

·   Could the free educational materials that were given out have caused the participants to start leading healthier lives which may contribute to the reduction of GDM and Type 2 diabetes?

·   Were the women using the pool or doing other exercise at times other than in the program?

·   Are the women on any type of medication that may interact with GDM or Type 2 diabetes?

·   Could the act of forming relationships with the other women in the program promote a healthier lifestyle and a support network causing the participants to be less stressed and less prone to GDM or Type 2 diabetes?

·   Or, is there a combination effect where exercise along with these factors contributes to decreased GDM or Type 2 diabetes?

Since this program has already been completed, and the goal of the present evaluation is to determine if the exercise program was effective in reducing GDM and Type 2 diabetes in future generations, it makes sense to include a product evaluation in this program evaluation. Questions that would assess short term and long terms outcomes include:

·   Do the participants feel healthier after being part of this program?

·   Do participants feel supported after being part of this program?

·   Has there been a change in diet/exercise habits of the participants since the program?

·   Do the participants have more knowledge about pregnancy, exercise, GDM, and/or Type 2 diabetes since this program?

·   Is there a reduction in GDM increasing to Type 2 diabetes in the women since taking this program?

·   Is there a reduction in the propensity of GDM and/or Type 2 diabetes in the offspring of the women who were involved in this program?

I would have liked to have included Lincoln and Guba’s naturalistic model as well, but due to the nature of the program, having been completed prior to the evaluation, this is not possible. I think that if this model had been part of the program evaluation, the evaluator could recieve valuable first hand information about the interactions between the women in the program as well as interactions between program facilitators and participants. By including this component, more clarity may have been possible in terms of interacting or confounding variables affecting the outcomes of this program.

In conclusion, it is my opinion that the integration of Scriven’s model, Stake’s countenance model, and Stufflebeam’s CIPP model ensure a comprehensive program evaluation. By utilizing elements of various models, and by combining various methods of data collection, a clearer picture can emerge of the relationship between exercise and GDM or Type 2 diabetes in the offspring of Aboriginal women. Furthermore, the utilization of these models allows for more clarity about the variables that could interact with or influence exercise and GDM or Type 2 diabetes. Lastly, the models that were chosen for this program evaluation were chosen specifically for the goal of this program: to reduce incidence of GDM and/or Type 2 diabetes in the offspring of Aboriginal women.

Reference

Klomp, H., Dyck, R., & Sheppard, S. (2003). Description and evaluation of a prenatal exercise program for urban Aboriginal women. Canadian Journal of Diabetes, 27, 231-238.

Assignment 1

            The program evaluation (PE) that is being reviewed is entitled Process and outcome evaluation of an emergency department intervention for persons with mental health concerns using a population health approach. In this article the authors sought to evaluate the introduction of a mental health triage and a mental health counsellor on an emergency department’s (ED) dealings with people with mental health concerns. This review will outline the models that were used in formulating this evaluation, will explore various strengths and weaknesses, and I will mention some personal thoughts on this PE.

            Rarely only one model is used when formulating a PE; four models were used in the current PE. Stake’s countenance model states that PEs should include both anecdotal data and descriptive data. In the present PE qualitative anecdotal evidence was obtained from various stakeholders and descriptive data was also obtained through a quantitative component. Another component of the countenance model that is relevant to the current PE is the question of if the intervention is being implemented in relation to the objectives. This directly relates to the question posed by the countenance model: is there congruence between what is intended and what is observed? A third component of the countenance model that is applicable to the current PE is the question of if there are logical connections between an event and purpose. This is seen in the current PE through the observation of the connection between the mental health triage and a mental health counsellor and the PE goals, or purpose.

            An element of Stufflebeams’s CIPP model was also used in the current PE. A premise of the CIPP model is that PEs can look at any of the four components (C-context, I-input, P-process, and/or P–product) that are part of a program. The current PE focused on the process and outcome, or product of the program. In this sense, the CIPP model was utilized.

            Scriven’s model is also apparent in the current PE. Scriven stated that PEs can be goal or role focused. The current evaluation is goal focused in that, part of the focus is on the outcomes of the program. Furthermore, there appeared to be a focus on what the goals of the program were when identifying the measures and research questions that were used in this PE.

            Lastly, Rippey’s transactional model has appeared to influence the current PE. Part of the transactional model is to include all those who are affected by an issue. The current PE included many of the individuals who are affected by the program, from psychiatric consultants, to the families of individuals with mental health concerns. Another component of the current PE that falls in the realm of the transactional model is that the program was implemented because of the potentially damaging situation of misuse/non-use of resources for people with mental health concerns in the ED.

Even though this PE used multiple models to inform its implementation, there are still weaknesses of both the PE and the models that were used.  In terms of weaknesses of the PE, a number of the pre and post intervention participant characteristics were significantly different from each other and therefore, the results of this PE need to be seen through this lens. Secondly, although long-term data were reported, the authors caution relying on this data since not enough time has elapsed to truly measure long-term outcomes. In terms of weaknesses of the models that were used for the PE, although there was mention that the program was implemented partially due to the fact that treating people with mental health concerns the same as those without was creating a strain financially, there was no analysis of the change in cost with the implementation of the new program. This may have been analyzed had Provus’s discrepancy model been used. Furthermore, the goals of the program were known a head of time and therefore, there is the possibility that the authors may have been affected by ‘tunnel vision’.

Strengths were also present both in terms of the PE and the models that were used. One of the strengths of the PE was the use of mixed methods. The inclusion of both qualitative and quantitative data allows for triangulation and therefore more validity. Another strength includes utilizing components of many models, allowing for a more inclusive PE. One of the strengths of the models that were used was to include many stakeholders in the evaluation. By including many stakeholders, a truer picture of the program is obtained. Lastly, the connection that was made between the intended use of the program and what was observed in the PE can be seen as a strength.

Personally, although I think that the program that was evaluated was a valuable contribution to the mental health care field in Ontario, Canada, I thought that this PE could have been written more clearly. I found that I had to re-read parts of the evaluation to truly understand what the authors were trying to say. Furthermore, I think that the PE would have been more valuable had it been conducted during the program implementation as opposed to after. This would have allowed the people who were involved in the program to respond in real time to the needs of the clients. Lastly, although the authors reported the results of the PE, and the results appeared to be positive in terms of the usefulness of the new program, the authors did not mention if this PE was used in changing the current program, or if this PE had any effect on the program at all. It is my view that PEs should be conducted with the intent of having an actual impact on the program, and it would have been beneficial if the authors would have explored this. On a positive note, I thought that the author’s use of five different datasets to be valuable because this allowed for more rich data and results. Furthermore, I found the background section of the program very helpful as setting up why this program was needed, and I enjoyed that the PE was Canadian research. This PE is applicable to me because of my thesis topic and work that I am doing with Community-University Institute for Social Research. It was great to be able to read an article and have it be utilized in a few areas of my life.

Many PEs make use of different models for implementation. The current PE utilized Stake’s countenance model, Stufflebeam’s CIPP model, Scriven’s model, and Rippey’s transactional model. Although this program evaluation had some weaknesses, not only in terms of the PE, but also in terms of the models that were used, this PE also had many strengths, including using mixed methods and mixed models.

Reference

Vingilis, E., Hartford, K., Diaz, K., Maitchell, B., Velamoor, R., Wedlake, M., & White, D. (2007). Process and outcome evaluation of an emergency department intervention for persons with mental health concerns using a population health approach. Administration and Policy in Mental Health and Mental Health Service Research, 34(2), 160-171.