Demonstrating Quality and Impacts of Engaged Scholarship

Description: 

This annotated list of scholarly articles offers various approaches to  demonstrating quality and impact in relation to the practice and production of engaged scholarship. 

• Braun, K. L., Nguyen, T. T., Tanjasiri, S. P., Campbell, J., Heiney, S. P., Brandt, H., …Herbert, J. R. (2012). Operationalization of community-based participatory research principles: Assessment of the National Cancer Institute’s community network programs. American Journal of Public Health, 102(6), 1195-1203. 
  • This study examined how National Cancer Institute-funded Community Network Programs (CNPs) operationalized principles of CBPR. The authors reviewed the literature and extant CBPR measurement tools. On the basis of that review, they developed a questionnaire for CNPs to self-assess their operationalization of 9 CBPR principles. Twenty-two CNPs completed the questionnaire. This study suggests that the CBPR processes can be assessed in a variety of settings, and may help others develop and test CBPR measures.
• Brown, P., Brody, J. G., Morello-Frosch, R., Tovar, J. Zota, A. R., & Rudel, R. A. (2012). Measuring the success of community science: The northern California household exposure study. Environmental Health Perspectives, 120(3), 326-331.
  • In this paper, the authors review the methods used to evaluate their CBPR project, “Linking Breast Cancer Advocacy and Environmental Justice”, in order to assist other teams in evaluating CBPR effectiveness. Their evaluation strategy assessed how the CBPR partnership met the goals of all partners, including scientific, educational, policy, community engagement and capacity building goals. The strategy also had partners talk to each other about these issues frequently throughout the project, to collect ongoing evaluation data and stimulate changes to address problems. The authors share ten questions that can be used by other teams to guide such an evaluation.
• Doberneck, D. M., & Schweitzer, J. H. (2012). Who’s publishing what? Publication patterns in seven community engagement journals. Proceedings from the 13th Annual National Outreach Scholarship Conference. 
  • This conference presentation shares a three phased research study that investigated the following questions: (1) What types of articles are published in the community engagement journals? (2) Who is publishing in the community engagement journals? And, (3) How rigorous is the research published in the community engagement journals? The findings of this study are discussed. Implications of the findings for authors, community engaged scholarship journals, and the field of engaged scholarship are also discussed.
• Drew, C. H., Pettibone, K. G., O’Fallon, L. R., Collman, G. W. & Birnbaum, L. (2012). Measuring partnership activities: Partnerships in environmental public health evaluation metrics manual. Environmental Health Perspectives, 120(7), A261-A262.
  • The “Partnerships in Environmental Public Health (PEPH) Evaluation Metrics Manual” was developed by the National Institute of Environmental Health Sciences (NIEHS) in 2012, in order to build the evaluation capacity of the PEPH grantees. The manual identifies metrics to measure and demonstrate success in five areas: partnering, leveraging, disseminating findings, training, and capacity building. For each of these five areas, the manual provides an illustrative logic model to demonstrate connections among project activities, outputs, and impacts; over 80 different metrics for each activity, output, and impact are provided as examples, to enable grantees to develop metrics relevant to their specific programs.
• Jagosh, J., Macaulay, A. C., Pluye, P., Salsberg, J., Bush, P. L., Henderson, J.,…Greenhalgh, T. (2012). Uncovering the benefits of participatory research: Implications of a realist review for health research and practice. The Milbank Quarterly: A Multidisciplinary Journal of Population Health and Health Policy, 90(2), 311-346.
  • In this study, the research team, selected, and appraised a large-variety sample of primary studies describing participatory research (PR) partnerships, and in stage, two team members independently reviewed and coded the literature. The team used a realist approach to analysis, in order to embrace the heterogeneity and complexity of the PR literature. This theory-driven synthesis identified mechanisms by which PR may add value to the research process. Using the middle-range theory of partnership synergy, the review confirmed findings from previous PR reviews, documented and explained some negative outcomes, and generated new insights into the benefits of PR.
• Khodyakov, D., Stockdale, S., Jones, A., Mango, J., Jones, F., & Lizaola, E. (2013). On measuring community participation in research. Health Education & Behavior, 40(3), 346-354. 
  • This article presents two complementary approaches to measuring the level of community participation in research—a “three-model” approach that differentiates between the levels of community participation, and a Community Engagement in Research Index (CERI) that offers a multidimensional view of community engagement in the research process. The article discusses the strengths and limitations of each approach, summarizes the lessons learned, and offers directions for future research.
• Kneipp, S. M., Lutz, B. J., Levonian, C., Cook, C., Hamilton, J.B., & Roberson, D. (2013). Women's experiences in a community-based participatory research randomized controlled trial. Qualitative Health Research, 23(6), 847–860. 
  • This study conducted focus groups with 31 disadvantaged women who participated in a CBPR-driven randomized controlled trial (RCT), to explore their study experiences. Analysis revealed that the tailored health questionnaire, treatment by study staff members, and RCT participants’ understandings of and responses to randomization were salient to what the women described as transformative experiences that occurred during the RCT. These findings have implications for understanding how CBPR and non-CBPR aspects of interventions and study designs have the potential to affect both process and endpoint study outcomes.
• Leisey, M., Holton, V., & Davey, T. L. (2012). Community engagement grants: Assessing the impact of university funding and engagements. Journal of Community Engagement and Scholarship, 5(2). 
  • This study investigated the importance and consequences of a series of CBPR projects funded by Virginia Commonwealth University. The study provides insight into the different ways that the university and community partners understood the research project outcomes. It also raises important questions about the relative importance of the outcomes of the project, when compared to the impact of the relationship between the university and community partner.
• Malone, R. E., McGruder, C., Froelicher, E. S., & Yerg, V. B. (2013). Being part of something: Transformative outcomes of a community-based participatory study. Health Promotion Practice, 14(2), 205-212. 
  • This study investigated the secondary (i.e. non-research) outcomes of Protecting the ‘Hood Against Tobacco (PHAT), a CBPR project conducted in San Francisco, California. An analysis of quasi-ethnographic documentation of the PHAT project was conducted. Analysis revealed that PHAT participation encouraged healthier behavior and public health promotion among community research partners, prompted academics to confront power asymmetries and recognize community knowledge, and widened social networks. The authors conclude that systematically capturing secondary outcomes, perhaps through wider use of ethnographic approaches, could help enhance understanding of CBPR’s true contributions.
• Ozer, E. J. & Douglas, L. (2013). The impact of participatory research on urban teens: An experimental evaluation. Journal of Community Psychology, 51(1-2), 66-75.
  • This study investigated the effects of youth-led participatory research on the psychological empowerment of 401 students attending urban public schools. The authors found that attending a participatory research elective class during the school day was associated with increases in students’ sociopolitical skills, motivation to influence their schools and communities, and participatory behavior. The implications for participatory research and related youth development interventions are discussed.
• Salimi, Y., Shahandeh, K., Malekafzali, H., Loori, N., Kheiltash, A., Jamshidi, E.,…Majdzadeh, R. (2012). Is community-based participatory research (CBPR) useful? A systematic review on papers in a decade. International Journal of Preventive Medicine, 3(6), 386-393.
  • This paper presents a systematic review of the CBPR literature that was conducted to examine the effectiveness of current CBPR intervention studies in creating positive change in target communities. The findings showed that collaboration among community partners, researchers, and organizations led to community-level action to improve the health and wellbeing and to minimize health disparities, and also enhanced the research of leadership capacity of the community. Based on their review, the authors recommend that future assessments of CBPR projects evaluate not only health outcomes, but also “how much the target community has been empowered.”
• Stedman-Smith, M., McGovern, P. M., Peden-McAlpine, C. J., Kingery, L. R., & Draeger, K. J. (2012). Photovoice in the Red River Basin of the north: A systematic evaluation of a community-academic partnership. Health Promotion Practice, 13(5), 599-607. \
  • This paper presents the evaluation of a community-academic project that facilitated mothers to use Photovoice to document pathways to pesticide exposure for their children. Surveys were administered to mothers and the research team of local stakeholders and academics to assess their perception of the process and short-term outcomes. The results of this evaluation provided insight on the strengths and weaknesses of the Photovoice project and demonstrated to team members and funders that formative and summative outcomes were met.