REU students Saloni Shah and Emilio Arias look at a tree map to find the correct trees to measure with mentor Audrey Barker Plotkin for the project: “Shifting dominance of red maple and red oak”.
Credit: Harvard Forest Archives: Sarah Plisinski CC BY-ND
Many scientists have pivotal experiences during their undergraduate education that lead them to choose a career in science, such as opportunities to conduct hands-on research or work closely with mentors. Unfortunately, it’s a challenge to measure the direct impact these foundational experiences have on participants. In a new paper, however, researchers from Harvard Forest LTER present a comprehensive theoretical framework that they argue makes it possible to improve how we evaluate the effectiveness of undergraduate research programs, and enhance their success in the process.
For more than 30 years, the US National Science Foundation’s Research Experiences for Undergraduates (REU) program has provided opportunities for thousands of undergraduate researchers annually to have their first research experiences in scientific fieldwork. REUs embed students in scientific communities where they work alongside experienced researchers, build networks with their peers, and come to understand research cultures and how to work within them. REUs are thought to provide formative experiences for developing researchers that differ from experiences in college classrooms, laboratories, or field trips. The ability to test this hypothesis, however, has been hindered by both a lack of an appropriate theoretical framework and of long term data.
The Summer Research Program in Ecology at the Harvard Forest LTER has begun to address this by using cultural-historical activity theory (CHAT) to measure social and cultural factors that influence learning at the REU site. Their new paper describes how CHAT, together with long term data collection, has provided focus for evaluating and improving the Harvard Forest Summer Research Program in Ecology (HF‐SRPE) by enhancing the program leaders’ understanding of barriers to student and mentor successes and assessment priorities. The process has also revealed important oversights in prior data collection.
CHAT embodies a systems approach to assessment that accounts for social and cultural factors that influence learning, such as a student’s prior education and research experience, or program leaders’ styles of mentorship. To help generate hypotheses about how REU programs may impact a participant’s persistence in STEM, the authors used the CHAT framework to prioritize collection of data including: the skills and knowledge a student brings with them to the research program; the professional development opportunities available to them during their research experience; the interactions students have with other members of the research community; and the goals of the research experience.
The paper provides three examples that demonstrate the flexibility of CHAT for assessing different aspects of REU programs: recruitment and hiring practices, student learning gains, and the impact on participant persistence in STEM. Based on the benefits of using CHAT for evaluating the Harvard Forest SRPE, the authors advocate for integration of sociocultural frameworks such as CHAT in future assessment and evaluation of REU programs more broadly. They believe that CHAT is flexible enough that it can guide the summative and formative evaluations for most aspects of REU programs in away that helps characterize and identify important components and interactions within the complex learning environment – with the ultimate goal of creating REU experiences where undergraduates of all backgrounds feel supported, enriched, and valued.
-Kristen Weiss
Reference: McDevitt, AL, Patel, MV, Ellison, AM. Lessons and recommendations from three decades as an NSF REU site: A call for systems‐based assessment. Ecol Evol; 00:1-29. https://doi.org/10.1002/ece3.6136
