ASSESSMENT

Assessing Undergraduate Research Experiences: An Annotative Bibliography

Mary Crowe, Florida Southern College

David Brakke, James Madison University

Abstract

In the past decade, the assessment of undergraduate research experience (URE) and course-based undergraduate research (CURE) has evolved and significantly expanded, with hundreds of studies published in books, white papers, technical reports, and academic journals. Much of the work has focused on the impact of URE and CURE on students, leading to new insights about the importance of mentoring and student self-efficacy and the identification of essential features of URE and CURE. Studies focusing on the impact of URE on faculty members and institutions have remained limited. The advent of a variety of assessment instruments and the spread of this high-impact practice across all academic fields suggest that the timing is ripe for new areas of study.

Keywords: assessment, faculty, mentoring, student learning, undergraduate research

doi: 10.18833/spur/3/2/3

A decade ago, the authors provided an overview of the peer-reviewed literature related to assessing undergraduate research experiences (URE; Crowe and Brakke 2008). At that point, robust assessment of this high-impact practice was in its infancy. In the past decade, the assessment of URE has evolved and significantly expanded, with hundreds of studies being published in a wide variety of formats including books, white papers, technical reports, and academic journals. The report of the Council on Undergraduate Research (CUR), Characteristics of Excellence in Undergraduate Research (COEUR; Hensel 2012), was used as a road map for this updated annotative bibliography. The review is organized to examine the impact of URE on three stakeholders: students, faculty, and institutions. It was difficult to decide which of many high-quality studies should be highlighted here. Selection criteria encompassed the following:

1. Sources from diverse fields and journals were included.

2. Sources selected had undergone a rigorous review.

3. Sources selected offered a comprehensive overview of a selected area of URE assessment or provided new or targeted assessments leading to new knowledge.

Additional resources include the Zotero bibliography catalogs curated by CUR, begun in 2012. One relates to the development, implementation, and assessment of course-based undergraduate research experiences (CURE), and the other is devoted to the assessment of varied UREs. Each of the bibliographies contain more than 300 references and is updated regularly by experts in the fields. They are available to CUR members through the Member Resource page “Bibliography Catalogs” (CUR n.d.).

Selected Comprehensive Sources

Gentile, Brenner, and Stephens 2017

A diverse team of experts in the field of STEM undergraduate research contributed to this report. The nine chapters address a variety of factors surrounding URE, including how to design and implement programs, examples of different URE models, how to assess them, and how UREs fit within the higher education landscape. This report also examines the role of mentoring in URE as well as the impact of URE on student learning and educational pathways. Finally, the report addresses weaknesses in the current offering of UREs, including limited funding models, lack of equity in student participation, and the need for coordinated assessment of outcomes. The literature cited for each chapter is robust and extensive.

Gourley and Jones 2018

This book provides in-depth analysis of CURE in chemistry. The editors of the book bring examples that span the educational pathway of students from a wide range of institutions. Each chapter provides an overview of why and how a given CURE was implemented and an assessment of outcomes (along with the assessment tools for a few). The four chapters on program and curricular reform inform on piloting an initiative in just one section of a course to department-wide CURE.

Hensel 2018

The overall goal of this book was to provide details on how to integrate research into first- and second-year courses. The editor has done an admirable job of providing examples from across disciplines and institutional types to show that CURE opens up URE to include all students. The chapters contain information on “how to” and “why,” with assessment practices included when appropriate.

Linn et al. 2015

This article begins with an overview of shortcomings in the assessment of URE, particularly that much of what is published about the benefits of URE is based on student self-reported surveys. It also suggests that students self-reporting about their experiences tend to be well prepared, with an affinity for STEM. The authors then compare URE with CURE, focusing on the duration, access (types of students), type of work, and amount of one-on-one mentoring received by the student. They provide a good overview of the published literature on URE and CURE that examines programs with respect to retention in STEM, understanding the nature of science, practices of scientists, conceptual understanding, and the role of mentoring. The authors underscore the need for better training of the mentors to undergraduate researchers, whether those mentors are faculty members or others along the STEM educational pathway.

Lopatto 2010

This evidenced-based book summarizes the extensive and excellent qualitative and quantitative work conducted by the author in examining the impact of URE and CURE on students. The book provides a historical overview of the development of URE, an in-depth treatise on how standardized surveys were developed, the results of those surveys, and the ways in which those surveys have helped to define essential characteristics of URE. It also addresses the importance of the mentoring component of URE and the way in which faculty members at various institutions view the teacher-scholar model.

Shanahan et al. 2015

This reference is an extensive review of the literature regarding best practices for mentoring undergraduate students on original work. The authors reviewed more than 100 peer-reviewed articles and books to develop a top-10 list of effective mentoring practices, which involved preplanning, scaffolding, teaching techniques, building community, devoting time, encouraging independence, networking, and using peers.

Whitmeyer, Mogk, and Pyle 2009

This important Geological Society of America report brings together a wide range of field-based geology experiences and their evaluations. Several of the programs occur in the United States, but some are international. The topics include the integration of field experiences in a project-based curriculum, long-term field-based studies, and the incorporation of various approaches in advanced project options. The group of chapters illustrates the evolution of traditional geology field courses into more project-based experiences incorporating a range of geophysical, hydrological, and other techniques in combination with geospatial technologies. A final chapter covers the evaluation of field course experiences and course-embedded projects.

Impact on Students

Thirty years ago, citing gains in student competencies and confidence, the Boyer Commission on Educating Undergraduates in the Research University (1998) recommended that mentored undergraduate research be a standard pedagogical practice in all undergraduate disciplines. It took some time to bring CURE to scale and involve more, if not all, students. The difficulty lies in assuring authenticity of the research experiences while simultaneously institutionalizing them. The CURE assessment literature suggests that students gain in most (but not all) of the same skills and competencies as their peers who are individually mentored in traditional stand-alone URE (Auchincloss et al. 2014; Foster and Usher 2018; Jordan et al. 2014; Kinner and Lord 2018; Schaffer et al. 2010). STEM CURE is now fully embedded in a wide range of institutions, across disciplines, and in varied places in the curriculum such as in first-year courses, in mid-level courses, and at some institutions, as at Florida Southern College, required of all students as a capstone course. Faculty outside of STEM are developing CURE- and URE-like programs for students (Dvorak and Hernandez-Ruiz 2019; Lima and Tsiang 2017; McNary-Zak and Peters 2011; Sternquist, Huddleston, and Fairhurst 2018).

Determining the impact of URE on students is the most advanced area in the field of assessing URE. Educational researchers are asking more nuanced and targeted questions about CURE and URE. A number of studies have focused on the mentoring aspect of the URE (Aikens et al. 2016; Bhattacharyya, Chan, and Waraczynski 2018; Hall et al. 2018; Vandermass-Peeler, Miller, and Moore 2018) and the impact of URE on underrepresented minority students (URM; DeFeitas and Bravo 2012; Hurtado et al. 2009; Hurtado et al. 2010; Komarraju, Musulkin, and Bhattacharya 2010; Slovacek et al. 2011).

Frantz et al. 2006

The participants in this study were assigned either to a traditional summer URE or to a collaborative learning model in which they worked as a group on a directed research project in a classroom setting over the 10-week summer period. The authors carried out this study with four different cohorts of students, including 76 students involved in the classroom model and 79 engaged in the traditional URE model. Students in both groups devoted the same number of hours to research. Those in the collaborative learning model worked as a group with one to three faculty members who rotated in and out of the classroom space over the course of 10 weeks, teaching new techniques, reviewing protocols, and providing support and guidance. The last three weeks of the 10-week classroom experience had groups designing and carrying out a unique experiment. Students assigned to a traditional URE spent the 10 weeks in a faculty member’s research laboratory conducting research in a manner that many equated to the Research Experiences for Undergraduates (REUs) funded by the National Science Foundation (NSF). Students from both groups completed surveys about self-efficacy, teamwork, science identity, anxiety around neuroscience, and commitment to a scientific career. Each student completed the survey at the start of, during, and end of the summer research period. To ascertain career status, the authors also followed up with participants after the summer research experience was over, in some cases more than seven years afterward. The majority of results showed no difference between the classroom-based group research model and the traditional undergraduate research model in terms of effect on long-term career paths. However, students in the classroom-based group research project reported higher self-efficacy gains than students in the traditional model. There were differences between genders; female participants entered the program with lower research self-efficacy than their male peers but caught up to them during the program. Women also reported less strong science identities than men at the beginning and end of the program, suggesting that more must be done to encourage women to remain in STEM fields.

Haeger and Fresquez 2016

The authors were interested in determining what mentoring models were related to student improvement in research skills and academic and professional growth. They also wished to examine whether the length of the mentored research experience affected these factors. This study was conducted at a public minority-serving institution and involved examining the GPA and graduation rate of nearly 348 students, comparing those students who engaged in mentored research to students with similar profiles who did not engage in a URE. The authors were very careful to match students in the two groups on the impact factors of high school GPA, gender, race/ethnicity, Pell Grant eligibility, and intended major. Mentored undergraduate research did not significantly impact GPA or graduation rates in a negative way. In fact, students who devoted extra time to research graduated at the same rate as their peers who did not engage in this time-consuming activity. The authors examined the postresearch surveys of the 138 students who engaged in mentored URE. Research mentors who received culturally relevant mentoring that addressed the emotional and social issues of students more positively impacted the research skill development of students than mentors who did not.

Mahatmya et al. 2017

Much is known about what students report as a result of their URE. What is not clear is how novice students find URE opportunities and decide to apply. The authors were interested in determining how college and university structures impact the identification process and how characteristics of individual students contribute to the motivators for and barriers to participating in a URE. They developed a web-based survey and administered it to students on four different campuses, collecting responses from more than 11,000 students. Students at Research 1 institutions were less likely than their peers at liberal arts and master’s degree institutions to be involved in URE. Regardless of type of institution, upper-division students were more likely to be participating in UREs than lower-level students. Students engaged in URE reported that instrumental factors (such as employment that would preclude involvement) and experience factors played a role in their involvement because they recognized that, to successfully apply to and complete postbaccalaureate study, they needed URE. Although all students reported lack of information, time, and financial resources as barriers to involvement, students who did not participate in URE were more likely to cite a lack of research readiness than their peers who were engaged in a URE. Both first- and fourth-year students who were not participating in URE also cited social reasons for not being involved. Regarding the apparent importance of a mentoring relationship to the URE, the study showed that the availability of a mentor, either as a motivator or a barrier, was not cited frequently by students as a factor influencing their participation.

Mraz-Craig et al. 2018

The authors were interested in documenting how an authentic research experience in a two-semester course influenced the development of scientific identity and in determining the relationship between student identity and career aspirations. The study was done of first- and second-year students at a large public research institution. At the end of the course, researchers conducted one-on-one interviews with 48 students enrolled in the course. Nearly 80 percent of students had developed scientific identities by the end of the course, expressed confidence as problem solvers and team members, and viewed themselves as the owners of their real-world problems. Sixteen percent of students saw themselves as doing what they needed to pass the course, and 4 percent of students reported little to no interest because of encountering repeated stumbling blocks. An interest in a STEM career was expressed by 77 percent of students.

Parker 2018

Students at universities in the United Kingdom are usually required to complete a year-long independent research project. This study examined whether there was a correlation between six variables and the grades the students received on their capstone projects. About 40 percent of the projects were in the field of natural sciences, 40 percent in social sciences, and 20 percent in the humanities. The results showed that students with lower academic achievement benefited more from the experience, natural science students were graded higher on their projects than students in the humanities, and women improved their research grades more than males. The grades of black students and white students did not differ, but Asian students received significantly lower grades than their peers.

Portillo et al. 2013

Faculty members in a criminology department report on student learning gains in a capstone research course. The data were collected during one academic year from eight different sections of the capstone course and included approximately 100 students. Each student did an independent research project that involved a literature review, interviews, and courtroom observations. The independent work of students was informed by the literature in the field. The authors used three methods to evaluate the experience: (a) students filled out pre- and post-surveys, (b) a graduate student annotated classroom discussions between the undergraduate students about the research process, and (c) the graduate student took notes about the one-on-one tutoring sessions she conducted. Students reported gains in their writing ability but were less likely to trust information generated by their own observations and interviews. They struggled to analyze and synthesize their work. They reported that the experience helped them understand the court system and processes much better.

Robnett et al. 2018

The authors were interested in examining what type of mentoring contributed most to the development of scientific identity in undergraduate researchers and the mentor’s motivation for working with undergraduate students. The focus here is on the first objective. This study examined student–research mentor dyad responses related to the student’s science identity and the types of mentoring the student received during the URE: socioemotional (motivational talks), instrumental (how to do an analysis, operate equipment, etc.), and negative (behaviors that undermine the relationship). The 66 student-mentor dyads were recruited at a national meeting at which STEM undergraduate students presented the results of their work. Participants hailed from a wide variety of two- and four-year campuses. The student and research mentor independently completed online surveys and disclosed demographic information about themselves. Instrumental mentoring was more important than socioemotional mentoring in helping students identify as scientists. Dyad responses to the instrumental mentoring were positively correlated, but the dyad responses to the socioemotional and negative mentoring were not. Mentors who said they had higher levels of negative mentoring tended to have students with a lower scientific identity; however, student responses were less predictable. Some students with a strong science identity reported a higher level of negative mentoring, and some students with a low science identity reported low levels of negative mentoring.

Stanford et al. 2017

Since 2003, Drexel University has run a faculty-mentored summer research program for high- achieving rising second-year students. It is open to students from all academic disciplines and is set up much like traditional summer STEM REU. In 2012, a modified undergraduate research student self-assessment (URSSA) was sent to alumni of the program. Students in the 2012 and 2013 summer cohorts also completed URSSAs. Thirty-six percent of alumni responded. The authors reported the survey results and provided information about participants’ persistence in college and graduation rates. The highest rated self-reported growth, for both STEM and non-STEM students, occurred in the following categories: understanding what everyday research work is like, increasing ability to work independently, and explaining their project work to people outside their fields. Overall, there were no significant differences between STEM and non-STEM students to survey questions. More than 900 students have participated in the Drexel program, with 96 percent of those students either still enrolled or graduated.

Impact on Mentors

UREs are dependent on the availability of mentors. A variety of individuals fulfill this role: peer mentors, graduate students, postdoctorate students, staff, faculty members, and researchers at government laboratories and in private industry. Faculty members, however, have been considered the primary source of mentors for URE.

Hayward, Laursen, and Thiry 2017

The authors begin the article citing the benefits of URE for students and make the argument that URE offerings must be increased so that as many students from diverse backgrounds can participate as possible. One way to accomplish this is to offer CURE. Another method is to increase the number of individuals who are willing to mentor students. In this article, the authors describe conducting one-on-one interviews with 30 research advisers at a research institution to uncover what factors motivated them to work with students. They also were interested in determining whether an individual’s career stage (early career versus experienced) influenced the decision to work with undergraduate researchers. Their results showed that both early career and experienced research advisers were intrinsically motivated to direct URE, that they felt a desire to help students develop and grow, and that they recognized that undergraduate researchers brought energy into the research setting. Early career research advisers, however, were the only group that was motivated by the belief that undergraduate students could further research productivity in their laboratories or that reported that it was an institutional expectation. Both early career and experienced research advisers cited intrinsic benefits of working with undergraduate students, such as the personal rewards of new friendships and better understanding of content because of the need to explain things to those with limited backgrounds. Both groups also cited other important benefits of working with undergraduate students, such as long-term networking and the role undergraduate researchers played in recruiting other novice students into the research setting.

Morrison et al. 2018

The authors were interested in identifying the factors that either encouraged or discouraged faculty members to mentor students. The introduction summarized previous work that had identified the two most consistent reasons faculty cite for not mentoring students: time and the level of preparation of students. For this study, the authors surveyed faculty on three campuses that were diverse in relation to private/public status, degrees granted, and size of the student population. Faculty were asked to report their motivation for mentoring students, the barriers to participating, and the extent to which they involved students in their work. Results from 240 faculty showed that 77 percent had mentored students, with nearly half of these working with one or more students per semester. These respondents were more likely to report that they knew how to involve undergraduates than their colleagues who reported that they did not involve students in URE. Both groups agreed that they did research for themselves and that one did not receive enough credit for working with undergraduate students. All respondents reported that lack of time followed by inadequate funding and the level of student preparation were barriers to involving undergraduate students. Respondents who involved students in their work reported that the primary motivator was helping students, followed by helping to create the next generation of scholars within their fields. Of note, there were no significant differences among faculty by institution, tenure status, gender, or scholarly discipline.

Prunuske et al. 2013

The authors were interested in determining how mentors influence the development of a scientific identity in their undergraduate researchers. They conducted interviews with 15 faculty mentors at a Research 1 (R1) institution in the Midwest about mentoring undergraduate researchers from URM groups in both summer and academic year programs. Faculty reported that there was intrinsic value in working with undergraduate students and that students gained in confidence and aptitude as a result of URE. The study found that faculty members had varied ideas on what composes diversity; some identified individuals from minority groups, whereas others identified individuals who were first-generation college students, from rural areas, or of lower socioeconomic status (regardless of race or ethnicity). One dominant idea emerged from the interviews—faculty members believed that diversity did not affect the mentoring relationship. Faculty members were unaware of the factors that contributed to URM leaving STEM fields and were interested in learning more about what they were. They also were interested in learning how to be more effective mentors.

Schwartz 2012

The subjects of this study were faculty-student undergraduate research dyads at a public, urban, diverse college. The author used CHAT (cultural historical activity theory) as a theoretical framework. Survey responses, interviews, and “check-ins” were analyzed to learn what students and their mentors gained from the URE. Students reported similar gains to those described in other studies. Of note, however, were the faculty responses that involve the cost of mentoring undergraduate students, the impact of URE on publication rates, the methods used, and the questions asked. Faculty noted the benefits of new ways of looking at their research and the satisfaction of helping students find, and be successful on, their educational paths.

Impact on Institutions and Programs

The assessment of the impact of undergraduate research on institutions is challenging. For some institutions that have long required research experiences for all students, such as the College of Wooster, the story of undergraduate research becomes part of the institutional culture and the way it presents itself to prospective students. In many other situations, undergraduate research may be evident in part of a campus but not uniformly developed across the institution. Given the differential implementation of URE and CURE across disciplines, it is not surprising that there are few comprehensive studies of institutional impacts.

Carpi et al. 2017

John Jay College is a minority-serving institution that offers a comprehensive, iterative, undergraduate research experience in which students spend between one to three years working with a faculty member on original research and earn financial or educational credit for involvement. The students are involved in a broader peer community and have regular programmatic interactions with student support offices such as the writing and career centers. The authors surveyed 47 alums, interviewed faculty and student participants, analyzed programmatic data, and examined student artifacts. Students and faculty reported on the importance of mentorship to building skills and confidence in abilities. Student efficacy related to career development was aided by the experience of presenting work at off-campus venues, the networking opportunities conferences provided, and the experience of coauthoring peer-reviewed articles. Students who graduated from the program were significantly more likely than their peers to pursue postbaccalaureate study. Of note, the authors addressed how the program impacted the department. It was asserted that the changes in the job expectations of departmental new hires and in the department’s mission statement were directly attributable to an increased focus on faculty-student research collaborations. The department also experienced a significant increase in external funding in support of its research.

Ghee et al. 2014

The Leadership Alliance Program (LAP) has been in existence since 1993. LAP is a consortium of Ivy League and R1 institutions partnering with minority-serving institutions to provide summer research opportunities for URM across all academic disciplines (although the majority of students hail from and conduct projects in STEM). There are two components to the program: an 8- to 10-week summer research program (SR-EIP) and the annual Leadership Alliance National Symposium (LANS) that occurs at the conclusion of the summer. LANS provides the SR-EIP participants with an opportunity to disseminate their work to others within the LAP network, attend educational sessions about planning for postbaccalaureate opportunities, and network with past SR-EIP students who return as near-peer mentors to share their experiences in graduate and professional schools and employment. At the time of publication, more than 2,600 students participated in the SR-EIP, with equal numbers of male and female students. Data for this study came from post SR-EIP participant surveys, internal evaluation activities, and the tracking of SR-EIP students through educational programs. Participants found the SR-EIP to be very effective in helping them understand what it took to be a researcher and select and apply to graduate programs, as well as the varied career choices. Participants cited similar learning gains to those reported in other URE programs (critical thinking, writing, working in a group). Eighty-eight percent of participants completed their undergraduate degrees, with 53 percent of these participants pursuing and completing postgraduate study.

Harsh, Maltese, and Tai 2011

The authors were interested in determining what makes URE so powerful. They conducted interviews with more than 100 graduate students in chemistry and physics. They used the results of that qualitative study to develop a survey and invited 13,000 professional chemists and physicists who were at various stages of their careers to complete it. They used data from 3,014 of the respondents, 75 percent of whom were men. Twenty-four percent of respondents were faculty members, 26 percent were graduate students, and 35 percent of all respondents were employed in STEM fields. Nearly half of respondents said the biggest benefit of the URE was doing genuine scientific research; they reported that their research experiences helped to build skills related to using laboratory equipment and led to gains in self-confidence. Approximately 5 percent of respondents had negative URE experiences, with the narratives of these respondents citing the same two themes: they either worked in a major research laboratory where the principal investigator passed them on to untrained graduate students or they were not given any independence on their projects. Respondents reported that a direct relationship with a research mentor and meaningful work were the two most important elements of a positive research experience.

Junge et al. 2010

This article examined the first 15 years of the 10-week, full-time, residential summer research program at Emory University. The authors had four sources of data: transcript analysis, two post-program participant surveys, and post-program career pursuits. During the 15-year period examined, 320 faculty mentored 822 students from more than 200 colleges and universities. Nearly all participants were rising third- or fourth-year students. The transcript analysis revealed that Emory STEM students involved in the URE took more high-level STEM courses and had higher GPAs than Emory STEM students who did not participate. Two hundred and fifty participants responded to the survey sent out for this study. Respondents indicated that they experienced the greatest growth in skills related to presenting their work to others and preparing for graduate school interviews. Participants also reported significant growth in skills associated with research design, data collection, and analysis. Forty percent of participants later pursued graduate training. Of those who completed at least a master’s degree, approximately 85 percent were in a STEM field, which mirrored the employment focus of participants who reported that they were employed. Nearly 50 percent of participants indicated that they were employed full-time, with a significant majority (77 percent) reporting that they were satisfied with their employment. Of particular note, students were less likely to want to pursue a K–12 teaching position as a result of participation in the program.

Rodenbusch et al. 2016

This article begins with a thorough introduction to the evolution and assessment of CURE in introductory biology using two national models: the HHMI Phage and the Genomics Education programs. The authors then explain the Freshman Research Initiative (FRI), in which first-year students are introduced to research and complete two other structured CURE in a subdiscipline. FRI has been in existence since 2005; the study used a subsample of nearly 5,000 students of the 75,000 who had completed FRI. A second subsample consisted of the records of 2,600 students who had not participated in FRI. Care was taken to match demographics that have been shown to influence persistence in STEM. Results showed that students who completed FRI were more likely to earn a STEM degree (94 percent) than students who started but did not complete the three-semester sequences or non-FRI students (71 percent). FRI students were more likely to graduate in six years (83 percent) than non-FRI students (66 percent). Gender, race/ethnicity, and first-generation status did not differentiate between the FRI and non-FRI groups.

Zimbardi and Myatt 2014

Standalone undergraduate research experiences have been shown to benefit students, but the scalability is limited. Throughout the world educators are working to make undergraduate research experiences equitable and accessible by implementing CURE. Using existing broad-based frameworks, the authors developed one-page descriptive summaries of undergraduate research and distributed them to 130 individuals across 28 schools in Australia. More than 90 percent of the undergraduate research investigations reported were embedded within coursework and open to all students, and 83 percent involved a literature review of work within the student’s field of study. Seventy-percent of fields required students to collect data, and more than 80 percent required analysis. In approximately 10 percent of fields, students analyzed existing data rather than generating their own. Just as many schools reported students working in an apprenticeship as reported students working in an industry-provided model. All students were required to communicate results of their work, with the majority of students communicating their findings in final research papers.

Current Status

Although much progress has been made in the assessment of URE, and there has been development and analysis of CURE over the past decade, there remain gaps and limitations in current understanding. First, it is believed that the process of students self-selecting into stand-alone summer and academic year–long UREs is systemic and foundational. Thus, it is unlikely that “student self-selection into URE” biased results will be overcome. Second, although a great deal of assessment of URE relies on self-reported responses from students, a fair number of studies are expanding the lines of evidence to include independently evaluated final posters and papers, mentor surveys, and institutional data regarding the GPA, retention within a major, progress toward graduation, and graduation rates of undergraduate researchers. It is recognized that longer-term longitudinal tracking is difficult and time consuming. Third, it is suggested that many factors contribute to the outcomes of URE, such as student motivation, self-efficacy, the design of the space, the timing and frequency of student interactions with their research mentors, and the presence of peers. Students applying for prestigious scholarships, including the Graduate Research Fellowship Program of the NSF, most often have their applications enriched by strong research experiences, signaling another incompletely documented benefit to students from their undergraduate research experience. Some of these factors are beginning to be investigated and will lead to further insights into this important high-impact practice.

The assessment of CURE potentially removes the self-selection bias, and published assessment in this area includes indicators of student success other than student self-reported surveys, such as performance on exams, GPA, and retention within a STEM major. What remains missing is robust assessment of mid- and upper-division CURE and capstone courses.

There is an incomplete picture of how URE impacts a faculty member’s career or direction of inquiry. The recent attention to mentoring, the importance of professional development for faculty who serve as mentors, and the reasons faculty involve undergraduate students in their scholarship is encouraging. There are a number of areas ripe for study. These include the impact of UREs on scholarly activity (types and rates of publication, success in promotion and tenure); the impact of engaging with undergraduate students on the mentor’s attitudes and behaviors; the way in which a faculty member’s mentoring types and styles has changed with time; the effect of an organized summer URE on the types of work undertaken in a URE; and measurement of how targeted faculty development affected the structure of URE and CURE.

There also is inadequate information on how robust implementation of URE and CURE impacts institutions. Although the Campus-Wide Award for Undergraduate Research Accomplishments (AURA) given by CUR brings awareness to institutions that have successfully executed components of the COEUR road map, the applications for those awards are not public. It would be worthwhile to have narratives of how campus-wide URE programs have affected recruitment, retention, graduation rates, changes in student demographics, alumni giving rates, and fundraising that supports URE. It would be interesting to learn how robust undergraduate research programs have affected an institution’s culture, practices, and climate.

The timing is prime for some institutional studies to emerge, as URE and CURE have spread to business, education, social sciences, the humanities, and other fields. Campuses with central offices supporting undergraduate research should have an advantage when gathering data, and there are some good tools that can guide planning for evaluation and assessment (Brown, Lewis, and Bevan 2016; Della-Piana, Della-Piana, and Gardner 2014; Engström 2015; Rorrer 2016). Several URE consortia are emerging (Blockus 2016) that have the ability to implement comprehensive assessment studies, as evidenced by the Meyerhoff Scholars group (Domingo et al. 2019). Given the long history of support for URE by the Goldwater Scholarship program, the National Science Foundation, and the National Institutes of Health (as well as other federal programs), it is recommended that special longitudinal studies be undertaken on the outcomes of these long-standing programs (Beninson et al. 2011).

The efforts made over the past decade to inform the undergraduate research community by paying additional attention to the assessment and documentation of learning outcomes and other impacts of URE and CURE are laudable. Although the full impact of undergraduate research may be unknown, much more is understood about its many positive effects than was the case a decade ago. Refinement of the story of the outcomes of undergraduate research on students, mentors, and campuses should continue.

References

Aikens, Melissa L., Sona Sadselia, Keiana Watkins, Mara Evans, Lillian T. Eby, and Erin L. Dolan. 2016. “A Social Capital Perspective on the Mentoring of Undergraduate Life Science Researchers: An Empirical Study of Undergraduate-Postgraduate-Faculty Triads.” CBE–Life Sciences Education 15(2): ar16. doi: 10.1187/cbe.15-10-0208

Auchincloss, Lisa C., Sandra. L. Laursen, Janet L. Branchaw, Kevin Eagan, Mark Graham, David I. Hanauer, Gwendolyn Lawrie et al. 2014. “Assessment of Course-Based Undergraduate Research Experiences: A Meeting Report.” CBE–Life Sciences Education 13: 29–40. doi: 10.1187/cbe.14-01-0004

Beninson, Lida, Jessica Koski, Erika Villa, Ronnie Faram, and Sally E. O’Connor. 2011. “Evaluation of the Research Experiences for Undergraduate (REU) Sites Program.” CUR Quarterly 32(1): 43–48.

Bhattacharyya, Prajukti, Catherine Chan, and Meg Waraczynski. 2018. “How Novice Researchers See Themselves Grow.” International Journal for the Scholarship of Teaching and Learning 12(2): art. 3. doi: 10.20429/ijsotl.2018.120203

Blockus, Linda. 2016. Strengthening Research Experiences for Undergraduate STEM Students: The Co-Curricular Model of the Research Experience. Washington, DC: National Academies of Sciences, Engineering, and Medicine.

Boyer Commission on Educating Undergraduates in the Research University. 1998. Reinventing Undergraduate Education: A Blueprint for America’s Research Universities. Stony Brook: State University of New York, Stony Brook.

Brown, Anne M., Stephanie N. Lewis, and David R. Bevan. 2016. “Development of a Structured Undergraduate Research Experience: Framework and Implications.” Biochemistry and Molecular Biology Education 44: 463–474. doi: 10.1002/bmb.20975

Carpi, Anthony, Darcy M. Ronan, Heather M. Falconer, and Nathan H. Lents. 2017. “Cultivating Minority Scientists: Undergraduate Research Increases Self-Efficacy and Career Ambitions for Underrepresented Students in STEM.” Journal of Research in Science Teaching 54: 169–194. doi: 10.1002/tea.21341

Council on Undergraduate Research (CUR). n.d. “Bibliography Catalogs.” Accessed November 26, 2019. https://www.cur.org/resources/faculty/bibliographies

Crowe, Mary, and David Brakke. 2008. “Assessing the Impact of Undergraduate Research Experiences on Students: An Overview of the Current Literature.” CUR Quarterly 28(1): 43–50.

DeFeitas, Stacie C., and Antonio Bravo. 2012. “The Influence of Involvement with Faculty and Mentoring on the Self-Efficacy and Academic Achievement of African American and Latino College Students.” Journal of the Scholarship of Teaching and Learning ١٢(٤): 1–11.

Della-Piana, Gabriel M., Connie K. Della-Piana, and Michael K. Gardner. 2014. Evaluating the Undergraduate Research Experience: A Guide for Program Directors and Principal Investigators. Charlotte, NC: Information Age.

Domingo, Mariano R. Sto., Starlette Sharp, Amy Freeman, Thomas Freeman Jr., Keith Harmon, Mitsue Wiggs, Viji Sathy et al. 2019. “Replicating Meyerhoff for Inclusive Excellence in STEM: Undergraduate Diversity Is Fostered across Many Contexts.” Science 364: 335–337. doi: 10.1126/science.aar5540

Dvorak, Abbey L., and Eugenia Hernandez-Ruiz. 2019. “Outcomes of a Course-Based Undergraduate Research Experience (CURE) for Music Therapy and Music Education Students.” Journal of Music Theory ٥٦: 30–60. doi: 10.1093/jmt/thy020

Engström, Henrik. 2015. “A Model for Conducting and Assessing Interdisciplinary Undergraduate Dissertations.” Assessment & Evaluation in Higher Education 40: 725–739. doi: 10.1080/02602938.2014.950552

Foster, Stephanie L., and Bethany M. Usher. 2018. Comparing Two Models of Undergraduate Research Using the OSCAR Student Survey. Scholarship and Practice of Undergraduate Research 1(3): 30–39. doi: 10.18833/spur/1/3/6

Frantz, Kyle J., Robert L. DeHaan, Melissa K. Demetrikopoulos, and Laura L. Carruth. 2006. “Routes to Research for Novice Undergraduate Neuroscientists.” CBE–Life Sciences Education 5: 175–187. doi: 10.1187/cbe.05-09-0119

Gentile, James, Kerry Brenner, and Amy Stephens. 2017. Undergraduate Research Experiences for STEM Students: Successes, Challenges, and Opportunities. Washington, DC: National Academies Press.

Ghee, Medeva, Deborah Collins, Valeria Wilson, and Willie Pearson Jr. 2014. “The Leadership Alliance: Twenty Years of Developing a Diverse Research Workforce.” Peabody Journal of Education 89: 347–367. doi: 10.1080/0161956x.2014.913448

Gourley, Bridget L., and Rebecca M. Jones (Eds.). 2018. Best Practices for Supporting and Expanding Undergraduate Research in Chemistry. Washington, DC: American Chemical Society. doi: 10.1021/bk-2018-1275

Haeger, Heather, and Carla Fresquez. 2016. “Mentoring for Inclusion: The Impact of Mentoring on Undergraduate Researchers in the Sciences.” CBE–Life Sciences Education 15(3): ar36. doi: 10.1187/cbe.16-01-0016

Hall, Eric E., Helen Walkington, Jenny Olin Shanahan, Elizabeth Ackley, and Kearsley A. Stewart. 2018. “Mentor Perspectives on the Place of Undergraduate Research Mentoring in Academic Identity and Career Development: An Analysis of Award-Winning Mentors.” International Journal for Academic Development 23: 15–27. doi: 10.1080/1360144x.2017.1412972

Harsh, Joseph A., Adam V. Maltese, and Robert H. Tai. 2011. “Undergraduate Research Experiences from a Longitudinal Perspective.” Journal of College Science Teaching 41(1): 84–91.

Hayward, Charles N., Sandra L. Laursen, and Heather Thiry. 2017. “Why Work with Undergraduate Researchers? Difference in Research Advisors’ Motivations and Outcomes by Career Stage.” CBE–Life Sciences Education 16(1): ar13. doi: 10.1187/cbe.16-07-0229

Hensel, Nancy (Ed.). 2012. Characteristics of Excellence in Undergraduate Research (COEUR). Washington, DC: Council on Undergraduate Research.

Hensel, Nancy. 2018. Course-Based Undergraduate Research: Educational Equity and High-Impact Practice. Sterling, VA: Stylus.

Hurtado, Sylvia, Christopher B. Newman, Minh C. Tran, and Mitchell J. Chang. 2010. “Improving the Rate of Success for Underrepresented Racial Minorities in STEM Fields: Insights from a National Project.” In “Students of Color in STEM,” ed. Shaun R. Harper and Christopher B. Newman, spec. issue, New Directions for Institutional Research, 148: 5–15. doi: 10.1002/ir.357

Hurtado, Sylvia, Nolan L. Cabrera, Monica H. Lin, Lucy Arellano, and Lorelle L. Espinosa. 2009. “Diversifying Science: Underrepresented Student Experiences in Structured Research Programs.” Research in Higher Education 50: 189–214. doi: 10.1007/s11162-008-9114-7

Jordan, Tuajuanda C., Sandra H. Burnett, Susan Carson, Steven M. Caruso, Kari Clase, Randall J. DeJong, John J. Dennehy, et al. 2014. “A Broadly Implementable Research Course in Phage Discovery and Genomics for First-Year Undergraduate Students.” mBio 5(1): e01051-13. doi: 10.1128/mbio.01051-13

Junge, Benjamin, Catherine Quiñones, Jakub Kakietek, Daniel Teodorescu, and Pat Marsteller. 2010. “Promoting Undergraduate Interest, Preparedness, and Professional Pursuit in the Sciences: An Outcomes Evaluation of the SURE Program at Emory University.” CBE–Life Sciences Education 9: 119–132. doi: 10.1187/cbe.09-08-0057

Kinner, David, and Mark Lord. 2018. “Student-Perceived Gains in Collaborative, Course-Based Undergraduate Research Experiences in the Geosciences.” Journal of College Science Teaching 48(2): 48–58.

Komarraju, Meera, Sergey Musulkin, and Gargi Bhattacharya. 2010. “Role of Student-Faculty Interactions in Developing College Students’ Academic Self-Concept, Motivation and Achievement.” Journal of College Student Development ٥١: 332–342. doi: 10.1353/csd.0.0137

Lima, Victor, and Grace Tsiang. 2017. “Training Young Researchers: Successful Strategies from University of Chicago College Economics.” Journal of Economic Education 48: 310–316. doi: 10.1080/00220485.2017.1353467

Linn, Marcia C., Erin Palmer, Anne Baranger, Elizabeth Gerard, and Elisa Stone. 2015. “Undergraduate Research Experiences: Impacts and Opportunities.” Science 347: 1261757. doi: 10.1126/science.1261757

Lopatto, David. 2010. Science in Solution: The Impact of Undergraduate Research on Student Learning. Washington, DC: Council on Undergraduate Research and Research Corp.

Mahatmya, Duhita, Janet Morrison, Rebecca M. Jones, Pamela W. Garner, Shannon N. Davis, Jill Manske, Nancy Berner, Ann Johnson, and Jayna Ditty. 2017. “Pathways to Undergraduate Research Experiences: A Multi-Institutional Study.” Innovative Higher Education 42: 491–504.

McNary-Zak, Bernadette, and Rebecca Todd Peters. 2011. Teaching Undergraduate Research in Religious Studies. New York: Oxford University Press.

Morrison, Janet A., Nancy J. Berner, Jill M. Manske, Rebecca M. Jones, Shannon N. Davis, and Pamela W. Garner. 2018. “Surveying Faculty Perspectives on Undergraduate Research, Scholarship, and Creative Activity: A Three-Institution Study.” Scholarship and Practice of Undergraduate Research 2(1): 43–54. doi: 10.18833/spur/2/1/1

Mraz-Craig, Jennifer A., Kristy L. Daniel, Carrie J. Bucklin, Chandrani Mishra, Laila Ali, and Kari L. Clase. 2018. “Student Identities in Authentic Course-Based Undergraduate Research Experience.” Journal of College Science Teaching 48(1): 68–75.

Parker, Jonathan. 2018. “Undergraduate Research, Learning Gain and Equity: The Impact of Final Year Research Projects.” Higher Education Pedagogies 3: 145–157. doi: 10.1080/23752696.2018.1425097

Portillo, Shannon, Danielle S. Rudes, Lincoln B. Sloas, Kirsten Hutzell, and Paula Salamoun. 2013. “Students as Scholars: Integrating Independent Research into Undergraduate Education.” Journal of Criminal Justice Education 24: 68–96. doi: 10.1080/10511253.2012.655750

Prunuske, Amy J., Janelle Wilson, Melissa Walls, and Benjamin Clarke. 2013. “Experiences of Mentors Training Underrepresented Undergraduates in the Research Laboratory.” CBE–Life Sciences Education 12: 403–409. doi: 10.1187/cbe.13-02-0043

Robnett, Rachael D., Paul A. Nelson, Eileen L. Zurbriggen, Faye J. Crosby, and Martin M. Chemers. 2018. “Research Mentoring and Scientists’ Identity: Insights from Undergraduates and Their Mentors.” International Journal of STEM Education 5: art. 41. doi: 10.1186/s40594-018-0139-y

Rodenbusch, Stacia, Paul R. Hernandez, Sarah L. Simmons, and Erin L. Dolan. 2016. “Early Engagement in Course-Based Research Increases Graduation Rates and Completion of Science, Engineering and Mathematics Degrees.” CBE–Life Sciences Education 15(2): ar20. doi: 10.1187/cbe.16-03-0117

Rorrer, Audrey S. 2016. “An Evaluation Capacity Building Toolkit for Principal Investigators of Undergraduate Research Experiences: A Demonstration of Transforming Theory into Practice.” Evaluation and Program Planning 55: 103–111. doi: 10.1016/j.evalprogplan.2015.12.006

Schaffer, Christopher D., Consuelo Alvarez, Cheryl Bailey, Daron Barnard, Satish Bhalla, Chitra Chandrasekaran, Vidya Chandrasekaran, et al. 2010. “The Genomics Education Partnership: Successful Integration of Research into Laboratory Classes at a Diverse Group of Undergraduate Institutions.” CBE–Life Sciences Education 9: 55–69. doi: 10.1187/09-11-0087

Schwartz, Joni. 2012. “Faculty as Undergraduate Research Mentors for Students of Color: Taking into Account the Costs.” Science Education 96: 527–542. doi: 10.1002/sce.21004

Shanahan, Jenny Olin, Elizabeth Ackley-Holbrook, Eric Hall, Kearsley Stewart, and Helen Walkington. 2015. “Ten Salient Practices of Undergraduate Research Mentors: A Review of the Literature.” Mentoring & Tutoring: Partnership in Learning 23: 359–376. doi: 10.1080/13611267.2015.1126162

Slovacek, Simeon P., Jonathan C. Whittinghill, Susan Tucker, Kenneth A. Rath, Alan R. Peterfreund, Glenn D. Kuehn, and Yvonne G. Reinke. 2011. “Minority Students Severely Underrepresented in Science, Technology, Engineering and Math.” Journal of STEM Education 12(1–2): 25–36.

Stanford, Jennifer S., Suzanne E. Rocheleau, Kevin P. W. Smith, and Jaya Mohan. 2017. “Early Undergraduate Research Experiences Lead to Similar Learning Gains for STEM and Non-STEM Undergraduates.” Studies in Higher Education 42: 115–129. doi: 10.1080/03075079.2015.1035248

Sternquist, Brenda, Patricia Huddleston, and Ann Fairhurst. 2018. “Framing the Undergraduate Research Experience: Discovery Involvement in Retailing Undergraduate Education.” Journal of Marketing Education 40: 76–84. doi: 10.1177/0273475317753864

Vandermass-Peeler, Maureen, Paul C. Miller, and Jessie L. Moore (Eds.). 2018. Excellence in Mentoring Undergraduate Research. Washington, DC: Council on Undergraduate Research.

Whitmeyer, Steven J., David W. Mogk, and Eric J. Pyle. 2009. Field Geology Education: Historical Perspectives and Modern Approaches, Special Paper 461. Boulder, CO: Geological Society of America. doi: 10.1130/spe461

Zimbardi, Kirsten, and Paula Myatt. 2014. “Embedding Undergraduate Research Experiences within the Curriculum: A Cross-Disciplinary Study of the Key Characteristics Guiding Implementation.” Studies in Higher Education 39: 233–250. doi: 10.1080/03075079.2011.651448

Mary Crowe

Florida Southern College, mcrowe@flsouthern.edu

Mary Crowe is the associate provost of experiential education and oversees high-impact practices such as study abroad, internship, and undergraduate research programs at Florida Southern College. She is a past president of the Council on Undergraduate Research and has extensive experience evaluating and assessing undergraduate research programs. Crowe is a behavioral ecologist by training and has had multiple extramural grants in support of research experiences for undergraduate students.

David F. Brakke is a limnologist/ecologist who served as dean of the College of Science and Mathematics at James Madison University from 1999 to 2016. He served over that same period as a CUR councilor. Brakke is a professor emeritus of the Department of Biology and the Department of Geology and Environmental Science. He is an elected fellow of the American Association for the Advancement of Science and a recipient of other awards for teaching and research from campuses, student organizations, and federal agencies.