Full NSF Grant Description

Collaborative Research: Teaching for PROWESS (TfP): Increasing Student Success in Community College Mathematics through Facilitating Systemic Instructional Change

INTRODUCTION

The Teaching for PROWESS (PRoficiency, OWnership, Engagement, Student Success) project is situated within the Institutional and Community Transformation track of the NSF-IUSE program (Level 2) with the goal of transforming instruction to increase student success in community college mathematics. The project leverages the American Mathematical Association of Two-Year College’s (AMATYC, 2018) Improving Mathematical PROWESS And College Teaching (IMPACT) framework for organizing department change and the active learning principles promoted by Laursen and Rasmussen (2019) and the NSF-funded Student Engagement in Mathematics through an Institutional Network for Active Learning (SEMINAL) project (DUE-1624610). We envision a vibrant community of mathematics faculty, department chairs, faculty developers, and mathematics education researchers who will use AMATYC’s IMPACT framework to shift instructional practices to promote the 4 active learning guiding principles and IMPACT’s 4 pillars of PROWESS (PRoficiency, OWnership, Engagement, and Student Success): (1) students’ deep engagement in mathematical thinking (PRoficiency), (2) instructors’ interest in and use of student thinking (OWnership), (3) student-to-student interaction (Engagement), and (4) instructors’ attention to equitable and inclusive practices (Student Success). We envision a project that will first investigate two institution’s implementations of active learning in their mathematics courses (Phase 1, Years 1-3) – Chandler-Gilbert Community College (AZ) and Clackamas Community College (OR) – and these institutions will serve as case studies for Phase 2 (Years 3-5). The two Phase 1 colleges were selected based on their current and past work in improving mathematics teaching and learning in courses on the STEM pathway, as well as for their significantly different institutional contexts. Located in the Maricopa County Community College District, Chandler-Gilbert Community College serves over 15,000 students and has a track record of innovation towards developing courses which embrace active learning and problem-based curriculum. Chandler-Gilbert will focus their work in the TfP project on faculty development (including adjunct faculty) to connect the curriculum with high-impact instructional practices to support active learning. Located just outside Portland, Clackamas Community College is a research-friendly college that places a high value on faculty development, continuous improvement, and scholarship. Over the past decade, Clackamas has engaged in projects focused on engaging students in deep mathematical thinking and will leverage the TfP project to redesign courses in the STEM pathway to more actively engage students. The combination of Chandler-Gilbert’s focus on faculty development with Clackamas’s focus on curriculum development provide a unique opportunity to investigate these colleges as case studies for Phase 2 institutions. In Phase 2, we will select an additional 6 community colleges across the nation based on a variety of institutional contexts and will provide them the opportunity to propose their own tailored three-year plan for advancing their mathematics department’s commitment to increasing student success by embracing the 4 active learning guiding principles. This project is grounded on a theory of change which posits that a community of site-based IMPACT Teams (Phase 1 and 2 colleges) engaged in researcher-practitioner partnerships will serve as the lever for department-level change to support implementation of the active learning guiding principles. Each IMPACT Team will include mathematics faculty, an administrator, and either an institutional researcher or faculty developer. Using a design-based approach towards planning, enacting, and revising both course and department routines will lead to systemic change necessary for transforming students’ experiences in community college mathematics (Cobb, Jackson, Henrick & Smith, 2018).

The TfP project models its activities on three primary NSF-funded initiatives in education that focus on systemic change and leadership development: Middle-school Mathematics and the Institutional Setting of Teaching (MIST, ESI-0554535 and DRL-1119122), SEMINAL (DUE-1624610), and Partnership for Undergraduate Life Sciences Education (PULSE, DUE-1624182). The MIST project is highlighted as one of the first examples of design-based implementation research (DBIR) (Penuel, Fishman, Haugen Cheng & Sabelli, 2011) and informs the design of TfP as a researcher-practitioner partnership that uses findings from iterative cycles of data collection and analysis as the foundation for each IMPACT Team’s plan for change. The SEMINAL project informed the research focus and structure of TfP through using the framework of the 4 active learning guiding principles to anchor each IMPACT Team’s project and engaging in cross-institutional site visits to collect and analyze data in DBIR cycles that will be used to integrate findings among all IMPACT Teams. TfP research focuses on both the classroom and organizational levels, which we believe is critical to understanding the complexities in improving instructional practices in community colleges. The PULSE project embraced leadership development in creating ambassadors with a vision for change to be used as levers for departmental-level transformation (Goldey, 2014). This element of leadership development is woven into the fabric of the TfP project by creating ambassadors-for-change within the IMPACT Teams. 

VISION AND GOALS

We envision a community of mathematics educators committed to fostering a culture in which faculty are encouraged, supported, and expected to promote systemic change in the teaching and learning of mathematics in community colleges through the implementation of the 4 active learning guiding principles and influenced by AMATYC’s IMPACT. The project will attend to the need for faculty to collaborate within their institutions to guide their project, as well as with other IMPACT Teams in an effort of continuous improvement. Our 5-year program involves 3 key components:

• Content and Pedagogical Knowledge Development Activities: hosting IMPACT Summer Institutes and IMPACT Workshops, as well as providing on-going faculty training.
• Building Community Engagement Activities: engaging in the my.AMATYC Community (an online community, which facilitates the sharing of ideas and findings, hosted on AMATYC’s my.amatyc.org website) and building researcher-practitioner partnerships.
• Research Activities: investigating the effects of the project on key metrics of student success, as well as studying the impact of the two components above.

Each of these components will contribute to the following overall project goals:

• Goal 1:Focusing on the student and instructional level, we aim to improve STEM learning and learning environments at 8 colleges by transforming instructional strategies through leveraging the work of IMPACT Teams that promote the 4 active learning guiding principles, along with the four PROWESS pillars of AMATYC’s IMPACT. 
• Goal 2: Focusing on the departmental and institutional level, we aim to stimulate iterative and sustained changes by building capacity for conducting research and evidence-based decision making.
• Goal 3: Focusing on knowledge generation, we aim to understand change at three levels (classroom, departmental, and organizational) and how the changes have reciprocal effects at other levels.

MATHEMATICS AS A BARRIER TO BROADENING PARTICIPATION IN STEM

In 2012, the President’s Council of Advisors on Science and Technology (PCAST) (Olson & Riordan, 2012) issued a report titled Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics. This report not only identified the national challenges for supporting students through the STEM pipeline in post-secondary education, but it specifically called out mathematics as the primary barrier preventing students from pursuing STEM degrees. Further, to the immense chagrin of the mathematics community, the report also suggested that “faculty from mathematics-intensive disciplines other than mathematics” should serve as course developers and instructors for college-level mathematics as a means for addressing the barrier to success in collegiate mathematics (p. 30). This report led to several calls for action within the mathematics community. The Mathematical Sciences in 2025 report stated that the PCAST report “should be viewed as a wake-up call for the mathematical sciences community” and called for “a community-wide effort” for a deep re-thinking of the mathematical content in our courses and the instructional methods largely employed in higher education” (NRC, p. 123). To facilitate such change, the report also stated that “the professional societies should work cooperatively to spark this” (p. 123). As a result, we have partnered with AMATYC, the primary mathematics organization for community colleges, to lead the way for supporting instructional transformation in mathematics. The four pillars of IMPACT illustrate the foci for improving mathematics education in the first two years of college, and they will serve as the foundation for which the TfP project aims to facilitate institutional and departmental transformation within colleges by focusing on instructional shifts that leverage the 4 active learning guiding principles..

The Mathematical Association of America (MAA) report titled A Common Vision for Undergraduate Mathematical Sciences Programs in 2025 (Saxe & Braddy, 2015) advocated for institutional change in the way we approach and teach post-secondary mathematics by arguing that: (1) the status quo is unacceptable and (2) the teaching of mathematics should include less traditional lecture and more active learning techniques. These reports ultimately culminated in the active learning statement endorsed by 15 mathematics and statistics professional societies (Conference Board of Mathematical Sciences, 2016). This statement strongly advocates for institutions of higher education, funding agencies, and all stakeholders to implement active learning in post-secondary mathematics.

Although engaging students actively has been shown to lead to significant gains in student achievement (Freeman et al., 2014), the primary instructional strategy implemented in post-secondary education remains as lecture. The 4 active learning guiding principles advocated by Laursen and Rasmussen (2019) require significant and purposeful work by the instructor to implement. Research in mathematics education has supported the need for students to engage deeply in mathematical reasoning (Principle 1) as a means for gaining understanding of the content (Oehrtman, Carlson & Thompson, 2008; Rasmussen, Zandieh, King & Teppo, 2005). Focusing on the nature and cognitive-level of the tasks in which students engage requires that instructors make purposeful choices about which tasks to use as anchors during instruction and outside of the classroom. There has been significant research that demonstrates the power of high-level cognitive demanding tasks on student learning in mathematics, even when students are assessed using standardized tests (e.g., Boaler, 2002), yet other research has revealed how challenging it is for instructors to increase and maintain higher-level cognitively demanding tasks (Stein, Grover & Henningsen, 1996). Facilitating student-to-student interaction requires significant mathematical knowledge for teaching (e.g., Speer & Wagner, 2009) and is an important component to culturally relevant pedagogy (Ladson-Billings, 1995) and culturally responsive teaching (Gay, 2002).

Continuing with the notion of equitable and inclusive teaching, primary is the need for educators to undergo a perspective change. Recently published MAA recommendations for instructional practices (MAA, 2018) emphasize the importance of adopting an anti-deficit perspective (Adiredja, 2018) and acknowledging the role of power and identity in the mathematics classroom (Gutierrez, 2009). Researchers have found that the implementation of active learning practices vary widely and that doing so does not always lead to equitable learning outcomes (Brown, 2018; Johnson et al., In press). We contend that a purposeful approach to equitable and inclusive teaching must involve active learning at the core of instruction. We believe that the community college setting is well suited as a learning laboratory for faculty to embrace and implement the active learning guiding principles. 

WHY COMMUNITY COLLEGE MATHEMATICS

U.S. community colleges play a vital role for providing educational opportunities for students beyond high school and for developing students to enter the workforce with the skills needed in today’s society. In Fall 2015, there were over six million full-time or part-time students enrolled in community colleges, which accounted for 43% of all undergraduate students enrolled in public U.S. institutions of higher education (Snyder, de Brey, & Dillow, 2018). The 2017 American Association of Community Colleges report highlighted the diversity of community college students: 56% of all community college students are female, 36% are first generation students, 17% are single parents, 62% are enrolled part-time, and 63% hold full-time jobs. Students who attend community colleges are certainly highly diverse and have a wide variety of reasons for attending these institutions, which include obtaining an associate’s degree or certificate, completing college credits for university transfer, or simply acquiring new knowledge and skills for re-tooling and re-entry into the workforce. Unfortunately, when we consider completion of a bachelor degree as a goal for many students who attend a community college, the data show that less than 16% of students who started at a community college in 2012 completed a degree at a four-year institution within six years (Jaggars & Stacey, 2014). Mathematics is widely viewed as one of the primary barriers that contributes to this abysmal completion rate, yet community colleges play a significant role in broadening participation in STEM since learners who have not been provided opportunities to engage in meaningful learning opportunities and/or have been academically and socially marginalized in school are more likely to arrive on community college campuses as they benefit from the open-access admission process instituted in community colleges. For example, Van Noy and Zeidenberg (2014) found that STEM majors at community colleges tend to be older, part-time and first-generation students in comparison to their counterparts at four-year colleges and universities.

The most prolonged and pervasive challenge that community colleges are facing relative to mathematics is the extremely low student success, persistence, and throughput rates, where courses unfortunately serve more as leaky pipes than primed pumps in preparing and advancing students through mathematics. Blair, Kirkman, and Maxwell (2018) found that almost 41% of the 6 million community college students were enrolled in developmental mathematics courses. When researching student success in developmental mathematics at community colleges, Jaggars and Stacey (2014) shockingly found that only 11% of the 63,650 students in their study successfully passed the college-level introductory algebra (gatekeeper) course after completing three or more developmental mathematics courses (see Figure 1). This extremely low throughput rate is exacerbated by the low persistence rate of students exiting the three-course developmental sequence, where almost 25% failed to persist to the college-level gatekeeper course. Figure 1 below illustrates this leaky pipeline of developmental mathematics students. The lower the initial developmental mathematics course placement, the less likely it is for the student to eventually pass the college-level gatekeeper mathematics course. 

Figure 1: Student Progression Data Through Developmental Mathematics (Jaggars & Stacey, 2014).

Students in the first two years of college often attempt one course and not others, or do not take mathematics courses at all, which minimizes the chance for students to obtain a degree or certificate, or transfer to a four-year institution (Mills, 2016). With the vast majority of students falling through these pipeline crevasses, it is clear that mathematics represents an overwhelming barrier to degree completion, especially for students aiming to obtain a degree from a four-year institution, and contributes to a lack of persistence in STEM disciplines. This challenge fueled significant work, and met with impressive results, by the University of Texas Dana Center and the Carnegie/West Ed partnership to develop quantitative and statistical reasoning pathways to serve as more career-related pathways for non-STEM students (Sowers & Yamada, 2015). Despite efforts to also improve the STEM pathway, there is still much work to be completed in improving pathways and learning experiences for STEM students, specifically in mathematics, given that curricula and instructional practices that rely heavily on traditional approaches remain mainstream in community college mathematics (Cawley et al., 2018; Mali et al., 2019). 

THEORY OF CHANGE: LITERATURE REVIEW

This project is grounded on a theory of change which posits that a community of IMPACT Teams engaged in researcher-practitioner partnerships will be a lever for department-level change where the focus is on improving instructional practices to support active learning in mathematics. Using a design-based approach to planning, enacting, and revising both course and department routines will lead to the systemic change necessary to transform learners’ experiences in community college mathematics classrooms. Working within the my.AMATYC Community, both IMPACT Teams and TfP researchers will come to understand how changes in instruction and department routines and cultures are different across institutional contexts. This work will be driven by evidence and the 4 active learning guiding principles, thus leading to a transformed experience that improves all students’ success in mathematics by emphasizing proficiency, ownership, and student engagement of mathematics. We hold two perspectives on a theory of change: (1) a collection of change theories that support why we think this will work and (2) an explicit relationship between activities and outcomes. We address the first perspective here. We address the second perspective after the description of project activities in the next section.

The TfP project focuses on departments, since departments play a significant role in changes to instructional practice (Austin, 2011; Bressoud, Mesa & Rasmussen, 2015; Corbo, Rienholz, Dancy, Deetz & Finkelstein, 2016). Findings from a national study of successful college calculus programs (Bressoud, Mesa & Rasmussen, 2015) led to 7 recommendations for mathematics departments. One of these recommendations is to use student-centered pedagogies and active learning strategies — the goal of TfP, but 4 of the other recommendations are within the routines in which departments engage: attending to the effectiveness of placement; the design of challenging and engaging courses; coordination of instruction; and the regular use of local data to guide curricular and structural modifications. Austin (2011) highlights the importance of departments to the work of faculty, since department routines also include assigning faculty to courses, the evaluation of teaching, how content and courses are sequenced, and monitors the relationship of specific courses to others in the curriculum. Finally, the role of creating a common vision is essential for change (Quardokus Fisher & Henderson, 2018), whether this vision is articulated by leadership at the start of a change initiative or emerges from ongoing work. Developing a shared vision for mathematics teaching and learning is a task well suited to departments (Corbo et al., 2016).

Change efforts require both informal and formal leadership, which influenced our conception of the IMPACT Teams. Using social network analysis, Apkarian & Rasmussen (2017) identified instructional leaders at institutions that have successful college calculus programs and found that instructional leaders (that is, people who are able to influence instructional practices) included both informal and formal leaders. Quardokus Fisher & Henderson (2018) used two change frameworks to identify successes and missed opportunities within a STEM education initiative they studied. Change is initiated by formal leaders in one framework, but the second framework includes formal leaders as change agents, and also adaptive and enabling leaders, who could hold any position within an institution. We expect that the role of leadership — and who emerges as instructional leaders on each IMPACT Team — will vary across teams. This variety will be a source of learning for IMPACT Teams as they reflect on which strategies are best for reaching their shared vision. 

We adopt the approach of Design-Based Implementation Research (DBIR) both as a way to frame our research (see below) and but also as a change theory that guided the design of TfP. There are four principles of DBIR: (1) a focus on persistence problems of practice from multiple stakeholder perspectives; (2) a commitment to iterative, collaborative design; (3) a concern with developing theory and knowledge related to both classroom learning and implementation through systematic inquiry; and (4) a concern with developing capacity for sustaining change in systems (Fishman et al., 2013). In addition, the MIST project, on which we modeled how the researcher-practitioner partnerships will work, is cited as one of the first examples of DBIR (Penuel et al., 2011). DBIR offers a novel approach to the persistent problem of the leaky STEM pathway. In addition, we posit that what distinguishes DBIR from the Networked Improvement Communities (Bryk, Gomez, Grunow, & LaMahieu, 2015) that guided the development of the Carnegie Math Pathways is the development of theory, which we will discuss below.

Effective professional learning is sustained over time. In the postsecondary setting, Condon, Iverson, Manduca, Rutz & Willitt (2016) describe the connection between faculty development and student outcomes. The faculty development opportunities were centered on initiatives (writing across the curriculum, supporting students’ development of critical thinking and quantitative reasoning skills) on which faculty participated over time. Changes to instructional practice and the impact on student learning were supported by this long-term participation. As part of sustaining professional learning, TfP leverages a community of IMPACT Teams to support transformative learning and continuous improvement. 

Professional organizations are taking a leading role on STEM Education change initiatives (e.g., Project Kaleidoscope and the Association of American Colleges and Universities; and SEMINAL and Association of Public and Land Grant Universities). AMATYC is uniquely placed to lead initiatives that support change to postsecondary mathematics through the first two years of college. Although there is little research that examines the role of professional organizations on change initiatives, this area of inquiry is emerging. One recent study (Kezar, 2018) examined the impact of an organization of the Association of American Universities (AAU) on a STEM education initiation and found that the professional organization was able to leverage its influence to facilitate change; one way the AAU played a role was through its ability to use messaging strategies to communicate to its member organizations that postsecondary teaching needs to be valued and rewarded. Another NSF-funded project (Project SLOPE, DUE-1726891) leverages AMATYC as the home of a program that supports changes to instructional practices. Currently, researchers working on Project SLOPE are examining the role of AMATYC in this particular change initiative. These two AMATYC initiatives (Project SLOPE and Teaching for PROWESS) are posed to contribute to the emerging area of inquiry into the role of professional organizations in change initiatives. 

IMPLEMENTATION AND RESEARCH ACTIVITIES

The TfP project will engage a variety of activities to support the development of IMPACT Teams as they implement a project focused on improving instructional practices and invoking department-level change. The following are descriptions of each key component for which the TfP project is designed around, along with the specific objectives and assessment driving each activity.

Content and Pedagogical Knowledge Development Activities

The TfP project will leverage the IMPACT Summer Institutes and IMPACT Workshops at the Annual AMATYC Conference to provide project-led faculty development and facilitate instructional change to impact each institution’s plan that they choose to implement, which connect to project goal #1 (student and instructional change) and #2 (departmental and institutional change). While we recognize that each institution will focus on various characteristics of instructional change, all will focus on AMATYC’s IMPACT framework and the 4 active learning guiding principles.

IMPACT Summer Institutes: The Institutes will be held in Year 1 (for Phase 1 institutions, 2 IMPACT College Teams) and in Year 3 (for Phase 1 and 2 institutions, 8 IMPACT College Teams). The goals of the Institutes for the IMPACT Teams are to (1) learn about evidence-based practices which support active learning, (2) learn about creating inclusive learning environments and equitable instructional practices, and (3) work with project leadership to revise their plan for implementation. The Institutes will consist of 4 days of sessions and discussions. In a national study, teachers reported that they had an increase in knowledge and skills and changed classroom practices when their professional development included opportunities for active learning (Garet, Porter, Desimone, Birman & Yoon, 2001). Not only will the faculty development for TfP include the 4 active learning guiding principles, but the structure of delivery will utilize and model these same principles. The sessions will be facilitated by members of the Project Leadership Team. The topics of the sessions will include, but are not limited to, active learning, inclusive learning environments, conducting classroom research, and institutional change through increased course coordination. In addition, the Research Team will facilitate sessions on how to work in a researcher-practitioner partnership as well as hold discussions that focus on the college’s implementation plan. The Institute held in Year 3 will predominantly be centered on the Phase 2 IMPACT Teams. The content will be based on the sessions and discussions held during the first Institute with Phase 1. The content will be revised as necessary utilizing the feedback from the experience of the Phase 1 Teams. In addition, Phase 1 Teams will play an important role in the facilitation of both the sessions and discussions and they will also have sessions specifically designed for them as they complete their implementation cycle. By including members of the Phase 1 Teams, TfP is building capacity to continue this work within the my.AMATYC community and assist in dissemination of the Phase 1 project findings.

IMPACT Workshops: The workshops will be held in Year 1-5 at the AMATYC Annual Conference; and will provide an opportunity for sessions designed specifically for the IMPACT Teams as well as for any attendees at the conference. Topics for the workshops will include (1) understanding the 4 active learning guiding principles and how to implement them, (2) advocating for department-level change, (3) participating in a researcher-practitioner partnership, and (4) consuming research findings centered around characteristics of successful mathematics programs (e.g., Bressoud, Mesa & Rasmussen, 2015). In addition to providing faculty development for the IMPACT Teams, an IMPACT Showcase will be held at the national AMATYC Conference for members of AMATYC. The Showcase sessions will serve as a platform to disseminate the work being done by the IMPACT Teams as well as to provide training for AMATYC members on the topics of active learning and how members can become change agents at their own colleges. The sessions will be facilitated by the IMPACT Teams, as well as the TfP Faculty Development and Research Teams.

Building Community Engagement Activities

The my.AMATYC Community (https://my.amatyc.org/) is a virtual community of practice that will provide a forum for the IMPACT Teams to network with each other and with project leadership. It provides a space for sharing activities that support the idea of teaching with PROWESS (from AMATYC’s IMPACT document) and the implementation of the 4 active learning guiding principles. Teams will be able to share their successes and challenges that emerge as their teams enact their project plans, which will provide opportunities for team members to become agents of change at their institution. The platform will also be leveraged to support the work and facilitation of the researcher-practitioner partnerships, and will be used to share findings and products produced by the projects that can inform the future work across all researcher-practitioner partnerships. These community building efforts will support the TfP project goal #2 (departmental and institutional change) and #3 (knowledge generation).

Research Activities

There are two types of research activities associated with this project: (1) the research activities associated with the project researchers’ engagement in the researcher-practitioner partnerships, and (2) the research activities undertaken to answer the questions posed to meet the three research objectives of the project, described below. The TfP models its research activities on NSF-funded initiatives in mathematics education with a focus on systemic change: MIST (ESI-0554535 and DRL-1119122). MIST informed the design of TfP to include researcher-practitioner partnerships that use findings from iterative cycles of data collection and analysis to inform the design of IMPACT Teams’ plan for change. Each researcher-practitioner partnership will include two TfP Research Team members and members of the IMPACT Team. Each researcher-practitioner partnership will collaborate on the development of the Team’s annual plan for change to the mathematics courses on the STEM pathway, determine data the researcher-practitioner partnership needs to evaluate the enactment of the plan, and use findings from an analysis of these data to redesign the plan for the next year. One self-selected member of each IMPACT Team will be invited and supported to gather and analyze data from another IMPACT Team’s department. This element of our research plan serves two purposes: (1) it engages members of the IMPACT Team more directly in the research, and (2) participating in a site visit at another college provides an opportunity for cross-fertilization across IMPACT Teams in a different way from the cross-institutional gatherings at the IMPACT Summer Institute and at the annual AMATYC conference. A retrospective analysis of the data gathered in the researcher-practitioner partnerships will be undertaken by the Research Team and used to develop a contextualized theory of change framework for community college mathematics departments that highlights the similarities across contexts and differences across institutional contexts.

TfP has three research objectives: (1) understand how an instructor’s enactment of the 4 active learning guiding principles support students’ learning of mathematics in courses on the STEM pathway, (2) understand how IMPACT Teams’ participation in a researcher-practitioner partnership leads to department-level change that supports student success on the STEM pathway; and (3) understand the role a professional organization plays in convening a community of two-year college mathematics departments and propagating a model to transform departments. We hypothesize that transformative change to students’ mathematics learning on the STEM pathway will happen when a department commits to adopting and enacting the 4 active learning guiding principles and evidence-based decision making and that AMATYC’s role in this work will be important (see Tables 1 and 2).

Methods: Research Objective 1. We pose questions to examine how faculty’s enactment of the 4 active learning guiding principles leads to gains in students’ mathematical understanding. Several sources of data will be gathered to answer questions related to this research objective including (a) video recordings of selected IMPACT Team member’s classrooms during two annual lesson cycles (typically 2-4 class meetings focused on a particular topic), (b) tasks used during a lesson cycle, and (c) task-based interviews with at least two students on the mathematical content of each lesson cycle. These data will be gathered during the annual site visits to the college and by an IMPACT Team member at the college. Qualitative analysis of these data will be undertaken using an analytic framework selected based on the particular mathematical focus of the lessons analyzed (e.g., a covariational reasoning framework (Thompson & Carlson, 2017) understand the development of students’ understanding of function).

Methods: Research Objective 2. We pose questions that will help us understand how particular forms of participation in the researcher-practitioner partnership (e.g., participation in the my.AMATYC community) are related to department-level changes. Data gathered to answer questions related to this objective will include artifacts from researcher-practitioner partnership and department meetings, posts to discussion boards in the my.AMATYC community, and interviews with IMPACT Team members and other organizational members at the IMPACT Team’s college gathered during annual site visits. Qualitative analysis of the data will be grounded on frameworks that will help us explain a particular form of participation. For example, Risien’s (2019) thick description of the role of netweavers, curators and sojourners in learning communities can be used to the roles IMPACT Team members play in the my.AMATYC Community and how these roles are related to the work each undertakes at their own college.

Methods:Research Objective 3. We pose questions to help us understand AMATYC’s role in supporting change in their members’ departments. For example, one question might be, “How do IMPACT Team members view the work they undertake as an AMATYC activity?” Another representation research question related to this objective is included in the table below. Data gathered will include artifacts from the my.AMATYC Community, AMATYC conference activities related to the work on IMPACT Teams and AMATYC Board meetings; and, semi-structured interviews with IMPACT Team members and AMATYC leadership (e.g., incoming, current and past presidents, board liaison to TfP project; AMATYC’s professional development coordinator). Since scholarship into the role of professional organizations in STEM education initiatives is emerging (Kezar, 2018), grounded theory (Strauss & Corbin, 1997) will be used to develop frameworks for understanding the role of professional organizations in change initiatives. 

Table 1: Overview of Research Methods and Activities.

CONNECTING PROJECT GOALS, OBJECTIVES, AND ACTIVITIES

Our second perspective on a theory of change is to make explicit the relationship between project activities and objectives. We posit that if the IMPACT Teams participate in the following 2 activities:

1. Actively participate in well-designed, sustained professional development that increases their knowledge and use of the 4 guiding principles of active learning (Objectives 1.2), 
2. Engage in a researcher-practitioner partnership to design, implement and evaluate the implementation of their plan for department-level change, and revise plans as necessary, and share what they learned from this activity (Objective 2.1), 

Then the IMPACT Team’s department will be transformed into a department with a shared vision for instruction that incorporates both institutional data and research findings in instructional decision making. 

If the mathematics departments and their practices are transformed as stated above, then the following additional project objectives will be met:

1. Mathematics classrooms will become more student-centered through the use of the 4 active learning guiding principles (Objective 2.1),
2. Student success in targeted mathematics courses and mathematical understanding will increase (Objective 1.1),
3. Change agents (informal leaders) will emerge in the department, ensuring changes are sustained beyond the life of the grant (Objectives 2.2-2.3), and
4. Over time the math department transformation will positively influence other departments and the institution (Objective 2.2).

Table 2 below illustrates the mapping of the project goals, corresponding measurable objectives, data sources used to measure each objective, and the activities that support each objective.

Table 2: Mapping of Project Goals, Measurable Objectives, and Activities

EVALUATION PLAN

Chandra Lewis, a Senior Research Associate at the Portland office of RMC Research, will direct the external evaluation.

Evaluation Design. During each year of the project RMC Research will conduct an accountability evaluation to address the question, “To what extent has the project met its objectives with regard to the project activities described in the logic model? ”and a formative evaluation to provide project staff with information to refine and improve the project on an ongoing basis to address the formative evaluation questions, “How are department-level changes initiated and continued at the mathematics department and at the institution?” and “How is AMATYC, as a professional organization, supporting collaboration among the IMPACT Teams and propagating the research findings?” For the formative evaluation RMC Research will study Phase 1 IMPACT Teams in depth and use these findings to increase the likelihood of success for the Phase 2 IMPACT Teams. RMC Research will conduct a summative evaluation on the IMPACT Teams using a quasi-experimental nonequivalent group design to address the question, “To what extent do the IMPACT Teams’ colleges experience increased student success after the implementation of this project?” The treatment group will be the students in the college during project implementation and the comparison group are students in the same college prior to the implementation of this project. The hypotheses are that starting in Year 2 of a college’s participation (a) the treatment group will have significantly higher percentage of students passing the targeted math courses than the comparison group; (b) the treatment group will have a significantly higher percentage of STEM majors passing the math requirements for their major than the comparison group, (c) the treatment group will have a significantly higher percentage of STEM majors who pass all of the mathematics requirements for their major than the comparison group. The project anticipates at least a 10% increase for each outcome measure and that these success measures will not differ by gender, race, ethnicity, or socioeconomic status.

Instruments and Data Collection. RMC Research will collect data in all 5 years of the project. For the accountability evaluation, RMC Research will use three data sources: monthly Project Leadership Team meetings, attendance sheets for project events, and project records. For the formative evaluation RMC Research will inquire about AMATYC propagating the research findings at the Project Leadership Team meetings; review the my.AMATYC.org site at least once per year; develop two interview protocols in Year 1, and conduct interviews in Years 2-5 with each Phase 1 IMPACT Team and 2-4 faculty at the IMPACT Team’s college not on the core team. For the summative evaluation, RMC Research will work with the institutional research office at each college to obtain institutional data each summer and during Year 1 for the 3 years prior to project implementation.

Analysis. RMC Research will analyze data in all 5 years of the project. All qualitative data will be transcribed and coded using the software program NVivo and analyzed to identify themes. Logistic regression will be used to assess the odds of a treatment student passing the required math courses versus a comparison student of the same gender, race, ethnicity, and socioeconomic status. Survival analysis would be used both (a) to assess the survival rate of STEM students within the program (i.e., the percentage of students who enrolled as STEM majors who successfully complete the required courses) and also (b) to compare the survival rates for STEM majors in the treatment and comparison groups.

Reporting and Monitoring. RMC Research will report the results of each data collection activity in evaluation briefs provided to Project Leadership Team throughout the duration of the project and in annual performance reports provided each year to Project Leadership Team. RMC Research will create a dashboard to track the student outcome measures over time which will be discussed at the Project Leadership Team meetings. These reporting processes will provide accountability regarding the project’s progress. 

PROJECT MANAGEMENT PLAN AND PROJECT TIMELINE

The project will be managed and executed by a Project Leadership Team, Faculty Development Team, and a Research Team. The Project Leadership Team will consist of PI-Dudley and Co-PIs Gaines, Ebersole, Phelps, Sitomer, and Strom. Gaines will serve as Project Director with PI-Dudley serving as the outward face of AMATYC for the TfP project. They will oversee the functioning of the entire project and the management of the budget. The Project Leadership Team will handle logistics and ensure timely completion of each year’s activities and reports. The Faculty Development Team will be led by Co-PI Ebersole and will include Phelps, Strom and Gaines as members. They will design and facilitate all IMPACT Summer Institutes, IMPACT Workshops, and IMPACT Showcase. The Research Team, led by PI-Sitomer and including Strom and Phelps, will coordinate all data collection and data analysis activities, ensure the researcher-practitioner partnerships are thriving, and collaborate with the external evaluator. Table 3 below illustrates a project timeline, including teams and activities for carrying out the project goals.

Table 3: Project Timeline of Teams and Activities.

ADVISORY BOARD

Susan Bickerstaff is a Senior Research Associate at the Community College Research Center at Teachers College, Columbia University. Bickerstaff conducts qualitative research on developmental education reforms and on community college instructors’ professional learning. She is investigating the impact of lesson study on mathematics instruction at community colleges, and is currently working on a project with Oregon State University.

Linda Braddy is the President of Brookhaven College and past Deputy Executive Director of the Mathematical Association of America (MAA) (2012-2016). Braddy has overseen several externally-funded projects and various policy-driven initiatives. She will bring knowledge of community college leadership, research in mathematics education, and overall project oversight in her role as an advisor to TfP.

Chris Rasmussen is a Professor of Mathematics Education at San Diego State University and PI of the NSF-funded SEMINAL project, as well as an expert in inquiry-based instruction and active learning. Rasmussen has also overseen many externally-funded projects, several of which have taken a cross-institutional perspective on mathematics teaching and learning.

Nancy Sattler is a faculty Emerita at Terra State Community College and current faculty at Walden University. As AMATYC Past-President and the IMPACT Committee Co-Chair, Satter’s knowledge of AMATYC as an organization will inform the sustainability of the TfP’s efforts and will provide insights to organizational leadership. Sattler serves on the advisory board for Carnegie Math Pathways and Project SLOPE. The Advisory Board will be called upon each year of the project to review and reflect on project activities and findings, as well as to provide guidance for moving the project forward. The Advisory Board will also be leveraged to facilitate dissemination of the project outcomes in their respective networks and communities.

RESULTS FROM PRIOR NSF EFFORTS

Project SLOPE: NSF-DUE 1726891; $305,110; 8/15/17-7/31/20. Scholarly Leaders Originating as Practicing Educators in Two-Year College Mathematics. PI-Sitomer currently serves as a Co-PI on Project SLOPE. This ongoing project seeks to build a sustained professional development program within AMATYC and the findings from this project will be leveraged to support the building of researcher-practitioner partnerships within the TfP project and AMATYC. Project SLOPE provides the foundation for developing two-year college mathematics faculty researchers engaged in classroom inquiry through the Scholarship of Teaching and Learning (SoTL).

Intellectual Merit: Research conducted by practitioners for practitioners holds the promise of immediate relevance and great impact for the development and growth of empirically-based improvement efforts. Preliminary findings (Campo, Sitomer, Breit-Goodwin & Quardokus-Fisher, 2020) indicate that when a program emerges from the grassroots (in this case the AMATYC members who brought the idea for the program to AMATYC leadership), leaders need to develop a sense of ownership of a program in order to innovate and sustain the program.

Broader Impacts: This project addresses the research gap surrounding two-year college mathematics faculty SoTL research by first conducting a critical analysis and synthesis of existing programming and the literature base that focuses on the intersection of SoTL, two-year college faculty research, and mathematics education research in postsecondary contexts.

Products: Findings from Project SLOPE have been presented at two conferences. One publication has resulted from this project (listed in References from NSF Prior Efforts in the References Document).

AI@CC: NSF-DUE 1561436; $2,370,836; 5/15/16-4/30/19. Algebra Instruction @ Community Colleges: An Exploration of its Relationship with Student Success (AI@CC). PI-Strom currently serves as Co-PI on the AI@CC project, along with this proposed project’s Senior Personnel Laura Watkins (who is the PI for AI@CC). This ongoing Core Research project seeks to investigate the conditions under which instruction in community college algebra courses can be associated with student learning gains and course performance. Watkins and Strom’s work on AI@CC is focused on analyzing instruction through video cases.

Intellectual Merit: AI@CC is investigating the relationship between two characteristics of mathematics instruction at the community college level: (1) quality of teacher-student interaction and (2) quality of mathematics with student learning gains and course performance in community college algebra courses. The work will advance the understanding of teaching with diverse populations and provide information about possible ways to alter classroom interactions.

Broader Impacts: Findings from this projectwill contribute to the improvement of mathematics teaching at community colleges by investigating the impact of instruction on students’ learning and by refining quality measures of instruction and student learning in this setting. AI@CC impacted community college students, and establish models of instruction that translate to increased learning to better prepare students for future mathematics courses, thus increasing their chances of pursuing STEM careers.

Products: Findings from AI@CC have been presented at 5 conferences and resulted in 7 publications (listed in References from NSF Prior Efforts in the References Document).

Math Science Learning Community: NSF-DUE 1347786; $212,489; 1/1/14-12/31/16. Co-PI Ebersole served as the PI for this NSF-WIDER project focused on creating two faculty learning communities for increasing the use of evidence-based instructional practices in mathematics and science classes of both full-time and part-time instructors. Participants were required to implement an evidence-based practice in their courses, then share results in a Community of Practice. Learning community members explored inquiry-based learning, experiential learning, and modified STEM student learning communities in math and science gateway courses.

Intellectual Merit: The project showed that one way to remove the barrier of concern that using evidence-based practices would lead to poor student evaluations that affect employment is to have deans overtly state support for evidence-based practices and acknowledge they may result in temporary low evaluations.

Broader Impacts: Lessons learned from this project are impacting the outcomes of nearly half the nation’s STEM students by adding to the body of knowledge on the effect of increased use of inquiry-based learning, experiential learning, and learning communities on student learning, especially for members of underrepresented groups.

Products: Findings from this project resulted in one publication (listed in References from NSF Prior Efforts in the References Document).

DISSEMINATION PLAN

The project dissemination plan is designed to share findings and products, as well as propagate the proposed changes in instructional practices beyond the eight IMPACT Teams. Dissemination occurs at four levels: (1) departmental, (2) institutional, (3) AMATYC community, and (4) national level. 

Departmental Level.The IMPACT Teams will develop a departmental vision and update full- and part-time instructors on the project. Teams will also submit annual reports summarizing lessons learned at Summer Institutes and AMATYC conferences to college leadership and to the Project Leadership Team. Pedagogical changes will be propagated to mathematics instructors not on the team by sharing instructional materials developed as part of the project and inviting all instructors to participate in project-related Traveling Workshops and Webinars.

Institutional Level. The administrators on the IMPACT Team will rally support for the project and disseminate results, including annual reports to be shared with other IMPACT Teams and colleges. The Project Leadership Team will work with administrators on the IMPACT Teams to coordinate efforts to share project findings with other administrators and across disciplines within their own institution.

AMATYC Community Level. AMATYC has five structures in place to help disseminate findings, increase the use of active learning, and assist in departmental and institutional transformation: 

• Webinars: Project staff will develop TfP Webinars focusing on, but not limited to active learning, equity and inclusion, research, institutional change. They will be open to all AMATYC members and then archived on the my.AMATYC website.
• Traveling Workshops: The TfP Traveling Workshops will be available to any community college after they have been presented at the IMPACT Teams’ colleges. The topics covered will be similar to the Webinars but will provide the opportunity to delve into the content more deeply.
• Publications: A Teaching for PROWESS column will be included in the quarterly AMATYC Newsletter highlighting the activities and results. The journal MathAMATYC Educator will provide a forum for more detailed articles about the project and findings.
• my.AMATYC website: The sitewill house the project’s closed learning my.AMATYC Community where IMPACT Teams can share their findings and work together on common problems. The library capability on the platform will allow the instructional materials developed by the project, the Webinars, and videos of the IMPACT Summer Institute to be shared with all AMATYC members. In addition, blogs and discussion boards will inform members about the project and its findings.
• Conferences: The annual conference will provide a venue for IMPACT Team presentations, project posters, and networking opportunities. The IMPACT Showcase will provide a venue to educate more faculty about using research to make evidence-based decisions, making changes in their instructional practices, and changing institutional practices.

National Level.The IMPACT Teams are encouraged to make presentations at state and regional conferences and write articles for non-AMATYC journals that will reach a larger audience. The project Research Team will also publish articles and present at research conferences. The webinars on active learning developed through the project will be made available to non-members. It is the intention of TfP to create a network of experts (beginning with the IMPACT Teams and spreading through their school, state, and AMATYC Region) that will run the Traveling Workshops, thus creating local leaders.

INTELLECTUAL MERIT

This project adds to the current knowledge base on several levels, while focusing on community college settings with their unique challenges. At the classroom level, the research will focus on how the use of active learning improves student learning. In particular, what is the effect of each of the active learning principles on students’ knowledge and ability: (1) students’ deep engagement in mathematical thinking, (2) student-to-student interaction, (3) instructors’ interest in and use of student thinking, and (4) instructors’ attention to equitable and inclusive practices. At the department level, the project adds to research on what works to transform departments and departmental culture towards a common vision. In particular, what effect does the combination of sustained professional development, working as a team that includes an administrator and an institutional researcher on a common problem, contributing to an online community of practice with other IMPACT Teams, participating in a researcher-practitioner partnership, and the associated analysis of local data have on student success rates and retention. Finally, the project will add to the limited research on the role of a professional organization such as AMATYC in transforming departments and institutions, as well as propagating project findings beyond the 8 IMPACT Teams. The project will determine the impact of AMATYC structures, such as its online community (my.AMATYC) for collaboration, annual conferences for disseminating project findings, and Traveling Workshops and Webinars for faculty development.

BROADER IMPACTS

The project will address several societal needs in an effort to impact the community broadly. Greater use of active learning and evidence-based practices will increase success rates in the gateway STEM courses, such as College Algebra, Precalculus, and Calculus, as well as their pre-requisite courses, thus retaining more students in STEM programs. The use of equitable and inclusive practices will retain more under-represented students in the STEM pipeline. Through a focus on the 4 active learning guiding principles and department wide change, students not prepared for the first mathematics course in their program will complete these gateway courses on schedule. Research shows that students who complete the gateway courses on schedule are more likely to complete their program of study at the two-year college, keeping them in the STEM pipeline and helping the U.S. meet the workforce need for STEM graduates. Also, use of the 4 active learning guiding principles results in greater conceptual understanding and mathematical ability, better serving client disciplines. Our research findings will lead to a framework for systemic transformation of two-year college mathematics departments that will progressively impact the workforce by providing quality workers equipped with 21st-century skills.