South Dakota State University Homepage | South Dakota ...



South Dakota Working in STEM for Equity (SD-WISE): A System, Institutional, and Individual Level Approach to Policy ChangeOverview Systemic policy change is needed across South Dakota institutions of higher learning in order to pursue gender equity for faculty in the science, technology, engineering, and mathematics (STEM) disciplines. Influenced in part by the rurality of our institutions, women account for only 32% of the STEM faculty in the six-university South Dakota Board of Regents (SD BOR) system, and only 29% of the faculty in STEM disciplines outside of the social and behavioral sciences. Thus, there is an unmet need to establish policy that creates a climate to more effectively recruit, support, and retain women in STEM. This has been the central focus of the NSF ADVANCE initiative. In particular, notable successes from Montana State University's Project TRACS, and the University of Maryland's Program for Inclusive Excellence inform this work particularly as it relates to policy development and implementation for gender equity. Our long-term goal is to build upon this work to increase the participation, representation, and advancement of women in faculty positions in the STEM workforce throughout the South Dakota Board of Regents (SD BOR) university system.One element of change has proven the most recalcitrant across past programs: the urgent need to effectively and demonstrably change the institutional academic climate for women in the STEM sciences (Howe, Juhas and Herbers, 2014; Rankins, Rankins and Inniss, 2014). We propose that the reason for the lack of long-term success in sustained institutional change for gender equity is due to the fact, as noted by Gill, Sharp, Mills, and Franzway (2008), that: “Gender equity should be seen as a systemic rather than an individual issue” (p. 401, emphasis ours). In order to address equity in higher education across a state system, we propose a “top-down” approach. We use the “top-down” terminology to indicate that, although this proposed work builds directly upon past ADVANCE successes, our emphasis is in breadth of change through work with a central authority, rather than individual institutional initiatives or “bottom-up” approaches which predominate in past ADVANCE work. Past research (see, for example, Blackmore, 2011; Castilla, 2008; Fletcher, et al. 2007; and Kalev, Kelly and Dobbin, 2006) indicates that the central establishment of equity policy and indicators at the system level is required, not only to demonstrate improvement in diversity but to actualize it. Therefore, the rationale for this proposal is that because current policies may not adequately serve women faculty in STEM in our state, we need to engage top administration not only in institutions but also at the state system level in order to develop and implement effective policy. We are well-positioned to conduct the proposed work. We have secured the support of SD BOR administrators to guide uniformity of policy. In addition, we have also secured the support of all six provosts to expedite consistent implementation of policy innovation at the institutional level. Drawing from the breadth of knowledge and expertise of past ADVANCE research and recipients, we have enlisted the services of Dr. KerryAnn O’Meara (University of Maryland) as an expert policy consultant to guide our policy work, and the services of Dr. Elizabeth Birmingham and Dr. Rick Axelson (North Dakota State University) for external evaluation throughout the project.Current SD BOR policies relate to equal opportunity, sexual harassment, discrimination, stop the clock, FMLA, and lactation in the workplace. Although in accordance with federal and state law, few of these policies address complex gender issues beyond these mandates. However, the results of research conducted at the lead institution (2012-2013) revealed that although these policies are in place, there are problems with their dissemination and implementation at the institutional level. For instance, although there is a “stop the clock” policy in place through the SD BOR, there is no such policy stated in the institutional policy and procedure manual. Therefore, faculty awareness of the policy and the procedures for using it are minimal. Other internal research documented faculty concern about how applying for “stop the clock” may impact the tenure and promotion process and women faculty’s specific concerns about application in complex situations (e.g., miscarriage, a stillborn child or a special needs child). SD BOR system lactation policy mandates the provision of a private space for lactation and appropriate work breaks for pumping; yet there are few university designated lactation areas available at the lead institution and employees seem largely unaware of those that are currently in place. Most importantly, departments lack resources and guidance for providing faculty members release time under FLMA. The organizational culture reflects the expectation that faculty members continue with graduate student advising, grading, and teaching on-line while on leave. Therefore, the aims of this work are to gather information across the state system to better understand if existing policies are adequate for the needs of women in STEM, if such policies are being effectively and consistently implemented across institutions, and if there are areas that necessitate new policy development.?Working together, our institutions and the Board of Regents will be able to implement significant policy change initiatives for establishing gender equity norms and, in consequence, increased recruitment, promotion and retention of female STEM faculty in the state of South Dakota. This contribution is significant because it is expected to 1)?extend?Scott's (1987) multilevel?theory?using a "top-down" approach to produce?data-informed policy change in higher education; 2) build?upon past single institution ADVANCE?IT policy work and adapt?it for use across a state system;?and?3) examine?evidence of?gender equity in faculty annual evaluations and career planning documents to contribute not only to system and institutional?policy revision but?to extend existing literature to inform the experience of?women faculty in STEM.Goals and ObjectivesWe propose three primary goals to increase gender equity in STEM across all participating institutions in the state of South Dakota. We will pursue our goals through the following specific objectives:Goal #1: To establish and sustain system-wide priorities for gender equity in the STEM sciences.Objectives:For year 1, determine base line by collecting and analyzing data related to Professional Development Plans (PDPs), individual Faculty Annual Review documents (FARs), climate surveys, and system-level faculty plete initial and periodic outside policy review for models of best practices.By the end of year 1, PLAN working group, BOR representatives and institutional leaders identify policy targets with the guidance of our Policy Consultant Dr. KerryAnn O’Meara, drawing on baseline data reports. By the end of year 3, PLAN working group, BOR representatives and institutional leaders identify common “best practice” policies for STEM women faculty retention with the guidance of our Policy Consultant, drawing on baseline data reports.By the end of year 5, PLAN working group, BOR representatives and institutional leaders identify common “best practice” policies for career development with the guidance of our Policy Consultant, drawing on baseline data reports.Goal #2: To document and evaluate incremental change from data-driven policies for gender equity at the South Dakota State system level and in each of the six institutions. Objectives:Institutionalize “best practice” and common implementation of policy innovations in years 2, 4, and 5.By the end of year 3, individual FARs indicate improved equity practices in assessment.By the end of year 4 STEM women faculty PDPs reflect higher individual aspirations and gender equitable career planning.By the end of year 5, campus climate surveys indicate integration of gender equity principles at all institutions.Goal #3: To disseminate innovative systemic procedures for gender equity policy effectiveness and to contribute to knowledge of gender equity in higher education.Objectives:Assess evidence of system change from a top-down impetus across all three levels.Disseminate results pertaining to policy innovation at the state system level.Establish a public website for the duration of the project which will be used as a centralized way to report on project impact at all institutions.The proposed goals and objectives are potentially transformative because we adapt Scott’s (1987) theory of change to address equity in higher education across a state system (see Figure 1), emphasizing the transformation of system-level structures and subsequent change in the individual practices underlying them. Utilizing a top-down approach at the system level, the SD BOR has the power to enact policy initiatives that cultivate gender equity. Within the system, at the institutional level, administrative leaders have the power to create consistent models for gender equity policy implementation and practices. At the individual level, a coherent and consistent gender equity policy becomes a new norm in academic culture translating into change in individual practices by faculty and administrators (Nielsen, et. al, 2005; van de Ven & Poole, 1995). Our proposed three levels of transformation are expected to nurture a change in the culture of the SD BOR system and participating institutions as well as in the careers of women faculty in STEM through establishing a supportive policy environment for system-wide gender equity initiatives, university level changes in equitable implementation and increased faculty awareness of equity issues.Fig. 1: Scott’s model of progressive change from top-down authorityScott’s theory resonates in the management literature, although imperfectly. Lewin’s (1947) classic model of institutional change also requires three steps: 1) “Unfreezing or thawing” current system practice; 2) “Change or Transition”; and 3) “Re-freezing” system practice with new norms. Lewin suggests that systems become hidebound, and that effective change occurs at the systemic level, which complements Scott’s notion of “top-down” impetus for change. Lewin’s change or transition stage reflects the implementation at the meso-level where new practices are both translated and practiced, reflecting Scott’s institutional level. Lewin’s “re-freezing” stage in essence translates to creating the change goal as a “new norm.” Here, according to Scott’s theory of change, policy change from the macro-level, as implemented at the meso-level, must be integrated into individual daily practice at the micro-level to endure.We anticipate these changes, combined with our research innovations and drawing on best practices from previous ADVANCE recipients, will produce the following expected outcomes: We will have collected a common set of gender-relevant data indicators from universities across the system and identified common policy targets for the recruitment, retention, and career development of women faculty (goal 1); We will have institutionalized policy innovations, documented improved equity practices in faculty annual evaluations, documented higher aspirations in women’s career development plans, and will have observed the integration of gender equity principles in campus climate surveys (goal 2); and finally, we will have assessed evidence of “top-down” change across the system and disseminated results of policy innovations (goal 3). These results are expected to have an important positive impact by contributing a multi-level model of policy change, beginning from a system-level impetus at a state governing authority, spreading to state institutions of higher learning, and reaching to the individual level to improve the recruitment, retention, and career development of women in STEM disciplines at rural universities. ? Context and DataWe are uniquely situated for this project. As a small rural state, the higher education system in South Dakota requires frequent cooperation across institutions, and enables more “face-to-face” interaction among institutional leaders to effect change. Our state institutions have clearly defined strengths and foci in STEM sciences at undergraduate and graduate levels. Last, the moderate collective size of our six institutions allows for not only the timely implementation of policies, but also their assessment. Existing institutional relations, diversity in institutional missions, and a size effective for policy implementation make us an ideal “test case” for system-wide transformation (see Figure 2).Fig. 2: System, institutional, and individual partnerships in "top-down" changeThe six state institutions partnering in this grant provide a diversity of representation in the STEM disciplines and differing strengths in their curriculum and specializations. Three of the six institutions are leaders in multiple STEM areas for Ph.D. study, and the remaining three institutions represent single disciplinary strength in STEM study at the undergraduate level. Our inherent structural and curricular diversity allows us to expand knowledge on the impacts of system-level policy change in a variety of institutional environments (See Table 1 for STEM faculty data and project participants).Table 1: Institutional STEM faculty data and project participantsInstitution# faculty/ % female# female faculty/% term contract# tenured female/# tenured male Steering committee SD PLAN committee SDSU323/30%97/44%31/122ProvostPI serve as Chair; rotating female faculty USD144/37%53/53%16/49 ProvostOne female faculty SDSMT115/24%27/37%5/43ProvostOne female faculty NSU33/33%11/10%6/13ProvostOne female faculty DSU54/33%18/55%6/18ProvostOne female faculty BHSU51/43%22/45%7/15ProvostOne female faculty SDSU, USD and SDSMT are the premier doctoral institutions in the state. South Dakota State University (SDSU), the lead institution, for this grant, is a comprehensive doctoral-level, land-grant institution in Brookings, SD. As the state's largest university, SDSU has 12,583 undergraduates, and 1,348 graduate students. It specializes in the agricultural and biological sciences, environmental, civic and mechanical engineering, chemistry, and social sciences. SDSU is also home to the Geospatial Center of Excellence and offers advanced programs in the STEM sciences across four of its six colleges. The University of South Dakota (USD), located in Vermillion, SD, is the second largest university in the state, with a total student enrollment of 10,235 at the undergraduate and graduate level. USD is home to key professional schools in South Dakota: the Medical School, Law School and Business School. USD also offers Ph.D. degree programs in the physical and social sciences. The South Dakota School of Mines and Technology (SDSMT), located in Rapid City, SD specializes in the physical and natural sciences, and offers undergraduate and graduate degrees in these disciplines. The total 2013 student enrollment was 2,640. Most notably, SDSMT offers research opportunities for faculty and students at the nearby Sanford Underground Research Facility. This unique underground laboratory is broadly utilized by faculty in geological sciences, physics and natural sciences, environmental and mechanical engineering, and atmospheric and environmental sciences. Our other state partners excel in particular areas of STEM undergraduate education. Northern State University (NSU), located in Aberdeen, SD, enrolled 3,622 students during 2013. NSU specializes in education technology, including graduate and undergraduate degrees in STEM science education. Dakota State University (DSU), located in Madison, SD, enrolled 3,129 students during 2013. DSU specializes in graduate and undergraduate degrees in computer science and technology. Black Hills State University (BHSU), located in Spearfish, SD, offers baccalaureate and associate programs in education, business, technology, and liberal arts and sciences with master’s degrees provided in business services, science, and education. The institution also supports the Center of Excellence in Mathematics and Science Education. The total student enrollment during 2013 was 4,464. Commitment and Sustainability We have received enthusiastic support from all six Provosts of the participating institutions, and equally enthusiastic support from the SD BOR System Vice President for Academic Affairs (see letters of commitment included in the application). The support from these key administrative leaders is vital to successful system-wide gender equity policy evaluation and implementation. In addition, the Provosts will lead on their respective campuses, and have worked with the lead institution to identify key female STEM faculty participants for representation on the SD PLAN committee. Additionally, we have approval from the SD BOR legal counsel and IRB for research activities pertaining to personnel records in order to affirm both legality in process and anonymity of personnel.At the completion of the grant, we anticipate continued collaboration across state institutions to support on-going work for the development of STEM faculty diversity. Following Scott’s (1987) model of change, the presence of new gender equity policy, centrally-driven implementation at the institutional level, and the creation of new practices (new norms) for faculty will lead to enduring change. Therefore, the policy initiatives that are developed in this work will be sustained or refrozen (Lewin 1947) within existing system authority, through leadership of the Provosts at the institutional level, and through the lived experiences of faculty members in all of our institutions. The already existing strong connection among and between the SD BOR and the Provosts involved in this project will enable the continued sustainability of our efforts beyond the duration of the grant.Activities Description The project activities reflect “best practices” from previous ADVANCE work and link knowledge building activities to policy outcomes.PLAN Project Activities:We have established a Steering Committee which includes all six Provosts of the participating institutions. The Provosts will meet bi-annually for a report from the SD PLAN committee liaison, and will meet annually with the SD PLAN committee to ensure compliance with system objectives and effective project management SD PLAN Committee. In addition, we have established a SD PLAN Committee which includes one permanent or revolving STEM faculty member from each institution. They will meet quarterly to share institutional practices on implementation of existing policy and, over the five years, implementation processes of new or amended centralized policies. In years 1, 3, and 5, the SD PLAN committee will work with Dr. KerryAnn O’Meara, University of Maryland, who has the necessary expertise in policy change to assist us in assessing existing policies and drafting new policy recommendations. These policies will be driven by the data collected through on-going research activities for knowledge building (described below) and will address issues relevant to the restriction of women’s career development in STEM. The resulting policy recommendations will be shared and discussed with SD BOR key personnel, including the System Director of Human Resources, General Counsel, and the System Vice President for Academic Affairs. The SD PLAN Committee will work with the Steering Committee and the SD BOR to negotiate and create regental policies that will govern the six institutions in the state system.The SD PLAN committee will also serve as a data analysis team to identify common trends across the state system with specific emphasis on gender equity issues in the STEM disciplines, such as problems with retention or promotion of women scholars as reflected in findings from annual research. The SD PLAN Committee members will work annually with their home institutions to establish the collection of data on gender-relevant indicators identified in past research (Nielsen, et al., 2005; van de Ven & Poole, 1995). The operationalization of these variables will be standardized across the SD BOR system to allow for ease of comparison. In addition, random samples of annual faculty evaluations (FARs) and individual professional development plans (PDPs) at each institution will be numerically coded to determine degree of gender-specific critique of faculty performance, and improvement in the gender equalization of assessment. (Appropriate legal and IRB procedures for access to personnel documents have been agreed to by the IRB compliance officer and the SD BOR legal counsel.) The external and internal evaluators for this project will track and assess the impact of system change throughout the project (see detail in the evaluation section below). Results will be disseminated across the state system by our PLAN committee and shared with administrative leaders on our steering committee, and with SD BOR leaders. A public website will be used as a centralized strategy for reporting on the project activities and impact at the institutions. We will create a portal via the website for employees to propose recommendations for policy innovation (Best Practice from Montana State University TRACS). Research Activities for Knowledge-Building We will contribute new knowledge in the literature on centralized policy change and its impacts on institutions and individual faculty. This contribution will be fulfilled through a series of research activities. We will engage in annual data collection and analysis, which will inform project activities and progressive policy formation in years 1, 3, and 5. We will conduct the following:An analysis of existing policies and consistency in implementation across all six campuses. This policy review and comparison of implementation procedures will form necessary baseline data to ensure common interpretation and application of policy across the system, in line with Scott's (1987) model. Annual campus climate surveys inclusive of the “vital variables” (Nielsen, et al., 2005) indicating that gender equity objectives are being practiced and internalized as “new norms” throughout the system. This research activity establishes that the objective of the project remains salient to state faculty throughout the work, and reflects system-wide progress in gender equity. Annual content analysis on centralized forms of individual STEM faculty annual evaluations (FARs), as represented in random samples drawn from all institutions to establish equity in evaluation system-wide. This research establishes institutional-level implementation of gender equity policies for recruitment and retention (Scott, 1987). Annual content analysis on centralized Professional Development Plans(PDPs; composed by faculty) to establish that individual faculty are practicing the “new norms” of gender equity in their own career planning, thus addressing the individual level of Scott's (1987) model. An analysis of annual demographic faculty data obtained at the system level for improvement in recruitment, retention and promotion of female STEM faculty throughout the five years of the project. Instituting annual climate surveys with identical indicators across all institutions provides necessary data for establishing baseline data on practices and integration of gender equity principles. Although the five year timeline is limited, we would expect to see evidence of improvement in institutional climate begin to emerge by the end of the grant period. Similarly, annual evaluation of system-wide demographic data on STEM faculty members will allow tracking of both incremental and longitudinal change in faculty composition during policy implementation.Consideration of the FARs will allow us to distinguish unacknowledged or even subconscious practices in evaluation of women faculty with regard to “character flaw” rather than “professional accomplishment” (Fletcher, Boden, Kent and Tinson 2007; Castilla 2008). It is widely recognized in the literature and popular media that women faculty are more often critiqued for perceived character deficiencies than their male counterparts, or are more likely to have past successes work against them in evaluation (Cable and Inesi, forthcoming). Again, while we don’t argue that such critiques are either ubiquitous or constant in our system or any other, we feel that part of systemic change is to bring their documented presence and use to the forefront of gender equity policy at that level. This analysis will allow us to work more effectively in designing gender equity policies for annual evaluation, and increase not only faculty success, but retention in our system.Consideration of the three to six year PDPs will provide key information on how women and men are merging personal and professional lives. Of key interest here is being able to distinguish not only women’s different individual experiences in planning the progression of the career, but in identifying whether or not women may be underestimating their own activities and potential (Rankin and Nielsen, 2006; Babcock and Laschever 2007; Frieze, 2009; Smith & Huntoon, 2013; Howe et al, 2014). Literature also suggests (MIT 1999, from Frize 2009) that senior women engineers feel invisible; spend a greater percentage of time on general and non-disciplinary service commitments; are often required to be on hiring committees to achieve gender balance but rarely lead such committees; and suffer long-term inequity in salaries, professional recognition, and internal award recognition. From these plans, we expect to not only have data to identify the ambitions of women in the STEM sciences, but also to be able to identify necessary areas for equity in STEM women faculty careers for future success (Sagebiel and Dahmen 2006; Gill Sharp, Mills and Franzway 2008).Together, these research activities will contribute to knowledge building of systemic change in three primary ways.? First, we will expand and assess understanding of the probability of gender equity improvement in higher education state systems through policy implementation by a centralized authority.? As noted previously, recent research in industry indicates the effectiveness of “top-down” change for increased gender equity.? By implementing and assessing this process in academia, we will be better positioned to provide evidence and produce models for centralized change that may be applied beyond our state. Second, we will expand the literature on potential inequalities in assessment of professionals by gender.? The vast majority of the literature on gender differentials in assessment has been conducted on women and men in private industry.? Working with the SD BOR, we innovate by including specific, and longitudinal, data on gender differences in assessment across higher educational institutions and in the academic workplace more generally. ?Further, through the duration of the project we will be able to establish evidence for how centralized policies can change individual assessment practices across a higher education system, linking the system and institutional level in Scott’s theory of change.Third, we will expand the literature on the impact of women faculty’s patterns of negotiation, or not, to produce stronger career outcomes.? As we have common forms of professional development planning, we are uniquely able to assess not only differences in career aspirations across gender, but also across differing institutional environments. ?Again, through the duration of the project we will be able to establish evidence for how centralized policies can change individual norms in career planning and aspirations, linking the system, institutional and individual level in Scott’s theory of change.Beyond South Dakota, both the SD PLAN and Steering Committees will engage in dissemination of research and project results through publication and conference presentations. For peer review publications, the SD PLAN committee will submit results for publication in the following proposed journals: Gender, Work and Society; Signs, and the Journal of Career Development. For professional and technical reports, the SD PLAN committee, with assistance from the Steering Committee, will present results in the following professional outlets: The AACU newsletter and The Chronicle of Higher Education.Project ManagementTo effectively implement this project we will work with existing entities and will establish a new working group. Our Steering Committee (named below with description of responsibilities) will provide overall project management. The SD BOR representatives (named below with description of responsibilities) will work with the Steering Committee in implementing policy amendments or additions. The SD PLAN committee (a new working group, named below with description of responsibilities) will work with the Policy Analyst (named below with description of responsibilities) to bring forward policy innovations. The External and Internal Evaluators (named below with description of responsibilities) will assess the on-going progress of the project and report to the SD PLAN committee and the Steering Committee.Figure 3: Management structure for project organization and supervisionProject management will be under the supervision of the Steering Committee. The Steering Committee will work with all other work groups to not only ensure the progress of the proposed projects, but to work with internal and external evaluators to identify points for re-formulation of the project if needed. Current organizational structures include monthly meetings of the Provosts; for this project, one of the meetings annually will have time set aside to meet the SD PLAN Liaison and evaluators to analyze and reformulate project activities as needed for improvement and one additional meeting annually will have time set aside to meet with the SD PLAN committee to address and resolve project implementation issues. Members of the Steering committee are all six Provosts from the participating institutions: Provost Laurie Nichols, SDSU; Provost James Moran, USD; Provost Duane Hrncir, SDSMT; Provost Rod Custer, BHSU; Provost Judy Dittman, DSU; and Provost Tom Hawley, NSU. In addition, Dr. Meredith Redlin, (PI for this grant and SDSU Professor of Sociology) will serve as a Liaison between the Steering Committee and the SD PLAN committee.The SD BOR representative will work with the SD PLAN committee and the Policy Analyst to institute policy amendment and to lead system-wide implementation of policy initiatives. SD BOR representatives include: Dr. Paul Turman, SD BOR System VP for Academic Affairs; Dr. Jim Shekleton, SD BOR General Counsel; and Dr. Barbara Basel, SD BOR Director of Human Resources. Execution of project and research activities will be done by the SD PLAN committee. This committee is comprised of one permanent or revolving female faculty member from each participating institution. The committee will meet quarterly, and will complete the following duties: gather data for longitudinal study; work with respective Provosts to ensure gathering of institutional data; analyze the data; work with the Policy Analyst, compose and propose new system policy for gender equity in the STEM sciences; and work with the SD BOR representatives to refine and implement these policies. All SD PLAN members will participate in dissemination through scholarly publication and conference presentation. Members of the SD PLAN committee include: Dr. Meredith Redlin, Chair and Steering committee liaison; Dr. Amanda Jantzer (SDSU, revolving member); Dr. Elizabeth Tolman (SDSU, revolving member); Dr. Robin Miskimins (USD); Shaobo Huang (SDSMT); Dr. Ashley Podhradsky (DSU); Dr. Cynthia Anderson (BHSU); Alyssa Kiesow (NSU).Dr. KerryAnn O'Meara, University of Maryland, will serve as Policy Analyst. She will work directly with both SD PLAN and the SD BOR representatives to advise on data collection on campuses to inform policy development, the development of policy language, and assessment of whether new policies are working as intended. O'Meara's experiences as a scholar and Co-PI on the University of Maryland ADVANCE grant will be useful to us, including having studied work-life policy (O'Meara and Campbell, 2011) and served on the UM committee that brought parental leave to UM in 2012; studying academic reward systems (O'Meara, 2011; 2014) and issues within them for women and faculty of color and serving on UM's P&T Taskforce revising their policies 2012-2014, or studying faculty retention (O'Meara, 2014; O'Meara, Lounder and Campbell, 2014) and designing and implementing UMCP's faculty climate survey and corresponding action projects (Campbell and O'Meara, 2013; Terosky, O'Meara and Campbell, 2014). Dr. Mary Emery (SDSU) will serve as the Internal Evaluator. The internal evaluator will use a developmental approach to review what is working and to identify challenges and possible remedies. The focus of internal evaluation will be primarily formative measures and assessment, although Dr. Emery will work in close collaboration with external evaluators for summative evaluation.Dr. Elizabeth Birmingham and Dr. Rick Axelson, with the staff support of the Center for Social Research at NDSU, will serve as External Evaluators.? As past recipients of an ADVANCE IT grant, these colleagues are familiar with the requirements and goals of NSF evaluation and the work of institutional change around issues of gender. They will provide evaluation design, close collaboration with the internal evaluator concerning needed instruments and data collection, and consultation for all participating institutions regarding standardized collection of necessary data, including 12-indicator data. External evaluators will meet with the Steering Committee, BOR representatives and consult with the SD PLAN committee.Table 2: TimelineProject QuartersTasks1234567891011121314151617181920Steering committee (SC) and SD PLAN meetingsxxxxxxSD PLAN meetings, data analysis, policy review xxxxxxxxxxxxxxxxxxxxSD PLAN completes annual data report xxxxxSD PLAN works with policy consultant and BOR to implement new policyxxxAnnual reports to NSF program officerxxxxxIncremental change assessments on longitudinal dataxxxxPC and Steering Committee confer on findings and project progressionxxxxxDisseminate new knowledge xxxxxxxInternal evaluationassessment/reportsxxxxxExternal evaluation data gathering/ReportsxxxxxxxxxxProject EvaluationInternal EvaluationInternal evaluation will focus primarily on formative measures and concrete outcomes through the project timeline (See table 3). Dr. Emery proposes a developmental approach, which monitors both the timely completion of the proposed project and research tasks, as well as the number of products produced by the work. The internal evaluator will work closely with the external evaluation team to ensure appropriate linkages and documentation for summative evaluation throughout the project duration.Table 3: Internal evaluation procedures and formative measuresProject TasksMeasurementAnalysis/reportingOutcomesTimeframeDetermine base line data; annual collection and analysis of key dataNumber of FARs, PDPs analyzed,Climate survey response rate; faculty demographic changeDevelop coding protocol to analyze differences in FAR evaluations and PDP goals by gender; standardize climate survey variables and demographic data.Shared knowledge of collective and unique barriers to gender equity reported in study results on climate and gender differences.Data collected (2-8 months of each project year)Report submitted to Steering committee (Months 9-11 or each project year)Identify policy targets, complete initial and periodic outside policy review. In initial policy analysis by consultant in Year 1; follow up in Years 3and 5 to determine degree of policy change. Number of policy issues addressed annually.Report with recommendations related to policy development year 1 on PDPs, FARs; year 3 on climate survey, and year 5 on implementation of new policies in SD.BOR will institute new policies; institutions will and develop implementation strategies Years 1, 3, 5Meeting and relationship between Steering Committee, SD PLAN committee and SD BORNumber of meetings, attendance at meetings, recommendations made to steering committee.Steering committee response to recommendationsSD BOR response to recommendations.PLAN committee forwards policy recommendations to the BOR; Provosts incorporate findings into institutional policy OngoingEvidence of incremental change for increased gender equity.Repeated analysis of PDPs, FARs, climate, and demographicsData reported to steering committee to determine new steps.Improved outcomes from all measures.AnnuallyDisseminate new knowledge on system level change for gender equityNumber of articles, presentations completedInformation on dissemination will be included in all reports.Lessons learned from systems-level approach used by other systems.Year 4, 5External EvaluationThe potential impact of this project on the national-level will be strengthened by regular communication between the internal evaluation team and external evaluators and enhanced by an evaluation model that mimics and examines the efficacy of the cascading, top-down model of the program itself. ?To do this work, we propose external evaluators, Axelson and Birmingham, with the support of North Dakota State University’s Center for Social Research. These external evaluators have experience in implementing and assessing programs related to gender equity issues through their work with NDSU’s ADVANCE IT grant (2009).As extension of the internal evaluation, the external evaluation will provide periodic opportunities to step back from the project to critically and comprehensively assess its progress and direction. ?In addition to the annual reports that the external evaluators will complete with data from the internal evaluator and all participating campuses, the external evaluators will conduct site visits and in-depth data collection and analyses at critical junctures in the project’s implementation. These analyses, likely to be conducted at the end of Project Years 3 and 5, will be designed to gain deeper insight into the project’s operation and the programmatic adjustments that could further enhance its effectiveness. Additionally, at the conclusion of the project, the external evaluators will provide a summative evaluation of the program’s effectiveness and further recommendations for improvement as other institutions employ this approach.Because training and communication surrounding policy analysis and change in practices will be the central means of driving the implementation of the program, the external evaluation will initially focus on tracking the effectiveness of the policy change rollout. The evaluators propose Kirkpatrick’s four-level model for evaluating training effectiveness (c.f., Kirkpatrick 2006) provides a useful framework for the evaluation (See Table 4). Kirkpartick’s model examines the sequential process leading from the training session(s) where participants experience affect (reaction) toward the training and its content, develop new skills and knowledge (learning), and transfer these new skills and knowledge to their work practices (behavior/practice). Lastly, after there has been sufficient time for the changes to impact the organization, the question of the extent to which these new practices have led to the desired changes in the workplace environment and practices (results) is addressed. ?The information to assess the four-levels of the Kirkpatrick model will be collected for each unit of analysis as indicated by the cells in the following grid. To the greatest extent possible, the data for these cells will come via internal evaluation, but with instruments developed collaboratively between the internal and external evaluation teams. The external evaluation will conduct additional analyses of the existing internal data and collect supplementary data, when needed, to clarify understanding of the program implementation and results.This cascading model is especially appropriate for evaluating change in top-down hierarchies, because it offers the opportunity to pinpoint where change has stalled (or using Lewin’s term, where practices remain “frozen”) so that appropriate interventions can be applied. Moreover, this model measures this specific project’s goals extremely effectively: 1. it measures the extent to which system-wide priorities are accepted, understood, lead to changes in practice, and produce results at the system, institution, and individual level; 2. it gathers data to measure incremental change; 3. it offers a framework for analyzing and disseminating the efficacy of a top-down approach, and sharing with other state institutions areas of particular success and places in the hierarchy that created particular challenge. Because this is a multi-institution study, the results gathered by external evaluators are likely to be disseminated in peer-reviewed journals, making the work of the project team more helpful to individual members seeking tenure and promotion.Table ?4 : Using the Kirkpatrick model for external evaluation of a top-down system changeUnit of Analysis:Outcome:ReactionLearningPracticeResultsSystem (evaluate the process used to communicate the policy change from the system to the leadership of each institution Short training surveys for roll-out participants Interviews with institutional leadership about their acceptance of policy changesShort survey of project roll-out participants and group of similar non-participants.interviews with institutional leadership ?about their knowledge of policy changesDocument evaluation: follow policy changes through document updates, directives to institutions, processes and trainings 12-indicator data from?each institution demonstrates greater system-wide diversity, retention, promotion Institution (evaluate the process for relaying change from the institutional leadership to its members)Interviews with institutional leadership about their acceptance of policy changesInterviews with institutional leadership ?about their knowledge of policy changesDocument evaluation: follow policy changes through document updates, directives to institutions, processes and trainings12-indicator datacomparisons of Work/life data across institutionsIndividual (1. how message was received by the individuals and implemented into their workplace practices; 2. ?extent to which ?diversity in individual recruitment, retention and promotion can be correlated with systemic changes.)Training surveysFocus group on policy change (by gender and rank)Training surveysFocus group on policy understanding (by gender and rank)Workplace practice surveys?and Document analyses document updates, directives to individuals and departments, processes and trainings12-indicator data comparisonsInstitutional climate surveysWork/life surveysBroader ImpactsDespite the many successes of past ADVANCE IT implemented projects, a sustained and sustainable impact on change in academic climate at any institution has proven difficult to overcome (see, for example, the UM ADVANCE Program 2013 report on assessing change in academic climate in 2001, 2006 and 2012). In this proposal, we seek to establish that an effective change in climate (the individual experience) happens not through further programming at the institutional or individual level, even if that is where the effects of climate are both felt and reported. Rather, the impetus for effective change in academic climate may be a result of a “top-down” process implemented by a system-level centralized authority (Scott 1987). The establishment of progressive centralized policies and consistent institutional implementation of these policies results in the creation of new organizational norms; these organizational norms, of course, are the foundation of the working academic climate.We anticipate these findings, combined with our research innovations and drawing on best practices from previous ADVANCE recipients, will produce the following broader impacts. First, our work will provide evidence for the establishment of progressive centralized policies and consistent institutional implementation of these policies for sustainable change in the working academic climate. We expect to identify key points in hierarchical structures which both encourage and discourage change, and demonstrate programmatic and procedural strategies for success. Second, our research activities for knowledge building will also produce procedures and processes for establishing gender neutral evaluation and building women faculty skills in career negotiation and planning, also through a centralized policy impetus. We expect that both methodological procedures for capturing the individual experience, and the procedures and implementation processes arising to address gender bias in a system will be models for increasing equity in higher education. Third, by producing change in our systemic context—a state-university system with a strong central authority—the policies developed in this project can be transported as a model for other state higher education systems in both rural and non-rural environments.Results from Prior NSF SupportNone. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download