Course Evaluation Report - Eppley Institute



Course Evaluation Report

Advanced Facility Manger Practices

Timothy Harvey

Facility Management Officer

WASO – PFMD

National Park Service

Elizabeth Dodson

Training Manager

WASO – PFMD

National Park Service

October 2008

Completed in accordance with Sub Agreement 48 of the National Park Service-

Indiana University Cooperative Agreement CA 2670-97-001

Stephen A. Wolter

Executive Director

Christy McCormick

Project Team

Amy Gregor

Project Team

Zachary Carnagey

Project Team

Eppley Institute for Parks & Public Lands

Indiana University Research Park

501 N. Morton Street, Suite 100

Bloomington, IN 47404

812.855.3095

Acknowledgements

The following individuals contributed to the course development or instruction of the Advanced Facility Management Program.

Dana Anderson Indiana University

Dan Blackwell National Park Service Peggy Buchanan Vista Del Monte

Zachary Carnagey Eppley Institute

Teresa Dickinson National Park Service Betsy Dodson National Park Service

Russel Galipeau National Park Service Steve Homan National Park Service

Don Mannel National Park Service

Yvonne Menard National Park Service Jeri Mihalic National Park Service

Shawn Norton National Park Service

Dave Park National Park Service Sherry Plowman National Park Service

Bill Thompson National Park Service

Steve Wolter Eppley Institute

The following individuals assisted with the development of the Advanced Facility Management Program.

Matthew Berry Eppley Institute

Zachary Carnagey Eppley Institute

Amy Gregor Eppley Institute

Cary Hair Eppley Institute

Christy McCormick Eppley Institute

Polly Nuest Eppley Institute

Allison Parman Eppley Institute

Matt Wolf Eppley Institute

Steve Wolter Eppley Institute

This document may not be duplicated without the permission of the Eppley Institute for Parks and Public Lands, acting on behalf of Indiana University. The National Park Service and federal agencies may duplicate it for training and administrative purposes, provided that appropriate written acknowledgement is given. No other state or local agency, university, contractor, or individual shall duplicate the document without the permission of Indiana University.

Copyright 2008, the Trustees of Indiana University

on behalf of the Eppley Institute for Parks and Public Lands

TABLE OF CONTENTS

INTRODUCTION 1

Course Assessments 2

PROCESS OF EVALUATION 3

Methodology 3

Evaluations of the Instructors 3

Participant Course Evaluations 3

INSTRUCTOR EVALUATION 4

The Instructors 4

Course Agenda and Corresponding Guest Speakers 4

Quantitative Instructor Feedback 7

Qualitative Instructor Feedback 11

COURSE EVALUATION 13

Quantitative Course Feedback 13

Qualitative Course Feedback 18

RECOMMENDATIONS 20

Instructor Recommendations 20

Course Content Recommendations 20

Commitment of NPS Leaders to the FMLP 21

APPENDICES 22

APPENDIX A: Instructor Evaluation 23

APPENDIX B: Online Course Evaluation 25

APPENDIX C: One- Minute Papers 29

TABLE OF FIGURES

Figure 1a: Q#1- The instructor knew the subject matter well. 8

Figure 1b: Q#1-The instructor knew the subject matter well. (cont.) 8

Figure 2a: Q#2- The instructor made the subject matter interesting. 9

Figure 2b: Q#2- The instructor made the subject matter interesting. (cont.) 9

Figure 3a: Q#3- The instructor encouraged student involvement. 10

Figure 3b: Q#3- The instructor encouraged student involvement. (cont.)Qualitative

Figure 4: Overall Course Rating-About the Course 14

Figure 5a: Overall Course Rating- About the Learning 15

Figure 5b: Overall Course Rating- About the Learning (cont.) 15

Figure 6: Most Helpful Assignment/Class Activity 17

Figure 7: Least Helpful Assignment/Class Activity 17

INTRODUCTION

Both the Department of the Interior (DOI) and the National Park Service (NPS) have targeted the facility management profession as “mission-critical” due to the importance of maintaining the built environment and the high numbers of facility managers eligible to retire in the near future. In anticipation of the growing need for competent facility management, the NPS has developed a comprehensive set of facility manager competencies which form a roadmap for the development of the facility management workforce. Additionally, the Facility Manager Leaders program (FMLP), of which the Advanced Facility Management Practices (AFMP) is one of the core courses, aims to develop a new cadre of competent facility managers for the future.

The AFMP course builds on the knowledge gained by successful completion of the Principles of Asset Management course and courses completed during the first independent study period titled Distributed Learning Session One. At AFMP, students had the opportunity to use their knowledge, judgment, and experience to anticipate, prepare, and plan for critical issues facing NPS facility managers.

This course was designed to provide facility management personnel with an understanding of the importance of park strategic plans and mission goals in the development of an Annual Work Plan (AWP). The course demonstrates how mandates, regulations, and policies guide the application of the Facility Management Software System (FMSS) in the management of park assets.

Course Description and Rationale

AFMP was a two-week instructor-led course using a teaching team that provided lecture, discussion, and small group activities, concluding with the creation of an AWP. All coursework and instruction was couched in the simulated environment of a fictitious park, Sea Otter Island National Park (SEOT). A student completing this class met facility manager competencies in the NPS by being able to do the following:

• Plan operations, maintenance, repair, and alterations programs using a mission-focused Asset Management Plan (AMP), adjusting plans through evaluation to fit within budgeted funds.

• Utilize the AMP to link facility management and develop the most efficient organization, program accountability, and defendable management decisions for assets.

• Utilize data from the FMSS to develop a credible work program that manages budget allocations associated with setting priorities, meets required mandates, and responds to management decisions in the framework of the Stewardship of Federal Facilities.

• Identify and execute the required elements of the FMP Standard Business Practices (Directors Order No. 80), including related components of park unit business plans and industry standards.

• Lead colleagues, subordinates, and managing supervisors to understand the visitor, personal, park unit, and Service-wide benefits of proactive asset management and use of the FMSS.

• Coordinate the creation of an Annual Work Plan (AWP) to demonstrate and advocate the benefits of using the FMSS for park unit management through the use of data analysis and data driven decision making.

Course Assessments

Assessments for the course were both formal and informal, and individual and group. Formal assessments included the following:

|PMIS Project Submittal |

|Oral Presentation with PowerPoint |

|Project Plan: Annual Work Plan |

|Functional Analysis |

|Critical Systems Priority List |

|Oral Annual Work Plan |

|Written Annual Work Plan |

|Team Assessment |

|Class Participation |

Except for the Oral Presentation with PowerPoint, Team Assessment, and Class Participation, all formal assessments were group assessments. This group assessment approach was designed to reflect and foster the development of the facility manager competencies for Leading People (VI.B) and Building coalitions and communication (VI.E).

Individual assessments, besides those mentioned above, were primarily informal and reflected principles of adult learner theory, especially the concept that adult learners take more responsibility in their learning. These assessments, therefore, gave students the chance to assess their own knowledge with the expectation that, should they find some deficit, students would seek additional instruction from their cohort or instructors.

An example of these informal assessments was the homework from the Excel Workshop. At the completion of this session, the instructor gave the students homework which required the use of skills developed during the session. The following morning, the instructor discussed the answers and provided students the opportunity to ask questions if their answers differed. For two of the homework problems, some students did have different answers, and the class discussion that followed included remediating instruction from not only the instructor, but also from fellow classmates. Though informal and not graded, assessments such as these allowed instructors to gage student learning and modify instruction as needed, without reducing classroom instruction time or subjecting students to the added stress of formal assessments.

Additionally, mastery of the content presented in the sessions during the AFMP was required in order to produce a comprehensive AWP.

PROCESS OF EVALUATION

The information contained in this report comes directly from data collected in the form of individual student and team member evaluations of the Advanced Facility Management Program course, held in Thousand Oaks, California, from October 20th to October 30th, 2008.

Methodology

The instructor effectiveness was assessed using data collected during the AFMP course, while course evaluations occurred one week after course completion. The details regarding how each of these components was implemented are included below. Students were also asked to provide feedback at the end of each day in the form of “One Minute Papers”. (Appendix C). This feedback allowed the course coordinators to make minor course corrections immediately but is not part of the summative evaluation.

Evaluations of the Instructors

The two-week course included a great deal of new information, multiple sessions per day, and a variety of guest speakers per day. Students were provided with instructor evaluation forms on which they could write their feedback and observations regarding each instructor’s session(s). The instructor evaluation tool allowed for a quantitative measurement of the instructor’s effectiveness as well as space for qualitative comments about each instructor. These evaluations were collected at the end of each class day (Appendix A).

Participant Course Evaluations

At the conclusion of the course, students were asked to complete an online course evaluation. The evaluation tool, developed by the Eppley Institute, included both quantitative and qualitative questions regarding the course. The evaluation included questions regarding course organization and content, instructors, and improvements that could be made to the course (Appendix B).

INSTRUCTOR EVALUATION

The Instructors

The guest instructors were selected based on identified course and student needs, including subject matter expertise, training and education skills, and the instructors’ availability and willingness to participate in the course. The instructors were highly credible and represented all levels of the NPS, including the National, Regional, and Park levels. The many perspectives of the NPS, provided through the guest speakers’ variety of experience levels, allowed the students to gain a holistic view of facility management. Participants were able to develop a basic understanding of other divisions and areas in the NPS and their applicability to Facility Management. This enhanced understanding supports the development of leadership skills.

Course Agenda and Corresponding Guest Speakers

The following tables provide the course agenda and the guest speakers for each session.

Week I

Monday, October 20th, 2008

Time Session Title Guest Speakers

8:00 am-8:45 am Course Introduction and Welcome to the Park Betsy Dodson

Dan Blackwell

8:45 am-11:30 am Student Presentations Students

11:30 am-12:30 pm Lunch

12:30 pm-3:00 pm Student Presentations (continued) Students

3:00 pm-4:00 pm Asset Management Program Update Don Mannel

Dan Blackwell

4:00-4:15 pm Lecturette: Unplanned Work Don Mannel

4:15 pm-4:45 pm One-Minute Paper, Instructor Evaluation Christy McCormick

Tuesday, October 21st, 2008

Time Session Title Guest Speakers

8:00-8:15 am Announcements Course Coordinators

8:15 -9:30 am Introduction to Sea Otter Island Steve Wolter (webcast)

Don Mannel

9:45-11:45 am Discussion of Scientific Method and Zach Carnagey

Data Analysis Tools Steve Homan

11:45-12:45 pm Lunch

12:45-2:00 pm Work Types and their importance Jeri Mihalic

in developing an Annual Work Plan

2:15-2:30 pm Lecturette: Productive Hours of your Workforce Don Mannel

2:15-4:15 pm Project Management Information System Sherry Plowman

4:15-4:45 pm One-Minute Paper, Instructor Evaluation Course Coordinators

Wednesday, October 22nd , 2008

Time Session Title Guest Speakers

8:00-8:15 am Announcements, Excel Homework Course Coordinators

8:15 -8:45 am Squad Meeting: Visitation Trends SEOT Supt (Acting)

Don Mannel

8:45- 9:45 am Project Planning Christy McCormick

9:45 -4:15 pm Student Work on SEOT SIMULATION Students

4:15 pm-4:45 pm One-Minute Paper, Instructor Evaluation Course Coordinators

Thursday, October 23rd, 2008

Time Session Title Guest Speakers

8:00 -8:15 am Announcements Course Coordinators

8:15-10:30 am Cultural/Natural Resource Issues in FM Russell Galipeau

10:30- 12:30 pm Group Work on SEOT SIM Students

12:30 pm-1:30 pm Lunch

1:00 pm- 3:00 pm Informal Visitor Contact: Supporting your Employees Yvonne Menard

3:15 pm-4:30 pm Accessibility Dave Park

4:30 pm-4:45 pm One-Minute Paper, Instructor Evaluation Course Coordinators

6:30 pm-8:00 pm Time Management: How to Prioritize Work Jeri Mihalic

Christy McCormick

Friday, October 24th, 2008

Time Session Title Guest Speakers

8:00-8:15 am Announcements Course Coordinators

8:00 -9:00 am Squad Meeting: PAMP and PAMP Execution SEOT Supt (Acting)

Betsy Dodson

9:00-9:30 am Lecturette: Fleet, Fuel, and Other Fixed Costs Don Mannel

9:30-4:00 pm Group Work on SEOT Simulation Students

4:00-4:30 pm Lecturette: Seasonality and Special Events Jeri Mihalic

4:30 pm-4:45 pm One-Minute Paper, Instructor Evaluation Course Coordinators

Week II

Monday, October 27nd, 2008

Time Session Title Guest Speakers

8:00-8:15 am The Good of the Day & Announcements Course Coordinators

8:15 -9:30 am Squad Meeting: Solid Waste Reduction Teresa Dickinson

9:30-11:30 am Leadership Strategies and Skills -Part One Steve Wolter

11:30-12:30 pm Lunch

12:30 -2:00 pm Health and Wellness Peggy Buchanan

2:15-4:15 pm Leadership Strategies and Skills -Part Two Steve Wolter

4:15-4:45 pm One-Minute Paper, Instructor Evaluation Course Coordinators

Tuesday, October 28th, 2008

Time Session Title Guest Speakers

8:00-8:15 am The Good of the Day & Announcements Course Coordinators

8:15-8:45 am Squad Meeting: Cyclical Funding Bill Thompson

8:45 -11:30 am Writing for Impact-Part One Dana Anderson

11:30-12:30 pm Lunch

12:30 -2:30 pm Writing for Impact-Part Two Dana Anderson

2:30-4:30 pm Sustainability Policy and Regulations Shawn Norton

4:30-5:00 pm One-Minute Paper, Instructor Evaluation Course Coordinators

5:00- ?? Evening Social with Mentors

Wednesday, October 29th, 2008

Time Session Title Guest Speakers

7:00 am-12:00 pm Field Experience for Students Jeri Mihalic, Christy McCormick

8:00 am- 12:00 pm Mentor Workshop Betsy Dodson,

Steve Wolter

12:00 pm-1:15 pm Student Lunch with Mentors

1:15 pm-1:45 pm Mentoring Assessment—Protégé Steve Wolter

2:00 pm-3:30 pm Distributed Learning Session #2 Planning Christy McCormick

& Discussion of Mentoring Assessment

3:30 pm-4:00 pm One-Minute Paper, Instructor Evaluation Course Coordinators

7:30 pm-8:30 pm SEOT Presentation Dress Rehearsal

Thursday, October 30th, 2008

Time Session Title Guest Speakers

7:00 am- 8:00 am Individual and Group Photos

8:00 am-11:30 am Group SEOT Presentations Evaluation Panel

11:30 am-12:30 pm Lunch

12:30 pm-4:30 pm Group SEOT Presentations

4:30 pm-5:00 pm Course Conclusion and Evaluation

5:30 pm-8:30 pm Group Dinner

Friday, October 31st, 2008

Travel Day

Quantitative Instructor Feedback

The course evaluations asked three quantitative questions related to the teaching effectiveness of each session. Therefore, instructors who taught more than one session were evaluated as many times as they had session. Participants were instructed to indicate the degree to which they agreed with each statement using a five-point Likert scale (1=Strongly Disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly Agree). The evaluations included the following questions:

Question #1: The instructor knew the subject matter well.

Question #2: The instructor made the subject matter interesting.

Question #3: The instructor encouraged student involvement.

Overall, most instructors were ranked highly in each of these question areas. The grand mean for all instructors for Questions #1 – 3 were 4.83, 4.62, and 4.61, respectively (see Figures 1, 2, and 3). Students felt strongly that the instructors knew their subject matter well, as all individual mean scores on Question #1 were above 4.5. There was slightly more variety in the instructor scores in Questions #2 and #3. This is also reflected in the comments about individual instructors.

[pic]

Figure 1a: Q#1- The instructor knew the subject matter well.

[pic]

Figure 1b: Q#1-The instructor knew the subject matter well. (cont.)

[pic]

Figure 2a: Q#2- The instructor made the subject matter interesting.

[pic]

Figure 2b: Q#2- The instructor made the subject matter interesting. (cont.)

[pic]

Figure 3a: Q#3- The instructor encouraged student involvement.

[pic]

Figure 3b: Q#3- The instructor encouraged student involvement. (cont.)

Qualitative Instructor Feedback

Participants also had the opportunity to provide additional feedback on the instructor evaluation form. Specifically, they were asked, “Please comment on the overall effectiveness of this instructor and any suggestions you have for improvements.”

A total of 201 comments were analyzed. In general, students had positive comments about the overall effectiveness of the course instructors. Students most frequently mentioned the excitement and passion of the instructors, as well as their knowledge of the subject and presentation skills. Students also appreciated the high level of student involvement, use of practical examples, useful information taught, and the variety of perspectives of the instructors. Suggested areas of improvement included placing some classes earlier in the course (e.g. writing course), providing more time for certain sessions, and reducing purely lecture-based instruction.

A representative sample of comments is included, categorized by instructor effectiveness and suggestions for improvements.

Overall effectiveness of this instructor

• “Excellent Speaker. Incredible knowledge and experience. [It is] motivational to believe in and pursue challenges of the FMSS and facility management. Thank you!”

• “Information was very useful and was a new ways of looking at a task that I had previously performed.”

• “Great energy and great information. Using what she was teaching was great.”

• “Vital information, needed at my job!”

• Great presentation. A wonderful way to introduce and talk about a sometimes dry topic.”

• “Hands Down—Excellent. I learned new Information and learned new speaking skills as well.”

• “Excellent Speaker. Incredible knowledge and experience. Highly motivated to believe in and pursue challenge of the FMSS and facility management. Thank you!”

• “Good way to involve students and encourage group skills.”

• “Lots of great insight from someone in the trenches.”

Suggestions for Improvement

• “Enjoyed the presentation, but it was very fast. It was hard to follow. I [am] a little technically challenged.”

• “The assignment was wider than expected…hard to get a real product.”

• “Subject matter was very dry. Could have used more interaction. Presenter was too involved with the actual laws and not about the actual solutions that we need in the field. Topic was interesting, but not delivered in an engaging way.”

• “The instructor obviously knew the material, but some crucial technical points seemed glossed over. Good job overall.”

• “Very effective, I wish we heard this before we started the AWP.”

• “This was excellent; however, I wish it could have been earlier in the course because we really needed more time to work on our projects.”

• “A lot of info at one time. Great info, but a lot.”

COURSE EVALUATION

Quantitative Course Feedback

The first two sections of the course evaluation asked students to rate the course and their levels of learning on a five-point Likert scale (1=Strongly Disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly Agree). By asking questions in this quantitative format, mean scores were calculated, providing an overall picture of participant feedback. Figures 4, 5a and 5b show the results.

Quantitative questions were divided into two categories: 1) About the Course and 2) About Learning.

1. About the Course

a) The objectives for each session were clearly communicated to the students.

b) The course maintained my interest and participation through a variety of techniques and activities.

c) The course was well organized.

d) The student materials were useful and valuable during class.

e) The use of multiple instructors helped to facilitate my learning.

f) The SEOT simulation was valuable in helping me to learn how to create an annual work plan.

g) Overall, the National Park Service should rate the quality of this course as outstanding.

h) The field exercise was a valuable practical experience.

2. About Learning

i) As a result of this course, I feel more confident about speaking in front of a group.

j) As a result of this course, I have improved my writing skills.

k) I understand the steps involved in developing an annual work plan.

l) I understand the basic principles of Informal Visitor Contact as they apply to facility management.

m) I understand the basic principles of Cultural and Natural Resource Issues as they apply to facility management.

n) I can apply basic principles of leadership in my job.

o) I will be able to apply the information presented in this course in my job.

p) The mentor-student relationship is a valuable part of the Facility Manager Leaders Program.

[pic]

Figure 4: Overall Course Rating-About the Course

In general, students gave positive feedback about the course. The highest mean (m=4.59) showed that students were most satisfied with having multiple instructors to facilitate their learning. (Figure 4). This information is also reflected in the comments on the instructor evaluations. Students felt less strongly about the value of the field exercise (m=3.65).

In the evaluation of the learning that took place in the AFMP course, students gave positive feedback, with little variation among questions. As seen in Figure 5b, students did feel least comfortable (m= 3.82) with their knowledge about the steps involved in developing an annual work plan.

It is also important to note the change in students’ confidence about speaking in front of a group. Prior to attending AFMP, the students participated in the Principles of Asset Management course in April 2007, the first of the five-part course of study for the Facility Manager Leaders Program (FMLP). Although the overall mean for students’ confidence in speaking in public increased only slightly between the two courses, Principles, m=4.08 and AFMP, m=4.18, the number of 4 and 5 ratings increased dramatically.

[pic]

Figure 5a: Overall Course Rating- About the Learning

[pic]

Figure 5b: Overall Course Rating- About the Learning (cont.)

In addition to the “About the Course” and “About the Learning,” two further quantitative questions were posed to the students, “Which five assignments/class activities were MOST HELPFUL in this course?” and “Which five assignments/class activities were LEAST HELPFUL in this course?” For these questions, students were to indicate their choices by checking a box next to five of the activities from the following list:

|Functional Analysis (staffing model) (Homework) |

|PMIS Project Submittal (Homework) |

|Creating a project plan for the Annual Work Plan (Homework) |

|Writing a briefing statement (Group Presentation) |

|Creating and using the PowerPoints (Group and Individual Presentations) |

|Creating a written annual work plan (Group Presentation) |

|Giving an oral presentation (Group and Individual Presentations) |

|M&M Activity (Scientific Method Session) |

|Excel Tutorial (Data Analysis Tools Session) |

|Matching Work Types activity with sticky notes on flip charts (Work Types Session) |

|Instruction on creating a project plan (Project Planning Session) |

|Case Study (Russ Galipeau's session on Cultural/Natural Resource Issues) |

|Role playing in groups (Informal Visitor Contact Session) |

|Assessing your health/wellness activity sheet (Health/Wellness Session) |

|Trying exercises for the office (Health/Wellness Session) |

|Writing activities (Writing for Impact Session) |

|Mentor/Protege evaluation with transparency (Time in the class working with mentors) |

|Time management activities (Time Management optional session) |

The activities which students identified as most helpful included “Giving an oral presentation (Group and Individual Presentations),” chosen by 58.8% of students and “Creating a written annual work plan (Group Presentation)” with 52.9% (See Figure 6) – less than half of the responses correlated for the remaining activities/assignments. Considering that so many students had mentioned that, as a result of the course, they feel more comfortable speaking in public, it is hardly surprising that “Giving an oral presentation” was chosen most frequently.

By contrast, the assignments/activities that students identified as being least helpful demonstrated more consensus. Seventy-five percent of students chose “Role playing in groups (Informal Visitor Contact Session)” as one of the five least helpful activities/assignments, while 62.5% chose “M&M Activity (Scientific Method Session)” (See Figure 7). It is remarkable to point out that, in both the “Most Helpful” and “Least Helpful” lists, every assignment/activity received at least one vote.

[pic]

Figure 6: Most Helpful Assignment/Class Activity

[pic]

Figure 7: Least Helpful Assignment/Class Activity

Qualitative Course Feedback

The course evaluation also asked the students to reflect on two questions relating to their expectations and future improvements. The majority of students indicated that their expectations were met, and some exceeded. However, it is significant to point out that the majority of those expectations were that AFMP was a very challenging course. Nearly half of the suggestions for improvement concerned reducing the workload or introducing mandated relaxation time, while others concerned providing more preparation before the course. A review of the feedback will be useful in planning future training courses. Representative quotes are included for each question to provide a more detailed understanding of participant attitudes.

Please comment on your expectations of the course and were those expectations met?

• “I expected to work very hard and spend lots of time doing that work. Those expectations were certainly met. I did, however, think that there would be more classroom training.”

• “Wow.... my expectations are greatly exceeded. The class and homework sessions were fantastic. I can directly apply what I have learned in my home park.”

• “I expected the exercise to be time consuming and difficult however, I found the project well worth the time and difficulty. I am grateful to the staffs of FMLP and Eppley for providing the opportunity to go so far beyond my known skills, and abilities to accomplish the unique challenges of the task.”

• “Overall, I was pleased with the course in the context of my expectations. On the micro level, I was disappointed with some sessions that held great promise (Excel) and pleasantly surprised by others (Health and Wellness).”

• “My expectations were based on hearsay from previous participants, I heard it was going to be a very intense two weeks. Yes, those expectations were met.”

Do you have any suggestions as to how the course could be improved?

• “Mandatory time-off on the weekend--just 2-3 hours for mental and physical health”

• “Take more time before the course to become aware of what is going to happen at the course. Unless it is meant to be unknown.”

• “I would like to see more pre-course work that directly ties to what is expected in the class.”

• “…Lastly I would recommend an e-course for excel prior to this week. For those who identify themselves as novices in using Excel, they should either attend a traditional class or take an e-course during DLS#1 so they could be ready to go with the annual work plan when they reach Advanced Principles. It was very hard for some in my team to "understand the asset principles" let alone analyze the data when they didn't know the basics of filtering and interpreting the excel data.”

• “The assignments (staff analysis, PMIS project, critical systems list, and the annual work plan plan) were beneficial in the production of the Final Annual Work Plan but they added a stress level that was paralyzing at times. If a couple of them could've been done during classroom time rather than as homework - say the critical systems list and the AWP plan, that would've taken some of the pressure off and made the whole exercise more conducive to learning rather than cramming…. I refer to the AFMP as boot camp - now I'm ready for the real battle (I hope).”

• “I understand the course was serving dual masters in terms of providing a learning experience and informing a service-wide effort to produce an annual work plan template. Given that, I am worried the effort was slightly off balance, as the level ambiguity did not lend itself to providing valuable options for future annual work plan development efforts. The missing piece in my opinion was an understanding of PAMP's implications in real world terms. Essentially, the students were asked to use theoretic, macro-level sources (PAMP, GMP, EMS) to produce a practical, micro-level document without any examples of the methodologies and/or logic required. This was a tall order given the time allotted and the student's varying levels of understanding.”

• “I didn't feel like there was enough discussion about what the outcome of the simulation was supposed to be. I needed a tangible product that I could develop. Maybe discussing some different forms of what is desired with examples would be helpful if you have to use this submersive teaching technique….”

RECOMMENDATIONS

Based on the different evaluation methods used for the AFMP course, a variety of recommendations can be made to increase the effectiveness of this course in the future. What follows is a list of the evaluation team’s recommendations for the AFMP course. These conclusions represent the most significant concerns of the evaluation team, as developed through the course analysis.

Instructor Recommendations

1. Most instructors received positive feedback as to their knowledge, effectiveness, and instructional delivery methods. These instructors should be invited to return. Those instructors that received marks of lesser value or comments for improvement should be evaluated to determine future inclusion.

Course Content Recommendations

2. Reflected in lower scores in About the Course, About Learning, and student comments, students did not feel AFMP prepared them to create an AWP. Students mentioned wanting to have a clearer idea about what an AWP was and expressed frustration at having to simultaneously decide what an AWP should be and populate it with data. The development team should reevaluate whether this developmental aspect of AWPs will be valuable or whether an AWP template should be used in future courses.

3. As in past years, one of the most frequently mentioned issues was time management: students felt that the pace of AFMP was too fasted paced and did not feel that they had time to relax. One recommendation from a student for the design team to consider is to provide more classroom time for the supplemental assignments (e.g. staff analysis, PMIS project, critical systems list, the annual work plan project plan) to reduce the amount of homework time spent away from AWP development.

4. The three assignments/activities (designated by 50% consensus or higher) which students indicated were least helpful should be reviewed by the design team. These included activities from Informal Visitor Contact, Scientific Method, and Time Management optional session. These activities need to be reevaluated to ensure that they support the goals and objectives of these sessions and that they are relevant to facility management. They can either be modified or removed from the course.

5. Several students mentioned the need for more training on Excel. Since this tool is of particular importance for facility management, a more thorough training should be considered, occurring prior to AFMP.

6. Students did not feel strongly about the value of the field trip. Some issues may be that it fell at the end of the course and that it was not tied directly to course material. The objectives of the field trip and its placement in the agenda should be revisited to make it a more effective and valuable experience.

Commitment of NPS Leaders to the FMLP

7. The students valued the commitment of facility management leaders in the NPS to this course. They gained new information from the experience of these high-level individuals, and they expressed their thanks at the obvious commitment to this program. In future courses, National and Regional level leaders in facility management in the NPS should continue to be present.

8. The experiences and knowledge levels and varying perspectives of the guest speakers were also appreciated by the students. This high level of instructors should be continued in future courses.

APPENDICES

APPENDIX A: Instructor Evaluation

Bill Thompson

1. The instructor knew the subject matter well. 1 2 3 4 5

2. The instructor made the subject matter interesting. 1 2 3 4 5

3. The instructor encouraged student involvement. 1 2 3 4 5

Please comment on the overall effectiveness of this instructor and any suggestions you have for improvements.

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Dana Anderson

1. The instructor knew the subject matter well. 1 2 3 4 5

2. The instructor made the subject matter interesting. 1 2 3 4 5

3. The instructor encouraged student involvement. 1 2 3 4 5

Please comment on the overall effectiveness of this instructor and any suggestions you have for improvements.

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Shawn Norton

1. The instructor knew the subject matter well. 1 2 3 4 5

2. The instructor made the subject matter interesting. 1 2 3 4 5

3. The instructor encouraged student involvement. 1 2 3 4 5

Please comment on the overall effectiveness of this instructor and any suggestions you have for improvements.

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

APPENDIX B: Online Course Evaluation

[pic]

[pic]

[pic]

[pic]

[pic]

APPENDIX C: One- Minute Papers

One Minute Papers – AFMP October 2008

Monday – Week 1

|Question |Feedback |

|1. What has been the most important or useful information so far? | |

| | |

| | |

| | |

| | |

| | |

|2. Have there been any concepts or ideas that were unclear or incomplete?| |

| | |

| | |

| | |

| | |

| | |

|3. What else do you need to learn during this class? What else would you | |

|like to know? | |

| | |

| | |

| | |

-----------------------

Overall Instructor

Average: 4.83

Overall Instructor

Average: 4.83

Overall Instructor

Average: 4.62

Overall Instructor

Average: 4.62

Overall Instructor

Average: 4.61

Overall Instructor

Average: 4.61

National Park Service

U.S. Department of the Interior Instructor Evaluation

Facility Manager Leaders Program

National Park Service

U.S. Department of the Interior Instructor Evaluation

Facility Manager Leaders Program

Tuesday, October 28, 2008

Today, please take a few moments to comment on the effectiveness of the instructors for the Advanced Facility Management Practices course. All responses will be kept confidential and will only be used for future improvements in the Facility Manager Leaders Program (FMLP).

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download