1



Evaluation Report

Community of Learning Program of SEFLIN, Inc.

Submitted by

Minges & Associates

Jim Minges and Brenda Hough, Consultants

Table of Contents

Section Pages

Executive Summary 1

Introduction 2

Methodology 3

Conclusions 4-6

Recommendations 7-10

Appendices 10-13

I. Executive Summary

The Southeast Florida Library Information Network (SEFLIN) provides a strong program of training for member library personnel through its Community of Learning Program. This evaluative review, conducted by Minges & Associates at the request of SEFLIN, concludes that the SEFLIN training program can be further strengthened in several respects:

Content and Delivery

• Continuation and expansion of training programs for paraprofessional and support staff.

• Increased efforts to develop true Community of Learning programs emphasizing personal networking and special interest groups in addition to actual instruction.

• Continuing efforts to promote and explore the potential of the SEFLIN Connect service.

• Exploring innovative new approaches to training including:

o Hosting workshops presented by the American Library Association and its divisions, and other major external continuing education providers.

o Utilizing more local presenters.

o Exploring unconference and community of practice programs.

• Continuation of the annual conference, and exploration of additional programs that offer similar opportunity for networking and interaction

• Providing innovative and ongoing support for learning and using new technologies, including a “software sandbox” and loanable tech devices.

Communication

• Improving the usability of the Community of Learning web site.

• Replacing the social networking features of the Staff Networking Directory with standard social networking services in general public use.

Planning and Evaluation

• Making more effective use of committees and discussion/interest groups.

• Utilizing library visits and small discussion groups for improved continuing education planning and evaluation.

II. Introduction

SEFLIN has contracted with Minges & Associates to conduct an overview evaluation of the SEFLIN Continuing Education / Staff Development program in terms of library and library staff needs. The following questions were posed for this evaluative study:

1)      Is there a better way to identify topics and issues and then find speakers or programs?

2)      SEFLIN has good participation with self-paced learning but does not want to forget about instructor led learning. Is there a balance that SEFLIN should aim for?

3)      SEFIN uses self-selected “area of interest” mailing lists and announcements to the Training Administrators to pass along to their library/library system staff in order to advertise the courses. Are there staff members that SEFLIN is consistently missing?

4)      Is the SEFLIN/Community of Learning web page easy to use to locate and register for courses and events? What could be improved?

5)      What can SEFLIN do as a regional organization to support and provide follow-up for learners?

6) SEFLIN has a “standard” online evaluation survey – is it an effective tool? What improvements are suggested in evaluating programs?

Based on those stated issues, and further discussion with SEFLIN staff, Minges & Associates proposed a study with the goal of addressing those questions, and such other issues as might emerge during the course of the project. The review was conducted during August and September of 2010.

This report includes the methodology used in the evaluation, statements of conclusions and recommendations, and appendices with additional information and examples of products and services used by other regional library systems offering continuing education services.

III. Methodology

This review utilized the following personnel and methodology.

1. A critical review by Minges & Associates consultants of the SEFLIN web site and of extensive SEFLIN continuing education program documentation provided by SEFLIN including continuing education program descriptions, participant evaluations, and annual evaluation/needs assessment documents.

2. Comparison of the SEFLIN continuing education program with those of a peer group of at least three (3) other regional continuing education cooperatives. Peer cooperatives that were reviewed during the course of this project include the Northeast Florida Library Information Network, Tampa Bay Library Consortium, North Texas Regional Library System, Kansas City Metropolitan Library & Information Network, MINITEX, and SWON (Southwest Ohio and Neighboring Libraries).

3. Telephone interviews with twelve SEFLIN stakeholders including the SEFLIN Interim Director and the Assistant for Staff Development Services, eight members of the Training Administrator advisory committee, the Cataloging Discussion Group Leader, and the Annual Conference Committee Chair.

4. Development of discussion questions and guidelines for focus group discussions with SEFLIN member library personnel including both actual and potential continuing education participants. Because SEFLIN determined that there be no onsite visits by project consultants, it is proposed that SEFLIN personnel conduct those focus groups. Although ideally these focus groups would be an integral part of the evaluation, it is our opinion that this step can effectively be performed as a separate and subsequent activity.

IV. Conclusions

This section presents conclusions and findings we reached based on examination of SEFLIN web site and additional documentation, review of peer organizations, and telephone interviews. The perceptions of SEFLIN continuing education program varied widely among individuals interviewed, and in evaluative documentation such as user surveys and program evaluations. Therefore the conclusions stated below do not represent the views of any one person interviewed. However there are a number of themes and findings that recurred frequently enough to be stated as conclusions of this study.

General Findings

1. SEFLIN provides a strong program of instruction and training that compares favorably with similar regional library consortia. There is generally a very widespread and strong level of support for high level of support for the SEFLIN Community of Learning program. Among the unsolicited comments were: “training invaluable”, “SEFLIN does a good job of balancing the needs of large and small libraries”, “please be sure that the continuing education program continues”, “SEFLIN provides a very high quality education program.” This consistently high level of support and satisfaction is notable.

2. There is an equally consistent and widespread concern about the future of SEFLIN. This concern was expressed repeatedly and spontaneously. It appeared to be related to both uncertainties about continued funding, and a general feeling of uncertainty and anxiety as a byproduct of staffing changes during the past year. However, this issue is outside of the parameters of this review.

Course content and delivery modes

3. SEFLIN’s course topics appear to be very much on a par with the offerings of other library organizations that provide continuing education. There is no template or list of topics that can be recommended in this report. The appropriate selection of topics and presenters can only be determined through interaction with SEFLIN member libraries. When asked how they select the topics they cover in training, peer organizations said they asked for recommendations from members and watched what other organizations do. There will be several recommendations in this report related to course content. However, SEFLIN should generally have a high degree of confidence in the quality of its course content. SEFLIN has a reputation for delivering a solid menu of more traditional topics, but could be invigorated by additional types of innovative sessions, too. This report contains a number of recommendations in that regard.

4. There appears to be a quite equal degree of desire for instructor led onsite programs and self-paced online training. Particular persons strongly favor one method over the other, but overall no change in the basic balance between those delivery modes is recommended. It is important for SEFLIN staff to be familiar with the needs of each individual library because of the very strong preferences for delivery modes among different libraries. Although total attendance at web based training was relatively high, 80% of respondents to the 2009 CLP survey used online training for 25% or less of their training activities, and 50% did not use any online training programs.

5. SEFLIN evaluations included desires that are somewhat in conflict for: 1) more programming useful to support staff; and 2) more high level professional training. However the requests for training for paraprofessionals was repeated several times. In the course of interviews there was a very strong expression of interest in training that included all levels of library personnel or was specifically targeted at paraprofessional and support staff. It is the opinion of this report that this concern greatly outweighs the desire for specialized professional education. One interesting suggestion was that SEFLIN should develop programs with a targeted focus upon both of these needs, even if the total number of training programs with a broader focus was reduced. This report will contain recommendations related to this issue.

6. The annual regional conference appeared to be extremely well received. There appears to be a desire for additional events of this type that focus on expert presentations and networking interaction rather than course instruction per se.

7. There is an unrealized opportunity to form genuine communities of practice/learning. Such practices as a blog with open comments, unconferences, and community of practice workshops among personnel with similar knowledge and responsibilities were not a part of the CLP program. Discussion groups and committees appear to vary in the extent to which they are active Communities of Practice; the Cataloging Discussion Group appears to be one good example of an active group. There was an expressed desire for more networking opportunities for staff in addition to actual instruction.

8. The content of programming may be driven as much by requirements of funding sources as the actual needs and desires of member library personnel. It may always be a challenge for SEFLIN to balance grant and other funding requirements with the expressed needs of members.

Communication

9. The SEFLIN/Community of Learning web site is not engaging or interactive. It does not provide easily accessible information that is a motivation driving users to the site. During our interviews with SEFLIN members, some said the website is just fine, but many mentioned that it was difficult to use (or stated that they do not visit the site). A few comments from interviews that express the varied reactions include “Not attractive and intuitive”, “does not use often; too much password protection”, “use it and finds it user friendly”, “not very user friendly; hard to find calendar”, “not as welcoming as Tampa Bay Library Consortium site.” There seems to be an emphasis on complexity, password protection, etc. The website contains a great deal of information but complex navigation, small text size, limited use of color and photos do not encourage use. In part these are aesthetic and cosmetic changes that can be readily achieved. However, SEFLIN has limited IT staffing that effects its ability to directly manage its web site.

10. The current methods for publicizing training events do not reach all of the potential participants. There were a number of comments that current email notifications do not reach all interested persons. There was an expressed desire for notifications that crossover the subject specialties used for the email notification categories. That was balanced by the desire of some libraries to filter the announcements that are sent to their staff. The large number of email program announcements was a drawback to broader distribution.

11. There are varying opinions among members about the value of using a Facebook page, Twitter and broader email distribution to make information about SEFLIN programs more accessible. Although not all respondents are in agreement about the use of social networking tools by SEFLIN, there were comments supporting that direction: “Yes, not important personally to me but SEFLIN needs to be more contemporary”, “Yes, they need to do that to be more inclusive”, “They need to move into the 21st century.” There were multiple expressed comments in documentation and some interviews that the staff networking directory is not very open and user friendly.

Planning and Evaluation

12. The quantitative surveys and session evaluations indicate a generally high level of satisfaction. However the numeric ratings do not provide particularly useful information for program planning. Many organizations do an annual survey of members to ask for topic suggestions, as does SEFLIN. However, survey responses are only useful as a piece of a larger approach to determining member needs. There is a need for more qualitative assessment. A desire was expressed for more direct contact between SEFLIN staff and member library personnel.

13. The role of the Training Administrators needs to be clarified. Some Training Administrators would like to take a more active role in planning SEFLIN programs. There were comments that the Training Administrators meetings should be more active and participatory. Based on some comments from member surveys, it appears that some library personnel do not seem to be aware that they have a training administrator to inform them of SEFLIN events.

14. Staff in some libraries clearly have difficulty in getting release time to attend sessions. However it is doubtful that SEFLIN can effectively respond to that issue. There are apparent difficulties in scheduling conflicts between SEFLIN events and those of some member libraries. There is an expressed desire for SEFLIN to vary the location of major events to make them more accessible for libraries located throughout the region.

15. Members of committees and discussion groups place a high value on the networking and professional development gained through committee activity. There is an unrealized opportunity to more fully utilize committees and discussion groups to plan SEFLIN programs.

V. Recommendations

This section of the evaluative report provides a number of recommendations that respond to the above findings, grouped in general categories of Content and Delivery, Communication, and Planning and evaluation. However, it should be emphasized at the outset that the SEFLIN Community of Learning program is strong and effective. SEFLIN may choose to adopt some recommendations and decline or revise others.

Content and Delivery

1. SEFLIN should continue approximately its current balance of delivery modes and levels of training. We received many comments that only online courses were convenient for participants, many others that face to face instructor led courses and workshops were much more useful, and yet others that the current mix of both modes of delivery is needed. On balance, it seems that in order to serve all members SEFLIN must continue approximately its current mix of online self guided and instructor led programs.

2. SEFLIN should continue strong programs of training for paraprofessional and support staff. One possible approach to such training that would provide motivation and recognition of participants would be a paraprofessional certificate program with related training. An important step in proceeding with such a program would be gaining administrative endorsement from libraries with large numbers of support staff. A web site with an example of a paraprofessional certificate program is referenced in appendix 2 of this report.

3. SEFLIN should seek opportunities to co-sponsor/host workshops sponsored by American Library Association divisions or other major external education providers, possibly subsidizing the registration cost for those events. That would enable SEFLIN to provide high quality training events of interest to all levels of personnel without requiring significant resources.

4. Reactions to SEFLIN Connect were uniformly positive. SEFLIN should continue to promote and explore the potential of this medium.

5. SEFLIN should continue to offer the strong coursework for which it is well regarded. However we encourage SEFLIN to go beyond that and offer programs that develop a true community of learners among library staff. One current example of that is the highly regarded Annual Conference. Many participants cited this as their best opportunity to network with others. SEFLIN should certainly continue this program, and in addition should explore additional programs that offer the opportunity for networking, innovation and community building. A few options for such programs are in the following recommendation.

6. SEFLIN should more fully develop a community of learning, including these steps:

• Sponsoring unconferences. Without a great deal of planning, an unconference provides an opportunity for individuals to meet and discuss current topics. At the start of the event, participants meet to brainstorm and choose the specific topics to be discussed during break-out sessions. Once the schedule is set, participants can attend whichever sessions sound interesting. Each session can have a volunteer facilitator who starts the discussion and encourages sharing and brainstorming.

• Building time and opportunity for interaction/ networking into workshops.

• Community of Practice and Special Interest Group (SIG) programs. These are sessions in which library personnel working in particular area or specialty share their knowledge and information through meetings, discussions, or self-led workshops. They build relationships that enable them to learn from each other. A Community of Practice can be structured or unstructured, and would be ideal for the SEFLIN discussion groups.

• Blogs or other interactive communication channels

7. SEFLIN should explore taking advantage of more local presenters. A benefit of the ongoing communication mentioned later in this report is that SEFLIN may become aware of individuals with expertise in certain areas. However, having subject matter expertise and being able to effectively train others on that topic are two different things. One regional library system said they are addressing this challenge and trying to grow more presenter talent locally by providing classes and workshops regarding presentation skills. The North Texas system we talked to is even considering starting a Toastmasters group to improve presentation skills.

8. SEFLIN should pursue innovative approaches to providing training and ongoing support for new technologies. The approach SEFLIN is using for the Flip Video Cameras workshop is an excellent example of such innovation. Providing flip cameras for participants to use after the class represents a forward-thinking approach. SEFLIN has a reputation for delivering a solid menu of more traditional topics, but could be invigorated by more of these types of innovative sessions, too. Beyond providing Flip cameras, SEFLIN might consider creating a technology petting zoo, also referred to as gadget garage, with new technologies. This “garage” or “zoo” could be used by member libraries to help staff learn about new technologies. Survey respondents frequently express inability to apply knowledge learned in their library environment, particularly in regard to Web 2.0 applications. SEFLIN could provide an online “sandbox” with web 2.0 applications for staff to experiment with, including social networking tools, a website/blogging tool such as Wordpress, etc. The Conference Toolkit for the 2009 Annual conference provided some sandbox components but this needs to be done on an ongoing basis.

Communication

9. SEFLIN should update its web site for improved usability. Reviewing the websites of other library organizations that provide continuing education demonstrates how challenging it is to create a website that contains a great deal of information, yet is user-friendly and appealing, too. We did not find a perfect site to point you to as a model, but instead will be highlighting features and design elements that might be incorporated into your site to make it more effective. If a major site overhaul is not possible, a few minor tweaks could help the site feel more vibrant and contemporary. Suggestions for the SEFLIN website include:

• Increase the font size used throughout the site

• Feature photos of people “in action”. Use photos from SEFLIN members and SEFLIN events.

• “View featured courses” needs to be placed more prominently on the CLP

• Many people prefer to scan upcoming continuing education events by looking at them on a calendar. The calendar that is on the front page of the SEFLIN site also needs to be on the CLP page.

• Prominently feature archived materials from previous trainings

• Create SEFLIN Facebook and Twitter accounts and link to them from the website. (SEFLIN created a Facebook group page prior to completion of this report. The real value of an organizational Facebook page is in the automatic updates sent to Facebook users. Therefore a key step will be the promotion of this site leading SEFLIN library staff to link their personal Facebook sites to the SEFLIN group page.)

10. SEFLIN should address several issues that limit the effectiveness of the email program alert system. It seems that the specific interest areas may not match the responsibilities and interests of a given individual. That reality is reflected in a number of comments we noted in SEFLIN evaluations. In addition to subject specific interest areas, SEFLIN member staff should be able to sign up for a general category that will allow them to receive notifications of all continuing education events. A related issue is the sheer number of alerts generated due to the large number of training events provided. SEFLIN should investigate whether to utilize the monthly online newsletter as its primary alert of events provided that month, and reduce individual event alerts. That is not a decision to be made lightly because it is possible that it would lead to reduced communication with some persons who currently receive those alerts. There we cannot conclusively recommend this step. However it is our recommendation that the issue be examined by SEFLIN staff.

11. Consideration should be given to phasing out the social networking functions of the SEFLIN Staff Networking Directory in favor of standard, open social networking tools. It appears that the quality of standard mass market social networking tools is superior, and that library personnel should use the same services that their library users do and acquire knowledge of appropriate net safety skills. Confidential information such as social security numbers can remain in a secure password protected location on the SEFLIN web site.

Planning and Evaluation

12. The role of Training Administrators should be clarified, and changes should be made in the manner in which Training Administrator meetings are structured. Training Administrators approach that function in very different ways. Some Training Administrators would like to take on a more active and participatory role. Reports should be emailed to Training Administrators prior to meetings, and the meetings should be devoted primarily to actual discussion and planning of the SEFLIN Community of Learning program. It is suggested that as a first step the next Training Administrators meeting be devoted to a participatory discussion of the role of Training Administrators.

13. SEFLIN should engage in regular visits to member libraries, and in small group discussions with all levels of member library personnel. This is a complex recommendation, with several aspects, as discussed below with examples of potential approaches.

• Qualitative evaluation. SEFLIN does a great deal of quantitative assessment of individual programs, but we recommend a greater emphasis on qualitative evaluation. Many organizations do an annual written survey of members, as does SEFLIN. However, survey responses are only useful as a piece of a larger approach to assessment. Discussions with small groups of member library personnel to assess the effectiveness of the Community of Learning program are very likely to produce candid and subtle evaluative comments that would not be revealed in a written survey. These meetings should not be limited to library directors and training administrators. They should include both SEFLIN Connect and onsite discussions.

• Planning future training programs. Our review of other regional continuing education cooperatives, and our own experience, indicate that program planning is a matter of “trial and error.” Sometimes people are aware of training needs but frequently they are not sure what they want or need (“I don’t know what I don’t know”). It is necessary to be in ongoing communication about changes in member libraries, about challenges libraries are facing, etc. in order to provide suitable continuing education options. One regional library organization (The North Texas Library System, including Dallas) divides their geographic area up into 7 regions and holds a discussion group in each of those regions twice per year (14 discussion groups). The CE Coordinator said that is where she gains her most useful feedback. She finds out what is happening in member libraries and receives feedback about training that has been offered recently.

• Communication and member relations. There were some interview comments that SEFLIN staff should engage in a program of regular visits to member libraries to improve awareness of library needs and become acquainted with member library personnel. This seems to us to extend beyond continuing education planning as such, but that distinction is somewhat artificial. This type of outreach to member libraries is not the same thing as the small group planning discussions recommended above, but can provide valuable information regarding training needs. The same Texas continuing education coordinator said that she and her coworkers visit libraries regularly and sometimes just take a laptop and work from a member library in order to keep a finger on the pulse of the organizations. We are not suggesting any particular approach, and recognize that the relationship of SEFLIN to member libraries may differ from the system referred to above. We are only pointing out a need for enhanced personal communication.

The focus group discussions that will be a follow-up step to this evaluation report could provide an excellent opportunity to initiate such small group discussions with SEFLIN members.

14. With less staff available for CE planning, SEFLIN should consider making greater use of the committees and discussion groups. These groups could take a stronger role in identifying needs and speakers and even in planning events related to their interest areas, with SEFLIN providing the financial support and oversight.

XI. Appendices

1. Telephone interview list.

2. Annotated list of Library Organizations Providing Continuing Education Regionally

Appendix 1: Final List for SEFLIN Phone Interviews

|Library |Name |

|Training Administrators | |

|Miami-Dade Public Library System|Bonnie Rothschild |

| |Training Librarian |

|Broward County Library |Laura Connors |

| |Assistant Regional Library Manager |

|West Palm Beach Public Library |Britta Krabill |

| |Reference Librarian |

|Florida International University|Nancy Hershoff |

| |ILS Coordinator &Planning Officer |

|Nova Southeastern University |Carrie Gits |

| |Asst. director of Reference |

|Everest Institute |Melissa Del Castillo |

|Palm Beach Atlantic University |Bob Triplett |

| |Learning Resources consultant |

|Florida Memorial University |Lucy Osemota |

| |Head Reference Librarian |

|Committees and Discussion Groups| |

|Broward College |Suzette Spencer |

| |Reference/Instruction Librarian |

|Palm Beach County Library System|Sue Bermann |

| |Cataloging Supervisor |

Appendix 2: Library Organizations Providing Regional Continuing Education Programs

For the purposes of this evaluation, we reviewed the websites of numerous library organizations that provide continuing education to staff at libraries within a region. Some of the organizations reviewed serve a larger area or focus on serving different types of library staff than SEFLIN, however, exploring their websites and approaches to continuing education is still relevant. This list highlights items of interest on the websites of several organizations.

KCMLIN



Web site design is not ideal but note the items in the “Workshops/Training” column: particularly the speaker’s bureau, the certificate programs, and the two-day workshop on serving diverse populations cosponsored with ALA/PLA: .

Minitex



Website features photographs of local people and places. Featured classes are well highlighted.

NEFLIN



Many people prefer to simply browse a full calendar of upcoming events (as is easily available on the NEFLIN site). NEFLIN is also very willing to experiment with blogs and other 2.0 technologies. See NEFLIN blog for examples of numerous interest groups featuring local speakers.

North Texas Regional Library System



Another site where it is easy to browse a chronological list of upcoming classes. Site also contains educational/informational articles, podcasts, etc. directly on home page. See “Special Interest Groups and Regional Conferences” for an interesting approach:

Appendix 3: Consultant Resumes

Minges & Associates provided the services of two consultants for this project with substantial experience with regional library cooperatives, consulting, and continuing education.

Jim Minges is principal consultant and president of Minges & Associates. He brings more than 35 years of experience in library cooperation and development, consulting and continuing education, including extensive experience in leadership and evaluation of regional library cooperatives in several states, including programs for resource sharing, interlibrary delivery, shared library automation systems, library planning, continuing education and consulting. In addition to his current position as Director of the Northeast Kansas Library System, he has provided consulting services to private clients including the North Kansas City, Missouri Public Library and Southwest Kansas Library System.

Northeast Kansas Library System (NEKLS), Lawrence, Kansas

Director, 1996-present

Nebraska Library Commission, Lincoln, Nebraska

Director of Library Development, 1984-1996

Northwest Missouri Library Network, St. Joseph, Missouri

Director, 1976-1984

South Dakota State Library

Library Consultant, 1973-1976

Brenda Hough has pursued the training and development of library personnel through projects and services that have combined technology based and in-person continuing education approaches. She has presented and consulted throughout the United States, and is currently a doctoral student at Emporia State University in addition to her position as regional continuing education consultant.

Northeast Kansas Library System (NEKLS), Lawrence, Kansas

Continuing Education Consultant, August 2009 – present

Technology Consultant, 2005-2008

MaintainIT, a TechSoup Project, San Francisco, California

Library Training Specialist, April 2008-July 2009

School of Library & Information Management (SLIM), Emporia State

University, Emporia, Kansas

August 2003–December 2005

Instructor in an ALA-accredited library and information management program

US Library Program, Bill & Melinda Gates Foundation, Seattle, WA

Trainer (1999-2003)

Staff Development Coordinator (2001-2002)

Lake Agassiz Regional Library, Detroit Lakes, MN

Reference Librarian, 1996-1999

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download