On May 17 & 18, 2008, a workshop was conducted in …



[pic]

Development of Professional Certification

for

Information & Referral Specialist –Aging/Disabilities (CIRS-A/D)

The Alliance of Information and Referral Systems (AIRS at ) is a nonprofit association of over 1,100 organizations that provide I&R across North America. In addition to the creation of international standards, AIRS offers a professional credentialing program for individuals working within the I&R sector of human services.

AIRS Certification

Certification is a measurement of documented knowledge in the field of I&R reflecting specific competencies and related performance criteria, which describe the knowledge, skills, attitudes and work-related behaviors needed by I&R practitioners to successfully execute their responsibilities.

The AIRS Certification Program is operated in alignment with national standards for credentialing organizations. There are currently more than 5,100 AIRS Certified practitioners in the United States and Canada.

Candidates may only apply to take the certification examination following the establishment of eligibility which is based on a combination of I&R experience combined with educational background (for example, at least 1 year of I&R employment for applicants with a Bachelors or higher degree).

The examinations are administered by approved proctors. Once secured, AIRS Certification must be renewed every two years following an application requiring proof of continuous professional development.

AIRS Certification provides value to individuals, organizations and communities.

From CIRS-A to CIRS-A/D

AIRS in partnership with the National Association of States United for Aging and Disabilities (NASUAD) and the National Association of Area Agencies on Aging (n4a), has offered Certification in Information and Referral – Aging (CIRS-A) for the past ten years.

Professional certification needs to maintain relevance and as it became clear that the work of those within Aging was being aligned more closely with those working within Disabilities, it also became clear that AIRS needed to transform our existing CIRS-Aging (CIRS-A) certification to a CIRS-Aging/Disabilities (CIRS-A/D) certification.

This involved much more than a simple name change. It involved a psychometric process that stretched over several months and nine distinct stages (summarized below and described in more detail within the document).

Stage 1: Job Task Analysis

Stage 2: Job Task Analysis Validation

Stage 3: Assessment of Existing Questions

Stage 4: New Question Development

Stage 5: Question Review

Stage 6: Cut Score Development

Stage 7: Exam Creation

Stage 8: Exam Review

Stage 9: Final Exam Simulation

Volunteer Subject Matter Experts (SMEs)

Central to the process was the participation of more than 80 volunteers drawn from a wide range of aging and disabilities organizations and responsibilities (from directors to front-line staff) from 27 states who served as subject matter experts.

Each stage required the work of various volunteer groups chosen to reflect the diversity of our sector, both in terms of the type of work performed, the geographic region, the nature of the agencies (small and large/urban and rural), and in terms of the individuals themselves (from 2 to 20 years of I&R experience, different levels of education, and cultural backgrounds). Most groups also included members for whom English was a second language.

AIRS would like to thank all of the following individuals who volunteered for this challenging and highly confidential process, and their organizations for allowing their participation.

|Name |Organization |City |State |

|Abigail Musselman |ADRC of Dane County |Madison |WI |

|Alison Hammond |Prince William County Area Agency on Aging |Prince William |VA |

|Angel Jewell |CSI Office on Aging |Twin Falls |ID |

|Angela Brown |National Association of Area Agencies on Aging (n4a) |Washington |DC |

|Angela MacDonald |Disability Action Center |Dalton Gardens |ID |

|Angela R. Edwards |Trident Area Agency on Aging |North Charleston |SC |

|Anne Wildman |Northern Kentucky Area Development District |Florence |KY |

|AnnMarie Abbott |Olympic Area Agency on Aging |Aberdeen |WA |

|Barbara Gordon |KIPDA AAA, Independent Living/Aging and Disability Resource Center |Louisville |KY |

|Barbara Labosky |Northeast Iowa Area Agency on Aging |Decorah |IA |

|Basil Bailey, RN |Area Agency on Aging, District 7 |Rio Grande |OH |

|Beverly M. Huguley |Three River Area Agency on Aging |Franklin |GA |

|Beverly Waid |Coastal Regional Commission/Area Agency on Aging |Darien |GA |

|Bob Gorman |Independence Northwest |Naugatuck |CT |

|Bob Kyllonen |Olympic Area Agency on Aging |Aberdeen |WA |

|Celia Easley |Central Plains Area Agency on Aging |Wichita |KS |

|Chelsea B. Crittle |Central Mississippi Planning & Development District |Jackson |MS |

|Cheryl Gaines |Middle Alabama Area Agency on Aging |Calera |AL |

|Christa Edwards |Senior Services of Southeastern Virginia |Norfolk |VA |

|Christine Thompson |Southwest Missouri Office on Aging |Springfield |MO |

|Christine Welton |Senior Choices, Area Agency on Aging for SWFL |North Fort Myers |FL |

|Darlene Machin |Aging & Disability Resource Center |Ellensburg |WA |

|Deborah Danner-Gulley |Area Agency on Aging, District 7 |Rio Grande |OH |

|Deborah Seed |Illinois Department on Aging |Chicago |IL |

|Debra Morgan |Valley Area Agency on Aging |Flint |MI |

|Diane White |Southern GA Area Agency on Aging |Waycross |GA |

|Donna Bileto |Northwestern Illinois Area Agency on Aging |Rockford |IL |

|Donnieka Woods |Aging & In-Home Services |Fort Wayne |IN |

|Elizabeth Phillips |ADRC of Central Wisconsin |Wisconsin Rapids |WI |

|Gary Bolduc |East Tennessee Human Resource Agency |Knoxville |TN |

|Glenn A. Claflin |Access Independence, Inc. |Stratford |CT |

|Holly Anderson |AAA of Deep Texas |Jasper |TX |

|Janet Long |Washington County Disability, Aging & Veteran Services |Hillsboro |OR |

|Jennifer R. Craig |Kentuckiana Regional Planning & Development Agency AAA |Louisville |KY |

|June Renaud |County of Kauai Agency on Elderly Affairs |Lihue |HI |

|Karen Nelson |Northwest Georgia Regional Commission |Dalton |GA |

|Karen Woodley |211 Palm Beach/Treasure Coast |Lantana |FL |

|Kathryn Van Curen |West Central Florida Aging & Disability |Tampa |FL |

|Kathy Lyons |Heart of Texas 2-1-1 |Waco |TX |

|Katrine Colten |Olympic Area Agency on Aging |Aberdeen |WA |

|Kealoha Takahashi |County of Kaua’i, Agency on Elderly Affairs |Lihue |HI |

|LaMonica Upton |Wyandotte/Leavenworth Aging and Disability |Kansas City |KS |

|LaWanna Broderick |Knoxville - Knox County Office On Aging - Senior Citizens I & R |Knoxville |TN |

|Leah Garcia |Area Agency on Aging & Disabilities of Southwest Washington |Vancouver |WA |

|LeslieAnn Cioti |Jefferson Council on Aging - ADRC |Metairie |LA |

|Lynda Southard |Cajun Area Agency on Aging |Lafayette |LA |

|Marci D. Brown-McMurphy |Community Action Program of East Central Oregon, AAA |Pendleton |OR |

|Margaret Matthews |Area Agency on Aging of Central Texas |Belton |TX |

|Marie Haenelt |Douglas County Senior Services |Roseburg |OR |

|Marissa Whitehouse |Mississippi Department of Human Services |Jackson |MS |

|Mary Osborne |National Association of Area Agencies on Aging (n4a) |Washington |DC |

|Matt Romero |Central Oregon Council on Aging |Bend |OR |

|Matthew Yee |Central Illinois Agency on Aging |Peoria |IL |

|Megan Velasquez |LifeStream Services |Yorktown |IN |

|Nadine Autry |The Freedom Center, Inc. |Frederick |MD |

|Nanette Reclave |National Association of States United for Aging and Disabilities |Washington |DC |

|Olivia Harvey |Junction Center for Independent Living Inc & Mountain Empire |Norton |VA |

|Pamela Bilal |Howard County Maryland - Office on Aging |Columbia |MD |

|Patricia A Kepler |Independent Living Resources |Portland |OR |

|Patricia Parmeter |Milestones Area Agency on Aging |Burlington |IA |

|Philip Ana |Hawaii Executive Office on Aging |Honolulu |HI |

|Rachel Helton |Mountain Empire Older Citizens, Inc. |Big Stone Gap |VA |

|Rachel Kaehny |Washington County ADRC |West Bend |WI |

|Raquel Amaya |Multnomah County Aging & Disability Services |Portland |OR |

|Ronnie Gipson |Brazos Valley Council of Governments Area Agency on Aging |Bryan |TX |

|Rose Mary Dudley |Council of Community Services |Roanoke |VA |

|Sara Tribe |National Association of States United for Aging and Disabilities |Washington |DC |

|Sharon Williamson |Department of Behavioral Health and Developmental Disability |Savannah |GA |

|Sheila Casey |La Plata County Senior Services |Durango |CO |

|Shelly Ogata |Hawaii County Office of Aging |Hilo |HI |

|Sue Greeno |Senior Resources Area Agency on Aging |Norwich |CT |

|Tami L Wacker |East Central IL Area Agency on Aging |Bloomington |IL |

|Tammy L Johnson |United Way of Greater Cleveland |Cleveland |OH |

|Tracy Barrows |Area Agency on Aging of Pasco-Pinellas |St. Petersburg |FL |

|Valerie Ng Joe |United Way of Greater Houston |Houston |TX |

|Virginia Pack |Department of Oklahoma, Aging Services of Oklahoma |Oklahoma City |OK |

|Wenda Black |Harvey County Department on Aging |Newton |KS |

|Wendi Altman |NH DHHS, ServiceLink Program |Concord |NH |

|Yolanda Pearson |Olympic Area Agency on Aging |Aberdeen |WA |

|Yvonne Duhaylongsod |Hawaii Pacific University |Honolulu |HI |

Stage 1: Job Task Analysis

The first and most crucial stage involves the development of a Job Task Analysis (JTA). In order to create a testing instrument (i.e. an exam), you must know what needs to be tested. What does an I&R Specialist working in the Aging and/or Disabilities sectors, actually do? What skills and knowledge is needed to carry out those tasks competently? Which parts of the job are the most important?

The additional challenge of the JTA was to ensure that the skills and knowledge identified as necessary to handle the disability-focused work of ADRCs was the same as the attributes required to deal with aging issues.

Our guides through this and all of the subsequent stages were AIRS’ certification consultants, Michael Hamm and Dr. Gerald Rosen.

Michael Hamm managed the accreditation program for the National Organization for Competency Assurance (NOCA) for eight years and is an American National Standards Institute (ANSI) certified auditor for compliance determination. He is a national authority on certification and accreditation – indeed he has literally written the book on it, authoring Fundamentals of Accreditation.

Our psychometrician, Dr Gerald Rosen, is a consulting psychologist specializing in the design and administration of testing programs, test validation, statistical analysis, and vocational testing. Dr. Rosen has more than 20 years of experience in testing and measurement, and has produced numerous professional publications on these topics. He has managed the development of many national certification and licensure programs. His background also includes a slice of I&R as he once worked at a crisis center in Pennsylvania.

The volunteer subject matter experts shared their experience and insights over a two-day session. The process began with a blank slate as far as the expectations were concerned but as people talked about the work they did, it was clear that there was a commonality of knowledge, skills and process.

The draft document established 7 Domains, 18 Tasks, 32 Knowledge Areas and 24 Skills Sets.

The Tasks were weighted in terms of their relative importance to the job and to the client (for example, “Screening/Assessment” was determined to comprise 20% of the overall job).

Stage 2: Job Task Analysis Validation

However, the JTA only represented the combined views of the 13 SMEs who attended the session. Did those conclusions hold true for everyone else?

The draft document was shared with more than 2,000 current CIRS-A participants providing them with a chance to make general and specific comments through an online survey that yielded detailed feedback from 265 individuals which represented a response rate of around 12%.

One of the questions asked was “Do you support the idea of having a single Certification that covers the work of I&R/Specialists serving both older adults and persons with disabilities (and their families and caregivers)?” The response was an overwhelming 91.9% in favor of the initiative.

The results validated the draft and enhanced it through several minor word improvements and subtle adjustments to the weightings of the various tasks. Here are the final approved domains:

|Domains |Subject of Domain |Weight of Domain |

|1 |Rapport |18% |

|2 |Screening/Assessment |20% |

|3 |Identification of Resources and Preferences |17% |

|4 |Information, Assistance, Referral and Advocacy |19% |

|5 |Documentation |11% |

|6 |Follow-up |10% |

|7 |Ethics, Professional, and Legal Issues |5% |

The document was approved by the AIRS Certification Commission. To view the full JTA report:

.

Stage 3: Existing Question Assessment

Once we knew what needed to be tested, how many of our existing CIRS-A questions were still relevant and how many new questions would be required and in what areas?

Another volunteer SME team was assembled.

The challenge for this group was to go through the entire CIRS-A question database of more than 300 items and decide which questions should be kept and which should be removed. Any retained items had to be assigned to a Domain/Task within the new JTA. At the end of this process, we were able to quantify how many new questions were needed and in which areas (for example, we needed 12 new questions that tested the understanding of demographic documentation).

Only about 60% of the existing questions were deemed both still relevant and equally applicable to someone working with persons with disabilities as well as with aging adults. However, through the ensuing process, virtually all of the retained questions were edited to the point of being almost unrecognizable from their original form.

Stage 4: New Question Development

We now needed to write over 100 new questions and subject matter experts to write them.

These volunteers received training from Dr. Rosen on question writing techniques.

This was the most challenging stage. Writing good questions is hard. But what is really hard is writing good ‘wrong’ answers. This involves coming up with three distracters for each question that are plausible without being misleading while still leaving one answer that is obviously correct (but only obvious to the individuals that properly understand the issue).

The group spent two days together engaged in individual and collaborative exercises that facilitated the process of question writing.

The session resulted in a final bank of about 300 new and revised questions. Each question was linked to a specific Domain within the new JTA and had a verifiable source.

Stage 5: Question Review

It was now time for the entire item bank to undergo an extensive review from volunteers that had not been involved in any earlier stage.

As a result, some questions were completely eliminated while the majority benefited from detailed editing. Very few questions emerged unaltered. I

Stage 6: Cut Score Development

A cut score (or pass mark) is not a random number. It should represent the percentage of answers that most accurately reflects the ability of an individual who is competent in the issues being tested.

A cut score is never perfect. But it should be the score that eliminates as many false positives and false negatives as possible (that is, it tries to ensure that people who are competent, pass and that people who are not yet as competent, do not pass). Within this context, an exam might have a pass mark as high as 95 or as low as 40 if those marks represent the “border line” between someone with the desired amount of understanding and someone who has yet to reach that level.

The AIRS Certification Program, in common with many examinations, uses a methodology known as modified Angoff ratings to determine cut scores. Basically, this involves SMEs assigning a ‘degree of difficulty’ number to each question in the item bank. Technically, each exam can have a different cut score depending on the difficulty or otherwise of the questions within each exam. However, AIRS generates a mathematical formula to ensure that each exam contains the same balance of difficult and easy questions (that is, each exam has the same cut score).

As a natural part of this process, further changes were made to many questions to improve their clarity and some questions were eliminated.

Stage 7: Exam Creation

AIRS now had a database of more than 300 extensively reviewed questions but no actual examinations.

Using mathematical models from the cut score analysis, three new exams were created with an identical (to within 2 decimal points) balance of difficult/easy questions that accurately reflected the weighting of the JTA (for example, 10% of the questions involved a knowledge of the issues surrounding follow-up). By the end, each new exam had a cut score of 75.

Stage 8: Exam Review

But there were still two more stages required to ensure each exam was a fair test unto itself (for example, an exam did not contain two questions that addressed the same issue with similar words … or did not contain a question that inadvertently gave away another answer).

This process involved yet another opportunity to confirm and further hone item clarity and accuracy, and even at this stage, a handful of questions were removed as the SME team verified that there is one and only one correct answer for each question.

Stage 9: Final Exam Simulation

In the final stage, each of these three new exams underwent a simulation exercise which involved having ‘experienced CIRS-A holders who had not been involved in earlier stages, take the exam under ‘real’ conditions. This enabled us to analysis the performance of each individual question in terms of whether it was clear to examinees and did not inadvertently lead competent individuals to choose an incorrect answer.

The process also ensured that we could be confident that each of the three exams returned equal results. As a bonus, this group of volunteers provided further suggestions and recommendations on the need to change certain questions.

Once they had taken the exams, they completed an online survey that provided the following results:

[pic]

[pic]

[pic]

[pic]

Transition plan for existing CIRS-A holders

The AIRS Certification Commission consulted with stakeholders to determine the best way to grandparent existing CIRS-A holders to the new CIRS-A/D qualification. The challenge was to provide the best balance between recognizing the new influences, respecting existing professionalism, ensuring no additional expense while also being as equitable as possible.

The final decision will apply to all CIRS-A holders regardless of their current speciality and requires them to first take a free online training module which will be made available by NASUAD on issues relating to providing I&R to persons with disabilities and their caregivers, and will also touch on some basic awareness issues regarding persons with disabilities. The training would conclude with a self-administered online test of not more than 20 questions. This quiz would have to be passed before an individual could describe themselves as a CIRS-A/D.

Final Implementation

In order to provide agencies with some basic training/information on the issues covered in the new exams and lead time towards knowingly booking CIRS-A/D exams, the AIRS Certification Commission has decided that March 16th 2015 will be the first day on which the new CIRS-A/D exam will be available.

Continuous Cycle

The process of reviewing JTAs, creating new questions and refreshing existing exams occurs about every 4 years, takes about 12 months to complete and follows national standards for credentialing organizations.

Appendix

Benefits of AIRS Certification

For the individual, AIRS Certification:

• Adds professional recognition to what you do. It addresses the misconception that I&R people “just answer phones.”

• Provides a transferable qualification. Many job postings state a preference for applicants with AIRS certification. As a consequence, there is much more mobility of I&R staff.

• Some agencies provide a pay increase for Certified staff.

For the agency, AIRS Certification:

• Builds confidence among staff – they believe more in their skills if they have been validated by an external body.

• Enhances agency quality assurance and consistency of service levels within your own I&R.

• Helps funders and other stakeholders understand and appreciate the professionalism involved in I&R. It shows that there is an emphasis on quality as the competencies of I&R positions have been defined it and are being externally tested.

• For the majority of its frontline staff is, along with AIRS Accreditation, often one of the criteria for securing and maintaining funding.

• The process of studying forces people to understand the context in which they perform their job and the skills that they need in a more systematic way.

• The alignment of training resources (e.g. ABCs of I&R and online training) with the Standards and the Certification process provides a continual enhancement of service.

For the general public and the human services sector, AIRS Certification:

• Enhances agency quality assurance and consistency of service levels between different I&Rs.

• Improves customer service. Staff are aware of the requirements for quality performance and are more ready and capable of meeting them.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download