Analysis and Synthesis of Scholarly Project: Predictors of ...



Running head: ANALYSIS AND SYNTHESIS

Analysis and Synthesis of Scholarly Project: Predictors of Success on the NCLEX-RN

Janis A. Franich

Ferris State University

Abstract

This paper describes, analyzes, and synthesizes the scholarly project that evaluated student variables for a relationship to success or failure on the National Council Licensure Examination for Registered Nurses (NCLEX-RN). The project description includes the opportunity that prompted the project and its significance as well as the goal and objectives. The implementation of the project is described with attention given to legal and ethical concerns. The project outcomes are evaluated and recommendations for future implementation based on the results of the project are discussed.

Analysis and Synthesis of Scholarly Project: Predictors of Success on the NCLEX-RN

This scholarly project examined student variables to identify those attributes that may predict success or failure on the National Council Licensure Examination for Registered Nurses (NCLEX-RN). Student failure of the NCLEX-RN may have negative consequences for both the student and the reputation of the nursing program (Beeman & Waterhouse, 2001; see also Daley et al., 2003; Yin & Burger, 2003). This paper describes the scholarly project, analyzes the implementation and evaluation of goals and objectives, and then discusses recommendations for future implementation.

Description of Scholarly Project

The goal of this project was to identify the predictors of associate degree nursing students at risk for failure of the NCLEX-RN. This project examined differences in student and academic variables, Health Education Systems, Inc. (HESI) results, and Nurse Entrance Test (NET) scores between students who were successful on the NCLEX-RN, and those who were not, among the last cohort of graduates from the nursing program. The purpose of this project was to use the variables with a significant positive correlation to failure on the NCLEX-RN to develop an instrument that faculty could use to easily identify students at risk of failure.

Exploring predictors of success, and failure, on the NCLEX-RN is an important endeavor of academic nurse educators (Daley et al., 2003). One of the core competencies of academic nurse educators is to participate in curriculum design and evaluation of program outcomes (National League for Nursing [NLN], 2005). Student performance on the NCLEX-RN, as an indication of effective nursing program outcomes (Daley et al.; see also Norton et al., 2006; Seldomridge & DiBartolo, 2004), can assist nurse educators in making decisions about the curriculum revisions. Another application of acceptable student pass rates of the NCLEX-RN is for a nursing education program to maintain approval with the state board of nursing and to meet accreditation standards (Norton et al.; Seldomridge & DiBartolo). The student success rate on the NCLEX-RN is an often measured outcome of nursing programs, thus an important area of interest to academic nurse educators (Daley et al.).

The setting for this project was a community college, which was established in 1923 to make higher education available to residents of the county. The program offers an associate degree in nursing and admits cohorts of 80 students each fall and winter semester. The nursing curriculum was revised in 2006 because of concerns about the success rate of the students on the NCLEX-RN. This project focused on the first class of students that graduated from the new curriculum. The examination of the variables as predictors of NCLEX-RN results has significance in the evaluation of the curriculum revisions.

This project took place during the Fall Semester of 2008 under the guidance of a Professor of Nursing at the college. This faculty member earned her Doctorate in Education in 2007 from Wayne State University with a program emphasis in curriculum and instruction (Wayne State University, 2008). In 2004, she became a Program Evaluator for the National League for Nursing Accrediting Commission. In this role, she participates in the program review and evaluation of other nursing programs. Her education and experience in this area demonstrated her qualification to serve as a preceptor for this project.

Implementation of Scholarly Project

Two issues related to the implementation of this project arose during the proposal review by the instructor. One issue was obtaining institutional review board approval, the other was obtaining permission to use a framework developed by a nurse educator as a basis for the project. Both of these issues required action before commencing with the project as proposed.

In 1953, Institutional Review Boards (IRBs) were developed as a method to protect human subjects during research by the National Institutes of Health in the United States (Sims, 2008). An IRB is a group of at least five members of varying backgrounds who review research proposals to protect the rights of human subjects before, during, and after a research project. The three levels of review for a proposal are exempt, expedited, and full. Exempt research does not involve invasive procedures but requires examination of existing data. Expedited research poses a minimal risk for the subjects. Research projects that have more than a minimal risk to the subjects require a full review by the IRB. Any project that receives federal funding is subject to a review by an IRB (Wood & Ross-Kerr, 2006). Some healthcare institutions and universities have their own IRBs and others contract with an outside agency (Sims). This institution does not have an IRB, nor do they contract with an outside agency. This did not exempt my responsibility for assuring the anonymity of student data. The steps I took to keep student data secure included not allowing anyone access to the computer database, keeping records secure, and not using names when I was sharing results of the project.

The framework that was followed for this project was the evaluation scholarship model. This model provides a framework that allows the evaluation to be subjected to the same rigorous assessment as other types of systematic research (Eddy, 2007). The standards of the evaluation scholarship model include: (a) clear goals, (b) adequate preparation, (c) appropriate methods, (d) significant results, (e) effective presentation, and (f) reflective critique. Eddy suggested that the use of this model by nurse educators for program evaluation would validate and communicate this important aspect of the scholarship of teaching. I emailed Dr. Eddy and received permission to use her model in my project, providing I give appropriate credit (L. Eddy, personal communication, August 22, 2008). This framework guided the application of scholarship to the evaluation with the standards described.

Evaluation of Goal and Objectives

The goal of this project was to identify predictors of nursing students at risk for failure on the NCLEX-RN. The identified predictors were to be used to adapt the RAI for use at the college.

The objectives were as follows:

1. Based on available student data and consensus with the faculty, finalize selection of student variables to examine for relationship to success on the NCLEX-RN. These variables will include: (a) age; (b) gender; (c) English as a second language; (d) NET scores; (e) HESI scores; (f) number of times the NET was taken; (g) nursing program GPA; (h) course grades for NRSG-101 , NRSG-102, NRSG-201, and BIOL-152; and (i) number of college credit hours prior to admission into the nursing program.

2. Develop a spreadsheet and analyze data using the software Statistical Package for Social Services (SPSS).

3. Adapt the Risk Appraisal Instrument (RAI) to quickly and simply assess students’ risk for failure of the NCLEX-RN.

4. Prepare and present results of data analysis and the RAI to nursing faculty.

Evaluation of Goal

The data analysis was limited by the sample size. The analysis did indicate trends in the expected direction and correlations may have been found with a larger sample size (personal communication, J. Kozemi, University of Michigan expert statistician, November 20, 2008). According to Polit & Beck (2008), the larger a sample size is, the more representative it becomes of the population under study. The sample size for this project was 51. There were no other subjects to add to the sample because these are the first graduates of the program since the new curriculum and admission requirements were implemented. Some of the variables had categories that were too small for any generalizations to be made. For example, there were seven non-white students and six male students. Twenty-nine students comprised the sample of those students who were administered the NET. Two variables were found to have a significant, but weak, correlation to the NCLEX-RN results. With these results, the validity of applying two variables to the creation of a risk appraisal instrument for the students is problematic. Thus, the goal to identify student predictors with the purpose of developing a risk appraisal instrument was not met.

Evaluation of Objective Number One

Objective number one was to finalize the selection of variables to examine for success or failure on the NCLEX-RN. This objective was met. To meet this objective, a review of the literature was conducted to determine existing knowledge of this subject (Polit & Beck, 2008). The literature review found that multiple academic and student variables have been evaluated as predictors of NCLEX-RN success at the levels of baccalaureate, diploma, and associate degree nursing programs. A common theme in the discussion section of the articles was the need for each program to determine a model of variables specific to their program (Barkley, Rhodes, & DuFour, 1998; Beeman & Waterhouse, 2001; Hass, Nugent, & Rule, 2004). The literature review, the opinions of experienced faculty, and the availability of data discovered during the collection phase influenced the final selection of the variables for this project.

The student demographic variables that have been examined for a relationship to NCLEX-RN success and failure include age, gender, race, and foreign-born graduates. The accuracy of student demographical variables in predicting NCLEX-RN results have not been consistent. Daley et al. (2003) found no significant relationship between age or gender on student success on the NCLEX-RN, but did find a significant relationship between ethnicity and success. Haas et al. (2004) found that men had a significantly higher failure rate on the NCLEX-RN than women, and African Americans had a significantly higher failure rate than Caucasians. Yin & Burger (2003) found no correlation between age, gender, or ethnicity and success on the NCLEX-RN.

Academic variables that have been examined include high school Grade Point Average (GPA), overall college GPA, GPA in nursing and science courses, grades in nursing clinical and theory classes, failure of nursing courses, and number of C grades in nursing courses. Overall GPA was found to have a statistically significant correlation to NCLEX-RN success by Beeman & Waterhouse (2001) and Yin & Burger (2003). Studies by Daley et al. (2003) and Hass et al. (2004) reported a significant relationship between nursing program GPA and success on the NCLEX-RN. Individual courses that have been found to have a significant relationship with NCLEX-RN success include; (a) pathophysiology I and II, nursing foundations, wellness nursing, and restorative interventions I and II by Beeman & Waterhouse; (b) anatomy and physiology, pathophysiology, and theory and clinical medical-surgical nursing by Daley et al; and (c) biology, chemistry, anatomy and physiology, and psychology by Yin & Burger. Nursing program GPA and course grades for NRSG-101 (Foundations of Nursing), NRSG-102 (Basic Adult Nursing-Medical), NRSG 201 (Intermediate Adult Nursing), and BIOL-152 (Human Anatomy and Physiology II) were originally selected as academic variables for this project. These courses were selected because, according to a faculty member (personal communication, July 29, 2008), these nursing courses have a higher student attrition rate in the program than other core nursing courses.

Beeman & Waterhouse (2001) and Yin & Burger (2003) included variables related to previous college courses completed in their studies on predicting success on the NCLEX-RN. Beeman & Waterhouse reported that students in an accelerated nursing program for students with a bachelor degree in another major of study had fewer failures of the NCLEX-RN than those students who were in the traditional program. Yin & Burger used the number of college credit hours completed prior to nursing program admission as a variable. In their analysis, there was no statistical significance of the relationship of previous credit hours to failure or success on the NCLEX-RN. This project initially included the number of college credit hours a student had completed upon admission to the nursing program as a variable. This variable was dropped from this evaluation because the data was not readily available.

Standardized test scores as a predictor of success on the NCLEX-RN received considerable attention in the literature. The Scholastic Aptitude Test (SAT) verbal scores, American College Testing Assessment (ACT) scores, National League for Nursing test scores, Mosby Assess Test scores, NET scores, and HESI scores, have been examined as predictors of success on the NCLEX-RN. Daley, et al. (2003) found the HESI had greater sensitivity, specificity, and positive and negative predictive values than the Mosby Assess/Test for their baccalaureate-nursing program at Ohio State University College of Nursing. Ellis (2006) found a positive correlation between NET math and reading scores and success in the diploma program at Baton Rouge General Medical Center School of Nursing. Bondmass, Moonie, & Kowalski, (2008) found that scores on the NET categories of composite, composite reading, reading percentile rank, and critical thinking-inferential reading were significantly higher for those students who passed the NCLEX-RN than for those who did not pass. The standardized tests utilized by this nursing program are the NET and the HESI.

The variables English as a Second language and number of college credit hours prior to admission into the nursing program were considered for selection, but the data was not consistently available for all students, so these two variables were not included in selection for the project. Originally, only selected nursing course grades were considered for variables. However, the grades for all nursing courses were readily available in the student database, so all nursing grades were included as variables. The twenty-six variables that were finally selected for evaluation included: (a) age at graduation, (b) gender, (c) race, (d) nursing GPA, (e) overall GPA, (f) NET composite score, (g) NET math score, (h) NET reading score, (i) first HESI score, (j) last HESI score, (k) number of times the HESI was taken, (l) advanced standing admission status, (m) BIOL-151 grade, (n) BIOL-152 grade, (o) grades in each of the 10 nursing courses taken, (p) number of C or lower grades in nursing courses, and (q) number of times readmitted to the program after failure of a nursing course.

Evaluation of Objective Number Two

Objective number two, to develop a spreadsheet and analyze data using SPSS, was met. See Appendix A for a discussion of the project’s statistical analysis. This task seemed daunting at the beginning. The amount of data to be collected and not being able to collect the data from one source made the task more difficult. The task was made more manageable when I received clearance to access the electronic database. From reviewing my journal, I would estimate that I spent about 50 hours collecting and organizing the data.

Learning to work with the SPSS program was intimidating. I felt I just had enough knowledge and resources to understand what I wanted to do to accomplish the objective. The tutorials in the program were less than helpful at times and the assistance from the statistician was invaluable. I now realize that I should have had a clearer idea of what analyses I wanted to perform at the beginning of the project. It was challenging to approach this phase of the project because I had no experience with the process of complex statistical analysis. I do feel that this experience was a success because I now have a much greater understanding of statistics and would confidently approach a future project.

I think I may have selected more, or different variables, to evaluate had I known more about the process of how the nursing program administers the HESI. For example, the analysis revealed a positive correlation between the first HESI score and the NCLEX outcome. There was no correlation between the last HESI score and anything except the grade in NURS-205. The last HESI score might have been the second for some students and the fifth for others because the students were required to keep repeating the HESI until they achieved a score of 900 or better. I did not collect the data of every HESI score because not all scores were readily available. Now I am wondering if the second score might have had more correlations than the last score. It would make sense that the students score would increase each time they took the HESI and this would influence the average of the last score thus reducing the correlation it has with other variables.

Evaluation of Objective Number Three

Objective number three was to develop a risk appraisal instrument to quickly and simply assess students’ risk for failure on the NCLEX-RN. This objective for the project was not met. Barkley, Rhodes, and DeFour used SPSS to analyze variables of student academic performance for correlation to success and failure on the NCLEX-RN to develop the RAI. The variables with significant positive correlation with success were used to construct a table that academic nurse educators could adapt for their program to quickly assess student risk for failure on the NCLEX-RN. Waterhouse & Beeman (2003) adapted the RAI for the nursing program at the University of Delaware. Both studies noted the advantage of using an instrument that can quickly assess students’ NCLEX-RN risk status. These studies provided evidence of the usefulness of adapting the RAI for use by this nursing program.

As noted previously, only two variables were found to have a positive correlation to success on the NCLEX-RN. However, there is an alternative to present to the faculty for consideration. The variable of the score on the first HESI exam was found to have a significant correlation to thirteen other variables. One of the correlations was strong, seven were moderate, and five were weak. Based on an examination of the methods used to establish the reliability and validity of the HESI exams, Morrison et al. (2005) reported that scientific data exists to reassure nursing faculty that the HESI exams can be used confidently to assess student preparedness for the NCLEX-RN. Until further data can be obtained, the faculty could choose to utilize these variables in the creation of the RAI.

Evaluation of Objective Number Four

Objective number four, to prepare and present results of the project and the RAI to the faculty was met. A progress report was presented to the program effectiveness committee on November 12, 2008. The purpose was to obtain feedback and suggestions for the meeting with the statistician. The presentation of the project to the faculty took place on December 1, 2008. The faculty listened intently to the presentation. There were questions about how the variables were counted when data was not available, what the numbers meant in the correlation coefficients, why certain variables were not included, and what the sample size would be needed for an analysis that is more predictable. The faculty was in favor of continuing with this project.

Two tools were constructed to evaluate the project outcomes. The table in Appendix B summarizes the evaluation of the goal and objectives as presented in the previous paragraphs. Appendix C is an evaluation of the project presentation to the faculty that was completed by the preceptor for this project.

Recommendations

The strongest recommendation is to continue to collect and analyze data from successive classes of graduates. As the sample size increases, the estimates will become more accurate (Polit & Beck, 2008). It is possible that there are significant differences within the population sample size that are unobservable due to the small representation of some variables in the data (personal communication, J. Kozemi, November 20, 2008).

The correlation analysis was performed for all variables, and the results raised more questions than they answered. One of the surprising results was the lack of correlation between NURS-205 to any variable except the last HESI score. The other three 200 level courses all correlated to one another. An examination of the grade point averages of NURS-205 revealed that the students who failed the NCLEX-RN outperformed the students who passed the NCLEX-RN in this class. A more thorough investigation of this anomaly is recommended. The implication of an analysis of this data is that curriculum changes to incorporate NCLEX content into the nursing courses should be implemented.

Another area recommended for further analysis is the variable of race. Although there were only seven non-white students, there were significant correlations between race and age at graduation, nursing GPA, NET math scores, first HESI scores, number of times the HESI was taken, grade in NURS-203, and number of C’s or less in a nursing course. These analyses could have an implication to the retention and success of diverse students. Further data collection would add to the strength of these estimates. Additionally, community colleges, serve the population of the county where they are located. White students represent 86% of the students in the population of this study. In this county, white persons comprise 77% of the population (United States Census Bureau, 2008). Further evaluation of the data may reveal sources of the disparity.

A nursing program coordinator at the college has expressed concern about the continued use of the NET as a condition of admission. Her concerns include the sale of the company that provides the service to another entity, the fact that student payment for the exam is handled through the nursing department and the lack of a remediation program for those who score less than the admission standard. The program coordinator is evaluating the services that NET provides against the services HESI provides for an entrance exam. The recommendation is to use the statistical analysis from this project to add another method of evaluation for the reliability of the NET. Although the sample size for the NET scores was twenty-nine, significant correlations were found with race, students who took the HESI more than twice, and BIOL-151 grade. Additional data may reveal additional correlations that could influence the decision on the nursing admission testing process.

Conclusion

Identifying students at risk for failure on the NCLEX-RN is an important endeavor for academic nurse educators (Hass et al., 2004). Predictors of student success and failure on the NCLEX-RN are specific to each program. It is important for each program to determine which of its variables can be utilized as predictors for NCLEX-RN success because student success on the NCLEX-RN has implications for the nursing program as well as the students (Daley, Kirkpatrick, Frazier, Chung, & Moser, 2003; see also Norton et al., 2006; Seldomridge & DiBartolo, 2004). This project took the initial steps needed to identify those predictors for the students. Two variables were identified as predictors of success on the NCLEX-RN for the students, the course grade for NURS-201 and the score of the first HESI exam. The project findings add to the knowledge about program outcomes, but these two variables alone are not sufficient to adapt a risk appraisal instrument for use. Other significant relationships were found that were not considered during the design of the project. The success of this project can be measured through the questions it raised and the implications for future projects that evaluate the nursing program outcomes.

References

Barkley, T. W., Rhodes, R. S., & DuFour, C. A. (1998). Predictors of success on the NCLEX-RN among baccalaureate nursing students. Nursing and Health Care Perspectives, 19(3), 132-138.

Beeman, P. B., & Waterhouse, J. K. (2001). NCLEX-RN performance: Predicting success on the computerized examination. Journal of Professional Nursing, 17(4), 158-165. doi: 10.1053/jpnu.2001.24860.

Bondmass, M. D., Moonie, S., & Kowalski, S. (2008). Comparing NET and ERI standardized exam scores between baccalaureate graduates who pass or fail the NCLEX-RN. International Journal of Nursing Education Scholarship, 5(1), Article 16.

Daley, L. K., Kirkpatrick, B. L., Frazier, S. K., Chung, M. L., & Moser, D. K. (2003). Predictors of NCLEX-RN success in a baccalaureate nursing program as a foundation for remediation. Journal of Nursing Education, 42(9), 390-398.

Eddy, L. L. (2007). Evaluation research as academic scholarship. Nursing Education Perspectives, 28(2), 77-81.

Ellis, S. O. (2006). Nurse entrance test scores: A predictor of success. Nurse Educator, 31(6), 259-263.

Glassick, C. E., Huber, M. T., & Maeroff, G. I. (1997). Scholarship assessed: Evaluation of the professoriate. San Francisco, CA: Jossey-Bass.

Hass, R. E., Nugent, K. E., & Rule, R. A. (2004). The use of discriminant function analysis to predict student success on the NCLEX-RN. Journal of Nursing Education, 43(10), 440-446.

Morrison, S., Adamson, C., Nibert, A., & Hsia, S. (2005). HESI exams: An overview of reliability and validity. Computers, Informatics, Nursing, 22(4), 220-226. doi: 10.1097/00024665-200407000-00016.

National League for Nursing (2005). The scope of practice for academic nurse educators. New York: Author.

Newman, M., Robin, B., & Lauchner, K. A. (2000). Predictive accuracy of the HESI exit exam: A follow-up study. Computers in Nursing, 18(3), 132-136.

Nibert, A. T., & Young, A. (2001). A third study on predicting NCLEX success with the HESI exit exam. Computers in Nursing, 19(4), 172-178.

Norton, C. K., Relf, M. V., Cox, C. W., Farley, J., LaChat, M., & Tucker, M. et al. (2006). Ensuring NCLEX-RN success for first-time test-takers. Journal of Professional Nursing, 22(5), 322-326. doi: 10.1016/j.profnurs.2005.11.004

Polit, D. F., & Beck, C. T. (2008). Nursing research (8th ed.). New York: Lippincott, Williams, & Wilkins.

Seldomridge, L. A., & DiBartolo, M. C. (2004). Can success and failure be predicted for baccalaureate graduates on the computerized NCLEX-RN?. Journal of Professional Nursing, 20(6), 361-368. doi: 10.1016/j.profnurs.2004.08.005

Sewell, J., Culpa-Bondal, F., & Colvin, M. (2008). Nursing program assessment and evaluation. Nurse Educator, 33(3), 109-112. doi: 10.1097/00006223-200509000-00010

Sims, J. M. (2008). An introduction to institutional review boards. Dimensions of Critical Care Nursing, 27(5), 223-225.

Symes, L., Tart, K., & Travis, L. (2005). An evaluation of the nursing success program. Nurse Educator, 30(5), 217-220. doi: 10.1097/00006223-200509000-00010

United States Census Bureau (2008). Genesee County quickfacts from the US Census Bureau. Retrieved September 27, 2008, from

Waterhouse, J. K., & Beeman, P. B. (2003). Predicting NCLEX-RN success: Can it be simplified?. Nursing Education Perspectives, 24(1), pp. 35-39.

Wayne State University (2008). COE introduces 2006-2007 Doctoral graduates. Retrieved July 10, 2008, from

Wood, M. J., & Ross-Kerr, J. C. (2006). Basic steps in planning nursing research (6th ed.). Sudbury, MA: Jones and Bartlett.

Yin, T., & Burger, C. (2003). Predictors of NCLEX-RN success of associate degree nursing graduates. Nurse Educator, 28(5), 232-236.

Appendix A

Project Results

Fifty-one graduates of the May 2008 nursing class took the NCLEX-RN.

Forty-five students were successful on the exam. This equates to an 88% pass rate. Forty-six of the students were female, five were male, 44 were white, six were black, and one was Hispanic. The average age of the graduates was 33 and the age range was from 20 to 51.

Twenty-six variables were evaluated for a correlation to the NCLEX-RN results. Significance was at the 0.05 level. Two variables, the first HESI score and the NURS-201 grade, were found to have a significant positive correlation to success on the NCLEX-RN. The first HESI score correlation coefficient was .324 at the level of .020 indicating that the higher a student scored on the first HESI exam, the more likely they were to be successful on the NCLEX-RN (see graph #1). The NURS-201 grade correlation coefficient was .310 at the level of .027 indicating that the higher a student’s grade in this class, the more likely they were to be successful on the NCLEX-RN (see graph #2).

Although the sample size for black students was small, some interesting trends appeared in the data. The mean age for black students was 9 years higher than for the white students. The mean Nursing GPA, mean NET math scores, and the first HESI scores had a statistically significant correlation for the black students. There was no significant correlation between race and NCLEX-RN results (see table 1).

| |

|State Board Exam Result |grade201 |grade202 |

| | | |

|Adequate number of variables selected and | | |

|variables are appropriate for nursing program. |X | |

| | | |

|Data is complete. |X | |

| | | |

|Analysis provides results that are practical. |X | |

| | | |

|Predictors for risk of failure on NCLEX-RN are |X | |

|identified. | | |

| | | |

|RAI is simple to use for quick assessment of | | |

|student risk for failure on the NCLEX-RN. | |X |

-----------------------

Appendix C

Appendix C

Appendix C

[pic]

Graph # 1

[pic]

Graph # 2

[pic]

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download