Quantitative Validation Overview - pSMILE



|Author: |Document Number: |Equ51-04 |

|Validation Committee | | |

| |Effective (or Post) Date: |19 Dec 2008 |

|Review History |Date of last review: |13-Feb-2020 |

| |Reviewed by: |Heidi Hanes |

|SMILE Comments: This document is provided as an example only. It must be revised to accurately reflect your lab’s specific |

|processes and/or specific protocol requirements. Users are directed to countercheck facts when considering their use in other |

|applications. If you have any questions contact your SMILE representative. |

SMILE Accuracy Guidelines-Chemistry

|SMILE Accuracy Guidelines-Chemistry |Document Number |210 |

| |Effective Date |19 Dec 2008 |

|Subject |Page |1 of 1 |

|Guidelines for performing accuracy testing on a chemistry analyzer | | |

| |Supercedes |New |

|Review History |Date of last review: |19 Dec 2008 |

| |Reviewed by: |Erin Gover |

| |Version # [0.0] |Revision Date |Description (notes) |

|Revision History | |[dd/mm/yy] | |

| | | | |

| | | | |

| | | | |

| | | | |

| | | | |

| | | | |

[pic]SMILE Chemistry Accuracy Guidelines

ACCURACY is the true value of a substance being measured. Verification of accuracy is the process of determining that the test system is producing true, valid results.

I. Determine your comparison or reference method

A. The comparison method must be previously validated.

B. The comparison method must be currently performing successfully on EQA

C. The ideal comparison method is a similar instrument/method

D. Comparison to an in-house method is preferred if the in-house instrument meets the above criteria.

E. Samples with known values, such as proficiency testing samples or commercial standards, may be used as the reference method.

II. Sample Criteria

A. A minimum of 20 samples that cover the reportable range of the method and include points near the Medical Decision Points.

B. Patient, quality control, and proficiency testing materials may be used.

C. Testing: Run each sample in duplicate on each instrument

1. Ideally samples should be run within 2 hours of each other.

2. Duplicates should be averaged.

3. Data should be plotted immediately to identify and correct any outliers.

III. Acceptability criteria—evaluate the data using one of the options below:

[pic]

|If |Then |

| R 0.975 |Proceed with step B below to evaluate acceptability. |

B. Using this method, 95% of the data points from the comparative method must be within Total Allowable Error limit of the reference method (refer to SMILE Chemistry TE Limits table, Appendix 2). Use one of the methods below to evaluate.

C. Evaluate results using the table below:

|If |Then |

|95% of Error Indices are acceptable |Accuracy is acceptable |

| |Proceed with Linearity experiments |

|0.975.

1. Plot the reference method on the X axis and the method being validated on the Y axis.

2. Enter the Allowable Total Error (TEa) concentration and percent. Refer to SMILE Chemistry TE Limits table (Appendix 2).

3. Enter Medical Decision Points

i. A Medical Decision Point (MDP) is the concentration of the analyte at which a medical decision is triggered. You may enter up to five values. These values are plotted on the graph, and the program computes a predicted Y value and confidence interval at each X decision point.

ii. For many analytes, the medical decision points correspond to the lower and upper limits of the normal range. It is advisable to also include medical decision points at the DAIDS Toxicity limits and/or laboratory established critical values.

4. Evaluate using one of the methods described below:

5. Evaluate acceptability using the table below:

|If |Then |

|All error indices fall between -1|Proceed with step 6 below. |

|and 1 | |

|Any error indices fall outside -1|Consider possible causes of inaccuracy. |

|and 1 |Troubleshoot before repeating accuracy studies. |

| |Consult SMILE for assistance. |

6. Six Sigma method acceptability evaluation

a. Plot the data in a Six Sigma analysis program such as the SMILE Accuracy Pack (Appendix 1) or EP Evaluator Six Sigma Metrics module.

b. Enter the Allowable Total Error (TEa) concentration and percent. Refer to SMILE Chemistry TE Limits table.

c. Enter Medical Decision Points.

d. Enter the slope and intercept calculated in step D above.

i. If using EP Evaluator, use the Deming Regression slope and intercept calculated by the program.

ii. If EP Evaluator is not used, it is acceptable to use slope and intercept calculated using regular regression.

e. Enter the Means and %CVs from your Between Day Precision experiments.

i. If using EP Evaluator create the report by selecting the Print Preview button. (See Appendix 6 for an example EP report).

ii. The SMILE Accuracy Pack and EP Evaluator calculate a Sigma metric for each Medical Decision point.

iii. Sigma metric=(TEa-Systematic Error)/Random Error

f. Evaluate acceptability at each MDP using the table below:

|If |Then |

|The Sigma metric less |The method has unacceptable performance and does not meet your requirement for quality, even when the method is |

|than 2.0 |working properly. |

| |It is not acceptable for routine operation. |

|The Sigma metric is |The method has marginal performance and provides the necessary quality when everything is working correctly. |

|between 2.0-3.0 |This method will require: |

| |4-8 controls per run |

| |well-trained operators |

| |reduced rotation of personnel |

| |more aggressive preventive maintenance |

| |careful monitoring of patient test results |

| |continual efforts to improve method performance |

|If the Sigma metric is |The method has fair performance and meets your requirement for quality and can be well-managed in routine |

|between 3.0-4.0 |operation. |

| |This method will require 2 control measurements per run using standard Westgard QC rules. |

|If the Sigma metric is |The method has good performance and is clearly acceptable and can be well-managed in routine operation with only |

|between 4.0-6.0 |2 control measurements per run using standard Westgard QC rules. |

|If the Sigma metric is |The method has Six Sigma performance and is even easier to manage and control. |

|>6.0 | |

IV. References

A. GCLP Guidelines, Standard for Performance Specifications.

B. Clinical and Laboratory Standards Institute (CLSI). User Verification of Performance for Precision and Trueness: Approved Guideline-Second Edition. CLSI document EP15-A2 (ISBN 1-56238-574-7). Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA.

C. NCCLS. (Currently CLSI) Evaluation of Precision Performance of Quantitative Measurement Methods; Approved Guideline—Second Edition. NCCLS document EP5-A2 (ISBN 1-56238-542-9). NCCLS, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA.

D. Clinical and Laboratory Standards Institute (CLSI). Method Comparison and Bias Estimation Using Patient Samples: approved Guidelines- Second Edition. CLSI document EP9-A2 (ISBN 1-56238-472-4). Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA.

E. Clinical and Laboratory Standards Institute (CLSI).Preliminary Evaluation of Quantitative Clinical Laboratory Measurement Procedure: Approved Guidline – Third Edition. CLSI document EP10-A3 (ISBN 1-56238-622-0). ). Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA.

F. Clinical and Laboratory Standards Institute (CLSI).Estimation of Total analytical Error for Clinical Laboratory Methods. Approved 2003. CLSI document EP21-A (ISBN 1-56238-502-xX. Clinical and Laboratory Standards Institutes, 940 West Valley Road, Suite 100, Wayne, Pennsylvania 19098-1898 USA.

G. EP Evaluator Release 8, David G. Rhoads Associates Inc., .

H. James O. Westgard, Online Validation Training, Westgard QC, Inc. , Sections 11-Determining Bias,12- Estimating Trueness, and 13- Judging Method Acceptability.

-----------------------

A. Plot the data in EP Evaluator Two Instrument Comparison Module or a similar regression analysis program.

1. Plot the reference method on the X axis and the method being validated on the Y axis.

2. Enter the Allowable Total Error (TEa) concentration and percent. Refer to SMILE Chemistry TE Limits table. (Appendix 2)

3. Enter the results for the reference (X) and the new (Y) methods.

4. Evaluate the statistics by clicking on Print Preview or on the statistics tab.

5. The correlation coefficient (R) must be >0.975

I. Plot the data in a regression analysis program or spreadsheet. The SMILE Accuracy Pack is included for this use (Appendix 1).

1. Plot the reference method on the X axis and the method being validated on the Y axis.

2. Enter the Allowable Total Error (TEa) concentration and percent. Refer to SMILE Chemistry TE Limits table. (Appendix 2)

3. Enter the results for the reference (X) and the new (Y) methods.

4. Evaluate the statistics.

5. The correlation coefficient (R) must be >0.975

1. The EP Evaluator program uses a statistic called the “Error Index” to measure the difference between the two methods as a ratio of the Total Allowable Error.

a. The Error Index can be calculated by subtracting the reference method data point (X) from the method being validated data point (Y) and dividing by the Total Allowable Error (TEa). The equation is: (Y-X)/TEa.

b. EP Evaluator uses the percent TEa or the minimum detectable difference (absolute TEa), whichever is greater.

c. The absolute TEa value is calculated by converting the TEa percentage to a decimal and multiplying by the reference method data point.

d. The Error Index is measured for each X-Y pair, and must fall within -1 and 1. If more than 5% of the specimens have an Error Index of less than -1 or greater than 1, the accuracy experiment fails. (See Appendix 3 for an example EP report).

1. Using a regression analysis program or spreadsheet (such as the SMILE Accuracy Pack, Appendix 1) calculate the “Error Index” to measure the difference between the two methods as a ratio of the Total Allowable Error.

a. The Error Index can be calculated by subtracting the reference method data point (X) from the method being validated data point (Y) and dividing by the Total Allowable Error (TEa). The equation is: (Y-X)/TEa.

b. Use the percent TEa or the minimum detectable difference (absolute TEa), whichever is greater.

c. The absolute TEa value is calculated by converting the TEa percentage to a decimal and multiplying by the reference method data point.

d. The Error Index is measured for each X-Y pair, and must fall within -1 and 1. If more than 5% of the specimens have an Error Index of less than -1 or greater than 1, the accuracy experiment fails.

a. If using EP Evaluator program create the report by selecting the Print Preview button.

b. Review the Medical Decision Point Analysis data on page 1 of the report. (See Appendix 4 for an example EP report)

c. Subtract the X Method MDP from the low and the high 95% confidence limits for each Medical Decision Point.

d. Calculate Error Indices for each end of the 95% confidence limits for each Medical Decision Point as follows: Divide the difference (from step c above) by the SMILE TE Limit absolute value.

i. Use the percent TEa or the minimum detectable difference, whichever is greater.

ii. To calculate the absolute TEa value, convert the TEa percentage to a decimal and multiply by the reference method data point.

iii. All error indices must fall within -1 and 1. An Excel spreadsheet is provided to calculate Error Indices (See Appendix 5 SMILE Error Index Calculator).

a. If using a regression analysis program or spreadsheet calculate and evaluate the 95% confidence limits and error indices. The SMILE Accuracy Pack, (Appendix 1) performs these calculations automatically.

i. Subtract the X Method MDP from the low and the high 95% confidence limits for each Medical Decision Point.

ii. Calculate Error Indices for each end of the 95% confidence limits for each Medical Decision Point as follows: Divide the difference (from step i. above) by the SMILE TE Limit absolute value.

1. Use the percent TEa or the minimum detectable difference, whichever is greater.

2. To calculate the absolute TEa value, convert the TEa percentage to a decimal and multiply by the reference method data point.

i. All error indices must fall within -1 and 1. An Excel spreadsheet is provided to calculate Error Indices (See Appendix 5 SMILE Error Index Calculator).

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download