Serology Test Evaluation Report for “SARS-COV-2 ELISA” from United ...

[Pages:11]Serology Test Evaluation Report for "SARS-COV-2 ELISA" from United Biomedical, Inc.

Updated

December 30, 2020

Contents

1 Introduction

2

1.1 Panel composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 Important caveats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.4 Notes about the evaluation procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.5 Additional notes, anomalies, and clarifications . . . . . . . . . . . . . . . . . . . . . . 5

2 Results

6

3 Line Data

7

List of Tables

1 Summary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Summary Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 Line Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1 Introduction

The SARS-COV-2 ELISA from United Biomedical, Inc. was tested on 2020-09-01 at the Frederick National Laboratory for Cancer Research (FNLCR), a Federally Funded Research and Development Center (FFRDC) sponsored by the National Cancer Institute (NCI). Tests were from lot number 0153004. The SARS-COV-2 ELISA is intended to qualitatively detect IgG.

1.1 Panel composition

A IgM+ Titers in Panel 3

15

12 11

10

6

5

B IgG+ Titers in Panel 3

15

12 11

10

7

5

Number of Samples Number of Samples

0 100

1

400

1600

6400

Titer

0

0

100

400

1600

6400

Titer

Figure 1: Titer levels for (A) IgM+ and (B) IgG+ samples according to the CDC SARS-CoV-2 Spike antigen assay

The test was evaluated against "Panel 3," which includes frozen SARS-CoV-2 antibody-positive serum samples (n = 30) and frozen antibody-negative Anticoagulant Citrate Dextrose Solution Formula A (ACD-A) plasma samples (n = 80). The panel size and composition were chosen to enable a laboratorybased evaluation and to provide reasonable estimates and confidence intervals for test performance in the context of limited sample availability. The sample size is comparable to that of a typical sample size used to support Emergency Use Authorization (EUA) by FDA for tests of this type.

Page 2 of 11

1.1.1 Positive samples

Positive samples used in Panel 3 were from patients previously confirmed to have SARS-CoV-2 infection with a nucleic acid amplification test (NAAT). Time between symptom onset, NAAT testing, and sample collection is not known for all samples. Both SARS-CoV-2 IgM and IgG antibodies are present in all Panel 3 positive samples. The Centers for Disease Control and Prevention (CDC) detected the presence of IgG and IgM antibodies at their laboratory using their SARS-CoV-2 spike enzymelinked immunosorbent assay (ELISA) tests.1 The presence of antibodies was confirmed at FNLCR using CDC's developed ELISAs (Pan-Ig, IgG, and IgM) as well as an IgG Receptor Binding Domain (RBD) ELISA developed by the Krammer Laboratory at the Icahn School of Medicine at Mount Sinai.2 The positive samples selected may not reflect the distribution of antibody levels in patient populations that would be evaluated by such a test. Because all samples are positive for both IgM and IgG, this evaluation cannot verify that tests intended to detect IgM and IgG antibodies separately detect these antibodies independently.

Positive samples were assessed at dilutions of 1:100, 1:400, 1:1600, and 1:6400 by CDC on their Pan-Ig assay, their IgM assay, and their IgG assay. Some samples were run at additional dilutions. Any samples that were positive at a dilution greater than 1:6400 were assigned a titer of 6400 because 1:6400 was the highest dilution at which all positive samples used in these evaluations were assessed.

1.1.2 Negative samples

All Panel 3 negative samples were collected prior to 2020, before the SARS-CoV-2 virus is known to have circulated in the United States. Panel 3 groups include:

? "Negatives" (n = 70): selected without regard for clinical status. This group includes a sample, C0063, that showed reactivity in the Pan-Ig CDC spike ELISA at FNLCR.

? "HIV+" (n = 10): selected from banked plasma from HIV+ patients.3 This group includes 3 samples, C0018, C0155, and C0182, that showed reactivity in the IgG RBD ELISA at FNLCR.

All Panel 3 negative samples were assessed at dilutions of 1:100 and 1:400 by CDC on their PanIg assay. A subset of samples was assessed in parallel at additional dilutions and on the CDC IgM

1See , which notes "CDC's serologic test has been designed and validated for surveillance and research purposes. It is designed to estimate the percentage of the U.S. population previously infected with the virus ? information needed to guide the response to the pandemic and protect the public's health. The CDC test is not currently designed to test individuals who want to know if they have been previously infected with SARS-CoV-2. Commercial tests are available to provide test results to individuals." 2An implementation of this test, the COVID-19 ELISA IgG Antibody Test, has been granted an EUA authorization by FDA for use at the Mount Sinai Laboratory (MSL), Center for Clinical Laboratories, a division of the Department of Pathology, Molecular, and Cell-Based Medicine, New York, NY. See . 3HIV+ samples were deemed appropriate for inclusion in the panel: (1) to increase the sample size and reduce the confidence interval; and (2) to identify any possibility of cross-reactivity with HIV+ samples. It is anticipated that other types of samples, as they become available, may also be evaluated in any future analyses.

Page 3 of 11

and IgG assays. All Panel 3 negative samples were negative at a dilution of 1:100 on the CDC PanIg assay. These samples were assigned an undetectable titer (represented as zero (0) in the line data) for the Pan-Ig assay, the IgM assay, and the IgG assay.

1.2 Analysis

Samples used in this evaluation were not randomly selected, and sensitivity (PPA) and specificity (NPA) estimates in this report may not be indicative of the real-world performance of the United Biomedical, Inc. SARS-COV-2 ELISA. Sensitivity and specificity were calculated for each antibody (e.g., IgM, IgG, IgA, and Pan-Ig, as applicable) separately. In addition, sensitivity and specificity were estimated in a combined manner, where a positive result for any antibody the United Biomedical, Inc. SARSCOV-2 ELISA is intended to detect was considered as a positive test result and a negative result meant that a sample tested negative for all antibodies the United Biomedical, Inc. SARS-COV-2 ELISA is intended to detect. Positive and negative predictive values were calculated for combined sensitivity and specificity assuming a prevalence of 5%. Cross-reactivity with HIV+ was evaluated, and results are presented separately. If cross-reactivity was detected, the samples with HIV+ were not included in calculations of specificity.

Confidence intervals for sensitivity and specificity were calculated per a score method described in CLSI EP12-A2 (2008).4 Confidence intervals for PPV and NPV were calculated using the values from the 95% confidence intervals for sensitivity and specificity. For evaluation of cross-reactivity with HIV+, it was evaluated whether an increased false positive rate among antibody negative samples with HIV was statistically higher than the false positive rate among antibody negative samples without HIV (for this, a confidence interval for the difference in false positive rates was calculated per a score method described by Altman.5)

1.3 Important caveats

Sensitivity and specificity estimates in this report may not be indicative of the real world performance of the United Biomedical, Inc. SARS-COV-2 ELISA.

These results are based on serum and plasma samples only and may not be indicative of performance with other sample types, such as whole blood, including finger stick blood.

The number of samples in the panel is a minimally viable sample size that still provides reasonable estimates and confidence intervals for test performance, and the samples used may not be representative of the antibody profile observed in patient populations.

4CLSI. User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline--Second Edition. CLSI document EP12-A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2008. See . gov/scripts/cdrh/cfdocs/cfStandards/detail.cfm?standard__identification_no=31791. 5Statistics with Confidence: Confidence Intervals and Statistical Guidelines. (2013). Wiley.

Page 4 of 11

1.4 Notes about the evaluation procedure

? The United Biomedical, Inc. SARS-COV-2 ELISA was used per the manufacturer's package insert.

? Devices were tested within any expiration dates provided. ? Devices were not obviously defective / compromised. ? Devices were stored at the FNLCR within their labeled conditions. ? A single operator conducted the test. ? The personnel who performed the testing were blinded to the identity / code of the sample and

the expected results. ? The testing was performed in a non-clinical laboratory environment. ? Negative and positive samples were ordered randomly and then tested serially. ? The United Biomedical, Inc. SARS-COV-2 ELISA was run with positive and negative controls.

1.5 Additional notes, anomalies, and clarifications

The FNLCR provided the following additional information: The Instructions for Use describe the following testing strategy: the positives are marked as "Initially Positive". Follow up testing should be performed on the "Initially Positive" samples. These samples should be tested in duplicate to confirm the positive results. Follow up testing has not been performed at FNLCR as of 09SEP20.

Page 5 of 11

2 Results

SARS-COV-2 ELISA

IgG+ IgGTotal

Table 1: Summary Results

Comparator Method

Collected pre-2020

Antibody Positive

Antibody Negative

IgM+, IgM+, IgG+ IgG-

IgM-, Negative HIV+ IgG+

27

3

70

10

30

70

10

Total

27 83 110

Table 2: Summary Statistics

Measure

IgG Sensitivity IgG Specificity Combined Sensitivity Combined Specificity Combined PPV for prevalence = 5.0%

Combined NPV for prevalence = 5.0% Cross-reactivity with HIV+

Estimate

90.0% (27/30) 100% (80/80) 90.0% (27/30) 100% (80/80) 100%

Confidence Interval

(74.4%; 96.5%) (95.4%; 100%) (74.4%; 96.5%) (95.4%; 100%) (46.1%; 100%)

99.5%

0.0% (0/10), not detected

(98.6%; 99.8%)

Page 6 of 11

3 Line Data

Page 7 of 11

In the table below, "Days" refers to "Days from symptom onset to blood collection." Table 3: Line Data

Sample IgG Result Number

1 Negative 2 Positive 3 Negative 4 Negative 5 Negative

6 Positive 7 Positive 8 Negative 9 Negative 10 Negative

11 Negative 12 Negative 13 Positive 14 Negative 15 Negative

16 Positive 17 Positive 18 Negative 19 Positive 20 Negative

21 Negative 22 Negative

Control Sample Type ID

Pass Pass Pass Pass Pass

Pass Pass Pass Pass Pass

Pass Pass Pass Pass Pass

Pass Pass Pass Pass Pass

Pass Pass

C0105 D0209 C0137 D0023 D0086

D0326 D0168 D0049 C0133 D0113

C0121 D0007 D0040 C0089 D0192

D0386 D0405 C0185 D0307 C0182

D0056 D0166

Plasma Serum Plasma Plasma Plasma

Serum Serum Plasma Plasma Plasma

Plasma Plasma Serum Plasma Plasma

Serum Serum Plasma Serum Plasma

Plasma Plasma

CDC Spike Pan-Ig Titer

0 6400

0 0 0

400 400

0 0 0

0 0 100 0 0

6400 400 0

6400 0

0 0

CDC Spike

IgM Titer

0 1600

0 0 0

400 100

0 0 0

0 0 100 0 0

6400 400 0

1600 0

0 0

CDC Spike

IgG Titer

0 6400

0 0 0

1600 400 0 0 0

0 0 400 0 0

6400 400 0

6400 0

0 0

Days Group

Negative 42 Positive

Negative Negative Negative

Positive 24 Positive

Negative Negative Negative

Negative Negative 43 Positive HIV+ Negative

20 Positive 32 Positive

Negative 41 Positive

HIV+

Negative Negative

Page 8 of 11

Sample IgG Result Number

23 Positive 24 Positive 25 Negative

26 Negative 27 Negative 28 Negative 29 Negative 30 Negative

31 Negative 32 Positive 33 Positive 34 Negative 35 Negative

36 Positive 37 Negative 38 Negative 39 Negative 40 Negative

41 Negative 42 Negative 43 Negative 44 Negative 45 Negative

46 Negative 47 Negative 48 Positive

Table 3: Line Data (continued)

Control Sample Type ID

Pass Pass Pass

Pass Pass Pass Pass Pass

Pass Pass Pass Pass Pass

Pass Pass Pass Pass Pass

Pass Pass Pass Pass Pass

Pass Pass Pass

D0451 D0231 C0041

C0054 D0123 C0032 D0135 C0196

D0187 D0498 D0344 C0095 D0097

D0349 C0018 D0031 D0064 C0156

D0184 C0103 C0198 C0098 C0179

C0059 C0063 D0102

Serum Serum Plasma

Plasma Plasma Plasma Plasma Plasma

Plasma Serum Serum Plasma Plasma

Serum Plasma Plasma Plasma Plasma

Plasma Plasma Plasma Plasma Plasma

Plasma Plasma Serum

CDC Spike Pan-Ig Titer

1600 6400

0

0 0 0 0 0

0 1600 6400

0 0

1600 0 0 0 0

0 0 0 0 0

0 0 400

CDC Spike

IgM Titer

400 1600

0

0 0 0 0 0

0 400 1600

0 0

400 0 0 0 0

0 0 0 0 0

0 0 100

CDC Spike

IgG Titer

1600 6400

0

0 0 0 0 0

0 1600 6400

0 0

6400 0 0 0 0

0 0 0 0 0

0 0 400

Days Group

35 Positive 19 Positive

Negative

HIV+ Negative Negative Negative Negative

Negative 17 Positive 27 Positive

Negative Negative

19 Positive HIV+ Negative Negative Negative

Negative Negative Negative Negative Negative

Negative Negative 24 Positive

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download