February 10, 2020 Version 2.0, Change 1 - Under Secretary of ...

February 10, 2020

Version 2.0, Change 1

CLEARED

FOR OPEN PUBLICATION

FEB 06 2020

CASE # 20-S-0618

Department of Defense

OFFICE OF PREPUBLICATION AND SECURITY REVIEW

Questions and issues regarding the content and format of this document, please email

guidebookfeedback@

For technical issues and questions regarding Cybersecurity Developmental Test and Evaluation,

please email

osd.pentagon.ousd-re.munications@mail.mil

Office of the Under Secretary of Defense, Research and Engineering, Advanced Capabilities,

Developmental Test and Evaluation and Prototyping ()

For technical issues and questions regarding Cybersecurity Operational Test and Evaluation,

please email

Mr. David Aland (david.j.aland.civ@mail.mil)

Office of the Director, Operational Test and Evaluation ()

Cybersecurity Test and Evaluation Guidebook 2.0, Change 1

Table of Contents

Introduction ........................................................................................................................................ 1

1.1 Organization of This Guidebook .................................................................................................... 1

1.2 Audience ........................................................................................................................................ 2

1.3 Applicability ................................................................................................................................... 2

1.4 Terminology ................................................................................................................................... 2

Cybersecurity Policies and Guidance for Defense Acquisition Programs and Systems .............. 3

2.1 Operation of the Defense Acquisition System, DoDI 5000.02 ...................................................... 3

2.2 Fiscal Year 2016 National Defense Authorization Act (NDAA) Section 804 ............................... 3

2.3 Business Systems Requirements and Acquisition, DoDI 5000.75 ................................................. 4

2.4 Cybersecurity, DoDI 8500.01 ........................................................................................................ 4

2.5 Cybersecurity Activities Support to DoD Information Network Operations (DODIN), DoDI

8530.01 ..................................................................................................................................... 6

2.6 Joint Requirements Guidance......................................................................................................... 6

2.7 DOT&E Cybersecurity Procedures Memoranda ............................................................................ 7

Cybersecurity Test and Evaluation Overview ................................................................................. 8

3.1 Cybersecurity T&E Phases Overview ............................................................................................ 8

3.2 Cybersecurity Working Group ..................................................................................................... 11

3.3 Cybersecurity Threat Assessments ............................................................................................... 12

3.4 DT&E and SE Collaboration........................................................................................................ 12

3.5 Early Tester/Analyst Involvement................................................................................................ 14

3.6 Mission-Based Cyber Risk Assessments ..................................................................................... 15

3.7 Role of Cybersecurity Developmental Testing ............................................................................ 15

3.8 Integrated Testing......................................................................................................................... 20

Phase 1: Understand Cybersecurity Requirements (and Plan for T&E) .................................... 23

4.1 Schedule ....................................................................................................................................... 23

4.2 Inputs ............................................................................................................................................ 24

4.3 Tasks ............................................................................................................................................ 25

4.4 Phase 1 Data Requirements .......................................................................................................... 29

Phase 2: Characterize the Cyber-Attack Surface ......................................................................... 31

5.1 Schedule ....................................................................................................................................... 31

5.2 Inputs ............................................................................................................................................ 32

5.3 Tasks ............................................................................................................................................ 33

5.4 Phase 2 Data Requirements .......................................................................................................... 42

Phase 3: Cooperative Vulnerability Identification ........................................................................ 44

i

Cybersecurity Test and Evaluation Guidebook 2.0, Change 1

6.1 Schedule ....................................................................................................................................... 45

6.2 Inputs ............................................................................................................................................ 45

6.3 Tasks ............................................................................................................................................ 46

6.4 Phase 3 Data Requirements .......................................................................................................... 56

Phase 4: Adversarial Cybersecurity DT&E................................................................................... 58

7.1 Schedule ....................................................................................................................................... 59

7.2 Inputs ............................................................................................................................................ 59

7.3 Tasks ............................................................................................................................................ 59

7.4 Phase 4 Data Requirements .......................................................................................................... 65

Phase 5: Cooperative Vulnerability and Penetration Assessment ............................................... 66

8.1 Schedule ....................................................................................................................................... 66

8.2 Inputs ............................................................................................................................................ 67

8.3 Tasks ............................................................................................................................................ 67

8.4 Outputs ......................................................................................................................................... 69

Phase 6: Adversarial Assessment .................................................................................................... 70

9.1 Schedule ....................................................................................................................................... 70

9.2 Inputs ............................................................................................................................................ 71

9.3 Tasks ............................................................................................................................................ 71

9.4 Outputs ......................................................................................................................................... 72

Acronyms and Glossary of Terms .................................................................................................. 73

10.1 Acronyms ............................................................................................................................... 73

10.2

Cybersecurity T&E Glossary of Terms .................................................................................. 77

References ......................................................................................................................................... 83

Cybersecurity T&E Phase 1 through 6 Quick Look .................................................. A-1

Incorporating Cybersecurity T&E into DoD Acquisition Contracts ....................... B-1

Considerations for Tailoring the Cybersecurity T&E Phases .................................. C-1

Key System Artifacts for Cybersecurity T&E Analysis and Planning .................... D-1

Guidance for the Cybersecurity Portion of the Developmental Evaluation

Framework (DEF)......................................................................................................... E-1

Considerations for Staffing Cybersecurity T&E Activities ...................................... F-1

Considerations for Software Assurance Testing ........................................................G-1

Considerations for Cybersecurity Requirements and Measures for DT&E (FOUO

Document)......................................................................................................................X-1

Cyber Threat Assessment for Cybersecurity T&E (FOUO Document) ..................X-1

Mission-Based Cybersecurity Risk Assessments (FOUO Document) ......................X-1

ii

Cybersecurity Test and Evaluation Guidebook 2.0, Change 1

Cybersecurity Test Infrastructure and Environment Planning (FOUO Document) ...

.........................................................................................................................................X-1

Cybersecurity Test Considerations for Non-IP Systems (FOUO Document) .........X-1

iii

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download