DevSecOps Review - AF



DoD Enterprise DevSecOps Maturity ReviewVersion: 1.6Author: Nicolas Chaillan, Chief Software Officer, U.S Air Forcenicolas.m.chaillan.civ@mail.milQuestions listed are designed to give an in-depth understanding of the Software Factory’s approach to DevSecOps, as it relates to each topic area. This review will be used to proactively propose improvements and better understand tools/capabilities being used so enterprise capabilities can be deployed. The review covers both cybersecurity maturity and DevSecOps maturity for each topic area.Table of Contents TOC \o "1-3" \h \z \u DevSecOps Review PAGEREF _Toc20201453 \h 2DoD Specific PAGEREF _Toc20201454 \h 2Generic PAGEREF _Toc20201455 \h 3Team / Languages / Security PAGEREF _Toc20201456 \h 3Culture / Principles PAGEREF _Toc20201457 \h 4Container Orchestration PAGEREF _Toc20201458 \h 5Containers PAGEREF _Toc20201459 \h 6Backup / Resiliency / Disaster Recovery PAGEREF _Toc20201460 \h 6Opensource Software PAGEREF _Toc20201461 \h 6Continuous Integration / Continuous Delivery PAGEREF _Toc20201462 \h 7CI/CD solution PAGEREF _Toc20201463 \h 7Plan PAGEREF _Toc20201464 \h 7Develop / Integrate PAGEREF _Toc20201465 \h 8Build PAGEREF _Toc20201466 \h 9Artifacts PAGEREF _Toc20201467 \h 9Test PAGEREF _Toc20201468 \h 9Secure PAGEREF _Toc20201469 \h 10Deploy & Operate PAGEREF _Toc20201470 \h 10Monitor / Continuous Monitoring PAGEREF _Toc20201471 \h 10Metrics / Maturity PAGEREF _Toc20201472 \h 12DevSecOps Review DoD SpecificHave you reviewed the DoD Enterprise DevSecOps Ref Design document signed by DoD CIO and the Air Force Chief Software Officer for a DoD-wide DevSecOps architecture compliance?Do you believe you are compliant with its MVP requirements?CNCF Kubernetes compliant PlatformOCI Container complianceIs your CI/CD stack containerized?Are you leveraging the containers from the DoD Centralized Artifacts Repository (DCAR)?If not, what is your timeline to migrate to DoD-approved containers from DCAR?Are you using the Sidecar Container Security Stack?Service Mesh with Zero Trust Capability?EFK centralized logging?Twistlock?Anchore?OpenSCAP?Are you using Infrastructure as Code (IaC) to instantiate your environments and avoid drifts between environments? Is the IaC unclassified? Which Impact Level?Enumerate the different environments (development, testing, staging and production) Please provide an unclassified diagram of the interaction between environments and whether they are automated or manual.For each environment, please provide the following details for each (at the unclassified level):Name of the environmentInfrastructure provider (Cloud provider or on-premise provider)Whether the environment is a Government Furnished Equipment (GFE) infrastructure or an environment provided by a third party vendor, if so, which oneWhether the environment is maintained by a third party vendor, if so, which oneDiagram including ports, protocols and ingress/egress, particularly when it comes to accessing the platform from Internet, NIPRnet or other networksPlatform of choice (Kubernetes, PCF or other, if so, detail)Which Operating System is used to run your Platform and its STIG percentage levelClassification level for code and dataImpact Level (Clouds SRG)If Cloud environment, what provider is your Cyber Security Service Provider (CSSP)?Endpoints and connectivity: What connectivity/transport is used to access the environment for both developers and administratorsIs it encrypted?Are you using a VPN/SDP (Software Defined Perimeter)?What authentication factors are being used for developers and administrators?Are you using a Single Sign On capability?Are you using GFE endpoints? What operating system are you using on your endpoints?Are you using a replicable developer endpoint image?If not, what endpoint is used and how is it patched/updated?Are you using ACAS/HBSS on the endpoints and where are the logs forwarded to? If not, what is being used?Describe whether you have an automated way to push to the next environment (Diode, CDS) or a manual process (DVD/Blu-ray burning etc.)Did you decouple your data from the source code to ensure that Impact Level is accurate for your development environment?Are your base OS (containers and VMs) STIGed? At what percentage? Is this automated for new releases?Whether you believe you are compliant with the DoD Cloud SRG. If not, describe what is not compliantPlease list any issues you are facing with being compliant with STIG or DoD policies? What would need to be changed for you to be compliant?Who is your Authorizing Official?Do you have an Authority to Operate (ATO)? A Continuous ATO? What were the ATO conditions?Please list all the IaaS, PaaS used.Please list all SaaS used or planned on being used. Is there any SaaS that is not FedRAMPed?GenericTeam / Languages / SecurityWhat kind of workloads are you running?How many developers are you supporting?What kind of developers are they? (federal employees/contractors)?What is your Service Level Objective (SLO)?Do you offer a Service Level Agreement (SLA) with your users?What is the size of your team and geographic representation?What is your team’s composition (vendors, federal employees (military/civilian) etc.)?What programming languages are you currently using?What programming languages are you planning on using?What databases are you currently using?What databases are you planning on using?What OSes are you currently using (for both VMs and Containers)?What OSes are you planning on using (for both VMs and Containers)?What is your solution for identity management/identity provider?What is your solution for Single Sign On?How do you manage roles/access control?How do you get your updates/patches? How often are they applied to your platform?Do you preform penetration tests on your environments? If so, what is covered by that scope of work (endpoints, Cloud, which environments etc.) and how often?Do you possess a red-team? Please describe.Do you possess a blue-team? Please describe.How do you perform your continuous monitoring? Is it based on signatures or also leverages behavior detection?Do you have a Security Operations Center?Has a Security Champion (CISO) been established? Is he empowered to evaluate and mitigate security concerns below a specified risk threshold? Culture / PrinciplesAre you leveraging the DevOps/DevSecOps culture?What Agile framework are you using (Scrum, Kanban…)?Are you leveraging Microservices?Are you using the strangler pattern to move legacy applications to modern Cloud-native?Do you have your own “DevOps/DevSecOps Guide” to describe your process, team and various documentation information?Are you using the Zero Trust principle or a Perimeter approach?Do you use concepts such as Pair Programming, XP, Test Driven Development etc.? If so, which ones?Do you want to enable Continuous Deployment or simply Continuous Delivery?Do you leverage Infrastructure as Code to ensure your Infrastructure and CI/CD stack is immutable and automated?Can you easily deploy your entire platform (including CI/CD) on any environment?Do you use concepts such as Chaos Engineering to emulate crashes and become more resilient? If so, on which environments are you actively using them?Container OrchestrationWhat Container orchestration solution(s) do you use? Are your Kubernetes master/nodes/etcd hardened according to the CIS Kubernetes Benchmark?Is your Kubernetes cluster STIGed?Are you using specific Kubernetes Controllers such as Ingress Controllers, Admission Controller? Various settings to detect insider threat/containers being run on your cluster? What solution do you use to instantiate your Kubernetes cluster?How do you migrate to newer releases (blue/green etc.)?Do you leverage Kubernetes namespaces or separate clusters for segmentation? If using namespaces, how do you divide your namespaces?Do you use RBAC? If so, is it tied to your main Identity Provider?Do you use SELinux capabilities?Do you use Pod Security policies or anything that can limit container security risks, container privileges etc.? What other Kubernetes security concepts are you using?Do you use Kubernetes to abstract your storage needs or do you use Cloud/provider APIs etc?Did the cloud provider APIs you used provide some specific capability that you could not access through the Kubernetes API? If so, what?Do you use Kubernetes to abstract your network/ingress/load balancing needs or do you use Cloud/provider APIs etc?Do you use non CNCF-compliant Kubernetes features such as product specific features like OpenShift Routes or similar?What Container registry do you use?What Container Security Solution(s) do you use?How do you store your persistent data (databases, configuration files etc.)?Do you leverage Helm or Kubernetes Operators as a packager?How do you store secret keys, passwords (to login into databases etc.)?If you leverage, Kubernetes secrets, do you ensure that encryption at rest is used?Is your platform FIPS compliant?Do you monitor rogue containers?How do you monitor rogue containers?ContainersWhat Container technology do you use?What container base images do you use? Do you store your Container build file (such as Dockerfile) into your repository?Do you have proper health checks for each Containers/Microservices?Backup / Resiliency / Disaster RecoveryHow are you backing up your Kubernetes cluster (or other platform)? Does that include your etcd, etc.? How is your cluster resilient? Is it deployed on multiple zones? How is it geo-replicated if it is?What is your disaster recovery process? Do you have a written disaster recovery policy?Opensource SoftwareDo you keep a current inventory of opensource libraries/software used across the program?Do you include your opensource library/software into your artifact solution to ensure software is always built with latest versions?Do you monitor opensource libraries/software for vulnerabilities? Do you apply opensource libraries/software patches based on severity?How often do you update your opensource libraries/software? Is this automated (CI/CD)?Continuous Integration / Continuous DeliveryCI/CD solutionDoes it run inside of your Kubernetes CNCF compliant cluster?What Continuous Integration/Continuous Delivery (CI/CD) solution(s) do you use?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege”?Do you use CAC authentication?Is your Continuous Integration completely automated?Is your Continuous Software Security completely automated?Is your Continuous Delivery completely automated?Do you store your CI/CD pipeline files (such as Jenkinsfile) into your source code repository? If so, how do you control change management? Do you mandate multiple user approval for specific changes? If so, which ones?How do chosen CI/CD tools and processes feed back into the decision cycle? How tight is the decision cycle?PlanWhat planning solution(s)/program management solution(s) do you use?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?What Collaboration (chats included) solution(s) do you use?Are they containerized? Are they running as a SaaS product? If so, are they FedRAMPed at the right level?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?Develop / IntegrateWhat source code repository and/or version control system(s) do you use?Do you leverage a single source code repository per project to facilitate integration?Do you properly keep track of your programming languages and code dependencies used in your application?What IDE do you use?Is it web based or installed on endpoints?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?What solution do you use to generate code documentation?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?Are you storing all configuration files into your repository to allow for complete build dynamically?Do you hash and/or sign your artifacts to ensure integrity?Which workflow model do you leverage: trunk-based workflow, branching workflow or forking workflow?If you use branches, how long (in hours) do they usually last, in average, before being merged into trunk?Do you leverage a Service Mesh? If so what solution do you use?How do you use your Service Mesh? Is it used for your applications/containers layer and/or your CI/CD containers as well?Does your Service Mesh has strong baked-in security including mTLS tunneling, strong authentication using certificates per containers etc? Solutions such as ISTIO are compliant with such statement.Do you leveraging a Zero Trust model down to the container level? If so, how?What Bug tracking/Ticket solution(s) do you use?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?BuildWhat Build solution(s) do you use?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?ArtifactsWhat Artifact repository solution(s) do you use?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?Are you storing all artifacts/libraries into your artifact solution to allow for complete dynamic build and to ensure latest versions are used at Build time?TestWhat testing tools/test suites/solution(s) do you use?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?What code coverage solution do you use to keep track of your test coverage?What test coverage percentage are you mandating?Is your QA/Test environment the exact copy of your production environment? Do you use containers to ensure of the exact same replica for your test/production platforms?Do you leverage automated testing from your CI/CD tool?Are you creating Unit tests? Are you creating Functional tests? Are you creating Regression tests? Are you creating Integration tests?Are you conducting load testing?How do you ensure proper Quality Assurance (QA)?SecureWhat software security solution(s) (such as static/dynamic source code analysis, fuzzing, bill of materials, etc.) do you use?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?What container security solution(s) do you use?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?Do you scan your container base images for vulnerabilities prior to using them? Continuously?Deploy & OperateWhat configuration management solution(s) do you use (such as Helm or Kubernetes Operators for Kubernetes or Chef, Puppet, Ansible for VMs)?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?When you deploy a new release, do you use the blue/green or canary deployment concept to test it out and allow for easy revert?Monitor / Continuous MonitoringDo you properly centralize logs to feed your continuous monitoring solution(s)?Do you also collect instrumentation/telemetry logs from your own code?Do you use the SCSS EFK capability for this monitoring?Do you keep track of your software/hardware/Cloud assets? Are you using a specific solution?Do you have best practices defined to select what logs should be collected?What kind of software logs are you collecting?Do you monitor access to the CI/CD tools with automated alerts for privilege access and changes?What continuous monitoring solution(s) do you use (such as Nagios, Twistlock, Elk, Splunk, Prometheus, New Relic etc.)?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?What alerting solution(s) do you use?Do you use SSO/single identity to access it with the same identity across all your CI/CD solutions?Do you leverage Multi-Factor Authentication (MFA) to access it?Do you enforce the principles of “Need to know” and “Least privilege” on this solution?Do you leverage behavior detection in your continuous monitoring?Do you (continuously) scan your container registry? How often?Metrics / MaturityAre you a DoD Enterprise DevSecOps Initiative Pathfinder?Are you using DoD Enterprise DevSecOps Hardened Containers? How many?Are you tracking your maturity/progression in DevOps?If so, which metrics are you leveraging?Please provide the following:Mean-time to recovery: shows how long it would take applications in the production stage to recover from failureMean-time to production: show how long it takes when new code is committed into code repositoryAverage lead-time: shows how long it would take for a new requirement to be delivered and deployedDeployment speed: shows how fast you can deploy a new version of the application between staging, test and productionDeployment frequency: shows how often you can deploy a new release into production environment and testing / QA.Production failure rate: shows how often software fails during productionTime to detect failure: how well is the system instrumented to detect and diagnose failures ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download