Software Sustainment



Software SustainmentThe purpose of this article is to increase awareness of some of the central tenets related to software sustainment planning. This article is not comprehensive and will not create software sustainment planning Subject matter Experts (SMEs). However, it is intended to highlight some key concerns and risk areas and offer potential mitigations that PMOs may find useful to help promote effective lifecycle management. One critical, foundational concept is that when it comes to software, sustainment activities are nearly indistinguishable from development activities – they both require essentially the same environment, processes, resources, skillsets, techniques, application of best practices and test approaches. Although they have different terms, the activities and behaviors comprising the development and sustainment phases are virtually identical. To that end, software sustainment is essentially a continuation of the original development.According to the Software Engineering Institute and Carnegie Mellon University, the working definition of software sustainment is; Software sustainment involves orchestrating the processes, practices, technical resources, information, and workforce competencies for systems and software engineering, to enable systems to continue mission operations and also to be enhanced to meet evolving threat and capability needs. (Emphasis added)To help understand the relationships between terms, the definition of software maintenance is; The process of modifying a software system after delivery to correct faults, improve performance or adapt it to a changed environment (IEEE 12207, 1995).To that end, software maintenance is a subset of software sustainment. When hardware breaks, maintainers remove the faulty item and typically replace it with an identical functional item, restoring the asset to its operating functional baseline. However, when software fails it is due to a latent defect in the software code, and is rectified by developing and distributing different code to correct the defect, essentially creating a new version of the product baseline. Therefore, software maintenance drives additional considerations inherent to creating a new version (baseline), including configuration management/control, updated manuals and training, distribution process (release management), etc.Software sustainment planning and management has become increasingly important in recent years for several reasons. First, an increasing amount of functionality is allocated to software configuration items compared to hardware configuration items. For example, consider that in the F-4 Phantom in the 1960s, roughly 10% of its functionality was performed by software. However, in the modern F-35, roughly 90% of it functionality is provided by software! Additionally, as more systems enter the operation sustainment phase, more software development is being performed post-deployment than is occurring in the original development phase. According to data derived from the Federal Procurement Database System, in 2011 there was 143% more software development performed for systems undergoing maintenance and upgrades than was being performed for ‘new’ systems (). For those reasons, it is more important than ever to plan for software sustainment early in the program’s lifecycle. Additionally, the sheer volume of existing software code within the DOD is conservatively estimated at over one billion lines of code (SEI/CMU unpublished study, 2017).One of the first decisions programs face is whether they believe they will assume management responsibility for the source code for their software items. Unless programs are procuring commercial off the shelf (COTS) software they should strongly consider procuring the source code (at a minimum including it as a pre-priced option in the development contracts) if they are contracting a private developer to generate the source code. Even if unlimited rights are granted as a result of complete government funding, source code ownership may not be guaranteed if not explicitly detailed in the contract – the best practice is to ensure that government lawyers and contract officers (as well as Software Support Activities) are consulted to ensure adequate contract language is used to deliver source code. Failing to procure source code for developed (non-COTS) software will introduce significant technical and business risk throughout the operations and sustainment phase, and will result in ‘vendor lock’ with the developing organization. If programs are buying commercial software (COTS), they may be limited to purchasing licenses throughout the lifecycle for the software; in which case the developer retains responsibility for software sustainment. However, if the program believes they may want to (and have the opportunity to) assume management of the source code throughout the lifecycle, the number of additional considerations increases significantly. Additional areas that must then be considered include procuring the required data rights and assets related to replicating the development environment, which will be required to adequately sustain the software. Let’s take a deeper look at each of these areas.If a program is buying a commercially available solution (specifically COTS) and is positive they will not be assuming ownership of the source code (which is common when buying commercially available (COTS) solutions), than sustainment efforts will generally be focused in a few key areas. First, the program office must plan for the recurring acquisition of appropriate licenses to support sustained operations. Key facets to consider in this area include determining the renewal frequency and scope of the license. Many licenses require annual renewal, and due to the scheduling requirements to support budgeting and contracting activities, many of these efforts will need to be identified and initiated several months before the existing licenses expire. For National Security Systems (NSS) employing COTS solutions, the military needs an automatic license renewal process in place to ensure continuity of operations. National defense military operations cannot afford the risks associated with human-in-the-loop license renewal processes. Additionally, many licenses will have constraints regarding authorized use, such as allowing both government and contractor support usage, per-seat constraints, and so forth – all of these implications need to be understood and adhered to for licenses that are acquired. Another consideration when procuring COTS software is how the Program Management Office (PMO) will interface with the existing commercial support infrastructure (also including government-built integration code that must be designed properly). What contractor support and help desk support is available, and how will the PMO interface with that infrastructure? How will services be sought and delivered (i.e., proper performance-based Service Level Agreements (SLA))? How will performance metrics and measures be conveyed back to the PMO (who will be responsible for system performance and availability (cradle to grave)). How will software patches and upgrades be distributed and how will cybersecurity considerations (including vulnerability scans) be designed in and managed?If the PMO believes there is even the possibility of assuming ownership of the source code, several additional planning factors become relevant. This applies primarily when a commercial vendor is contracted to develop military-specific software (e.g. for a military weapon system). A ‘best practice’ for this approach would include initiating contact with a Software Support Activity (SSA) early in the acquisition process. A recent software sustainment study conducted by the Software Engineering Institute and Carnegie Mellon University in February 2017 identified over 30 DOD organic software engineering organizations, staffed by nearly 6,000 government engineers who are capable of serving as Software Support Activities.One reason to include the SSA early is to reduce the amount of ‘technical debt’ incurred during the development phase. In this context, technical debt refers to inadequate attention being attributed to software architecture and design that enables maintainability; such as clean, well-organized and documented code that is easily understood and maintained (SEI/CMU unpublished study, 2017). SSAs can help shape developmental contracts to ensure that maintainability considerations are factored into the original development contracts (pre-MS B) and reduce this aspect of technical debt. Additionally, early and continuous SSA involvement can help ensure that required architectures are developed to reduce programmatic risk and enable effective software development practices during requirements decomposition. SSA involvement can also help ensure that the correct data rights are obtained, and that assets and skillsets necessary to replicate the development environment into the production (or sustainment) environment that they will manage are identified in the development contract to ensure competitive pricing. Generally, SSA involvement early in the acquisition process can reduce risk by ensuring that deliverables that enable effective sustainment are present in the original development contract and are in the proper format to enable efficient and effective sustainment operations. Finally, SSA involvement at this stage can also establish them as participants and collaborators in the ultimate software development effort, increasing their familiarity with the code and enhancing their ability to assume sustainment responsibilities. SSA participation early in the process will likely increase quality, and improve reliability, maintainability, and sustainability. Even if the government PMO is not sure if they will be assuming management responsibility for the source code, SSA support and involvement can ensure the required contractual language is identified as ‘unfunded options’ in the development contract to capitalize on competitive pricing. Thus far, this article has discussed how SSA involvement early in the process can help position the PMO for success by identifying the right information and assets that will be required to enable efficient and effective lifecycle software sustainment. However, there are also many services that PMOs need to plan for throughout the lifecycle that the SSA may be able to perform, known as Post Deployment Software Support (PDSS) or Post-Production Software Support (PPSS). It is important to note that best practices, tools, techniques, artifacts and activities relevant in a development environment are also relevant (and equally required) in a sustainment environment. These areas include:Source code management – if the PMO is leveraging an SSA to manage the source code, they will be able to identify and respond to product deficiencies by developing and distributing new source code through packaged releases. In an agile environment, this becomes even more critical; as the PMO guides the development to the actual customer requirement via many field-able releases, the SSA must be kept informed as to what changed from release to release.Cybersecurity support – some SSAs have the capability to support cybersecurity activities by conducting vulnerability scans and promulgating code improvements to enhance security. This includes providing the Program’s Cybersecurity Strategy to the SSA.Help desk support – SSAs may have the capability to develop and manage tiered help desk functionality to enable cost-effective software sustainment activities.Development environment changes – Software tools are constantly changing. The PMO needs to ensure there is a process in place that includes the SSA when there are critical software tool changes made.Design Documents – Software Design Document (SDD), Interface Control Documents (ICD), and other software architecture documents describe the capability being fielded. The SSA needs these documents as reference materials during software sustainment.Software Test Report (STR) - It is a best practice to test “Mission Critical Threads (MCT).” You can’t test everything but you must test what the war-fighter will be executing. These MCT test results will help the SSAs understand the history and risks of each MCT that they will be supporting. In an agile environment, this becomes even more critical; as the PMO guides the development to the actual customer requirement via many field-able releases, the SSA must be kept informed as to the test results from release to release.For an SSA to assume software support authority, there are several activities that must occur. It must be recognized that SSAs will likely have software development responsibilities and must have the resources and information that would enable those activities throughout sustainment. To that end, it is considered a ‘best practice’ that the SSA would be identified early so they could be a partner in the original development effort to ensure sustainability and increase understanding of the source code.Similar to other Depot Source of Repair (DSOR) practices, a Business Case Analysis (BCA) should be considered prior to MS B to identify potential SSA sources that can advise on initial development and potentially support sustainment roles. Once a preferred SSA is identified a Memorandum of Agreement/Understanding (MOA or MOU) or Service Level Agreement (SLA) should be drafted that articulates their role, authority and scope, as well as establishing performance measures for each of their expected contributions. As the PMO progresses through development and approaches deployment, there are several products the receiving SSA may require. These include (Lapham & Woody, 2006):Source of Repair Assignment Process (SORAP) – this is essentially the BCA that was completed to identify the candidate SSA based on their skillset, capacity, and ability to provide PDSS/PPSS to the PMO.Test & Evaluation – demonstrated success that the software performs in accordance with its requirements.Stable Software Baseline – the software baseline is established and the source code is documented with the appropriate plete Documentation – This includes the source code documentation that enables maintainability (such as designer notes) as well as related operator and maintenance documentation.Authority to Operate (ATO) – granted by the Authorizing Official (AO) prior to deploymentSoftware Transition Plan (STP) – This plan documents how the software will be transitioned from the development environment (OEM) to the production/sustainment environment (SSA). For software, the production environment should be essentially the same as the development environment, so this document will contain detailed information about the development environment, including processes, test scripts, required skill sets, etc. The STP can be jointly developed between the OEM and the SSA with PMO oversight if the SSA is brought into the team early in the development cycle. NOTE: Many organizations are transitioning to an Agile Software Development environment. This transition must include the SSA software development environment.Staffing and Training Plan (MER/MP&TP) – This requirement may be met with a comprehensive Manpower Effectiveness Report (MER) and Manpower and Training Plan (MP&TP), if they considered software sustainment skillsets and requirements in the analysis. This is another area that an SSA can help inform during development if they are included early in the process. Needless to say, there are many very important reasons to identify candidate SSAs, and to include them on the PMO team as early as possible in the development effort!Once software is deployed, it will often require changes and upgrades to remain viable throughout its lifecycle. Software fails differently than hardware. Software does not ‘wear out’ – it suffers from latent defects that exist in the original code and emerge under certain circumstances post-deployment. Comprehensive testing and responsible and diligent coding practices can reduce the amount of latent defects – however, there is no such thing as ‘error-free’ software. Therefore, all software will require some form of modification post-deployment. Generally, there are 10 reasons that software can require changes post-deployment (expanded from NAVAIR Software Logistics Primer v2.0, 2010), including;Defect corrections -Fixing errors in design and coding Threats -Modifying the software to respond to new threats; new Cybersecurity functionality Policy and doctrine -Modifying the software as a result of changes to policy and doctrine Safety -Modifying the software to improve safety Interoperability -Modifying interfaces to stay interoperable with associated previously existing or new systems Hardware changes -Modifying software to interface or work with hardware upgrades Technology insertion -Improving software to incorporate new technologies Functional changes (user request) –Modifying the functionality or adding new functionality of the software based on user requests; this includes cybersecurity functional enhancements.Performance Enhancements (no user request) – Using more efficient coding algorithms that operate faster; functionality does not change but the operational speed does.Refactoring due to excessive complexity – Cleaning up the code/code optimization for better software sustainability. This is usually done as a result of a static code analysis.These ten areas align with the four types of maintenance; corrective, adaptive, perfective, and preventative.Corrective Maintenance; Reactive modification to correct discovered problems (1)Adaptive Maintenance; Modification to maintain usability in a changed environment (2, 3, 4, 5, 6, 7, 8) Perfective Maintenance; Provide functional enhancement to users (8, 9)Preventative Maintenance; Enhance maintainability (10)Surprisingly, corrective maintenance (fixing ‘bugs’) accounts for roughly 20% of all required software modifications (Grubb & Takang, 2003). The most common reason for software maintenance is actually perfective maintenance – providing functional enhancements based on user feedback.When software changes are required (the need for change having been approved through the appropriate process), they typically follow the below steps;Understanding Requirement – defining and understanding the required change; identifying derived requirements required to implement required changeUnderstanding Existing Code – reviewing existing documentation and tracing logicImplementing Change – Adjusting software architecture, identifying software items (SI) and software components (CSC), prioritizing the CSCs in the Product Backlog and identifying software release packages (a number of CSCs) and building release(s) as neededCheckout & DeploymentIf the code is well documented and understandable, the maintenance process is significantly easier. In those cases, the allocation of effort in each of the stages is roughly (Mosemann & June, 1996);Understanding Requirement – 15%Understanding Existing Code – 30%Implementing Change – 50%Checkout & Deployment – 5%However, if the source code is poorly documented and difficult to understand, it takes significantly greater effort (both cost and schedule) to understand the existing code (Lapham & Woody, 2006). In these cases, the effort may follow the below distribution depending on the extent of the documentation deficiency;Understanding Requirement – 10%Understanding Existing Code – 50%Implementing Change – 37%Checkout & Deployment – 3%Due to the additional effort required to understand the code, there will be some adverse effects. If cost and schedule are held constant, the result will likely be reductions in testing and debugging and subsequent documentation updates. This will result in increasing software failures experienced by the customer post-delivery (fewer bugs detected and corrected in the testing/debugging phase) and increasing time requirements for ‘understanding existing code’ in future maintenance cycles. Conversely, if performance is held constant instead of cost/schedule, poorly documented code can increase cost and schedule requirements for software maintenance by 20-30%, depending again on the extent of the documentation deficiency. It is truly a ‘pay me now or pay me later’ scenario when it comes to software documentation during the development phase – which may be an increased risk to be aware of if an Agile software development approach is being used. Agile requires that the documentation be just enough for the sustaining organization to use; it is difficult to define what “just enough is”; documentation to enhance understandability should be developed as part of the effort and retained. Please see: ConclusionThis article is not a comprehensive accounting of all facets related to software sustainment planning and management. A complete conversation in that area would greatly exceed the scope of this paper. However, this paper is intended to highlight some of the critical planning factors related to increasing software sustainment efficacy. Key points included identifying the PMOs potential sustainment strategy for software early in the effort (certainly pre-MS B); including Software Support Activities in PMO planning early in the process so they can help influence contract language, collaborate in the software development effort and ensuring the right deliverables (and data rights) are considered in early production contracts; and highlighting the importance of creating and maintaining understandable source code (through effective documentation and responsible coding practices) so that cost-effective sustainment activities can occur throughout its lifecycle.There are several resources available that can provide additional information. Crosstalk – The Journal of Defense Software Engineering.CrossTalk, The Journal of Defense Software Engineering, is an approved U.S. Department of Defense journal. CrossTalk's mission is to encourage the engineering development of software in order to improve the reliability, sustainability, and responsiveness of our warfighting capability and to inform and educate readers on up-to-date policy decisions and new software engineering technologies. Acquisition Community Connection (ACC) Community of Practice (COP)An Information Technology COP is active, and a subordinate IT Maintenance/Software Sustainment COP is being populated. Contains many useful links and information resources.(SW)-Sustainment-.aspxReferences:Grubb, P., & Takang, A. A. (2003). Software maintenance: concepts and practice. World Scientific.ISO, I. (1995). IEC 12207 Information technology-Software life cycle processes. International Standards Organization: Geneva, Switzerland.Lapham, M. A., & Woody, C. (2006). Sustaining Software-Intensive Systems (No. CMU/SEI-2006-TN-0007). CARNEGIE-MELLON UNIV PITTSBURGH PA SOFTWARE ENGINEERING INST.Mosemann, L. K., & June, I. I. (1996). Guidelines for Successful Acquisition and Management of Software Intensive Systems.Related Training:CLL056; Sustainment of Software Intensive SystemsISA101; Basic Information Systems AcquisitionISA201; Intermediate Information Systems AcquisitionISA301; Advanced Enterprise Information Systems AcquisitionISA320; Advanced Program Information Systems AcquisitionLOG211; Supportability Analysis ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download