Measures of Effectiveness for Supportability - DAU



Measures of Effectiveness for SupportabilityContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 1. Lesson 3: Measures of Effectiveness for Supportability Welcome to Lesson 3: Measures of Effectiveness for Supportability.IntroductionContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 2. Topic 1: Introduction Content870005200356Technology Maturation & Risk Reduction00Technology Maturation & Risk ReductionSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 3. Life Cycle Management Framework: Where Are You? What Influence Do You Have? Measures of Effectiveness (MOEs) are meaningful measures defined by user needs. They facilitate and sustain the right system outcomes over the life cycle. MOEs become metrics as they are refined and become measurable. This lesson illustrates the MOE refinement process and its relevance in defining the Technical Performance Measure (TPM) - it enables Systems Engineering feedback to assure design meets the user needs and the support system and design are in balance.MOEs span the entire life cycle to:Ensure the system design meets requirements prior to fielding Pre-Milestone C Ensure the support system is adequate to sustain the system design through fielding Post-Milestone CAs a program matures, so do the MOEs.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 4. MOE Life Cycle Role MOEs are an integral part of the life cycle. They are the foundation upon which a system is designed and by which the system evolves. MOEs ensure a system synchronizes user requirements from design through Sustainment. The Closed-loop Systems Engineering process provides an overarching framework that is only as good as the MOEs that support the product, program, and its processes. This process ensures disposition is considered during system design.Sustainment objectives are set in the Materiel Solution Analysis (MSA) phase of the program’s Life Cycle, and then transformed into outcomes through compliance with the Key Performance Parameter (KPP)/Key Sustainment Attributes (KSAs) and TPMs. The MOEs assure program and product compliance with user needs as key features of major program documents. MOEs enable system Sustainment as criteria for agreements, surveillance, and execution of the Life-Cycle Sustainment Plan (LCSP).ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 5. Affordable System Operational Effectiveness Model The Affordable System Operational Effectiveness (ASOE) Model illustrates the relevance of Sustainment MOEs at a system level. Operational effectiveness requires balance between Mission Effectiveness (i.e., can the system do it) and Life Cycle Cost (i.e., can the program afford it) and Process Efficiency (i.e., are the processes responsive to the equipment and user needs). ASOE is achieved through influencing early design and architecture and focusing on Supportability outputs. MOEs enable testing feedback to assess whether design meets user needs. Additionally, MOEs link user needs to design, design to support, and enable trade-offs to achieve the best balance of capability for acquisition and support costs. This traceable path to user needs is supported through TPMs.Content699102857344SAE GEIA-STD-0007020000SAE GEIA-STD-00073716835333878Technology Maturation & Risk Reduction00Technology Maturation & Risk Reduction1370330108889Technology Maturation and Risk Reduction Phase00Technology Maturation and Risk Reduction PhaseSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 6. Technology Maturation and Risk Reduction (TMRR) Phase The previous lesson introduced the Materiel Solution Analysis phase; the Technology Maturation and Risk Reduction (TMRR) phase begins at Milestone A when the Analysis of Alternatives (AoA) is complete and the Acquisition Strategy is approved. The goal of the Technology Maturation and Risk Reduction (TMRR) Phase is to determine the risk associated with proposed technology and to design and test prototypes that reflect the technologies that enable system performance. Although a specific physical platform solution is not yet identified, there are several important Design and Sustainment activities conducted.The Technology Maturation and Risk Reduction (TMRR) phase is complete when the following occur:An affordable increment of capability is identified.The technology is demonstrated in a relevant environment.A system is able to be developed for production within a short timeframe (i.e., less than five years).A preliminary Maintenance Task Analysis and Level of Repair Analysis are accomplished.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 7. Technology Maturation and Risk Reduction (TMRR) Phase: Milestone B Activities Recall that the Materiel Solution Analysis phase ended with initializing the Logistics Product Database. Important Operations and Maintenance goal data were entered into Table A of SAE GEIA-STD-0007 Logistics Product Database and validated by the LCL. In the Technology Maturation and Risk Reduction (TMRR) phase, the LCL performs several analyses with inputs to the database; they include:Comparative AnalysisMaintenance AnalysisLevel of Repair AnalysisThe Technology Maturation and Risk Reduction (TMRR) phase has Entry and Exit documentation and corresponding activities that incorporate or address Supportability and logistics considerations. Entry Documents are completed when the phase is initiated (Milestone A) and Exit Documents/Activities are completed or updated during the phase, prior to exit (i.e., Milestone B).Entry Documents include the following:Systems Engineering Plan (SEP) with the preliminary Reliability, Availability, Maintainability, and Cost (RAM-C) Rationale Report AoAMarket Analysis / Consideration of Technology Issues Test and Evaluation StrategyExit Documents/Activities include, but are not limited to the following:Updated SEP with the updated Reliability, Availability, Maintainability, and Cost (RAM-C) Rationale Report AoATechnology Readiness Level (TRL) AssessmentCapability Development Document (CDD)Test and Evaluation Master Plan (TEMP)LCSPProject Management Plan (PMP)Content Slide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 8. MOE Lesson Approach This lesson is structured in three phases: Set Up, Analyze, and Report Findings. Each phase applies MOE fundamentals to programs and processes and illustrates the importance of feedback in measuring success.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 9. Topics and Objectives Understand Measures of EffectivenessContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 10. Topic 2: Understand Measures of Effectiveness This topic provides a general overview of MOEs, their definitions, and classifications:Key Performance Parameter (KPP)Key System Attribute (KSA)Measure of Performance (MOP)Measure of Suitability (MOS)Technical Performance Measure (TPM)It is important to keep in mind, that as MOEs are refined across the Life Cycle, there are different ways to which they are referred (e.g., TPMs, metrics, etc.).ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 11. MOE Set Up The first step in the Set Up phase is to classify MOEs into categories. ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 12. Set Up – Classify MOEs into Categories: MOE Before classifying MOEs into categories, it is important to have a clear understanding of key terminology and how these concepts relate to each other. These concepts are integral to the entire Life Cycle; consequently, they appear throughout this course.MOEs are:An expression of the minimum level of proposed capability requirementsA tool used to measure results achieved in the overall mission and execution of assigned tasksThe data used to measure the military effect (mission accomplishment) that comes from the use of the system in its expected environmentA prerequisite to performance measurement activitiesContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 13. Set Up – Classify MOEs into Categories: KPPKey Performance Parameters (KPPs) are performance attributes of a system considered critical to developing an effective military capability. A system’s failure to meet a validated KPP threshold (i.e., minimum acceptable performance), brings the military utility of the associated system(s) into question, and may result in a re-evaluation of the program or modification to production increments. A KPP is usually comprised of KSAs, which contribute to the overall performance required for the KPP.Key System Attributes (KSAs) are attributes or characteristics considered essential to achieving a balanced solution or approach to a system, but not critical enough to be a KPP.The Sustainment KPP, Availability, is derived from the operational capability requirements of the system, assumptions for use, and the planned Sustainment strategy. The Sustainment KPP has two Key System Attributes and a Metric that provide an integrated structure that balances Sustainment with capability and Affordability across a system’s life cycle. The KSAs and Metric are listed in this slide and described on the following page.Availability KPP – Is defined in terms of two perspectives:Materiel Availability (AM) – Is the measure of the percentage of the total inventory of a system operationally capable (i.e., ready for tasking) of performing an assigned mission at a given time, based on materiel condition (fleet-wide Availability)Can be expressed as the number of operationally available end items divided by the total population of itemsOperational Availability (AO) – Indicates the percentage of time that a system or group of systems within a unit are operationally capable of performing an assigned mission (operational unit Availability) and can be expressed mathematically in two ways:Uptime/(uptime + downtime)) where uptime is the time the system is available to perform designated mission and downtime (total time – uptime) is the time the system is unavailable for taskingAs ((MTBM)/(MTBM + MDT)), where MTBM is Mean Time between Maintenance and MDT is Mean Down TimeKey System Attributes – two supporting attributes are defined:Reliability – Measures the probability that the system will perform without failure over a specified interval under specific conditionsIs usually referred to as Mean Time between Failure (MTBF) which is expressed mathematically as (total operating hours/total number of failures)Operation and Support Cost (O&S Cost) – Provides balance to the Sustainment solution by ensuring that Availability Costs (e.g., maintenance, spares, fuel, support) are considered in making program decisionsSustainment MetricMean Down Time (MDT) is not a KSA, but the Defense Acquisition Guidebook (DAG) categorizes it as a Sustainment metric. MDT is the average time an end item is unavailable to perform its assigned mission after it experiences unscheduled or scheduled maintenance actions. MDT is expressed as (total down time for all failures/total number of failures) and encompasses the following:The time from reporting an asset being down to the asset returning to operation/productionThe administrative time of reporting, logistics, and materials procurement and lock-out/tag-out of equipment for repair or preventive maintenanceNote: See Appendix E to Enclosure B of the revised JCIDS Manual dated January 19, 2012 () for additional information about the Sustainment KPP and supporting KSAs.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 14. Set Up – Classify MOEs into Categories: MOP, MOS, and TPMMeasure of Performance (MOP) is expressed as speed, payload, range, time on station, frequency, or other distinctly quantifiable performance features (i.e., how to design to perform a mission). MOPs are primarily the focus of System Engineers. Note that several MOPs may be related to a specific MOE or metric.Measure of Suitability (MOS) is the measure of a component’s ability to be supported in its intended environment. MOS typically relates to readiness or Operational Availability (AO), Reliability, Maintainability and the component’s support structure (i.e., how to design to perform a mission AND be supportable). Several MOSs may be related to a specific MOE or metric. MOSs are primarily the focus of LCLs. MOS examples include, but are not limited to, the following:ReliabilityMaintainabilityAccessibilityModularityTestabilityTechnical Performance Measures (TPMs), also referred to as metrics, measure the program uses to monitor the progress of the design in relationship to Supportability (e.g., Mean Time between Failure (MTBF) and Mean Time to Repair (MTTR). TPMs assess design and risk, validate requirements, and test KPPs/KSAs. Both System Engineers and LCLs share a joint focus on TPMs.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 15. Set Up – Classify MOEs into Categories: TPMTPMs are valuable tools for measuring a program’s progress and ensuring the program meets design (i.e., user) requirements. TPMs provide greater value by including limits (upper and lower boundaries). MOEs having a time dimension (i.e., measuring values over time) will vary from the target mean value. The mean value is the average between the upper and lower limits. How much the parameter varies from the target mean and which way the line is trending are the important parts of charting the TPM. The TPM trend line becomes important information during design reviews. It informs management about where to place emphasis on design requirements.At the beginning of a program, the difference between the mean value and the plotted TPM data points are likely to be large (i.e., of significant variance) when (a) design issues are investigated and resolved; (b) actual testing is not performed; and (c) the program is operating on estimated values. This uncertainty is represented by a wide span between limits. Toward program completion, there is more certainty in the design meeting its performance objectives (i.e., TPM targets) because actual testing is conducted and measured. This greater certainty in achieving the objective is represented by narrowing the span between the limits.As this slide illustrates, at SFR, the MTBF TPM is within the upper and lower limits allowed but below its planned progression (i.e., variance).Supportability reviews conducted by Life Cycle Management phase use TPMs to determine progress and emphasis, and to evaluate the maturity of design and the Product Support Strategy:Materiel Solution Analysis Phase (MSA)Alternative Systems Review (ASR)Technology Maturation and Risk Reduction (TMRR) Phase (TD)System Functional Review (SFR)Preliminary Design Review (PDR)Engineering & Manufacturing Development Phase (EMD)Critical Design Review (CDR)Developmental Test and Evaluation (DT&E)Functional Configuration Audit (FCA)Production Readiness Review (PRR)Production & Deployment Phase (PD)Initial Operational Test and Evaluation (IOT&E)Physical Configuration Audit (PCA)ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 16. Set Up – Classify MOEs into Categories: Inputs and Outputs This slide provides a high-level view of the inputs, process, and outputs pertaining to MOEs. Inputs:The AoA compares the Operational Effectiveness, Suitability, and Life Cycle Cost of alternatives that satisfy established capability needs. User needs are captured in the Initial Capabilities Document (ICD) which becomes the Draft CDD at Milestone A. Throughout a program’s life cycle, there are checkpoints at each milestone for ensuring user needs are being met. TPMs measure the program’s success of meeting the users’ needs and are the language that is used to visualize and discuss the program goals. Process: The LCL uses these MOE inputs to analyze the Sustainment TPM candidates and their attributes, and then to establish appropriate evaluation criteria.Outputs:An IPT meeting to discuss the findingsRefined TPMsRefined Logistics Product DataNew/Refined requirements and program documentsContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 17. Set Up – Classify MOEs into Categories: Attributes for Good Measures/Metrics For a measure to be effective, it must be meaningful and actionable to the decision maker. The data must have context and be customer oriented, relate to a product or service, link to the process generating that product or service, and support one or more organizational or program objectives.MOE requirements are usually based on needs and, therefore, come with specific context – instructions on the format and criteria/priority of important outcomes. Many MOEs (e.g., those relating to cost, schedule, and performance) may be used throughout a program’s life cycle, while others may be tied to only one portion of a program.Good measures/metrics have the following attributes:Express how well organizational goals and objectives are being met through processes and tasksAre simple, understandable, logical, and repeatableEnable timely collection, analysis, and transformation for reportingProvide insight for actionable outcomesNow that MOEs are defined, turn your attention to categorizing them.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 18. Set Up – Classify MOEs into Categories MOEs are classified into three categories:Product DevelopmentProgram ManagementProcess ManagementEach category is associated with defining artifacts, planning documents, and agreements. ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 19. Set Up – Classify MOEs into Categories: Product Development The most important product oriented MOEs map to KPPs/KSAs. These MOEs are elevated to criteria for measuring success. Product Development MOEs are used to:Measure the program's design capability to meet a specific mission scenario Indicate whether or not the desired technical performance is achievable given program's budget, scope , and schedule constraintsFor development programs, product performance (i.e., MOP) and Suitability (MOS) measures are documented in the CDD as objectives (i.e., performance the customer really wants) and thresholds (i.e., absolute minimum performance the customer will accept) and in the Test and Evaluation Master Plan (TEMP) as critical technical parameters. Note: Objectives and thresholds are addressed in Topic 3: Select Measures of Effectiveness for Supportability.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 20. Set Up – Classify MOEs into Categories: Program Management Program Management metrics are used to do the following:Monitor the health and progress of program activitiesServe as indicators for adverse trendsThese MOEs are the Program Manager’s trip wire to bring early detection of adverse trends that can be reversed if detected early. As such, the Program Manager establishes control limits to indicate level of action needed: No action required – continue to monitorMove from surveillance to active engagementReassign tasks / resources and shift priorities /activity sequence to bring program back on trackEarned Value is a powerful Program Management tool - it enables early detection of cost and schedule variance which are required for Major Defense Acquisition Programs meeting certain thresholds specified in Interim DoDI 5000.2 – Operation of the Defense Acquisition System, November 25, 2013, Enclosure 1, Acquisition Program Categories And Compliance Requirements, Table 8, EVM Reporting Requirements, page 71.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 21. Set Up – Classify MOEs into Categories: Process Management Process Management MOEs are used to do the following:Assess the quality and productivity of a program’s processes. A process must be defined, understood and measured in order to improve it.Collect data at specific milestones or checkpoints in the process and analyze them.Predict quality at later stages in the pare current or predicted performance to performance objective targets.Process MOEs are important not only to the Integrated Product Teams (IPTs) measuring them, but also to the functional areas (e.g., budgeting, contracting, or testing) that own the processes being measured.Now turn your attention to MOE alignment to the life cycle milestones.Content4001087609600Technology Maturation & Risk Reduction00Technology Maturation & Risk ReductionSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 22. Set Up – Align MOEs to Milestones: The Life Cycle and ArtifactsThe second step in the Set Up phase involves aligning MOEs to Life Cycle Milestones. As a system matures through its Life Cycle, the JCIDS process reviews the progress of the requirements in meeting user needs in terms of both achievement and relevance. As this diagram illustrates, there are three overarching activities that relate MOEs to program phases and artifacts: Understand the user needs as documented in the ICD.Measure whether the program is meeting the user needs using CDD/Capability Production Document (CPD)/KPPs/KSAs/TPMs.Monitor the performance of metrics in the CDD.The program creates and maintains documentation in parallel with JCIDS requirements:At Milestone A, the Program Management Plan (PMP), the RAM-C Rationale Report, the Systems Engineering Plan (SEP), and a high-level draft of the LCSP are created.The PMP manages and tracks program progress and schedule.RAM-C Rationale Report monitors the program’s technical performance through the Sustainment KPP/KSAs.The SEP is a living document that is periodically updated to capture the program's current status. The SEP:Tracks its evolving systems engineering implementation progressIncludes the processes for achieving the required Sustainment performance Provides contractor reporting requirementsDocuments the process (e.g., modeling, simulation, etc.) for deriving the AM metricThe LCSP is a living document which monitors the Sustainment of a system across its life cycle; at this point, it contains essential Product Support Strategy elements, to include the Concept of Operations (CONOPs).At Milestone B, the PMP, RAM-C Rationale report, SEP, and LCSP are updated and the TEMP is created.Generally speaking, the LCSP serves as the Sustainment PMP and contracting strategy.The TEMP is a top-level master planning document that describes testing required to verify the capability being developed during the EMD phase.Also, the TEMP focuses on the overall structure, major elements, and objectives of the T&E program and must be consistent with the Acquisition Strategy, approved CDD or CPD, System Threat Assessment, Information Support Plan, and SEP.At Milestone C, the PMP, RAM-C Rationale Report, SEP, LCSP, and TEMP are updated.Post-Milestone C, the SEP, LCSP, and TEMP are updated throughout Sustainment.Content490555351131Technology Maturation & Risk Reduction00Technology Maturation & Risk ReductionSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 23. Set Up – Align MOEs to Milestones: MOE Maturity and the Life Cycle MOEs progress into TPMs, which are continuously refined and tested and evaluated (T&E). Periodic Supportability reviews ensure the TPMs are on track:Preliminary Design Review:Evaluates the subsystem requirements via TPMs to determine whether they correctly implement all allocated system requirementsReviews Supportability requirements/MOEs and the Product Support Strategy contained in the LCSP are consistent with the evolving designCritical Design Review:Results in an initial product baseline for the system, hardware, software, Maintainability, Supportability, and the Integrated Product Support Elements, including support equipment, training systems, and technical dataEnsures Supportability requirements and the Product Support Strategy contained in the LCSP are consistent with the product baseline and the projected Sustainment metrics (e.g., Reliability and Maintainability) and other Supportability featuresDT&E is an engineering-focused test which demonstrates functionality and provides data on the achievability of KPPs and KSAs.IOT&E is an independent operational test and evaluation of a system’s Operational Effectiveness and Suitability before entering Full-rate Production (FRP).Note: Table 2-1 Sustainment Requirements and Measures by Phase in the Reliability, Availability, Maintainability, and Cost Rationale Report Manual provides additional information.You have just completed Topic 2: Understand Measures of Effectiveness. Next, you will learn about selecting MOEs for Supportability.Select Measures of Effectiveness for SupportabilityContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 24. Topic 3: Select Measures of Effectiveness for Supportability As the program progresses through its acquisition life cycle, MOEs become more refined (e.g., TPMs/metrics) and designed for traceability to the original user needs.This topic discusses TPM candidates and attributes in the context of the three MOE categories described earlier:Product DevelopmentProgram ManagementProcess ManagementRecall that the program manager and lead LCL use TPMs to monitor the progress of the design in relationship to Supportability and to ensure the system is in conformance with user needs and defined MOEs.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 25. MOE Analyze – TPM Candidates and AttributesThe first step in the Analyze phase is to identify the best Sustainment TPM candidates. Designating TPM candidates include the following considerations:MOPsPhysical size and stability (e.g., useful life, weight, volumetric capacity)Functional correctness (e.g., accuracy and power performance)MOSsSupportability, Maintainability, dependability, and Reliability (i.e., Mean Time To Failure (MTTF))Both MOPs and MOSsEfficiency (e.g., utilization, response time, throughput)Suitability for purpose (i.e., readiness)For Supportability considerations, TPMs provide program-level visibility on the progress of design interface with sustaining engineering requirements. Progress is measured by comparing planned to actual values. This difference in value is called variance, and the TPM variance impacts serve as inputs for the PM to balance cost, schedule, performance, and risk.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 26. Analyze – TPM Candidates & Attributes: ASOE Model and TPMs TPMs are the means (measuring stick) that enable trade-offs among competing priorities to accomplish the best outcome possible. This slide translates the ASOE Model into a table and indicates the role TPMs play in balancing the performance and Sustainment objectives. Subsequent slides use this table to illustrate the relationship of Mission Effectiveness and Suitability to TPM candidates.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 27. Analyze – TPM Candidates: ASOE and Product DevelopmentThe ASOE Model is distilled into two tables:The top table illustrates the ASOE model’s primary intersections by assigning a number to each and listing its related MOEs. The bottom table relates an MOE capability (i.e., user need) to an ASOE concept/description (i.e., KPP/KSA or higher order metric) and then provides a corresponding TPM (i.e., lower order metric). This example illustrates the Product Development MOE category by diagramming user need to specific TPM candidates. This lesson highlights four Product Development MOE capabilities:Be on station for ‘n’ hour duration relative to the ASOE Supportability conceptRespond to operational tasking within ‘n’ hours relative to the ASOE Process Efficiency conceptWeigh no more than ‘n’ pounds relative to the ASOE Supportability conceptCarry and deploy payload within weight and loiter parameters relative to the ASOE Supportability conceptNote: The first two are highlighted on this slide and the remaining two Product Development MOEs are addressed on a subsequent page.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 28. Analyze – TPM Candidates: Product Development and Logistics Product DataTPMs are part of the SAE GEIA-STD-0007 Logistics Product Database. Logistics Product Data are data elements used when assessing support requirements. The SAE GEIA-STD-0007 specifies how Logistics Product Data relates to support requirements by linking TPMs to levels of indenture for the system product structure. As this slide illustrates, the first TPM candidate applicable to Product Development corresponds to the third level of indenture in the Strike Talon UAV Hierarchy. Note: Lesson 4: Logistics Product Data/SAE GEIA-STD-0007 Database addresses how to generate Logistics Product Data.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 29. Analyze – TPM Candidates: ASOE and Product Development (cont.)This slide highlights the two additional Product Development MOE capabilities:Weigh no more than ‘n’ pounds relative to the ASOE Supportability conceptCarry and deploy payload within weight and loiter parameters relative to the ASOE Supportability conceptTPMs can be specific to program milestone events, as is the case here. The following list defines the program event terminology associated with the TPM timeframe:Preliminary Design Review (PDR)Critical Design Review (CDR)Initial Operational Test & Evaluation (IOT&E)ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 30. Analyze – TPM Candidates: ASOE and Program ManagementProgram Management is the second MOE category examined to form TPM candidates. This lesson highlights two Program Management MOE capabilities:Deliver Supportability analyses by “xQFYyy” relative to the ASOE Process Efficiency conceptDeploy ‘n’ prototypes within budget relative to the Process Efficiency/Life Cycle Cost conceptsNote: xQFYyy designates the applicable Quarter and Fiscal Year (e.g., 1QFY12 is first Quarter of Fiscal Year 2012).ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 31. Analyze – TPM Candidates: ASOE and Process ManagementThis example highlights the Process Management MOE category by determining TPM candidates supporting two capabilities. This lesson highlights the following two Process Management MOE capabilities:Deploy ‘n’ prototypes with ≤ ‘m’ requirement changes relative to the ASOE Process Efficiency/Life Cycle Cost conceptsDeploy ‘n’ full rate production units with ≤ ‘m’ defects/unit relative to the ASOE Process Efficiency conceptNext, turn your attention to analyzing TPM attributes.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 32. Analyze – Sustainment TPM Attributes: Thresholds and Objectives After the TPM candidates are identified, analyze achievable outcomes within Sustainment TPM constraints/attributes. Attributes may include, but are not limited to, the following:Achieved to date – measured technical progress or progress estimateCurrent estimate – value of a technical parameter that is predicted to be achievedMilestone – a point in time when the evaluation of a measure is accomplishedPlanned value – predicted value of the technical parameterPlanned performance profile – profile representing the project time phased demonstration of a technical parameterTolerance band – management alert limitsThreshold – the limiting acceptable value of a technical parameterVariances – demonstrated technical variance, predicted technical varianceThe RAM-C Rationale Report and CDD include KPPs and KSAs expressed using threshold and objective values. These values drive the most important mission requirements into the design and have the following attributes to better enable program decision making:QuantifiableTestableAchievableTPM threshold and objective are defined as:Threshold is the minimum acceptable value considered achievable within the available cost, schedule, and technology at low-to-moderate risk. Performance below this value is not operationally effective or suitable or may not provide an improvement over current capabilities.Objective is a higher level of performance representing significant increase in operational utility. It is the desired operational goal achievable but at a higher risk in cost, schedule, and test resources. Performance above the objective is desirable but should not equate to additional cost.The graph on this slide uses MTBF (measured in hours) to illustrate the relationship between objective and threshold. It is preferable that the MTBF objective be a higher number than threshold since the goal is to have as much time lapse between failures as possible. Objective and threshold do not always have this type of relationship. If the TPM in question pertains to weight or defect rate, a lower number is preferable for the objective since the goal is to have as low a weight or defect rate as possible.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 33. Analyze – Sustainment TPM Attributes: Thresholds and Objectives (cont.)This graphic illustrates the relationship between Reliability, Sustainment, and operational effectiveness. The difference between the threshold and objective values sets the trade space for meeting the thresholds of multiple KPPs/KSAs. This graphic emphasizes the need for balance between Reliability, Sustainment Cycle Time, and Life Cycle Cost.Achieving the threshold and objective targets is facilitated by conducting trade-off analyses. Using available trade space, the Program Manager determines the most effective path to a successful program outcome. In this example, the difference between threshold and objective represents the trade space in which design Reliability (i.e., performance), Life Cycle Cost (i.e., Affordability), and Sustainment Cycle Time (i.e., efficiency) are analyzed to achieve the optimum solution.A trade study is the analysis and identification of potential trade-offs within a trade space. The Program Manager makes trade-off decisions throughout the development process and assures they are three-dimensional, rather than limited solely to design decisions. The three dimensions are Design, Sustainment, and Cost Requirements. A successful program is defined as having these trade-off dimensions in balance.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 34. Analyze – Sustainment TPM Attributes: TPM Promotion to Program Documents This step also involves promoting Product TPMs to the appropriate requirement and program documents (e.g., CDD, CPD, SEP, TEMP, LCSP) depending upon the Life Cycle phase. In this example, the first Product Development TPM candidate introduced earlier is promoted to the CDD. It also illustrates the tabular format of the corresponding TPM information in the document.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 35. Analyze – Sustainment TPM Attributes: TPM Promotion to Program Management PlanThis slide provides an example of a Program Management TPM candidate promoted to the Program Management Plan (PMP). It also illustrates how the corresponding TPM information appears in the document.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 36. Analyze – Sustainment TPM Attributes: TPM Promotion to ContractsIn order to produce a reliable system, it is important to set measurable goals on quality outcomes. The Program leadership must set quality objectives to assure the design is implemented correctly in its physical form. TPMs are useful in specifying an exacting level of quality. This slide provides an example of a Process Management TPM candidate being promoted to a Development Contract Post-Milestone C. The context for defect rate is a run chart, plotting performance over time within specific limits. In the Development Contract, the TPM’s objective is production focused and provides visibility into adverse defect trends early in the development process.You have just completed Topic 3: Select Measures of Effectiveness for Supportability. Next, you will learn how to translate Measure of Effectiveness into achievable outcomes.Translate Measures of Effectiveness into Achievable OutcomesContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 37. Topic 4: Translate Measures of Effectiveness into Achievable Outcomes This topic discusses establishing evaluation criteria.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 38. MOE Analyze – Establish Evaluation CriteriaThe last step in the Analyze phase is to establish evaluation criteria by describing the Sustainment TPM’s role in design performance. ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 39. Analyze – Establish Evaluation Criteria: TPM Role in Design Performance It is helpful to represent visually the evaluation criteria in order to examine its trend over time. TPMs must track in a certain slope direction (rise over run) to reach the planned target at the appropriate milestone event. As the graph in this slide illustrates, the weight trade-space is less prevalent in the latter stages of the program. This lack of trade-space is depicted by a flattened curve as the design is locked down for production. TPMs can provide predictive power, especially if the slope is unchanged. The Program Manager would extend the line to estimate the probability of achieving the objective within cost and schedule.Content624265213108Technology Maturation & Risk Reduction00Technology Maturation & Risk ReductionSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 40. Analyze – Sustainment TPMs: TPM Maturity As the life cycle progresses, TPMs become more refined. TPMs mature and increase in number as they are promoted into requirements and planning documents:At Milestone A, the initial MOEs are captured in the PMP, RAM-C Rationale Report, and LCSPAt Milestone B, new and refined Sustainment KPP/KSAs and TPMs are:Promoted to the CDDUpdated in the PMP and RAM-C Rationale ReportDocumented in the LCSPAt Milestone C, new and refined Sustainment TPMs are:Promoted to the CPDUpdated in the PMP, RAM-C Rationale Report, and the LCSPPromoted to the Production ContractYou have just completed Topic 4: Translate Measures of Effectiveness into Achievable Outcomes. Next, you will learn about validating a supportable design.Validate a Supportable DesignContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 41. Topic 5: Validate a Supportable Design This topic discusses how to validate a supportable design.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 42. MOE Report Findings The first step of the Report Findings phase is to develop and employ a surveillance process to monitor Sustainment metrics. ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 43. Report Findings – Monitor Sustainment Metrics: Surveillance Process The surveillance process should be data driven and the data should do the following:Address both actual and predicted problemsRemain connected to process changes through scheduled data revisionsThe Failure Reporting, Analysis, and Corrective Action System (FRACAS) is used as part of a surveillance process. FRACAS is a closed loop process for reporting, classifying, and analyzing failures to determine their root causes, and to plan corrective actions. The program creates a data repository from which statistics can be derived. Systems Engineers, the Product Support Manager, and LCLs use these statistics to quantify system Reliability, safety, and quality, as well as the effectiveness of the Product Support mon FRACAS outputs may include the following: Field Mean Time between Failure (MTBF)Mean Time between Removal (MTBR)Mean Time to Repair (MTTR)Spares consumptionFailure/Incident distribution by classification of severity, type, location, part number, serial number, symptom, etc.Potential training deficienciesContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 44. Report Findings – Report Deviations: Communication Structure The second step in the Report Findings phase is to report deviations. All of the Integrated Product Support (IPS) Elements involve MOEs/TPMs. Since IPS elements are so interrelated and impact each other, globally important MOEs/TPMs are reported to all of the IPTs. MOEs are documented and updated in several program documents which are driven by the life cycle phase.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 45. Report Findings – Report Deviations: Design Reviews When a program goes through a design review, the TPMs/metrics should be trending in a direction that intersects the objective line at the appropriate milestone event. The example in this slide illustrates the downward trend in the UAV’s weight as it progresses through the life cycle. Specifically, at Post-Milestone B, its weight is between the threshold and objective value, which is appropriate.ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 46. Lesson Exercise ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 47. Lesson Exercise Debrief: Question 1 Graph ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 48. Lesson Exercise Debrief: Question 2 Graph You have completed Topic 5: Validate a Supportable Design. Next, you will conclude the lesson.SummaryContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 49. Topic 6: Summary Content3050882447350Technology Maturation & Risk Reduction00Technology Maturation & Risk ReductionSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 50. Takeaways ContentSlide STYLEREF 1 \s 3 SEQ Slide \* ARABIC \s 1 51. Summary Congratulations! You have completed Lesson 3: Measures of Effectiveness for Supportability. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download