EXECUTIVE OVERVIEW
Applied Information Economics Analysis
of
Phase I of the Virtual Plant Project
(The Document Management System)
for
Midwest Electric
February 29, 1996
This is an example deliverable for a typical AIE-driven cost/benefit analysis.
Notice:
The contents of this document are considered valuable and proprietary and contains trade secrets of Hubbard Decision Research, Inc. It is offered to selected companies for internal use only and may be distributed only within said purposes.
Copyright 1999 Hubbard Decision Research, Inc. All rights reserved. This document may not be reproduced in whole or in part, by any means, without the written permission of Hubbard Decision Research, Inc.
EXECUTIVE OVERVIEW
Hubbard Decision Research, Inc. is pleased to announce completion of the analysis of Phase I of the Virtual Plant Project. This document contains the methods of analysis, the findings and the recommendations.
Background
Hubbard Decision Research, Inc. was originally asked to analyze Phase I of the Virtual Plant project after the Information Systems Steering Committee recognized the potential for utilizing Applied Information Economics (AIE) in ascertaining the economic feasibility of the Virtual Plant concept. The steering committee recognized that traditional Information Engineering (IE) and Cost Benefit Analysis (CBA) methods were unable to place a dollar value on benefits that had been categorized as intangible. In addition, IE and CBA techniques were unable to give a meaningful measurement of the risks involved or approaches to be taken to reduce those risks.
Approach
Applied Information Economics utilizes the best available metrics and mathematical models. We interview and held workshops with over 50 individuals and statistically sampled the activities of 500 engineers. These findings were incorporated into advanced statistical simulations so that risk and return could be quantified. A systematic method called the Economic Information Quantity (EIQ) calculation was used to conduct the most cost effective information gathering strategy. The findings were incorporated into a Modern Portfolio Theory analysis of the risk and return.
Findings and Recommendations
Hubbard Decision Research, Inc. is pleased to announce completion of the analysis of Phase I of the Virtual Plant Project with the following recommendations:
The Document Management System (Phase I) should be implemented according to the priorities outlined in Section V (Recommendations).
In order to reduce uncertainty and aid in the planning of future phases, data should be gathered on the variables Data Entry Labor and Scan Rate by observing implementation of Phase I.
Phase I of the Virtual Plant project has an anticipated Internal Rate of Return (IRR) of 41.58%. The anticipated Net Present Value (NPV) is equal to $9,781,356 when discounted at the risk free rate of 5.5% over five years. The probability of loss for Phase I is near risk free at 0.12%.
Value of This Information
The value of this deliverable can be calculated in the same way that AIE uses decision theory to calculate the value of any type of information. If the recommendations of this document are implemented the net value of this analysis is approximately $9,586,000.
TABLE OF CONTENTS
I. Background
II. Approach
III. Financial Assumptions
IV. Findings
A. Workshops
1. Clarification Workshop
2. The New Cost Benefit Model
3. Estimating Workshops
a) Engineering Group
b) Human Resources Group
c) Application Development Group
B. Phase I’s Initial Risk and Return Relationship
1. Initial Cost Benefit Analysis (CBA)
2. Initial Probability of Loss and Investment Region
C. Value of Information Calculations
1. Initial EVPI Calculations
2. EIQ Analysis on Percentage Time Spent Searching for Technical Drawings
3. EIQ Analysis on the Percentage Time Spent in Activities that Can Be
Eliminated by a Document Management System
4. EIQ Analysis on Quality Assurance Rate
D. Results of Sampling (Uncertainty Reduction)
1. Survey Results of Percentage Time Spent in Activities that Can Be Eliminated by a Document Management System
2. Time Requirements to Conduct a Quality Check on One Scanned Document
3. Final EIQ Calculations
E. Phase I’s Post Sampling Risk and Return Relationship
1. Post Sampling Cost Benefit Analysis
2. Post Sampling Probability of Loss and Investment Region
3. Solution Space
F. Scanning Priorities
V. Recommendations
VI. Appendixes
A. Initial CBA with Detail
B. Post Sampling CBA with Detail
Section I
Background; The internal Cost Benefit Analysis (CBA) performed on Phase I of the “Virtual Plant” Project did not justify the expenditure. Due to the potential value of the “intangibles” not taken into consideration within the internal CBA, Hubbard Decision Research, Inc. was asked to analyze the problem utilizing the tools within its Applied Information Economics (AIE) Practice.
In response for the need to cut costs, Midwest Electric had decided to investigate ways in which to utilize modern information technology to reduce its operating costs. A steering committee was formed to preside over this effort. The steering committee decided to utilize Hubbard Decision Research, Inc. in the initial planning process. A little over a year ago, Hubbard Decision Research, Inc. conceived the concept of a “Virtual Plant”.
The “Virtual Plant” would be an electronic model of currently existing plants. From this model all levels and types of technical drawings could be accessed and modifications made using Computer Aided Design (CAD) software.
After the project was fully completed real time sensory outputs from the physical plants would be added into the model. The system would also allow for scenarios to be run on proposed designs. When compared to the current paper based filing and design system, the “Virtual Plant” concept would presumably result in a reduction in staff and an increase in the productivity of engineers.
The steering committee working with Hubbard Decision Research, Inc. realized that transforming the current operations into the desired end state of a “Virtual Plant” would take several years and would best be divided up into multiple phases. The steering committee then decided upon the following division:
Phase I - Scan all of the technical drawings from plant to subsystem level. This would result in a read only system at the user level.
Phase II - Scan all component level drawings. The system at the user level would still be read only.
Phase III - Make all drawings modifiable by CAD with approved CAD drawings automatically posted into the system.
Phase IV - Integrate the real-time sensor readings of individual plants into the system to complete the model of the physical plant. The system at this point would be capable of running scenarios independently of the physical plant.
The steering committee made the decision to only proceed with a given phase if that phase was economically justifiable independently of the other phases. An internal CBA performed on Phase I claimed that the Phase I was not economically feasible.
After reviewing the internal CBA, the steering committee recognized that the two “intangibles” excluded from the project (Improved Engineering Quality and Availability of Information) potentially held great value. However, both of these “intangibles” were not accounted for in the internal CBA because they were considered difficult or impossible to measure.
The steering committee was interested in discovering if Phase I would be economically justifiable if the “intangibles” could be defined and measured. Having already received a presentation and White Paper regarding the Applied Information Economics (AIE) Practice at Hubbard Decision Research, Inc., members of the steering committee were already familiar with the vast amount of tools available within AIE. After a thorough review, the committee decided to commission the AIE Practice of HR&C & Associate, Inc. to perform a study on Phase I of the Virtual Plant Project.
Section II
Approach; The Applied Information Economics approach uses the best available metrics and mathematical models for optimizing the IS investment decision. The rigorous methods of AIE include decision theory, Modern Portfolio Theory, and computer simulations of statistical models
The AIE approach focuses on clarifying ambiguously defined intangibles, measuring with scientific methods and optimizing the decision with provable mathematical methods.
The AIE approach:
1. Clarify
2. Measure
3. Optimize
The clarification step is required when there is perceived to be several “intangible” costs or benefits of a proposed system investment. Facilitated workshops have proven to be a successful method for identifying less ambiguous and more measurable costs and benefits from a list of intangibles.
Participants invariably find that what they really meant by an intangible is some observable quantity. For example, by “employee empowerment” a participant may really be referring to “reduced management overhead per employee” or “reduced decision delay in expenditure approvals”. These are measurable quantities that have a measurable economic impact. Clarification Workshops help people find the measurable that lie underneath the “intangibles”.
Only after the nature of the costs and benefits are defined unambiguously can a financial model be completely formulated. This financial model must incorporate the investment preferences and the financial decision criteria of the organization.
Once each variable is defined as part of a financially complete cost/benefit model, we can begin the measurement process. The measurement process usually begins with some very broad initial estimates. This is used to calculate the risk as well as the return in the decision. The quantified risk and return provide an initial financial assessment within a Modern Portfolio Theory framework.
At this point we identify the variables have the highest payoff for further analysis by calculating the Economic Information Quantity (EIQ) for the unknown variables.
We usually find that one or more variables merits further analysis. When this happens we conduct a scientifically designed study of the unknown quantity. We only attempt to achieve a precision as dictated by the EIQ.
With the refined estimates from the scientific studies we recalculate the risk and return of the proposed investment as well as the EIQ’s for every variable. If the all EIQ’s indicate that no further analysis is economically justifiable, then a decision is made based on the risk/return position of the investment. If some of the EIQ’s are still positive then further analysis should be conducted before a decision is made.
[pic]
Section III
Financial Assumptions; For Phase I to be implemented the financial analysis must plot the risk and return of the project within the agreed upon “investment region” of Midwest Electric. The historical risk/return behavior of Midwest Electric, the financial markets current risk free rate, and the steering committee’s input were taken into account to construct the following graph of acceptable risk/return relationships:
After meeting with the steering committee the following assumptions were reached:
3. Phase I of the Virtual Plant project would have to be economically justifiable independently of other phases.
4. Due to the high profile and length of the project, a decision to enact Phase I would not occur until the probability of the * project failing (achieve a return less than the risk free return) was less than 1 in 20 (5%).
5. At the request of the client, all analyses were performed on a pre-tax basis.
6. The values associated with the following two variables were treated as known values: Number of Lost Clerical Positions and Hardware Lease Costs.
7. All dollar values assume that, if economically justifiable, Midwest Electric, as an organization, will be 100% effective and efficient while implementing Phase I.
8. Any IRR or NPV will be calculated with a 5 year horizon.
Working in conjunction with Midwest Electric’s Finance Department and the Director of Finance, an “investment region” was constructed. The investment region is a portion of a risk/return graph which illustrates the economic validity of a given project. The investment threshold or boundary is constructed by several known points: the risk free rate, Midwest Electric’s historical risk/return requirements (hurdle rates) and projects historically taken despite their risk due to higher anticipated returns.
[pic]
[pic]
Since the investment boundary is composed of only few points and not a complete function, there are varying levels of confidence to areas within the investment region. The area of the highest confidence is that area displayed in the two darker shades of gray. This area is anchored to Midwest Electric’s historical risk/return requirements. Areas of lesser confidence exist in the light gray regions to the right of the investment boundary. These areas are not as confident for investment purposes because Midwest Electric is not yet entirely clear about what its investment strategy should be.
* The probability of project failure is defined as the probability of loss. The probability of loss is the percentage of all possible outcomes that have a negative net present value after being discounted at the risk free rate.
Section IV.A.1
Findings: Clarification Workshop; The clarification workshops analyzed benefits that were previously only thought of as “intangibles”. The key intangible benefits were believed to be “Availability of Information” and “Improved Engineering Quality”. Once unambiguous definitions of these intangible benefits were constructed, the workshop participants agreed that these “intangibles” really had measurable consequences.
During the initial inquiry and the Clarification Workshop, Hubbard Decision Research, Inc. conducted over 200 surveys, 20 personal interviews and performed an in-depth analysis of the internal CBA. From these efforts, we discovered that there were four distinct categories of anticipated benefits from Phase I of the Virtual Plant project.
The first tangible benefit was reduced clerical staff. This benefit was the only one of the four that was taken into account during the internal CBA. The internal analysis of Phase I listed increased “Availability of Information” and “Improved Engineering Quality” as intangible benefits. During the Clarification Workshop, a third tangible benefit was discovered - “Avoid Reproduction of Lost Documents”. This new finding was discovered while placing a unit of measure upon the two previous intangibles. One of the workshop participants had previously assumed that “Improved Engineering Quality” included the avoidance of having to reproduce lost technical drawings while the rest of the participants didn’t even recognize this as a potential benefit.
The following table shows the identified “unit of measure” definitions. The factors in italics are some of the identified quantities that would be used in the cost/benefit calculations.
[pic]
Section IV.A.2
Finding: The New Cost Benefit Model; The clarification workshops analyzed benefits that were previously only thought of as “intangibles” and placed them into measurable units. These units were then used to derive seven categories of costs and benefits.
As a result of the Clarification Workshop, we now have a new model for the components that have an impact on the decision making process. The various 20 elements that have been defined have been divided up into two groups: Annual Costs & Benefits and Startup Costs. Components of these two groups are then divided up into categories and the individual elements within that category:
Annual Costs & Benefits Startup Costs
Reduction in Filing Room Staff: Cost of Initial Development:
Number of Lost Positions Hardware Lease Costs
Annual Salary with Load Developer Costs (cost per day)
Work (in man hours)
Reduction in Document Search Time: Cost of Initial Data Entry:
Percent Time Spent Searching Number of Documents
Percent Reduction in Time Spent Data Entry Labor
Average Engineering Hourly Rate Scan Rate
Number of Engineers Quality Assurance Rate
Avg. Engineering Hourly Rate
Avoid Reproduction of Lost Documents: Retrieval Data Entry Rate
Number of Annual Occurrences
Engineering Hours to Reproduce
Average Engineering Hourly Rate
Reduced Engineering Errors:
Annual Number of Errors/Redesigns
Average Engineering Hourly Rate
Average Engineering Hours to Correct
Hours Lost on Original
Annual Entry/Maintenance Costs:
Annual Number of New Documents
Cost per Document (function of other variables)
All of the sub-categories that are in italics are variables with standard distributions.
Section IV.A.3
Estimating Workshops: Concept Overview; We utilize a coached estimating method that trains internal experts to be better estimators. Individuals with relevant expertise have their estimating habits “calibrated” through assessment and training. The workshop participants are then asked to apply their calibrated estimating skills to the variables in the cost/benefit model. These estimates are the basis for determining which variables require additional research.
The following sections use some methods that require some explanation. First, the reader should understand that estimating quantities is a general skill that can be measured and refined. In other words, experts can measure whether they are systematically “underconfident”, “overconfident” or have other biases about their estimations of quantities. Once this self assessment has been conducted they can learn several techniques for achieving a measurable improvement in estimating.
This initial “calibration” process is critical to the accuracy of the estimates later received about the project. The methods used during this “calibration” have been designed in the recent past by such well known academics as
Dr. Shoemaker from the University of Chicago.
|Definition |
|Overconfidence: The individual routinely puts too small of an|
|“uncertainty” on estimated quantities and they are wrong much|
|more often then they think. For example, when asked to make |
|estimates with a 90% confidence interval much fewer than 90% |
|of the true answers fall within the estimated ranges. |
| |
|Underconfidence: The individual routinely puts too large of |
|an “uncertainty” on estimated quantities and they are correct|
|much more often then they think. For example, when asked to |
|make estimates with a 90% confidence interval much more than |
|90% of the true answers fall within the estimated ranges. |
Few individuals tend to be naturally good estimators. Most of us tend to either be biased toward over or under confidence about our estimates.
Academic studies by Dr. Shoemaker and others have proven that you can receive better estimates by putting proposed estimators through a workshop designed around removing personal estimating biases. This workshop begins by asking the participants to make a 90% confidence interval to describe their personal knowledge about a given set of general knowledge questions.
Since the original estimates were made with a 90% confidence, an average of 1 in 10 should be incorrect. By reviewing the participants answers to these questions we can derive and illustrate their over or under confidence. By performing this process of answer and review several times, participants become “calibrated” to the level of their personal confidence that corresponds to a 90% level of statistical confidence.
The estimates contained in the following sections were made by Midwest Electric employees during a workshop facilitated by Hubbard Decision Research.
The following sections include the results of Estimating Workshops with:
1. The Engineering Group
2. The Human Resources Department
3. The Information Systems Department
Section IV.A.3.a
Estimating Workshop: Engineering Group; As a result of this Estimating Workshop 9 of the total 18 variables were assigned probability distributions. The initial estimates received from this workshop will be used to determine if more information is required about the disposition of these variables.
| |90% Confidence Intervals | | |
| |[pic] | | |
|Cost/Benefit Variable |Lower 5% |Mean |Upper 5% |
|Percentage Time Spent Searching |5% |15% |25% | |
|Percentage Reduction in Time Spent |5% |30% |50% | |
|Number of Annual Occurrences |10 |50 |90 | |
|Engineering Hours to Reproduce |60 |150 |240 | |
|Annual Number of errors/redesigns |3 |15 |25 | |
|Average Engineering Hours to Correct |100 |300 |550 | |
|Hours Lost on Original |15 |40 |65 | |
|Annual Number of New Documents |38,000 |50,000 |62,000 | |
|Number of Documents |42 |50 |58 | |
The Estimating Workshop was performed with 17 engineers from various departments within Midwest Electric. Hubbard Decision Research, Inc. through a series of questionnaires taught the participants to become better estimators by understanding their own individual biases.
The engineers spent a half day in the type of training described in Section IV.A.3. The remaining half day was spent in a facilitated estimating workshop.
Each of the cost/benefit variables listed above was discussed in turn. In each case the engineers chose to describe their uncertainty about the quantity with a normal distribution. In some cases, however, the normal distribution had to be truncated so that known limits of the quantity could be taken into account. Specifically, it was immediately agreed that none of the variable listed could be less than zero.
For most of these variables it was easier for the engineers to estimate their own experiences instead of the “company average”. We decided not to attempt to estimate the company average but to treat the workshop as a sample of 17 engineers. Each engineer focused on reporting their own experiences with “Percent Time Spent Searching”, “Average Engineering Hours to Correct”, etc. The distributions shown above were statistically derived from the 17 responses.
Other variables, such as “Annual Number of New Documents” were based on an “indexed estimate”. Engineers were informed about the total number of documents in a nuclear power plant, the average number of documents generated per project, etc. This gave the engineers an informed scale by which to base an estimate. This is another proven estimating technique.
Section IV.A.3.b
Estimating Workshop: Human Resources Group; As a result of this Estimating Workshop 5 of the total 18 variables were assigned probability distributions. The initial estimates received from this workshop will be used to determine if more information is required about the disposition of these variables.
| |90% Confidence Intervals |
| |[pic] |
|Cost/Benefit Variable |Lower 5% |Mean |Upper 5% |
|Annual Salary with Load |$32,500 |$41,250 |$50,000 | |
|Average Engineering Hourly Rate |$42 |$50 |$58 | |
|Number of Engineers |460 |500 |540 | |
|Developer Costs |$600 |$700 |$800 | |
|Data Entry Labor |$8 |$10 |$14 | |
The Estimating Workshop was performed with eight representatives from Midwest Electric’s Human Resources Department. Hubbard Decision Research, Inc. through a series of questionnaires taught the participants to become better estimators by understanding their own individual biases.
The workshop participants spent three hours in the type of training described in Section IV.A.3. Another two hours was spent in a facilitated estimating workshop.
Each of the cost/benefit variables listed above was discussed in turn. In each case the workshop participants chose to describe their uncertainty about the quantity with a normal distribution. In some cases, however, the normal distribution had to be truncated so that known limits of the quantity could be taken into account. Specifically, it was immediately agreed that none of the variable listed could be less than zero.
Most of the variance in these variables was due to uncertainty about future hiring practices and the economy-wide supply and demand for engineers. These uncertainties were discussed and some bounds were placed on the possible future scenarios.
For example, no one believed that the proportion of employees relative to contractors would increase significantly. This affects the average hourly rate of engineering labor. All indications were that the flexibility requirements of projects and the number of engineering specialties ensured that a certain percentage of the staff would be contractors. This belief was reinforced with the knowledge that several long-term contracts are currently in force with outside professional firms.
Similar factors affected the expected cost of system developers. Fortunately this quantity does not have to be estimated for a very long period of time since the development costs are expensed early in the five year planning window.
These discussions facilitated the estimation process by finding necessary upper and lower bounds to each of the above variables.
Section IV.A.3.c
Estimating Workshop: Application Development Group; As a result of this Estimating Workshop 4 of the total 18 variables were assigned probability distributions. The initial estimates received from this workshop will be used to determine if more information is required about the disposition of these variables.
| |90% Confidence Intervals | | |
| |[pic] | | |
|Cost/Benefit Variable |Lower 5% |Mean |Upper 5% |
|Development Work |4,355 |6,000 |7,645 | |
|Scan Rate |460 |500 |540 | |
|Quality Assurance Rate |0.5 |1.5 |3.0 | |
|Retrieval Data Entry Rate |0:30 |0:45 |1:00 | |
The Estimating Workshop was performed with six member of Midwest Electric’s software development group. Hubbard Decision Research, Inc. through a series of questionnaires taught the participants to become better estimators by understanding their own individual biases.
The workshop participants spent two hours in the type of training described in Section IV.A.3. Another two hours was spent in a facilitated estimating workshop.
Each of the cost/benefit variables listed above was discussed in turn. In each case the workshop participants chose to describe their uncertainty about the quantity with a normal distribution. In some cases, however, the normal distribution had to be truncated so that known limits of the quantity could be taken into account. Specifically, it was immediately agreed that none of the variable listed could be less than zero.
The estimate for the amount of development work was based in part an a previously conducted “function point analysis”. This function point analysis was not universally trusted by the workshop participants.
Furthermore, the function point analysis (as it was conducted) gave no basis for quantifying the uncertainty in terms of some kind of range.
The estimate for the project development work was further gauged with estimates of the number of modifications required and the productivity (in change orders per day) of the IS staff. These and other related variables can be used to calculate another estimate of the development effort (with an associated probability distribution)
Since the function point method and the “work per module” method differed by about 40%, a probability distribution was made that would include both estimates.
The other variables (scan rate, QA rate and retrieval data entry rate) were not familiar quantities to several of the participants. Only two of the six participants said they had worked in a similar project where such activities as scanning could be observed. The other participants deferred to the estimates of the two that had some related experience.
Section IV.B.1
Finding: Initial Cost Benefit Analysis (CBA); The initial Net Present Value (NPV) and Internal Rate of Return (IRR) are $14,963 and 25.14% respectively. The Net Expected Annual Benefit is $2,971,042 with anticipated Total Startup Costs of $7,975,000.
The initial CBA was constructed after both the Clarification and Estimation Workshops. The Clarification Workshop provided the framework by defining the cost and benefits into measurable quantities that could be estimated. The Estimation Workshop then place means and ranges as well as specifying the type of probability distribution for each of the defined variables.
The initial CBA utilizes the average of each probability distribution provided by the Estimation Workshop. With this, we can derive critical estimates such as the Net Expected Annual Benefit and anticipated Total Startup Costs. Utilizing this information, we were able to calculate both the IRR and NPV.
For both the IRR and NPV, a time horizon of five years is used. The five year time horizon applies to the duration of the proposed hardware lease. The discount rate of 25.00% used for the NPV was derived in conjunction with the Finance Department and reflects the return required by Midwest Electric for investing in a project with an associated probability of loss of 16.91%. (See Section IV.B.2)
To illustrate the current uncertainty surrounding the project, a Monte Carlo Analysis was performed on the initial CBA. The Monte Carlo Analysis provided 10,000 independently run scenarios by randomly generated values across the probability distributions to each variable in the CBA. The results of simulation were then discounted to derive 10,000 independent NPVs. A histogram was then performed on the resulting values and a graph prepared.
[pic]
Initial Cost Benefit Analysis
|BENEFIT AND COST CATAGORIES |MEAN VALUES |
|Annual Benefits: | |
|Reduction in Filing Room Staff | $165,000 |
|Reduction in Document Search Costs | $2,250,000 |
|Avoid Reproduction of Lost Documents | $375,000 |
|Reduced Engineering Errors | $255,000 |
|Total Expected Annual Benefits | $3,045,000 |
|Annual Entry/Maintenance Costs |$73,958 |
|NET Expected Annual Benefit |$2,971,042 |
|Startup Costs: | |
|Cost of Initial Development | $6,200,000 |
|Cost of Initial Data Entry | $1,775,000 |
|Total Startup Costs | $7,975,000 |
| | |
|NET Present Value @ 25.00% | $14,963 |
|Internal Rate of Return |25.09% |
Appendix A contains additional sub-category level information about the Initial CBA.
Section IV.B.2
Finding: Initial Probability of Loss and Investment Region; Further Study is required before being able to make the decision to invest. In accordance with the steering Committee’s guidelines, Phase I of the Virtual Plant project should not be undertaken until the probability of loss is less than or equal to 5%. At the current state of uncertainty the probability of loss is 16.91%.
The Finance Department agreed that risk should be quantified as the chance of obtaining a return on any given project less than the risk free return for the same time period. We will refer to this as the “probability of loss”. The procedures used in deriving the probability of loss takes advantage of current computing power by performing a Monte Carlo Analysis.
The Monte Carlo Analysis generates 10,000 random numbers for each of the variables across its predetermined probability distribution. These variables are then used to simulate 10,000 potential cash flows for the project and discounts them at the risk free rate of 5.50%. These 10,000 NPVs can then be used to determine the probability of loss for any given project.
As described in Financial Assumptions (Section III), the investment region is a portion of a risk/return graph which illustrates the economic validity of a given project.
Phase I’s current risk return relationship places the proposed investment in an area of the investment region with relatively low confidence. As is illustrated by the graph, the probability of loss for this project is greater than that specified by the steering committee.
The resulting recommendation is to not invest at this time and to utilize the findings of the EIQ analysis to reduce the uncertainty surrounding the returns of the project. This will potentially reduce the probability of loss. [pic][pic][pic]
Section IV.C
Information Value Calculations: Concept Overview; We utilize a formal scientific approach to identify the highest payoff information gathering strategy. This method calculates the Economic Information Quantity (EIQ) of each unknown variable. The EIQ tells us which variables in the cost benefit analysis should be studied further and exactly how much analysis is needed.
The following sections use some methods that require some explanation. First, the reader should understand that the value of information can be calculated for a specific variable in an investment decision. Perhaps we are uncertain about the actual development labor of a proposed information system. This uncertainty can be expressed in a range such as “The development effort must be somewhere between 100 and 250 work months”. The value of a more precise answer can be calculated using some methods from decision theory.
Furthermore, we can calculate an economically optimal amount of precision (i.e. information) that gets the most value without spending too much on analysis.
|Definition |
|Economic Information Quantity (EIQ): The economically optimal|
|amount of information to gather about an uncertain quantity |
|that affects an investment decision. |
The Economic Information Quantity (EIQ) is calculated using a formal mathematical method from decision theory. The methods from decision theory quantify the following:
1. How much is a quantity of information worth?
2. How much does a quantity of information cost?
3. What information quantity gives the economically optimal balance of costs and benefits?
The EIQ can be a difficult and time consuming calculation. In order to spend time effectively we can make a “first cut” elimination of unknown variables before we do a complete EIQ calculation. We do this by calculating the Expected Value of Perfect Information (EVPI).
|Definition |
|Expected Value of Perfect Information (EVPI): The economic |
|value of perfect information about an uncertain quantity that|
|affects an investment decision. |
Perfect information is, of course, never achievable. However, the calculation is relatively easy and it can eliminate many variables from consideration for more analysis. If, for example, the EVPI of a variable is very low then we may conclude that it is unlikely that any analysis would yield useful information.
If the EVPI of a variable is high then we calculate an EIQ to find the optimal information gathering approach.
The following sections include:
1. Initial EVPI calculations for each of the 18 variables in this cost/benefit analysis.
2. EIQ analysis on the variable “Percentage of Time Spent Searching for Technical Drawings”.
3. EIQ analysis on the variable “Percentage of Time Spent in Activities that Can Be Eliminated”.
4. EIQ analysis on the variable “Quality Assurance Rate”.
Section IV.C.1
Finding: Initial EVPI Calculations; Results of the initial Expected Value of Perfect Information (EVPI) calculations show that there are only three key variables with EVPI’s large enough to justify an EIQ analysis. These three key variables are the percentage time spent searching for information, the percentage reduction in time spent and the quality assurance rate for scanned documents.
The chart to the right lists the Expected Value of Perfect Information (EVPI) for all of the variables associated with Phase I. The EVPI is the dollar value that a rational person would be willing to pay for information that would make he or she 100% confident of that variable’s value.
Knowing the EVPI, we can compare the anticipated costs of the sampling technique that would be effective in measuring this variable within an anticipated degree of confidence needed to justify the expense. If the degree of confidence achievable by a given sampling technique is both cost effective and greater than the Lower Bound of Valuable Information then it may be worthy of further study in the form of an EIQ analysis.
The Lower Bound of Valuable Information (LBVI) is designed to identify the minimum amount of information needed before additional information would have value. The more uncertain a decision, the more likely that a small amount of information about that decision would have value. Therefore, when a decision is surrounded with uncertainty, the LBVI for a given variable will be lower than it would if there were less uncertainty about the decision.
Since the initial CBA has shown the NPV of this project to be near zero, there is a great deal of uncertainty surrounding the decision at this point. Therefore, the LBVI for the majority of these variables is very low.
The further study of any of these variables has the potential of lowering the overall uncertainty surrounding this project. If uncertainty were lowered via sampling, it would have two effects, the LBVIs would increase for all of the variables and the EVPI’s for all of the variables would decrease. As the uncertainty surrounding the project is decreased the value of additional information will decrease.
|VARIABLES | EVPI |
|Annual Salary with Load | 16,055 |
|% Time Spent Searching | 633,136 |
|% Reduction in Time | 949,371 |
|Cost of Eng. | 233,843 |
|Number of Engineers | 112,784 |
|# of Annual Occurrences | 152,124 |
|Eng. Hrs. to reproduce | 126,067 |
|Annual # of errors/redesigns | 101,144 |
|Avg. Eng. Hrs. to Correct | 110,379 |
|Hours lost on Original | 6,655 |
|Developer Cost | 144,290 |
|Software Development (Work) | 280,564 |
|Number of Documents | 41,500 |
|Data Entry Labor | 11,143 |
|Scan Rate | 10,697 |
|QA Rate | 445,024 |
|Retrieval Data Entry Rate | 4,824 |
|Annual # of New Doc. | 11,161 |
The three variables listed above which are in bold and italics are variables for which an EIQ analysis is justified. Two other variables, software development and cost of engineers had high EVPI’s and were not chosen for further study. The precision of the estimate for software development was very close to that which has been historically achievable. Therefore, further expenditure studying this variable is unjustified.
The precision of the estimate for the cost of engineers is attributable to controversy surrounding the appropriate load and mix of internal and contract engineers beings used. The costs of further studying this variable is prohibitive.
However, EVPI’s will be calculated again after additional information is gathered. Further EIQ’s could be calculated at that point.
Section IV.C.2
Finding: EIQ Analysis on Percentage Time Spent Searching for Technical Drawings; Additional information gathering on this variable is justified. The economically optimal amount of additional information corresponds to a sample size of 310.
The following Economic Information Quantity (EIQ) analysis was performed on the variable with the second highest EVPI out of all of the defined variables in Phase I. The Percentage Time Spent Searching for technical drawings appeared in the CBA under annual benefits.
The proposed sampling technique was a binomial “spot-sampling” of engineers’ activities. This is designed to obtain a simple “yes” or “no” answer to the following question. “At [some specified time] were you searching for a technical drawing?”
The EIQ analysis derived an optimal sampling size of 310 by taking the following Economic Cost of Information (ECI) assumptions into account. The estimated fixed costs of the sample equaled $34,000 while the variable cost per sample was estimated at $50. These cost assumptions come from the expected time to do the analysis and the cost of the labor.
The maximum Net Benefit of Information (NBI) is reached at a sample size of 310. The optimal sampling size is derived by utilizing a finite element analysis. The distribution of all possible values of Percentage Time Spent is divided up into 1,000 increments along its probability distribution. The probability of the number of successes given a specific sample size are then summed along with the Expected Value of Information (EVI).
The EVI for various sample sizes can then be compiled and graphed. The NBI function, which is also graphed for an EIQ analysis, is equal to the EVI less the ECI. The NBI will reach its maximum at the EIQ.
For the EIQ analysis performed on the Percentage Time Spent searching for technical drawings, the maximum Net Benefit of Information has an anticipated dollar value of $568,486. Further, the average confidence associated with a sample size of 310 is 88.3%.
[pic]
While performing this analysis, one of the analysts conceived the idea of combining the sample for the Percentage Time Spent with a sample for the Percentage Time Reduced.
The response to the spot sample would simply be categorized more precisely. The answer to the survey would be analyzed to determine if the engineer was performing a task that would be explicitly eliminated by the document management system. If an engineers survey response was analyzed as performing a task that would be eliminated, that response would be considered a success using a binomial sampling method. An EIQ analysis combing these two variables is promising and merits additional EIQ analysis (see Section IV.C.3).
|EIQ Summary: | |
|Sample Size: |310 |
|Net Benefit of Information (NBI): |$568,486 |
|Cost of Information (ECI): | $49,500 |
|Value of Information (EVI): |$617,986 |
|Average Confidence: |88.3% |
|Fixed Cost of Sampling: |$34,000 |
|Variable Cost of Sampling: |$50 |
Section IV.C.3
Finding: EIQ Analysis Performed on the Percentage of Time Spent in Activities that Can Be Eliminated by a Document Management System; Additional information is economically justified. The economically optimal amount of information would be obtained by a study with 950 samples.
The results of the following EIQ analysis combines 2 of the previously defined 18 variables being studied during Phase I. While analyzing the EIQ’s, the two variables with the highest expected payoff were the percentage time spent searching for information and the percentage reduction in time spent due to the proposed document management system. Both of these variables alone justified further study with optimal sampling sizes in excess of 300. The sampling techniques used for both the percentage time spent searching for information and the percentage reduction in time spent due to the proposed document management system were found to have methods that were nearly identical.
A system for performing a combined study was then formulated and an EIQ analysis was performed to compare the results of a single study of both variables to the individual EIQ findings. The results clearly showed that the most economically justifiable sampling method was the one in which both of the variables percentage time spent searching for information and the percentage reduction in time spent due to the proposed document management system were combined into the same study.
The proposed survey method was a binomial sampling designed as a “spot sample” survey that would take a detail snapshot of what an engineer was doing at a particular point in time. Engineers from the total pool of 500 would be randomly sampled at different points in time over a 2 week period. Each engineer would give a detailed description of the activity engaged in at the time the sample was taken. Some of these activities would be interpreted as one of the activities that would be eliminated by the proposed document management system. The number of activities in this category would be counted and reported by the survey.
[pic]
The estimated costs of the survey to be used within the EIQ calculation had a fixed cost of $37,000 with a variable cost per survey of $50. These costs were derived from historical data on the use and costs of such sampling techniques. Using these cost inputs into the EIQ calculation we found that the optimal sampling size was 950. Further, the average confidence associated with a sampling size of 950 was approximately 91%.
|EIQ Summary: | |
|Sample Size: |950 |
|Net Benefit of Information (NBI): |$1,021,303 |
|Cost of Information (ECI): | $84,500 |
|Value of Information (EVI): |$1,105,803 |
|Average Confidence: |91.0% |
|Fixed Cost of Sampling: |$37,000 |
|Variable Cost of Sampling: |$50 |
Section IV.C.4
Finding: EIQ Analysis on Quality Assurance Rate; Additional information gathering on this variable is justified. The economically optimal amount of additional information corresponds to a sample size of 180.
The following Economic Information Quantity (EIQ) analysis was performed on the variable with the third highest EVPI out of all of the defined variables in Phase I. The Quality Assurance Time needed per scanned document appeared in the CBA under both startup costs and annual maintenance costs. The sampling technique which was assumed for this analysis was a discrete sampling of 12 engineers who take part in a simulated scanning operation. This simulation would be organized with the assistance of the vendor of the scanning hardware and software utilizing their training facilities which are located at their corporate headquarters.
The costs used to derive the Economic Cost of Information (ECI) function of the EIQ analysis was a fixed sampling cost of $30,000 and a variable sampling cost of $10 per document. These costs assume that all 12 engineers could be taught how to use the system, practice to the point where their individual learning curves where relatively flat and perform 180 samples within the same day. The vendor of the hardware and software assured us that this type of efficiency is reached during their standard six hour system orientation.
The maximum Net Benefit of Information (NBI) is reached at a sample size of 180. The optimal sampling size is derived by utilizing a finite element analysis. The distribution of all possible values of the Quality Assurance Time is divided up into 1,000 increments along its probability distribution. The probability of the number of successes given a specific sample size are then summed as well as the Expected Value of Information (EVI).
[pic]The EVI for various sample sizes can then be compiled and graphed. The NBI function, which is also graphed for an EIQ analysis, is equal to the EVI less the ECI. The NBI will reach its maximum at the EIQ.
For the above EIQ analysis performed on the Quality Assurance Time to review a scanned document, the maximum Net Benefit of Information has an expected dollar value of $134,212. Further, the average confidence associated with a sampling size of 180 is approximately 89.5%.
|EIQ Summary: | |
|Sample Size: |180 |
|Net Benefit of Information (NBI): |$134,212 |
|Cost of Information (ECI): | $31,800 |
|Value of Information (EVI): |$166,012 |
|Average Confidence: |89.5% |
|Fixed Cost of Sampling: |$30,000 |
|Variable Cost of Sampling: |$10 |
Section IV.D.1
Finding: Survey Results of Percent Time Spent in Activities that Can Be Eliminated by a Document Management System; The study of how engineers spend their time shows that about 7.2% of their time is spent in activities that will be automated. This is well above the amount of time needed to justify an investment in a document management system.
Due to the findings of the Economic Information Quantity (EIQ) analysis performed on the survey results of percentage time spent in activities that can be eliminated by a document management system, we decided that a survey of how engineers spend their time would have a high payoff relative to other analyses. This survey actually attempts to measure two uncertain variables in the cost/benefit analysis of the document management system - percent time spent searching for information and the percent reduction in time spent due to the proposed document management system. It was found during the EIQ analysis that surveying both of these variables together had the highest expected information gathering payoff.
Consequently, we designed a “spot-sample” survey that took a detailed snapshot of what an engineer was doing at a particular point in time. Almost all of the 500 engineers participated in the survey.
Each engineer was sent instructions regarding this survey. They were told that they would be paged or called (the majority had pagers) one to three times during a two week period. As soon as possible after notification, the engineer filled out a survey describing exactly the activity that they were involved in at that moment. A sample survey is shown on the following page.
The initial EIQ analysis showed that a survey that included 950 spot-samples gave the optimal amount of information. However, a mid-point EIQ calculation showed that we should terminate the study after 621 samples were taken. At that time, 45 of the 621 samples of activities were activities that would be eliminated by a document management system. The most common examples were:
Signing out documents from the document management center
Tracking down a misrouted drawing
Sorting through unindexed files of drawings.
[pic]
Concerns that responses may not be honest or that the survey was intrusive were addressed by the fact that each engineer would only be answering one to three spot surveys. This is not enough statistical data to draw any conclusions about how an individual engineer spends time. This may also encourage honest answers.
Another concern was that since the study was only over a two week period that cyclical activities may be over or underrepresented. This effect is minimized since the study covered all eight nuclear power plants. The maintenance cycles of each plant are deliberately scheduled so that at any time only two plants are at the same maintenance phase. The natural result of simultaneously measuring all the plants is that all maintenance cycles are represented in the study.
| Survey Summary | |
|Number of samples taken: |621 |
|Number of samples of activities that will be |45 |
|automated: | |
|Number of engineers included in study |439 |
|% samples of automated activities: |7.2% |
|Standard deviation of the result: |1% |
Section IV.D.2
Findings: Time Required to Conduct a Quality Check on One Scanned Document; A controlled test showed that it takes two minutes to conduct the quality checks for the average scanned document. This is slightly higher than expected and it raises the expected implementation and maintenance costs of the document management system. But it is still well within an acceptable cost range when compared with the new decision threshold point.
Due to the findings of the Economic Information Quantity (EIQ) analysis performed on the time required to conduct a quality check on one scanned document, we decided that more information about this variable would hold great value. Consequently, we designed a controlled experiment that measured the amount of time that it took to QA a scanned document in a realistic simulation of a scanning operation. The vendor of the scanning hardware and software agreed to set up a small-scale version of a scanning operation at their headquarters. Twelve engineers were then chosen to take part in a one-day long experiment.
The engineers were given two hours of training on how to look for defects in a scanned document. The engineers were tested by reviewing 30 documents with various scanning defects. The engineers were retrained until the error rate (the chance of accepting a flawed document) was minimized and relatively uniform across all 12 engineers.
The engineers were then asked to inspect 200 previously scanned in documents. The scanned documents were randomly selected and included samples from systems diagrams, text documents and mechanical drawings.
Although the EIQ analysis required only 180 samples of scanned documents, we decided to continue sampling as long as the engineers were available. We finally sampled about 250 QA reviews of documents (note the 200 scanned documents were each reviewed more than once but by different engineers).
There was a concern that the relevant sample size here was not the 250 document reviews but the 12 engineers conducting the inspections.
[pic]This type of sampling could be called a type of “clustered” sampling. The findings of the survey showed, however, that trained engineers differed insignificantly on defects detected and the average amount of time spent reviewing each document. Since the “sample variance” was small for differences among engineers, we can conclude that the population of engineers will not be much different. A more significant variance is found among the various types of documents. More complicated documents simply take longer to inspect.
The findings are still conclusive. The above graph indicates the adjusted “threshold” (the point at which the QA time can change the project investment decision) for the risk free rate. The distributions for either assumption about the relevant sample size is also shown. Regardless of the distribution, it is virtually impossible for the remaining uncertainty in QA time to make a difference in the final decision.
|Survey Summary | |
|Document inspections sampled: |250 |
|Engineers sampled: |12 |
|Average time to inspect: |2 minutes |
|Standard deviation of the result: |0.2 minutes |
Section IV.D.3
Findings: Final EIQ Calculations; All of the Economic Information Quantities (EIQ) are now zero. As a result of the uncertainty reduction resulting from the earlier studies performed the Expected Value of Perfect Information (EVPI) for all of the variables has been reduced to zero. This indicates that no further analysis of uncertain quantities is economically justified.
The following chart compares the initial EVPI’s and EIQ’s to the post sampling EVPI’s and EIQ’s:
| | Initial Values |After Sampling |
| VARIABLES | EVPI |NBI @ EIQ | EVPI |NBI @ EIQ |
|Annual Salary with Load | 16,055 |0 |0 |0 |
|% Time Spent Searching | 633,136 |*1,021,303 |0 |0 |
|% Reduction in Time | 949,371 |*1,021,303 |0 |0 |
|Cost of Engineers | 233,843 |0 |0 |0 |
|Number of Engineers | 112,784 |0 |0 |0 |
|# of Annual Occurrences | 152,124 |0 |0 |0 |
|Eng. Hrs. to reproduce | 126,067 |0 |0 |0 |
|Annual # of errors/redesigns | 101,144 |0 |0 |0 |
|Avg. Eng. Hrs. to Correct | 110,379 |0 |0 |0 |
|Hours lost on Original | 6,655 |0 |0 |0 |
|Developer Cost | 144,290 |0 |0 |0 |
|Work | 280,564 |0 |0 |0 |
|Number of Documents | 41,500 |0 |0 |0 |
|Data Entry Labor | 11,143 |0 |0 |0 |
|Scan Rate | 10,697 |0 |0 |0 |
|QA Rate | 445,024 | 134,212 |0 |0 |
|Retrieval Data Entry Rate | 4,824 |0 |0 |0 |
|Annual # of New Doc. | 11,161 |0 |0 |0 |
* This NBI @ EIQ is for both the percentage time spent and percentage time reduced variables. Data was presented in this fashion due to the sampling technique used. This technique gathered data on both variables simultaneously.
Variables for which studies were performed are in bold and italicized.
Section IV.E.1
Finding: Post Sampling Cost Benefit Analysis (CBA); The initial Net Present Value (NPV) and Internal Rate of Return (IRR) are $9,781,356 and 41.58% respectively. The Net Expected Annual Benefit is $4,275,208 with anticipated Total Startup Costs of $8,475,000.
The post sampling CBA was constructed after all of the variables associated with the CBA had an Economic Information Quantity (EIQ) equal to zero. When all of the EIQ’s became zero, this indicated that any further analysis of the variables was no longer economically justified.
Two major studies were undertaken to reduce the uncertainty surrounding Phase I. The first was a spot-sampling performed to measure both the percentage time spent and the percentage time reduced looking for technical drawings. The second, was a controlled test to measure the amount of time it would take for an engineer to perform quality assurance procedures on a scanned document. For the remaining variables, their original estimates derived from the Estimating Workshop are sufficient.
The post sampling CBA utilizes the means provided by the Estimation Workshops and analysis performed in conjunction with the EIQ. These figures are then utilized to derive critical estimates such as the Net Expected Annual Benefit and anticipated Total Startup Costs. From this, the IRR and NPV can be calculated.
For both the IRR and NPV, a time horizon of five years is used. The five year time horizon applies to the duration of the proposed hardware lease. The discount rate of 5.50% used for the NPV was derived in conjunction with the Finance Department and reflects the return required by Midwest Electric for investing in a project with an associated probability of loss of 0.12% (near risk free).
To illustrate the range of potential outcomes, a Monte Carlo Analysis was performed on the post sampling CBA. The Monte Carlo Analysis provided 10,000 independently run scenarios by randomly generated values across the probability distributions to each variable in the CBA. The results of these
[pic]
simulation were then discounted to derive 10,000 probable NPV’s. A histogram was then performed on the resulting values and a graph prepared.
Post Sampling Cost Benefit Analysis
|BENEFIT AND COST CATEGORIES |MEAN VALUES |
|Annual Benefits: | |
|Reduction in Filing Room Staff | $165,000 |
|Reduction in Document Search Costs | $3,575,000 |
|Avoid Reproduction of Lost Documents | $375,000 |
|Reduced Engineering Errors | $255,000 |
|Total Expected Annual Benefits | $4,370,000 |
|Annual Entry/Maintenance Costs | $94,792 |
|NET Expected Annual Benefit | $4,275,208 |
|Startup Costs: | |
|Cost of Initial Development | $6,200,000 |
|Cost of Initial Data Entry | $2,275,000 |
|Total Startup Costs | $8,475,000 |
| | |
|NET Present Value @ 5.50% | $9,781,356 |
|Internal Rate of Return |41.58% |
Appendix B contains the Post Sampling CBA down to the sub-category level.
Section IV.E.2
Finding: Post Sampling Probability of Loss and Investment Region; Phase I of the Virtual Plant project is economically justifiable. The current risk/return relationship of Phase I plots well within the investment region of Midwest Electric and meets the steering Committee’s guideline of the project having no more than a 5% probability of loss. At the current state of uncertainty, the probability of loss is 0.12% with an IRR of 41.58%.
The probability of loss (POL) is defined as the chance of obtaining a return on any given project less than the risk free return for the same time period. The procedures used in deriving the probability of loss takes advantage of current computing power by performing a Monte Carlo Analysis.
The Monte Carlo Analysis generates 10,000 random numbers for each of the variables across its predetermined probability distribution. These variables are then used to simulate 10,000 potential cash flows for the project and discounts them at the risk free rate of 5.50%. These NPV’S can then be used to determine the probability of loss for any given project.
As described in Assumptions (Section III), the investment region is a portion of a risk/return graph which illustrates the economic validity of a given project. The investment threshold or boundary is constructed by several known points: the risk free rate, Midwest Electric’s historical risk/return requirements (hurdle rates) and projects historically taken despite their risk due to higher anticipated returns.
Since the investment boundary is composed of several points and not a function, there are varying levels of confidence to areas within the investment region. The area of the highest
confidence is that area displayed in the two darker shades of gray. This area is anchored to Midwest Electric’s historical risk/return requirements. Areas of lesser confidence exist in the light gray regions.
Phase I’s current risk return relationship places the proposed investment well within the investment region with the highest confidence. As is illustrated by the graph,
[pic]
the probability of loss for this project is within the 5% allowed by the steering committee. The resulting recommendation is to proceed with Phase I .
[pic]
[pic]
Section IV.E.3
Finding: Solution Space for The Variables of Percentage Time Spent and The Percentage Time Reduced Searching for Technical Drawings; As graphically illustrated by the Solution Space below, all potential combinations of Percentage Time Spent and Percentage Time Reduced within a 90% confidence plot well above a 10% IRR, holding all other variables at their mean. Although the mean IRR for Phase I is 41.58%, the Solutions Space is able to graphically depict the impact of potential outcomes for the two most sensitive variables in the project.
A Solution Space is a graphical depiction of two or more variables and there potential impact on any given project dependent upon their outcomes. Although an IRR is a very good measure of return, it does not contain an uncertainty or risk component. The IRR measures a given return at the mean of all possible outcomes and does not take the range of potential outcomes into consideration.
Our experience has taught us that generally there is a limited number of variables within a problem that have a truly dynamic impact on the outcome. Through the AIE approach, we can identify these highly sensitive variables and depict their potential outcomes across a given confidence graphically.
The typical Solution Space is comprised of several curves and an ellipse. The curves can be made to represent any potential IRR threshold, such as the risk free rate or an internally used hurdle rate. The ellipse contained within the Solution Space represents a project’s potential outcomes across any given confidence.
The Solution Space for Phase I graphically illustrates the potential project outcomes by depicting the probable values of the variables Percentage Time Spent and Percentage Time Reduced within a 90% confidence ellipse. This ellipse is plotted along with three IRR curves. These curves are constructed by holding all less sensitive variables at their mean values, applying the given discount rate and solving for possible combinations of Percentage Time Spent and Percentage Time Reduced that yield a zero NPV.
[pic]
The three curves contained within the Solution Space represent the boundary for three specific IRR’s. From left to right, these IRR boundaries are 5.5% (risk free), 10% and 20%. All portions
of the solution ellipse that plot to the left of one
of the given boundaries illustrates the potential for obtaining a return on Phase I less than that boundary’s IRR. All portions to the right of a boundary represent the potential for obtaining a return on Phase I equal to or greater than that boundary’s IRR. The ellipse contained within the Solution Space for Phase I clearly shows that all potential outcomes of Percentage Time Spent and Percentage Time Reduced within a 90% confidence plot above both the 5.5% risk free boundary and the 10% boundary. The vast majority of the solution ellipse is to the right of the 20% boundary as well.
Section IV.F
Finding: Document Scanning Priorities; A “Solution Space” display of major document types shows a definite economic benefit for scanning some types of documents sooner than others. Furthermore, the solution space shows that all document types are certain to have a high positive net benefit of being scanned into the Document Management System.
A method of prioritizing the scanning of documents has been formulated as a “Solution Space”.
Most of the documents (92%) that would be scanned into the proposed Document Management System fall into one of four categories: system schematics, structural and mechanical drawings, regulatory documents and vendor specifications.
Each of these document types differ by the number of documents of that type and the amount of time spent searching for a document of that type.
The number of documents of each type was estimated using the results of an earlier document management study (May 1994). This study provided statistical confidence levels for the number of documents of each type. The time spent searching for that type of document was estimated from the findings of the study detailed in Section IV.D.1.
With these two variables, a “marginal net benefit” of scanning can be calculated for each type of document. This means that we can calculate for each type of document a separate “value added”. If a document type has a high marginal net benefit of scanning then we should scan it sooner than later. Scanning low marginal net benefit documents can be deferred. This maximizes the value of the document management system for a given period of time.
The following solution space indicates that the best approach to scanning is to scan in the following order.
Scanning Priority
1. System Schematics
2. Structural and Mechanical Drawings
3. Regulatory Documents
4. Vendor Specifications
Note that all of the document types fall well above the break-even line. This shows that all of these document types have a net positive benefit to being scanned. Therefore, all four document types should eventually be scanned.
[pic]
Section V
Recommendations: Phase I of the “Virtual Plant” Project; Investing in Phase I of the Virtual Plant Project is economically justified. The sequence of scanning the document categories should be prioritized in order of anticipated benefit. To aid in the justification for and the planning of future phases, the following variables should be observed while implementing Phase I: Software Development Work, Scan Rate and Data Entry Labor.
As discussed in Section IV.E.2 (Post Sampling Probability of Loss and Investment Region), the Probability of Loss (POL) for Phase I is 0.12% while the anticipated Internal Rate of Return (IRR) is 41.58%. This risk and return relationship plotted well within the “invest” area of the investment region.
Per the Finance and Information Systems (IS) Departments, Phase I of the Virtual Plant Project compares favorably to both Information Technology (IT) and other company wide uses for the capital. The overall IS Portfolio will also be favorably impacted by this investment, due to its size and outstanding risk/return relationship.
The Ghantt chart located at the bottom of this page illustrates the steps needed to implement Phase I in a time series based upon quarters of a given calendar year. This time series takes into account the anticipated benefit of each document category as discussed in Section IV.F (Document Scanning Priority).
In order to aid in the justification for and implementation of future phases, the following three variables should be observed when implementing Phase I: Scan Rate and Data Entry Labor. The first variable, Software Development Work had a very high initial Expected Value of Perfect Information (EVPI). An Economic Information Quantity (EIQ) analysis was not performed on this variables due to the standard deviation being very close to that which has been historically achievable. Since the sample technique to be used is simply observing the are
implementation of Phase I, the relevant sample costs extremely low and will likely reduce the future uncertainty of Software Development Work.
The other two variables which should be observed while implementing Phase I are the Scan Rate and the Data Entry Labor. While implementing future phases, the level of detail and the number of documents will increase. Having an engineer perform quality assurance (QA) on each document is the most costly portion of the overall document scanning. Although it is not currently economically justifiable for Phase I, it is conceivable that in future phases utilizing a QA technique used in Information Theory may be of great value.
This technique would scan the same document multiple times (the number of times depends upon the future cost benefit structure) and electronically compare all of the versions of the same document. If all of the versions were identical one copy of the scanned document would automatically be posted. If any of the versions of the same document were different from each other, all versions of the document would be forwarded to an engineer’s workstation. The engineer performing the QA would simply have to choose which version of the document was correct and post it, or instruct the scanners to scan the document again. The overall economic validity of this method as well as the number of versions to be scanned in order to optimize the cost/benefit relationship will depend greatly upon the variables of Scan Rate and Data Entry Rate.
|YEAR |1996 |1997 |1998 |
|QUARTER |I |II |
|Annual Benefits: |Mean Value |Source |Formula |
| | | | | |
|Reduction in Filing Room Staff | | | |
| |Number of lost positions | 4 |Internal CBA |A |
| |Annual Salary with Load | $ 41,250 |EWS-Human Resources |B |
| |Total Savings from Filing Staff | $ 165,000 |Formula |A*B |
| | | | | |
|Reduction in Document Search Costs | | | |
| |% Time Spent Searching |15.0% |EWS-Engineering Group |C |
| |% Reduction in Time |30.0% |EWS-Engineering Group |D |
| |Cumm. data (%s*%r) |4.50% |Formula |C*D |
| | Avg. Engineering Hourly Rate | $ 50.00 |EWS-Human Resources | E |
| | Avg. Savings per Avg. Hour | $ 2.25 |Formula | C*D*E |
| |Cumm.data (S per Eng per Yr) | $ 4,500 |Formula | C*D*E*2000Hrs/Yr |
| |Number of Engineers | 500 |EWS-Human Resources |F |
| |Savings per year X # of Eng. | $2,250,000 |Formula | C*D*E*F*2000Hrs/Yr |
| | | | | |
|Avoid Reproduction of Lost Documents | | | |
| |# of Annual Occurrences | 50 |EWS-Engineering Group |G |
| |Eng. Hrs to reproduce | 150 |EWS-Engineering Group |H |
| |Cumm. data (#*Eng Hrs) | 7,500 |Formula | G*H |
| |Avg. Engineering Hourly Rate | $ 50.00 |EWS-Human Resources | E |
| |Avg. Annual Cost | $ 375,000 |Formula | E*G*H |
| | | | | |
|Reduced Engineering Errors | | | |
| |Annual # of errors/redesigns | 15 |EWS-Engineering Group |I |
| |Avg. Engineering Hourly Rate | $ 50.00 |EWS-Human Resources | E |
| |Cumm. data (errors*Eng Rate) | 750 |Formula | E*I |
| |Avg. Eng. Hrs. to Correct | 300 |EWS-Engineering Group |J |
| |Hours lost on Original | 40 |EWS-Engineering Group |K |
| |Avg. Annual Lost Costs | $ 30,000 |Formula | E*I*K |
| |Avg. Annual Redesign Costs | $ 225,000 |Formula | E*I*J |
| |Avg. Annual Total Lost/Redesign | $ 255,000 |Formula | E*I*(J+K) |
| | | | | |
|Total Expected Annual Benefits: | $3,045,000 |Formula |A*B+E*G*H+E*I*(J+K)+ |
| | | | | C*D*E*F*2000Hrs/Yr |
| |Continued on following page | | | |
| | |Initial Cost Benefit Analysis (continued) |
|Annual Entry/Maintenance Costs |Mean Value |Source |Formula |
| |Annual # of New Doc. | 50,000 |EWS-Engineering Group | T |
| |Cost per Document | $ 1.48 |Formula | (P*Q+E*R+P*S)/60 |
| | | | | |
|Total Annual Cost: | $ 73,958 |Formula | (P*Q+E*R+P*S)/60*T |
| | | | | |
|NET Expected Annual Benefits: | $2,971,042 |Formula |A*B+E*G*H+E*I*(J+K)+ |
| | | | | C*D*E*F*2000Hrs/Yr- |
|Startup Costs: | | | (P*Q+E*R+P*S)/60*T |
| | | | | |
|Cost of Initial Development | | | |
| |Required Hardware Investment | $2,000,000 |Internal CBA/Offer | L |
| |Developer Cost | 700 |EWS-Human Resources | M |
| |Work | 6,000 |EWS-Application Dev. |N |
| |Total | $6,200,000 |Formula | L+M*N |
| | | | | |
|Cost of Initial Data Entry | | | |
| |Number of Documents | $1,200,000 |EWS-Engineering Group | O |
| |Data Entry Labor | $ 10 |EWS-Human Resources | P |
| |Scan Rate | 0.6250 |EWS-Application Dev. |Q |
| |Cost to Scan a Doc. | $ 0.1042 |Formula | P/(60min/Q) |
| |Quality Assurance (QA) Rate | 1.5 |EWS-Application Dev. |R |
| | Avg. Engineering Hourly Rate | $ 50 |EWS-Human Resources | E |
| |Cost to QA a Doc. | $ 1.25 |Formula | E/(60min/R) |
| |Retrieval Data Entry Rate | 0.75 |EWS-Application Dev. |S |
| |Data Entry Labor | $ 10 |EWS-Human Resources | P |
| |Cost to Enter a Doc. | $ 0.12500 |Formula | P/(60min/S) |
| |Time per Document | 2.88 |Formula |Q+R+S |
| |Cost per Document | $ 1.48 |Formula | (P*Q+E*R+P*S)/60 |
| |Total | $1,775,000 |Formula | (P*Q+E*R+P*S)/60*O |
| | | | | |
|Total Startup Costs: | $7,975,000 |Formula | (P*Q+E*R+P*S)/60*O+ |
| | | | |M*N+L |
|NET Present Value @ 25.00% |$14,963 |Formula |Net Annual Benefits * |
| | | | |2.68928-Startup Costs |
|Internal Rate of Return |25.09% | | |
Section VI.B
Appendix B: Post Sampling Cost Benefit Analysis (CBA); The initial Net Present Value (NPV) and Internal Rate of Return (IRR) are $9,781,356 and 41.58% respectively. The Net Expected Annual Benefit is $4,275,208 with anticipated Total Startup Costs of $8,475,000.
The following chart provides detailed information on the mean value of all of the variables as well as their source and formula if applicable. Please note, that since the initial CBA, several mean values have changed from their initial Estimating Workshop (EWS) values due to sampling:
| | | Post Sampling Cost Benefit Analysis |
|Annual Benefits: |Mean Value |Source |Formula |
| | | | | |
|Reduction in Filing Room Staff | | | |
| |Number of lost positions | 4 |Internal CBA |A |
| |Annual Salary with Load | $ 41,250 |EWS-Human Resources |B |
| |Total Savings from Filing Staff | $ 165,000 |Formula |A*B |
| | | | | |
|Reduction in Document Search Costs | | |
| |% Time Spent Searching |22.0% |Spot-Sample |C |
| |% Reduction in Time |32.5% |Spot-Sample |D |
| |Cumm. data (%s*%r) |7.1500% |Formula |C*D |
| | Avg. Engineering Hourly Rate | $ 50.00 |EWS-Human Resources | E |
| | Avg. Savings per Avg. Hour | $ 3.58 |Formula | C*D*E |
| |Cumm.data (S per Eng per Yr) | $ 7,150 |Formula | C*D*E*2000Hrs/Yr |
| |Number of Engineers | 500 |EWS-Human Resources |F |
| |Savings per year X # of Eng. | $3,575,000 |Formula | C*D*E*F*2000Hrs/Yr |
| | | | | |
|Avoid Reproduction of Lost Documents | | |
| |# of Annual Occurrences | 50 |EWS-Engineering Group |G |
| |Eng. Hrs to reproduce | 150 |EWS-Engineering Group |H |
| |Cumm. data (#*Eng Hrs) | 7,500 |Formula | G*H |
| |Avg. Engineering Hourly Rate | $ 50.00 |EWS-Human Resources | E |
| |Avg. Annual Cost | $ 375,000 |Formula | E*G*H |
| | | | | |
|Reduced Engineering Errors | | | |
| |Annual # of errors/redesigns | 15 |EWS-Engineering Group |I |
| |Avg. Engineering Hourly Rate | $ 50.00 |EWS-Human Resources | E |
| |Cumm. data (errors*Eng Rate) | 750 |Formula | E*I |
| |Avg. Eng. Hrs. to Correct | 300 |EWS-Engineering Group |J |
| |Hours lost on Original | 40 |EWS-Engineering Group |K |
| |Avg. Annual Lost Costs | $ 30,000 |Formula | E*I*K |
| |Avg. Annual Redesign Costs | $ 225,000 |Formula | E*I*J |
| |Avg. Annual Total Lost/Redesign | $ 255,000 |Formula | E*I*(J+K) |
| | | | | |
|Total Expected Annual Benefits: | $4,370,000 |Formula |A*B+E*G*H+E*I*(J+K)+ |
| | | | | C*D*E*F*2000Hrs/Yr |
| |Continued on following page | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | Post Sampling Cost Benefit Analysis (continued) |
|Annual Entry/Maintenance Costs |Mean Value |Source |Formula |
| |Annual # of New Doc. | 50,000 |EWS-Engineering Group | T |
| |Cost per Document | $ 1.90 |Formula | (P*Q+E*R+P*S)/60 |
| | | | | |
|Total Annual Cost: | $ 94,792 |Formula | (P*Q+E*R+P*S)/60*T |
| | | | | |
|NET Expected Annual Benefits: | $4,275,208 |Formula |A*B+E*G*H+E*I*(J+K)+ |
| | | | | C*D*E*F*2000Hrs/Yr- |
|Startup Costs: | | | (P*Q+E*R+P*S)/60*T |
| | | | | |
|Cost of Initial Development | | | |
| |Required Hardware Investment | $2,000,000 |Internal CBA/Offer | L |
| |Developer Cost | 700 |EWS-Human Resources | M |
| |Work | 6,000 |EWS-Application Dev. |N |
| |Total | $6,200,000 |Formula | L+M*N |
| | | | | |
|Cost of Initial Data Entry | | | |
| |Number of Documents | $1,200,000 |EWS-Engineering Group | O |
| |Data Entry Labor | $ 10 |EWS-Human Resources | P |
| |Scan Rate | 0.6250 |EWS-Application Dev. |Q |
| |Cost to Scan a Doc. | $ 0.1042 |Formula | P/(60min/Q) |
| |QA Rate | 2 |Controlled Test |R |
| | Avg. Engineering Hourly Rate | $ 50 |EWS-Human Resources | E |
| |Cost to QA a Doc. | $ 1.7 |Formula | E/(60min/R) |
| |Retrieval Data Entry Rate | 0.75 |EWS-Application Dev. |S |
| |Data Entry Labor | $ 10 |EWS-Human Resources | P |
| |Cost to Enter a Doc. | $ 0.12500 |Formula | P/(60min/S) |
| |Time per Document | 3.375 |Formula |Q+R+S |
| |Cost per Document | $ 1.896 |Formula | (P*Q+E*R+P*S)/60 |
| |Total | $2,275,000 |Formula | (P*Q+E*R+P*S)/60*O |
| | | | | |
|Total Startup Costs: | $8,475,000 |Formula | (P*Q+E*R+P*S)/60*O+ |
| | | | |M*N+L |
|NET Present Value @ 5.50% |$9,781,356 |Formula |Net Annual Benefits * |
| | | | |4.2703-Startup Costs |
|Internal Rate of Return |41.58% | | |
-----------------------
The Applied Information Economics Company
Decision Research
Hubbard
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related searches
- overview of starbucks
- starbucks overview of the company
- overview of photosynthesis
- overview of photosynthesis quizlet
- activity overview of photosynthesis
- brief overview of starbucks
- overview of photosynthesis review worksheet
- overview of philosophers beliefs
- overview of photosynthesis 4.2 answers
- overview of photosynthesis worksheet
- brief overview of a meeting
- section 4.2 overview of photosynthesis