Use of Computer-Assisted Audit Tools and Techniques ...



. 4, October 1, 2001Audit ToolsUse of Computer-Assisted Audit Tools and Techniques (CAATTs), Part 1Readers' rating: 5 out of 5 By Charles Le Grand, CIA, CISA, CDP CAATTs may be classified in the following groups: Electronic Working Papers????? Information Retrieval and Analysis????? Fraud Detection????? Network Security????? Electronic Commerce and Internet Security????? Continuous Monitoring????? Audit Reporting????? Database of Audit History????? Computer Based Training????? Time TrackingAs audit tools grow more powerful and sophisticated, they are also becoming easier to learn and use. And, at the same time, they also must fit into a complex and ever changing environment. Features of audit software can easily conflict with features of other software on the computer or network, and must be carefully managed.As tools become more powerful, auditors may use features or services provided in the software that command considerable system resources (memory, processing cycles, communication bandwidth, and storage) and compete with other users of those resources. For example, an auditor may request access to a file with a program that will examine each record in the file and may lock other users out until the process is complete. The processing could also require large amounts of network storage space at a time when it is in short supply and could cause a server to crash. It is important to schedule such processing at times when other system users will not be delayed or prevented from performing their work. Alternatively, many audit organizations perform their audit analyses using files copied or archived from the live production files.CAATTs may also be large, powerful, or specialized enough to require a dedicated server for audit purposes. A server may be needed to support the audit website, or just to assure the independence and security required by audit functions. And, as evidenced by the list of software tools attached to this document, there are more tools available than the amount of time an auditor may have to learn to use those tools. So the need for software specialists to support internal auditing is increasing even as the software is getting easier to use.Risks associated with software tools and techniquesSoftware ease of use may also result in the implementation of features that unintentionally weaken information security provisions. While software vendors may not be particularly open about their potential weaknesses, a growing body of websites documents software weakness and available corrections. This provides both positive and negative opportunities.As weaknesses in software are discovered and documented, the vendors of those software products develop corrections or patches that may be applied until the weakness is corrected in the next formal release version of the software. However, many organizations do not apply such patches, for a variety of reasons. Hackers know software frequently goes unpatched, so they search for particular versions of software with known weaknesses. They may then launch an attack against that system using software developed to exploit known weakness. Such software, called a "script," may require little or no knowledge to use. The successful attack using a script may give the attacker unlimited (or root) access to the target system. Normally, root privileges are reserved for system administrators and are closely monitored. Once an attacker has root access they have virtually unlimited access to the system, and may also obtain access privileges to other systems with an established trust relationship.Another element contributing to risk in information systems and networks is the configuration of systems as provided by vendors. Frequently systems are initially installed with the security and control features turned off. System and network engineers and administrators must select the appropriate mix of control features they need and turn them on when the system is installed. Sometimes security and control features will conflict with features of other system components or may add considerable overhead to system processing, such as through the use of system logging. When security components conflict with operations, the typical response is to turn those components off. Unless the organization provides strong security policy administration and/or auditing, management may be unaware security features are not being used. Therefore, frequent assessment and monitoring are important elements of information security management.The Center for Internet Security (CIS) (see , a not-for-profit organization) has developed benchmarks for identifying the security features that should be activated for specific operating environments, and publishes the specific settings for individual operating systems. These benchmarks are available on their website. CIS also provides downloadable software to check system configurations against the benchmark.Electronic working papersThe capability to search for information in text, databases, or other audit records is giving auditors great ability to coordinate their efforts and to examine findings from prior or concurrent audits. The ability to require standardized audit forms and formats can improve both the quality and consistency of audit working papers. The management of current and archived working papers in a centralized audit file or database can make it easier for audit management to coordinate concurrent audits and assure they consider findings from prior or related projects.Expert systems provide an opportunity to add broad support and increased functionality to audit working paper tools. For example, an expert system may evaluate responses to a questionnaire and automatically generate links to additional related questions. Expert systems may also look at patterns in information, findings, recommendations or related concurrent or previous audits, and provide reports indicating potential related or systemic problem areas.As audit work paper tools provide the ability to include supporting information other than text or numbers — such as pictures, sound, and video — the methods for organizing and providing access to such information must adapt accordingly. In future, auditors may discover that a great deal of information needed in audit reviews may exist in forms other than text, numbers, or graphical characters.A word of caution is in order: As you consider commercial solutions for managing electronic working papers, consider the environment in which the software will operate. Some packages require environments that may be inconsistent with the systems and networks maintained by the rest of the organization. Consider also flexibility. Some packages may be limited in the options available for different types of working papers that can be used and communicated among audit team members. Some packages may need modifications to suit the needs of your organization. Modifications may result in difficulty applying new releases of the software and/or may void the vendor’s warranty of features and functionality. These considerations are certainly not unique to audit software tools and are part of the complexity routinely managed by information services professionals and rmation retrieval and analysisTo sample or not to sampleHistorically, auditors have relied on samples of transactions to perform their tests. With the use of automated retrieval and analysis tools, it may be easier to assess all records than to evaluate a sample. Furthermore, auditors can set parameters in software to identify all records meeting selection criteria. Full selection of known error type records can eliminate the problem of estimating error rates. Instead, error analysis can focus on those records with data that are outside the range of expected transaction values but still within the limitations that define error conditions.Actual sampling techniques may be applied at the time records are selected from the production system, or all records of a given type may be selected and sampling or more detailed selection may be applied in the analysis process.Record selection criteria may be based on prior audits, but auditors should continuously assess opportunities to improve audit coverage — especially if this can be accomplished at reduced overall cost. Automated selection and analysis tools can facilitate improvements, but will not automatically assure such improvements.Retrieval and analysis softwareIdentifying and accessing informationInformation retrieval and analysis tools can present significant technical challenges to auditors as information subject to audit may reside in diverse and distributed system types with varying degrees of control and standardization. Data may be stored under the control of various machine types and operating systems using differing formats; it may move across telecommunications environments using different protocols; it may be stored or archived by various database management systems using fixed or variable length fields or records and subject to differing database standards; and it may even reside in numerous physical locations as in a distributed database or data warehousing environment. Particularly sensitive data may only be available in encrypted form and may be subject to government regulations regarding its transmission, storage, controlling software, encryption key management, and import / export or transmission across national borders.Many auditing departments use technical specialists to locate and evaluate data sources and provide the software tools to extract data and convert it into a form that can be used by audit analytical tools. Because there are so many forms and formats for information and so many proprietary standards for information storage, and because information systems environments change frequently, it may be necessary to maintain significant technical expertise among the audit team members responsible for using retrieval software. People with such expertise may be difficult to recruit or afford, and providing training to audit staff for such skills may make them highly marketable.In some organizations or industries information is stored according to specified standards that do not change frequently, and multiple audits may be performed on information in a common format. In such cases libraries of information retrieval routines can be maintained, accessed, and executed by any auditor. In other organizations the frequency of change may be greater than the frequency of audits and preparation of retrieval software routines may preclude the use of pre-programmed routines.Once information is stored in a form usable by audit analytical tools, auditors with varying degrees of technical expertise may actually perform and review the results of analysis. Many ordinary office software tools such as spreadsheets or databases may be able to access and analyze information stored in an open database compliant (ODBC) format.Some audit organizations not only maintain automated routines for information retrieval and analysis, but they deploy such software via telecommunications to allow reviews of remote systems without the time and expense of staff travel. Organizations with centralized controls and standards management are best suited to remote auditing, but auditors may also use some of the same types of software as deployed by hackers to assess security and control in distributed systems environments without centralized rmation analysisAccumulation of information about business data over a period of time may allow analysis software to identify patterns, shifts, or trends in the data indicating changes in the business, the business environment, the customer base, the economy, changing competition factors, etc. Such pattern analysis may be important to business planning and competitive advantage, and may be performed by groups outside of internal auditing. However, if audit analysis recognizes such patterns then the auditors may be able to provide a valuable contribution to the organization.Audit analysis of data patterns may be focused on shifts that indicate a need to redefine record selection criteria, quality management mechanisms, error threshold monitoring, or review of records and transactions that fall outside the normal realm of events (possibly defined in standard deviations). But audit analysis can also target certain data patterns such as identification of artificial numbers. For example, Benford’s Law defines a natural distribution of numbers common to all large bodies of numbers. In circumstances where individuals make up or modify numbers due to fraud or errors, the resulting set of numbers will not follow Benford’s Law and may be detected and investigated via audit analysis software. (For more information on this subject there are several articles in ITAudit Forum’s archives. Mark Nigrini wrote a series on Benford’s Law and Digital Analysis – published in the Emerging Issues department; and Rich Lanza wrote an article on Continuous Monitoring – published in the Audit Tools department.)More common audit data analysis routines include matching employee data to customer or vendor records, duplicate payments, payroll and overtime, approvals versus authorization levels, force codes, system overrides, access authorities, telephone usage, and much, much more. Examples abound in auditing literature.Trends in information retrieval, analysis, and monitoringA trend in auditor information retrieval and analysis is to include greater intelligence in auditing or monitoring software embedded in business systems and networks. As auditors identify risk elements and develop software to detect errors, suspicious transactions, or unusual data patterns, it is often a relatively simple process to embed such tests or monitors into production systems. In these cases, auditors can then be informed of errors or changes in data patterns soon after they occur throughout the operating life of the system or monitor.Auditors planning to deploy embedded system audit features can be identified as "users" of systems under development. Rather than functioning on the design and development team only as control specialists, they function as any other system user or interfacing system representative. The auditors specify the record selection and data format criteria for embedded monitors, as well as any special features such as logging, or the ability to modify, expand, or suspend audit monitoring.For example, auditors may expect certain systems to process transactions at expected volumes or within certain monetary ranges. Embedded monitors may alert the auditor by triggering an alarm if transactions exceed expected threshold boundaries and may gather and store copies of the related transactions. The auditor can then evaluate the data and determine if the fluctuations are normal or require additional appraisal. In either case, the audit software may be provided additional logic or intelligence to enhance such selection or appraisals in the future.Typically, when audit monitors become more sophisticated than the tools used by managers responsible for the systems, the managers will request that they also be provided such functionality. After all, no one wants the auditors to come in asking questions about problems before management is even aware of the problems. As management controls and monitoring tools become more sophisticated to match or exceed the auditing tools, then auditors can shift their emphasis to areas of greater risk, or can increase the sophistication or intelligence of their monitors. In either case, the overall control environment is enhanced.In the future, the logic used by auditors to trace transactions and events forward and backward within computer systems, networks, and files will also be embedded in sensitive systems. Then sensitive transactions flowing through systems can carry with them embedded information indicating the source(s) of the transactions and all routes taken through processing, networks, or files. Such "audit tags" will be most useful in the case of monetary transactions such as payment processing or funds transfers and will provide vital information needed to detect or deter fraud.With the decreasing costs and new capabilities of information processing and storage systems and media, it is becoming feasible to capture and archive sensitive information at all points of entry, processing, transfer, or storage. The availability of "massive redundancy" in data management will enable monitoring and analytical tools to track, in great detail, the changes applied to data throughout its life cycle. Massive redundancy can also provide for data analysis using "voting" and other analytical or statistical methods. Thus appraisals of information integrity in the future could be based on complex data analysis and proceed to controls analysis only as anomalies are encountered. This is the opposite of how traditional audit appraisals are applied and may require some process reengineering within the auditing profession.About this articleThis article is extracted from a paper prepared for an "International Seminar on IT in Audit" hosted by the National Audit Office of China (CNAO) September 16-21, 2001 in Beijing. The larger paper, titled "Information Technology in Auditing," incorporates updated material from audit software articles originally posted in the ITAudit Forum on September 1 and October 15, 1998. This article and the two subsequent companion articles replace the older ones found in the archives. An updated list of audit and risk management software and related tools and services and their providers is also provided.The IIA’s work with the CIAO and PCIS continues with the PCIS supporting the "National Plan for Information Systems Protection" and working to facilitate information sharing across sectors of the critical infrastructures and extending outreach to other nations to improve global security practices and help ensure protection of the global economy. For more information, or to participate in this activity, contact Charles Le Grand at The IIA. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download