Software Quality Assurance: Techniques and Tools



Software Quality Assurance: Techniques and Tools

Matt Heinzelman

Department of Software Engineering

University of Wisconsin – Platteville

Platteville, Wisconsin 53818

heinzelm@uwplatt.edu

Abstract

Software Quality Assurance is essential to every software development process. SQA improves development to deployment time, reduces the number of incomplete or missed deadlines, and reduces time spent on making sure that requirements, design, code, and documentation look the same by ensuring consistency without doing it all manually. Using SQA audits to evaluate that guidelines have been met before advancing in the development process is essential. Also, audits can be used to show stakeholders that continuous progress is being made on a project. There are many tools that can be used to achieve Quality Assurance on a project including Borland StarTeam and Lucent Technologies Sablime for configuration, change, and problem management, and Programming Research QA C++ and Parasoft Jtest for code testing/metrics. Therefore, Software Quality Assurance is a very essential part of the development process.

What is Software Quality Assurance?

Software Quality Assurance involves the entire software development process – monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with [1]. It is oriented to ‘prevention’ [1]. There are many standards that are followed in this area, involving the most known and recognized IEEE IS0 9000 software quality and management guidelines.

Software Quality Assurance can also be defined more in depth as a planned and systematic approach to the evaluation of the quality of and adherence to software product standards, processes, and procedures [2]. SQA includes the process of assuring that standards and procedures are established and are followed throughout the software acquisition life cycle. Compliance with agreed-upon standards and procedures is evaluated through process monitoring, product evaluation, and audits. Software development and control processes should include quality assurance approval points, where an SQA evaluation of the product may be done in relation to applicable standards [2].

Software Quality Assurance is found in almost every computer product that is developed today. Each software development company develops their own system of standards and procedures to follow using the IEEE IS0 9000 guidelines. They use this system to help improve the development of their software system as a whole, as reviews are made during and at the end of each step in the software development process.

Standards and Procedures

Every software development company must establish standards and procedures to aid in the development of software, because these standards are the framework for which software evolves [2]. Standards are the established criteria to which software products are compared. Procedures are the established criteria to which the development and control procedures are compared [2]. The whole role of Software Quality Assurance is to make sure that these standards and procedures are developed and followed exactly in the development of software. Proper documentation of standards and procedures is necessary since SQA activities of process monitoring, product evaluation, and auditing rely on unequivocal definitions to measure project compliance [2].

There are many different standards and procedures that can be implemented into a software development system; however, this paper will only define and refer to four major areas. These areas include: Requirements, Design, Code, and Documentation standards.

Requirement standards specify the form and content of how requirements in the system will be defined. The normal standard in industry today is to establish a numbering system for each major requirement and sub-requirements. Requirement standards will also establish a system on how to write each requirement. Almost every system uses the method of writing a short phrase to summarize a requirement and then elaborates it into more detail. Some software companies like Rational have requirement development software to aid in the organization and writing of requirements.

Design standards specify the form and content of how design documents will be developed. They provide rules and methods for translating the software requirements into the software design and for representing in the design documentation [2]. Some software companies like Rational and Microsoft have design development software to aid in the organization and development of diagrams and charts.

Code standards specify the language in which the code is to be written and define any restrictions on use of language features. They define legal language structures, style conventions, rules for data structures and interfaces, and internal code documentation [2]. Coding standards can be a good thing but too many standards will force productivity and creativity to suffer. Using methods as ‘peer reviews’, ‘buddy checks’, and code analysis tools can help to enforce standards. Some examples of what good standards would be: the reduction or elimination of global variables, function and method sizes should be minimized; each line of code should be seventy characters maximum, one code statement per line, etc [2].

Documentation standards specify form and content for planning, control, and product documentation and provide consistency throughout a project [2]. Documentation is critical in a system. It could be written in any form, such as electronic (e.g. comments in a program), or in paper form (e.g. a manual). Each practice should be documented clearly so it can be repeated or changed later in the development process if needed. Specifications, designs, business rules, inspection reports, configurations, code changes, test plans, test cases, bug reports, user manuals, etc. should all be documented in some form [1]. There should ideally be a system for easily finding and obtaining information and determining what documentation will have a particular piece of information. Change management for documentation should be used if possible [1].

Software Quality Assurance procedures are explicit steps to be followed in carrying out a process. All processes should have documented procedures. Examples of processes for which procedures are needed are configuration management, nonconformance reporting and corrective action, testing, and formal inspections.

Techniques

The major technique that is used in Software Quality Assurance is the audit. They are used to perform product evaluation and process monitoring. Audits are performed routinely throughout the software development process. Their job is to look at a process and/or a product in depth, comparing them to established procedures and standards. Audits are used to review management, technical, and assurance processes to provide an indication of the quality and status of the software product [2].

The purpose of using an audit is to assure that proper control procedures are being followed, that required documentation is maintained, and that the developer’s status reports accurately reflect the status of the activity. The SQA product is an audit report to management consisting of findings and recommendations to bring the development into conformance with standards and/or procedures [2].

Tools

There are many different tools that have been developed for problem, change, and configuration management along with many different testing tools. Each tool works differently than each other; however, they accomplish the same goal: help improve the development process of a computer system. There are so many different tools out on the market today to accomplish and, this paper will look at two different tools from configuration and problem management, and two different tools for testing software.

Borland StarTeam

Borland’s process based configuration management called StarTeam is a comprehensive system that can be tailored to any software development team. They offer a range of solutions that can meet any development team according to size, distribution, and work style. StarTeam can coordinate and manage the whole development process by promoting team communication and collaboration through a centralized control of all project assets [3]. StarTeam offers integrated change management, defect tracking, file versioning, requirements management, threaded discussion, and project and task management. Every project module is put into one repository so each user is able to access it and make changes as needed. It can be customized to fit your needs, thanks to an open API, highly customable forms, and workflow [3]. StarTeam uses a Windows server interface, but is multiplatform for each of the clients.

StarTeam is offered in three different versions varying on the development teams’ size and type of project. Borland’s biggest version, StarTeam Enterprise Advantage, is used to support the configuration and management needs of large, widely distributed teams work on enterprise-level projects. This version supports the whole development cycle by delivering a flexible, customizable solution with integrated requirements management, change management, defect tracking, file versioning, threaded discussions, and project and task management [4]. Enterprise offers project trend analysis and reporting to enhance visibility for business stakeholders to optimize the delivery of software [5].

Enterprise Advantage has four unique features that are designed to meet the needs of large, geographically distributed teams. Advantage has search and query capabilities that allow team members to search for features across multiple objects and repositories for reuse and sharing. Second, a MPX server that provides multicast communication to ensure that teams are always up-to-date by broadcasting all events and keeping data for the project current. Third, a cache agent that supports distributed development with multisite repositories to ensure maximum scalability and high availability across global terms. Fourth, a Web edition that lets users work wherever and whenever they would like.

Borland’s middle of the road version is called StarTeam Enterprise. Enterprise is an ideal solution for medium-to-large development teams [4]. Blending ease of use with the power of its integrated file, change request, task, and discussion components – in a single interface – StarTeam Enterprise is an excellent next step for organizations that have outgrown their current SCM technology [4]. Enterprise uses an unified repository to manage shared and reusable components to increase team productivity by promoting parallel development. Enterprise enhances control of the development process by ensuring that all assets are versioned and changes automatically tracked, making it easier to monitor project status and keep all team members up to date [5]. Enterprise includes Web client support, a completely browser-based Web client that offers users access to work wherever and whenever they like. Enterprise Web Edition allows users to access StarTeam without installing a client application, increasing the client choices available to users.

Borland’s smallest, or standard version is referred to as StarTeam Standard. Standard is an easy-to-use, entry-level SCM tool with integrated components for file versioning, defect tracking, and threaded discussions [4]. StarTeam Standard helps small project workgroups and distributed teams to effectively collaborate and efficiently manage change across the application development lifecycle [4]. Some key development tasks with Standard are enhanced file check-in, checkout, and labeling. Also the ability to have change request functionality and able to roll back to previous file versions is a feature of StarTeam Standard. Standard promotes better team productivity, enabling rapid response to change and reducing the overall burden of development task management [5].

Borland also offers two other, smaller, software tools to aid in process and configuration management. StarTeam Server provides a unified collaborative process repository for all application development assets [4]. StarTeam Datamart is a decision support system, provides in-depth query and analysis capabilities for SCM and change management environments [4]. Datamart offers data into resource allocation, change requests, potential defects, and task management. Using this information, it is easier to monitor progress and perform accurate impact analysis for effective project management. Datamart includes three different features that can aid in the analysis of process and configuration management. Datamart Extractor takes the selected StarTeam project data and prepares it for analysis. Second, Datamart Synchronizer opens the Business Objects Universe and refreshes the data to reflect any field changes, thereby ensuring that the most up-to-date information is always used for analysis [5]. Third, Datamart Viewer provides easy access to any reports stored in StarTeam, and the ability to launch them in a reporting tool such as Business Objects and Crystal Reports [5].

Lucent Technologies Sablime

The Sablime Configuration Management System is a powerful and efficient tool that provides integrated version control and change management of your software artifacts such as source files and documentation [6]. Sablime is easy to learn and maintain, and can be used on any size project with any number of people. Sablime incorporates an effective development process that balances the needs of managers, developers, testers, and integrators, and provides workflow management so the stakeholders are kept informed [6]. Sablime provides version, configuration, and change management to help improve product quality and shorten release cycles. Sablime also supports concurrent development, so files can be worked on by two or more developers at the same time. This improves productivity of the development team. Sablime will automatically merge the files reliably and efficiently. Sablime was developed at Bell Labs and is a multiplatform software environment. It can be incorporated into many different development environments including Microsoft Visual Studio and Eclipse. Also, Sablime’s analysis tools can be incorporated into many different spreadsheet programs, notably Microsoft Excel and WordPerfect Quattro.

Sablime has a distinctive approach in the configuration management system by building its development process around the Modification Request. Any changes that are proposed by a team member or customer must be done by creating a MR. When MRs are reviewed they can be deferred, killed, assigned for study or accepted for implementation in one or more generics (codelines) [6]. Sablime tracks changes using MRs, and makes sure that all changes are delivered together at integration time. Program versions are assembled based on these MRs states. Each MR is assigned to one or more project members with appropriate priority and due date. If an MR requires extensive effort or spans several areas of responsibility, it can be divided into smaller requests and independently assigned, tracked and managed in different generics [6]. As an MR passes through its life cycle, appropriate project members are notified of events via email. This communication promotes clear responsibility and prompt, timely actions. Sublime keeps track of who, when, and why each action was taken [6].

Sablime has many different benefits in using it to help in the software configuration management process. Software changes constantly and is often difficult to manage. Different artifacts (such as source code, documentation, etc.) exist in different versions at different times [6]. Refer to the list below for benefits of using Sablime [6].

• Helps improve product quality and shortens release cycles

• Supports an out-of-the-box process model

• Tracks, coordinates and integrates product changes and change requests

• Helps prevent fixed bugs from getting reintroduced

• Enables you to reconstruct versions sent to customers

• Makes project status and source files accessible to all team members, even when geographically distributed

• Enables managers to control and characterize contents of each release, and track release status

• Enables testers to see what features or fixes are ready for testing, and to review the requirements, notes, and implementation associated with each change request

• Enables integrators to create consistent product versions automatically, based on readiness for integration

Sablime provides comprehensive configuration management and version control [6]. Refer to the table below for key features of Sablime [6].

• Coordinates change requests and actual changes

• Supports multiple active codelines (releases) per product

• Supports concurrent development, with less need for merging

• Detects dependencies automatically

• Integrates with the Eclipse development platform

• Integrates with Visual Studio and other IDE’s

• Integrates with Excel, enabling status reporting and management

• Guides teams to consistent results using defined roles and workflow with email notification

• Scales easily from small to large objects

• Supports local and web-based users

• Allows scripting and customization

• Easy to learn and use

• Simple to install and maintain

• Does not require dedicated hardware

• Available on UNIX, Linux, and Windows

Borland StarTeam and Lucent Technologies Sablime are just two tools that can be used for configuration and problem management, and both of these can be used in the software quality assurance process. They help improve the quality of software by able to organize everything about a development process (requirement, design, code, documentation) into one area. However, to complete software quality assurance on a system, some kind of testing needs to be implemented. There are many different testing/metrics tools out on the Web today, and I will discuss two of them: Programming Research QA C++ and Parasoft Jtest.

Programming Research QA C++

QA C++ quickly ensures code quality while enhancing productivity in the development process. QA C++ is fast, non-disruptive, easy-to-use and can be quickly integrated almost everywhere [7]. QA C++ provides a fully automated environment to introduce and enforce custom coding standards, those maintained by the software development company, and required by customers. QA C++ documents and proves compliance – a growing customer requirement especially for contract developers and offshore development firms [7]. QA C++ helps to ensure that coding standards that were set have been followed; a key in Software Quality Assurance.

QA C++ can parse any size project quickly and easily. QA C++ detects many different problems and defects like language implementation errors, inconsistencies, obsolescent features, and coding standard violations [7]. Detecting these early can prevent delays later in the development cycle where problems are costly to fix. QA C++ reports many industry-standard code metrics into graphs, diagrams, and HTML output. These metrics can be exported into an open format so they can be used in applications such as Microsoft Office and StarOffice to analyze, share, and present information.

QA C++ includes a huge knowledge base built around it. Any problems that are discovered are explained in a message browser with a drill-down environment. QA C++ explains why problems it discovers need to be corrected and then provides detailed examples of how to fix them [7]. QA C++ helps identify software defects and non-compliance issues early in the development cycle and prevents them from entering production code – thwarting potentially huge problems in your deployed products. QA C++ provides the ability to limit complexity so you can develop code that is truly testable and easier to maintain [7].

QA C++ is a market leader in analyzing source code and enforcing coding standards [7]. Refer to the table below for product highlights of QA C++ [7].

• Identifies coding problems early in the development cycle

• Accelerates the code review process – improves teamwork

• Ensures quality code and coding standard compliance

• Educates and raises programmer awareness

• Reduces the risk of program failure

• Enhances reliability, portability, and maintainability

• Lowers software development costs – increases productivity

• Improves time-to-market while reducing costs

• Allows instant and repeatable code audits and reviews

• Delivers unmatched technology & strong ROI

Parasoft Jtest

Jtest is an automated Java unit testing and coding standard analysis product that improves Java code reliability, functionality, security, performance, and maintainability. Jtest checks whether code compiles with over 500 built in development rules and corrects many reported violations automatically. User defined code guidelines (rules) can be produced without using code; they can be created graphically or automatically [8]. To expose reliability problems, Jtest examines each class, then generates and executes JUnit test cases designed to achieve high coverage and expose uncaught runtime exceptions. To expose functionality problems, Jtest provides fast and easy ways to add and execute realistic user-defined test cases – including a Test Case Sniffer that monitors a running application and generates JUnit test cases that capture application behavior [8]. To ensure continued functionality, Jtest’s automated regression testing identifies problems introduced by code modifications. Jtest improves Java code quality throughout the software lifecycle and improves productivity. Quality Assurance team members can use Jtest to identify critical problems before an imminent release/deployment deadline [8].

Jtest analyzes code in two different ways. First, Jtest verifies whether code complies with development rules for preventing functional errors, security vulnerabilities, performance problems, and pitfalls [8]. Many of these violations can be corrected automatically. Second, Jtest automatically generates and executes JUnit test cases designed to achieve high coverage, expose uncaught runtime exceptions and memory leaks, and describe the code’s current behavior [8]. Test findings are reported as a prioritized task list for quick review and response. Test cases can be added automatically (using Test Case Sniffer), graphically (using a graphical object editor and graphical test case editor), or programmatically (by modifying the generated JUnit test cases) [8].

Jtest’s Test Case Sniffer is the first technology that makes realistic functional unit test practical for development and Quality Assurance organizations [8]. Test Case Sniffer monitors a running application and automatically generates unit test cases that capture application behavior. Simply exercise the application functionality you want to test, and Test Case Sniffer automatically generates JUnit test cases with real data that represents the paths taken through the application [8]. The result is a library of test cases against which new code can be tested to ensure that it meets specifications and does not “break” existing functionality [8]. Jtest is multiplatform software that can be used on Windows 2000 and XP, Solaris, and Linux. Refer to the table below for benefits of Jtest [8].

• Improve code reliability, functionality, security, and performance quickly and painlessly

• Obtain instant expert feedback on code quality and potential defects

• Prevent code modifications from breaking previously-verified functionality

• Perform extensive testing/debugging and more time on creative tasks

• Perform extensive testing with minimal user intervention

• Reduce the risks that cause late, over-budget, incomplete releases

• Identify errors lurking in existing applications

• Optimize code review time

• Ensure that best practices are applied consistently and uniformly across the team

• Monitor overall project quality, specific project segments, and progress toward quality goals

Benefit of Software Quality Assurance in Projects

Software Quality Assurance is essential to every software development process. Without SQA, many development groups would not reach their release goals/deadlines, or having incomplete releases because they would be spending too much time revising requirements, design, code, and documentation of each portion of a project to match. Establishing certain standards (rules) lowers the time spent working on making every part of the project look the same. Also, it decreases the time from development to deployment. Reviewing these standards against what has been developed can help catch any errors before they become very costly to fix. These standards can be reused across many projects, which will improve the development time and lower costs of each project involved.

References

[1] Hower, Rick (2006). Software QA and Testing Resource Center. Updated April 2006. Web site:

[2] Software Quality Assurance, NASA.

Web site:

[3] Buchanan, Ian (2005). Borland StarTeam. Updated January 2005.

Web site:

[4] CM Crossroads. Borland StarTeam Product Review.

Web site:

[5] Borland Software Corporation. Borland StarTeam.

Web site:

[6] Lucent Technologies (2006). Sablime.

Web site:

[7] Programming Research, Inc (2003). QA C++ Data Sheet.

Web site: -

%20DATASHEET%20FEB05%20HQ.pdf

[8] Parasoft (2006). Jtest Data Sheet.

Web site:

ParasoftJtestDataSheet.587.pdf?path=/jsp/products/-quick_facts.jsp&product=Jtest

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download