IEEE-730-2014 - 4. Key concepts of Software Quality Assurance

4.1 Organizational management responsibility

Management support of the SQA function is essential for SQA processes to be effective. This support minimally includes:

• Management is familiar with and understands the SQA function’s purposes, concepts, practices, and needs.
• Management provides the SQA function with an appropriate level of skilled resources (people, equipment, knowledge, methods, facilities, and tools) in order to accomplish their project responsibilities.
• Management receives and acts upon information provided by the SQA function throughout the course of a project.

The organization’s management supports the SQA function by providing the outcomes listed in 5.3.1.2.

4.2 Organization and project relationship

Projects are performed by people working within an organization. The scope of an organization can range from a very small group that has only one project to a multi-national conglomerate with many divisions andmany projects within each division. SQA can be described from the perspective of an acquirer or supplier. Except as specifically noted otherwise, this standard describes the SQA function from the supplier viewpoint.

Although this standard is focused on the project, this standard refers to some organizational-level activities. This standard presumes that some organizational-level activities establish the SQA processes when, or prior to, initiating the project. That initiating activity is the subject of 5.3.1. Additionally, if the organization expects to execute more than one project, then other organizational-level activities are concerned with capturing lessons from the current project and ensuring that those lessons are available for subsequent projects.

Figure 1 illustrates the time-oriented relationship of work performed at the organizational-level versus work performed at the project level. The figure emphasizes that the scope of this standard is a single project. This project is performed within an organization.

The project is performed to accomplish the business goals of the organization. The organization sponsors the project by providing people and resources. The organization initiates the project and, during project initiating, work at the organization level overlaps with work at the project level. During the course of the project, the organization exercises oversight of the project. The organizational-level work of capturing and applying lessons learned occurs during execution and continues through closing.

This standard does not presume or impose any particular project life cycle model. This standard is applicable to all software life cycle models. However, this standard does acknowledge and use generally recognized project management practices as described in the PMBOK® Guide.

In addition to being executed by an organization, a project can be part of a product life cycle, as illustrated in Figure 2. The software project is completed upon the acceptance and release of the product. Subsequently, life cycle support activities are part of the product life cycle model. As post-release problems are reported, new software projects might be executed within the product life cycle support phase to produce new versions of the product, whose purpose could be to fix problems or introduce new features. Software projects can involve developing software, acquiring software, supporting software, or combinations of all three.

The project SQA role ends with project closing. The organization can provide post-project SQA support. The organization-level SQA can provide continuity and communications among many projects.

This standard may be applied at the organization or project level. Organizations that produce software strive to develop a set of software life cycle processes that are consistently applied to the organization’s projects. An organization may define appropriate procedures, practices, and policies that conform to this standard. A software project would then conform to the organization’s processes rather than directly to this standard. Organizations can create and structure their quality management system in various ways; the specific structure of this standard is not imposed.

A project may be executed in an organization that does not have an organizational-level set of software life cycle processes. Such a project may apply the provisions of this standard directly to the project.

4.3 Software quality and relationship to requirements

Quality is the entire set of attributes that gives a software product the ability to satisfy expressed or implied stakeholder requirements. These stakeholder requirements become refined into software requirements, including functional requirements and performance attributes that specify how well the software performs the functional requirements.

The project determines that requirements satisfy criteria such as clarity and testability and that those requirements accurately represent stakeholder needs, wants, and expectations. This standard defines software quality as conformance to requirements that have been established by the project.

This standard defines the scope of SQA as: 1. Assessing the software development process; 2. Evaluating the conformance to software processes; and 3. Evaluating the effectiveness of the software processes. These processes include those that identify and establish the software requirements, develop the software product, and maintain the software product.

Software quality is achieved by conforming to established requirements as described in 5.4.3. In some industries, conformance to requirements also includes the constraint that the product meets its intended use and is evaluated in the actual use setting. Because requirements are the basis for quality, the SQA function pays special attention to those software processes that evaluate and respond to the requirements as described in 5.5.2. Those project processes could utilize ISO/IEC/IEEE 29148:2011 [B43], which defines verification criteria for requirements including clarity, consistency, completeness, and verifiability. The ISO/IEC/IEEE 29148:2011 standard presents detailed attributes that projects can use for verifying requirements.

The SQA function confirms that the software processes can and do produce software that conforms to requirements. Confirming includes evaluating intermediate and final software products along with methods, practices, and workmanship. Evaluation further includes measurement and analysis of software process and product problems and related causes as well as recommendations about ways to correct current problems and prevent future problems.

Requirements can be categorized as either process or product requirements. Process requirements specify the processes the project will use to produce the project outcomes. Process requirements may include specific processes mandated to be in place, specific tasks the project or organization is mandated to perform, and the manner in which specific tasks are to be performed. Product requirements specify the functions that a product is mandated to perform and attributes that a product is mandated to possess. These attributes include performance attributes that specify how well the product should perform. Product requirements, also called system requirements, can be allocated to software or non-software aspects of the project. Both process and product requirements are derived from and are a response to stakeholder requirements.

Figure 3 shows the processes from ISO/IEC/IEEE 15289:2011 involved in requirements derivation. The figure illustrates the distinction among stakeholder, process, and product requirements, and shows the process flow leading to both process requirements and software requirements.

This standard presumes that the requirements derivation activities shown in Figure 3 are executed, but may not be fully completed prior to the start of the software project. The SQA activities and tasks in this standard presume that software requirements are established prior to being used by the SQA function to evaluate conformance of the software product to those requirements. There is no implication that all software requirements are established at any point in the project, only that the subset used for a particular conformance evaluation is established. This standard recognizes that some software projects include activities that create or refine requirements. Some projects are initiated with only an informal and incomplete set of requirements. Regardless of the starting point, requirements can be continually elicited, documented, validated, and implemented throughout the project.

The SQA function’s responsibility is to produce and collect evidence that forms the basis for giving a justified statement of confidence that the software product conforms to its established requirements. This responsibility is based on the definition of quality adopted by this standard: quality is conformance to established requirements. Assuring stakeholders that their expectations will be met is based on the conformance relationships illustrated in Figure 4 as well as additional SQA processes that determine the efficacy of the software processes.

The ISO/IEC/IEEE 12207:2008 standard, in 7.1.2, Stakeholder Requirements Definition Process, describes activities and tasks that produce requirements for the software project. Once reviewed and accepted by the project, these requirements become established requirements. The SQA function reviews established requirements as well as changes to established requirements.

4.4 Overview of SQA activities

Once the contract has been approved and the requirements established, the SQA function evaluates the compliance and conformance relationships shown in Figure 4. The scope of Figure 4 is the project.

Figure 4 shows how project artifacts are derived from the contract. Process and product requirements are derived from the contract. Software requirements are derived from product requirements; software products are based on software requirements. Similarly, the project’s processes and plans are derived from the established process requirements. The project’s activity execution is based on the processes and plans. Figure 4 also shows that the contract is required to comply with some artifacts external to the contract: rules, regulations, and laws.

Figure 4 serves two purposes: 1. The figure illustrates the two fundamental conformance categories of product assurance and process assurance; and 2. The figure shows the transitivity of the conformance relationships. That is, if process requirements conform to the contract, and the project’s processes and plans conform to process requirements, then the processes and plans conform to the contract. This simplifies SQA’s job. Each project artifact need not be checked against the contract. Each artifact need only be checked against its immediate predecessor.

Software product assurance activities check that software products conform to software product requirements, and that those software product requirements conform to system product requirements that in turn conform to stakeholder requirements as shown in the contract.

Similarly, process assurance activities check that the execution of project activities conform to the project’s processes and plans, that those processes and plans conform to process requirements, and that those process requirements conform to stakeholder requirements as shown in the contract. In addition, process assurance includes checking that the process requirements comply with any laws, regulations, and rules imposed on the project.

In Clause 5, this standard describes the activities for these two categories. In particular, the project’s Software Quality Assurance Plan (SQAP) defines the project’s product and process assurance activities, tasks, and outcomes, as described in 5.3.3.

4.5 Acquirer and supplier perspectives

For the purpose of this standard, software development is considered to be carried out by a supplier and delivered to an acquirer according to the terms of a contract.

During the course of the software development life cycle, the contract may evolve, as long as both acquirer and supplier agree to the changes. This aspect is especially prevalent when iterative or incremental development strategies are used.

4.6 Key concepts of this standard

4.6.1 Defining the software quality assurance role

The roles and responsibilities of an organizational unit called Quality Assurance (QA), Software Quality Assurance (SQA), Quality Engineering (QE), or some other variant, differ across industry sectors and businesses. They might perform software testing, product assurance, process assurance, or some combination of these, or SQA as defined in this standard. Organizational units with the name SQA might or might not carry out the SQA role as defined in this standard.

While this standard does not require the prescribed activities be executed by any specific organizational unit, it does require that clear responsibility be assigned to resources to perform the SQA activities described.

The resources carrying out the SQA function determine whether project activities were performed in accordance with the project’s processes and plans. Organizational Quality Management may provide oversight of the SQA function.

4.6.2 Software product risks

Software systems are increasingly developed to perform tasks that can cause harm to living things, physical structures, and the environment. A fundamental principle of this standard is to first understand software product risk and then to ensure that the planned SQA activities are commensurate with product risk. This means that the breadth and depth of SQA activities defined in the SQA Plan are determined by and derived from software product risk.

Two techniques for addressing software product risks are discussed in Annex I.

4.6.3 Relationship between systems and software

This standard recognizes that software is part of a system. It is based upon the general principles of systems engineering. The allocation of functions to hardware or software is a design decision that occurs within the overall system development process. The SQA function includes aspects of the process leading to such allocation decisions. Software is treated as an integral part of the total system and performs certain functions in that system. Software is implemented by deriving the software requirements from the system requirements and design, producing the software, and integrating it into the system. A fundamental premise of this standard is that software always exists within the context of a system, even if the system consists of only the processor upon which the software is executed. Therefore, a software product is always treated as one item in a system. If a product is software-only, it would still be considered a system.

4.7 Software process improvement

This standard aids an organization’s software process improvement efforts. The measurement activities and tasks can supply objective data to an organization’s Life Cycle Management Process (6.2 of ISO/IEC/IEEE 12207:2008). Additionally, at all occurrences of evaluating the conformance and effectiveness of processes, improvement opportunities can be identified and reported to the organization’s process improvement function. Similarly, evaluating software products for conformance can identify improvement opportunities.

An organization should base process improvement efforts on the results of in-process as well as completed projects, gathering lessons learned, and the results of ongoing SQA activities such as process assessments and reviews. Process documentation should be updated to reflect the identified improvements.

Historical, technical, and evaluation data should be collected and analyzed to gain an understanding of the strengths and weaknesses of the employed processes. These analyses should be used as feedback to improve these processes, to recommend changes in the direction of ongoing or subsequent projects, and to determine technology advancement needs.

Quality cost data should be collected, maintained, and used to improve the organization's processes as a management activity. This data serves the purpose of establishing the cost of both the prevention and resolution of problems in software products, documentation, and services.

4.8 Maintaining quality management system consistency

The SQA processes described in this standard are intended to provide evidence that serves as the basis for a justified statement of confidence that the quality of the organization's (or the project's) quality management system can produce software that conforms to established requirements.

The ISO 9001:2008 standard specifies requirements imposed on quality management systems. Guidance for the application of ISO 9001 to software can be found in IEEE Std 90003™-2008. The ISO/IEC/IEEE 12207:2008 standard, in 6.2.5, provides a reference model for the activities of a quality management system.