IEEE-730-2014 - 5. Software Quality Assurance process

5.1 Purpose

This standard defines the purpose of SQA processes as: to specify the activities and tasks that enable software suppliers to produce, collect, and validate evidence that forms the basis for a justified statement of confidence that the software product conforms to its established requirements.

SQA ensures that processes are established, managed, maintained, and applied on projects by skilled and qualified staff and that the activities and tasks performed are commensurate with product risk.

5.2 Organization of SQA process outcomes

The outcomes produced by SQA processes are organized into three groups:

The 16 SQA activities are organized into three groups as shown in Table 1, below:

Table 1 — Organization of 16 SQA activities

Subclause Title
5.3 SQA process implementation activities
▻ 5.3.1 Establish the SQA processes
▻ 5.3.2 Coordinate with related software processes
▻ 5.3.3 Document SQA planning
▻ 5.3.4 Execute the SQA Plan
▻ 5.3.5 Manage SQA records
▻ 5.3.6 Evaluate organizational independence and objectivity
5.4 Product assurance activities
▻ 5.4.2 Evaluate plans for conformance to contracts, standards, and regulations
▻ 5.4.3 Evaluate product for conformance to established requirements
▻ 5.4.4 Evaluate product for acceptability
▻ 5.4.5 Evaluate product life cycle support for conformance
▻ 5.4.6 Measure products
5.5 Process assurance activities
▻ 5.5.2 Evaluate life cycle processes and plans for conformance
▻ 5.5.3 Evaluate environments for conformance
▻ 5.5.4 Evaluate subcontractor processes for conformance
▻ 5.5.5 Measure processes
▻ 5.5.6 Assess staff skill and knowledge

NOTE—5.4.1 and 5.5.1 contain introductory material concerning product assurance and process assurance, respectively, and are not numbered in the 16 SQA activities.

As specified in 1.7, each of the activities in Clause 5 of this standard is described by the following information:

a) Reference to ISO/IEC/IEEE 12207:2008 clause. The text of the ISO/IEC/IEEE 12207:2008 subclause relevant to the activity is presented in a boxed figure, for example:

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

b) Purpose. Defines the activity’s intention (e.g., “Determine the degree to which the software products and related documentation conform to established requirements.”).

c) Outcomes. Specific observable results of the successful achievement of activity’s purpose. An outcome may an information item (e.g., records, documents), a change in the state or an attribute of an information item, a change to a project constraint, or a change to an attribute (e.g., training, experience, capability) of a project team member. Information items are defined in ISO/IEC/IEEE 15289:2011. These outcomes are the basis for validated evidence to provide a justified statement of confidence in the quality of the software products. Outcomes are written as declarative sentences in the present tense (e.g., “Non-conformances are raised when software products do not conform to established software requirements.”)

d) Tasks. Specific actions that are intended to contribute to the achievement of one or more of the stated outcomes of the activity. Using the definitions from ISO/IEC TR 24774:2010, a statement in the task description can be:

  1. A required action,
  2. A recommended action, or
  3. A permissible action.

Task statements identify the role performing the task. The role is either the subject of the sentence or is in a clause that introduces a set of tasks. Task statements for required actions begin with an action verb and are written as declarative sentences in the present tense. Recommended and permissible action statements start with a phrase or clause that qualifies the conditions under which the task may be performed. There is not necessarily a one-to-one relationship between tasks and outcomes (e.g., identify software products and related documentation required by the contract).

5.3 SQA process implementation activities

5.3.1 Establish the SQA processes

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclauses:

5.3.1.1 Purpose

Define and establish documented SQA processes that exist separately from projects. When these SQA processes are applied to a project they enable the development of software that conforms to established requirements. These processes also provide projects with software quality measurements to help make cost, schedule, quality, and risk tradeoffs. The SQA processes define the SQA function's role, concepts, methods, procedures, and practices.

Effective SQA processes identify what activities to do, how to confirm the activities are performed, how to measure and track the processes, how to learn from measures to manage and improve the processes, and how to encourage using the processes to produce software products that conform to established requirements. SQA processes are continually improved based on objective measures and actual project results.

5.3.1.2 Outcomes

This activity shall produce the following outcomes, prior to executing the project:

5.3.1.3 Tasks

To accomplish this activity, the organization shall perform the following tasks: 1. Define an organizational quality policy statement that defines and governs SQA roles and responsibilities. 2. The organizational quality policy statement may be included in an organization Quality Manual for those organizations that have an existing quality management system. 3. Establish an organizational quality policy statement that defines the SQA process as an organization level process independent of SQA processes established for specific projects. 4. Create and assign tasks to those responsible for SQA activities and for implementing the organizational quality policy statement. 5. Define a mechanism for providing management oversight of SQA activities and tasks. 6. Assign responsibility for project SQA activities to individuals who are organizationally independent of project management and software development. 7. Review the organizational quality policy and identify gaps and inconsistencies between those organizational quality policies and proposed SQA roles and responsibilities. 8. The SQA process should be established prior to the acquisition phase, when contracts aretypically negotiated and agreements reached, so that the SQA function can assist in forming effective contracts.

NOTE: Table A.1 in Annex A shows the correlation of four outcomes as described in ISO/IEC/IEEE 12207:2008 and the clauses of this standard.

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.3.2.1 Purpose

Coordinate with the verification, validation, review, audit and other ISO/IEC/IEEE 12207:2008 processes that are relevant to the project to achieve project objectives. The SQA function works with project management to determine which of the other 39 processes defined in ISO/IEC/IEEE 12207:2008 should be coordinated with SQA activities.

The SQA function identifies project and organizational roles and responsibilities with respect to all of the following ISO/IEC/IEEE 12207:2008 processes and clauses that apply to the project:

At the organizational-level, the SQA function identifies organizational roles and responsibilities with respect to the following organizational project-enabling processes:

NOTE: Figure A.1 in Annex A shows the relation of the SQA function to all of the ISO/IEC/IEEE 12207:2008 processes.

5.3.2.2 Outcomes

This activity shall produce the following outcomes:

5.3.2.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Collaborate with other project and organizational elements to define roles and responsibilities with respect to all of the ISO/IEC/IEEE 12207:2008 processes listed above that are applicable to the project.
  2. Document the defined roles and responsibilities based on this collaboration in the SQA Plan (SQAP) (see 5.3.3).

5.3.3 Document SQA planning

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.3.3.1 Purpose

Document SQA activities, tasks, and outcomes that are commensurate with the software risk for a specific project. Project-specific SQA activities and tasks are derived from the “Coordinate with related software processes” activity described in 5.3.2 as well as from the organizational quality management plan. the SQA function’s planning includes adapting the generic SQA processes to the specific needs of the project in a manner commensurate with product risk. The Software Quality Assurance Plan (SQAP) documents the results of SQA planning.

5.3.3.2 Outcomes

This activity shall produce the following outcomes:

5.3.3.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Work with all stakeholders to determine which subclauses of this standard are relevant for the project.
  2. When the applicable subclauses are identified, use this standard and Annex C to help prepare an SQAP that is appropriate for the project, addresses the needs of all stakeholders, and is commensurate with product risk.
  3. Use outcomes of 5.3.2 to help prepare the SQAP in a manner that reflects the coordination with other processes areas.
  4. Review and update the SQAP as the project evolves.
  5. Present status information to Management in the agreed manner.
  6. Estimate the SQA function’s resources (including effort, schedule, people, required skills, tools, and equipment) needed to perform the SQA activities, tasks, and outcomes.
  7. Analyze the project and adapt SQA activities accordingly so they are commensurate with risk.
  8. Analyze product risks, standards, and assumptions that could impact quality and identify specific SQA activities, tasks, and outcomes that could help determine whether those risks are effectively mitigated.
  9. Define specific measurements for assessing project, software quality, and the SQA function’s performance against project and organization quality management objectives.
  10. Identify and track project changes that require further SQA function planning, including changes to: requirements, resources, schedules, project scope, priorities, and product risk.
  11. If process areas or activities are not adequately addressed by the organizational quality management function, or if there is no organizational quality management function, then these process areas may be in the SQAP.
  12. Address all of the topics listed in the normative SQA plan outline shown in Figure 5. Every section in the plan is to be included. The topics of the SQAP are normative but the section names and order are informative. If a section in the outline is not applicable on a given project, a placeholder for that section may be included along with a justification for why the topic is not applicable.
  13. If there is an organizational SQAP, or if organizational processes will be used directly, the SQAP may refer to those items. Additional sections and appendices beyond those listed below may be added.

NOTE: Refer to Annex C for additional guidance in preparing an SQAP.

NOTE: Specific content for quality records and reports are defined in ISO/IEC/IEEE 15289:2011.

5.3.4 Execute the SQA Plan

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.3.4.1 Purpose

Execute the SQAP in coordination with the project manager, the project team, and organizational quality management.

5.3.4.2 Outcomes

This activity shall produce the following outcomes:

5.3.4.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks: 1. Execute the activities and tasks defined in the SQAP, based on project schedules. 2. Create the outcomes identified in the SQA Plan. 3. Revise the SQAP in response to project changes. 4. Raise non-conformances when actual outcomes do not agree with expectations.

5.3.5 Manage SQA records

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.3.5.1 Purpose

Create records of SQA activities, outcomes, and tasks; manage and control these records; make these records available to project stakeholders.

5.3.5.2 Outcomes

This activity shall produce the following outcomes:

5.3.5.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks: 1. Create records as required by the SQAP. These records capture findings of SQA activities and tasks and provide evidence that SQA activities and tasks were performed. 2. Maintain records according to trustworthiness, security, and privacy requirements. 3. Identify records of quality assurance activities as accessible, deliverable, or internal use only to avoid contract non-compliance or inadvertent transfer of intellectual property. 4. Maintain the integrity of the SQA function’s records through a document control system to prevent their modification or inadvertent removal and release. 5. Supply specific records to authorized stakeholders defined in the contract. Records are made available subject to confidentiality and other constraints. The contract specifies what the acquirer will receive from the supplier; the SQAP specifies which SQA function records the acquirer’s internal organizations will receive.

5.3.6 Evaluate organizational independence and objectivity

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.3.6.1 Purpose

Determine whether those persons responsible for performing the SQA function have a position within the organization that provides an unimpeded communication mechanism with organizational management. Also, determine whether those persons have the resources and authority to make objective evaluations, initiate, effect, and verify problem resolutions.

5.3.6.2 Outcomes

This activity shall produce the following outcomes:

5.3.6.3 Tasks

To accomplish this activity, the following tasks shall be performed:

  1. The organization identifies the individuals and organizational entities responsible for the SQA function of the product and project.
  2. The organization, including SQA, determines whether those responsible for the SQA function have the organizational freedom to perform that activity in a manner that permits objective evaluations

5.4 Product assurance activities

5.4.1 Defining product assurance

An important aspect of SQA is the establishment of confidence in the quality of the software products produced by the project. The products are the software and related documentation. Related documentation may include all those documents associated with the development, operation, support, maintenance, and retirement of the software, including installation and administration. A product may also be a software service provided to the acquirer.

The outcomes of the product assurance activities provides evidence that the software services, products, and any related documentation are identified in and comply with the contract and any non-conformances are identified and addressed. Product Assurance comprises these activities:

The SQA function confirms that software products are in conformance with established requirements. That is, SQA enables software suppliers to produce, collect, and validate evidence that forms the basis for a justified statement of confidence that the software product conforms to its established requirements. For example, Product Assurance activities may include SQA personnel participating in project technical reviews, software development document reviews, and software testing.

5.4.2 Evaluate plans for conformance to contracts, standards, and regulations

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.4.2.1 Purpose

Determine whether plans required by the contract are documented. Determine whether plans conform to the contract. Determine whether plans required by the contract comply with applicable laws, regulations, and rules. Determine whether all the plans taken as a whole are consistent with each other.

5.4.2.2 Outcomes

This activity shall produce the following outcomes:

5.4.2.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Analyze the contract to identify the plans required by the contract.
  2. Evaluate (validate, verify, or review) project plans for conformance to the contract, determine whether plans conform to the established process requirements.
  3. Raise non-conformances when actual results do not agree with expectations.
  4. Evaluate (validate, verify, or review) plans for mutual consistency.

5.4.3 Evaluate product for conformance to established requirements

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.4.3.1 Purpose

Determine the degree to which the software products and related documentation conform to established requirements.

5.4.3.2 Outcomes

This activity shall produce the following outcomes:

5.4.3.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Identify software products and related documentation required by the contract.
  2. Identify the requirements allocated to the software products and related documentation. Evaluate (validate, verify, or review) the results of the allocation process.
  3. Evaluate (validate, verify, or review) software products for conformance against the established software requirements.
  4. Evaluate (validate, verify, or review) related documentation for conformance against the established software requirements.

5.4.4 Evaluate product for acceptability

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.4.4.1 Purpose

Prior to delivery, determine the degree of confidence the supplier has that the established requirements are satisfied and that the software products and related documentation will be acceptable to the acquirer. Collect measurement data sufficient to support these satisfaction and acceptability decisions. A contract may provide that the acquirer, prior to delivery, determine whether software products are acceptable.

5.4.4.2 Outcomes

This activity shall produce the following outcomes:

5.4.4.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Identify criteria for product acceptance. These conditions may be derived from the contract, project plans, documentation, SQA reports, and other sources.
  2. Determine whether the product conforms to the contract using techniques that include: reviewing, auditing, testing, or evaluating the results of these techniques.
  3. Determine whether the product conforms to the documented acceptance requirements.
  4. Determine whether acquirer has the means to determine the criteria to accept the product.

5.4.5 Evaluate product life cycle support for conformance

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.4.5.1 Purpose

Determine whether the product support requirements identified in the project plans are consistent with the contract and clearly identify the responsibilities of both the product delivery organization and the acquirer.

5.4.5.2 Outcomes

This activity shall produce the following outcomes:

5.4.5.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Analyze the contract to identify the required level of customer support.
  2. Review customer support plans and determine whether the level of customer support is consistent with contractual support requirements.
  3. Identify SQA requirements regarding software customer support and include them in the SQAP.
  4. Monitor software customer support activities and raise non-conformances when these activities are not performed as defined in the plan.
  5. Measure and evaluate, on a regular basis, the level of support and cooperation with respect to product support plan and identify issues.

5.4.6 Measure products

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.4.6.1 Purpose

Determine whether product measurements demonstrate the quality of the products and conform to standards and procedures established by the project.

5.4.6.2 Outcomes

This activity shall produce the following outcomes:

NOTE: Additional information about measurement can be found in IEEE 15939-2008

5.4.6.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Identify the standards and procedures established by the project or organization.
  2. Determine whether proposed product measurements are consistent with standards and procedures established by the project.
  3. Determine whether the proposed product measurements are representative of product quality attributes.
  4. Analyze product measurement results to identify gaps and recommend improvements to close gaps between measurements and expectations.
  5. Evaluate product measurement results to determine whether improvements implemented as a result of product quality measurements are effective.
  6. Analyze product measurement procedures to confirm they are sufficient to satisfy the measurement requirements defined in project’s processes and plans.
  7. Perform Task 1 through Task 6, above, for software products developed by all subcontractors.

5.5 Process assurance activities

5.5.1 Defining process assurance

The outcomes of SQA process assurance activities make certain that the processes used to develop, install, operate, and maintain software conform to the contract, comply with any imposed regulations, and are adequate, efficient, and effective. “Adequate, efficient, and effective” means that the processes can, and do, consistently produce software that conforms to established requirements and organizational considerations. Depending on life cycle model processes, these outcomes are reported to project management, quality management, and risk management.

Process Assurance comprises these activities:

5.5.2 Evaluate life cycle processes and plans for conformance

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclauses:

5.5.2.1 Purpose

Determine whether the project life cycle processes and plans conform to the established process requirements. Determine whether execution of software activities conform to the project’s processes and plans.

5.5.2.2 Outcomes

This activity shall produce the following outcomes:

5.5.2.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Identify applicable process requirements that may affect the selection of a software life cycle process.
  2. Determine whether the defined software life cycle processes selected by the project team are appropriate, given the product risk.
  3. Review project plans and determine whether plans are appropriate to meet the contract based on the chosen software life cycle processes and relevant contractual obligations.
  4. Audit software development activities periodically to determine consistency with defined software life cycle processes.
  5. Audit project team periodically to determine conformance to defined project plans.
  6. Perform Task 1 through Task 5, above, for subcontractor’s software development life cycle.

5.5.3 Evaluate environments for conformance

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.5.3.1 Purpose

Determine whether software engineering environments (SEE) and software test environments (STE) conform to project processes and plans.

5.5.3.2 Outcomes

This activity shall produce the following outcomes:

5.5.3.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Review the software engineering environments used by the project team to determine whether they conform to the contract.
  2. Review the software engineering libraries used by the project team to determine whether they conform to the contract and project plans.
  3. Review the software test environments used by the project team to determine whether they conform to the contract and project plans.

NOTE: SEEs and STEs often include software development tools. Such tools may require validation or evaluation as determined by the contract or industry regulations.

5.5.4 Evaluate subcontractor processes for conformance

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.5.4.1 Purpose

Determine whether the subcontractor’s software processes conform to process requirements that have been allocated from the acquirer.

5.5.4.2 Outcomes

This activity shall produce the following outcomes:

5.5.4.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Identify process requirements from the supplier and, if appropriate, from the acquirer that are allocated to the subcontractor.
  2. Determine whether subcontractor software processes are defined.
  3. Determine whether subcontractor software processes conform to project processes and plans.

5.5.5 Measure processes

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.5.5.1 Purpose

Determine whether the process measurements support effective process management and conform to the project’s processes and plans and conform to established standards and procedures.

5.5.5.2 Outcomes

This activity shall produce the following outcomes:

NOTE: Additional information about measurement can be found in IEEE 15939-2008

5.5.5.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Identify the standards and procedures established for the project or organization.
  2. Evaluate whether the process measurement activities are being executed in conformance to the project’s processes and plans.
  3. Evaluate whether the process measurement activities conform to standards and procedures established by the project and the organization.
  4. Analyze process measurement procedures to confirm they are sufficient to satisfy the measurement requirements defined in project’s processes and plans.
  5. Review the process measurement plan to determine whether the SQA function’s measurement needs are addressed.
  6. Analyze process measurement procedures to confirm they are sufficient to satisfy the measurement requirements defined in project plans and the contract.
  7. For each subcontractor, perform Task 1 through Task 6, above, for each of their software processes.

5.5.6 Assess staff skill and knowledge

This subclause addresses the following ISO/IEC/IEEE 12207:2008 subclause:

5.5.6.1 Purpose

Determine whether staff assigned to the project have the required knowledge, skill, competencies, and abilities to perform the tasks required for their roles.

5.5.6.2 Outcomes

This activity shall produce the following outcomes:

5.5.6.3 Tasks

To accomplish this activity, the SQA function shall perform the following tasks:

  1. Audit the skill and knowledge needs of the project and compare to skill and knowledge of the organization’s staff and identify any gaps.
  2. Determine whether skills and knowledge development plans are in place and are being executed to fill the identified gaps and to accomplish required knowledge transfer.
  3. Determine whether the skills and knowledge needs of the project have changed.
  4. Determine whether new team members are assessed and that all prepared individual skill and knowledge development plans are being monitored and tracked to completion.
  5. Review personnel training records on a regular basis.