2.5.14  Quality Assurance (QA)

2.5.14.1  (06-30-2010)
Introduction

  1. This IRM provides the framework for conducting Quality Assurance (QA) Audit activities within Applications Development (AD). It establishes a standard context for Project Teams, including contractors working for the Applications Development organization, to participate in the QA audit process. This IRM establishes:

    • the overall approach to Quality Assurance;

    • the applicable Quality Assurance standards; and

    • the reporting and control requirements for the QA program as outlined by the AD QA Directive and related processes and procedures.

  2. The Quality Assurance (QA) Program Office supports the delivery of high-quality products and services by ensuring that projects implement a coordinated set of activities that conform to organizational policies, processes and procedures.

  3. Quality Assurance is a systematic, planned set of activities necessary to provide adequate confidence that the product conforms to stated customer requirements. The activities are designed to evaluate the processes (i.e. Project Planning, Project Monitoring and Control, Requirements Management, etc.) by which products are developed.

2.5.14.2  (03-05-2008)
Quality Assurance Auditing Process

  1. QA Audit Process and Procedures are used to objectively and independently evaluate adherence of the process and work products to applicable directives, processes, standards, procedures, and guidelines. The objectives of the audit process are to:

    • identify and track noncompliance instances;

    • communicate and facilitate the resolution of noncompliance issues;

    • identify and communicate, to senior management, best practices and opportunities for improvement;

    • document Quality Assurance activities; and

    • report quality issues to relevant stakeholders.

  2. Benefits of the audit process are realized through:

    • consistency in assessing use of organizational processes;

    • improved compliance with enterprise-wide standards and directives;

    • facilitation of improvements; and

    • enhanced planning and resource allocation capability.

2.5.14.2.1  (06-30-2010)
Auditing Process Roles and Skills

  1. To meet the objectives and realize the benefits of QA auditing, the following roles and skill sets are needed to perform QA auditing activities.

    Roles Responsibilities
    AD Quality Assurance Program Office (Quality Specialist) Subject matter expertise on the objectives, procedures, and methods for performing the audit tasks, in order to:
    • Establish and maintain the AD QA Quality System artifacts (plans, procedures, templates, checklists, project folders, etc.)

    • Establish and clearly state criteria for audits on QA website

    • Establish and maintain the AD portfolio project database

    • Establish and maintain the Quality Assurance Training Materials

    • Establish and maintain a Master QA Audit Calendar

    • Compile and report quality information and trends

    • Use the stated criteria to audit performed processes for adherence

    • Maintain an audit and/or review calendar

    • Identify each non-compliance found during the audit/release review

    • Train and mentor project staff

    • Conduct audit orientations

    • Document audit findings

    • identify quality trends and make recommendations on all organizational levels

     
    Domain Director High level knowledge of the QA objectives in order to ensure Domain compliance with the applicable IRS directives, processes and procedures. Duties include:
     
    • Concurrence on Audit Calendar

    • Review Quarterly Domain Director Report and correct Domain level systemic problems identified

    • Ensure resolution of escalated of non-compliances

     
     
     
     
    Domain POC Working knowledge of the objectives, procedures, and methods for performing the audit tasks, in order to endure project compliance with the applicable IRS directives, processes and procedures. Duties Include:
    • Act as liaison between the Domain and AD QA.

    • Concurrence on Audit Calendar, if delegated by the Domain Director

    • Coordinate Domain level QA activities, at both the project and Division levels

    • Correct Domain level systemic problems identified during audits

    • Reconcile project inventories

    • Participate in audit opening and closing meetings, as appropriate

    • Oversee the resolution of audit non-compliances

    • Coordinate training for project staff to correct trends identified in the Domain Director Report

    • Attend Domain POC training and quarterly AD QA Domain POC meetings

     
    Project Manager/Branch Chief Overall responsible for quality on the project. They should have a working knowledge of the objectives, procedures, and methods for performing the audit tasks, in order to endure project compliance with the applicable IRS directives, processes and procedures. Duties Include:
    • Review audit report and provide project resources to correct non-compliances in the required timeframes

    • Participate in the opening and closing meetings

    • Identify the Quality Analyst(s) for the project

    • Attend QA Training, as appropriate.

     
    Project Team (Quality Analyst) Working knowledge of the objectives, procedures, and methods for performing the audit tasks, participating in normal quality control (i.e. development and review of program and project documentation, peer reviews of source code, unit testing, etc.) lifecycle tasks. The designated project Quality Analyst will:
    • Act as a liaison between the project and AD QA to coordinate the audit and document reviews

    • Ensure AD QA has access to the project's repository and/or artifacts

    • Oversee the Correction Action Plan process for the project(s)

    • Convey trends/problems to the Project Manager and/or Branch Chief

    • Monitor open non-compliances to ensure that they are closed in the required timeframes

    • Attend QA training

     

2.5.14.2.2  (06-30-2010)
Audit Process Control

  1. The audit process is controlled and driven by the Audit Checklists. The checklists are a series of questionnaires used to gather data, for the processes and products being audited, to evaluate the project’s level of compliance. The checklists can be accessed on the Intranet at http://mits.web.irs.gov/AD/PMO/QA/audit_checklist.html. There are seven process areas and three types of audits currently covered by the AD QA audit process. The process areas are:

    • Project Planning

    • Project Monitoring and Control

    • Configuration Management

    • Requirements Management

    • Measurement and Analysis

    • Software Development

    • Testing

    • Supplier Agreement Management

    The audit types are:

    • Process Audit - Evaluates a process area, in depth, based on organizational standards and requirements. This type of audit includes the procedures, requirements, and standards for the development lifecycle, project management, and support processes. Process audits are based on where a project is in the development lifecycle; however, the auditor can look at previous phases for adherence

    • Work Product Audit - Reviews work products for conformance to the Enterprise Life Cycle (ELC), Data Item Descriptions (DID) and templates. Work Products include all work products and deliverables, as well as the standards and/or procedures used to produce them. This type of audit is either performed as the project completes it during the lifecycle or in conjunction with the QA Release Review.

    • Release Review Audit - Evaluates a sampling of the processes used and work products produced during a given release. These audits can be seen as a combination of the process and work product audit that is normally performed at the end of a scheduled release for projects that have an accelerated release schedule.

2.5.14.2.3  (03-05-2008)
Auditing Process Tasks and Flow

  1. The following tasks constitute the flow of the QA auditing process:

    • Plan to Audit

    • Perform the Audit

    • Manage Documents

    • Monitor and Control Audit Findings

2.5.14.2.3.1  (06-30-2010)
Plan to Audit

  1. This task is performed by the QA Specialists in conjunction with the overall project level audit planning activities. Project teams and Management support QA Audit planning activities. The following steps describe the audit planning activities:

    1. Prioritize Programs and Projects for Audit Activities. The QA Specialist develops a prioritized audit plan based on the goals of the strategic plan, annual priorities and weighted project risk factors.

    2. Create Domain and Master Audit Calendar. The QA Specialist, in cooperation with the Domain Point of Contact and/or Project Quality Analyst, develops the AD Domain Audit Calendar, which is incorporated into the QA Master Audit Calendar.

    3. Notify stakeholders of final Audit Calendar. The QA Specialist shares the final Calendar with all stakeholders.

2.5.14.2.3.2  (06-30-2010)
Perform the Audit

  1. This task is performed in accordance with the Master Audit Calendar. The following activities occur during this task:

    1. Projects are formally notified of the audit and access to the project repository is requested.

    2. The QA Specialist reviews the Project Repository and makes a preliminary assessment of the quality status of the project.

    3. The Opening Audit meeting is conducted to establish the scope, objectives and observations from the preliminary assessment.

    4. The Quality Specialist, guided by the appropriate Audit Checklists, will evaluate the level of compliance with standards and processes as well as evaluate associated work products by reviewing instructions and procedures, checking records and through observation.

    5. If necessary, project staff will be interviewed to address questions that arise during the checks and observations.

    6. Subsequent to the observation and interviews a closing meeting is held to discuss audit findings and to allow the auditee to provide factual corrections and explanations, prior to issuance of the final audit report.

    7. The results of the observations, checks and interviews are documented in the Audit Findings Report and delivered to the auditee.

    8. The final Audit Findings Report is issued and stored in the appropriate QA repositories.

    9. When audit findings require corrective action, a Corrective Action Plan is requested, from the auditee, as a part of the Audit Findings Report.

2.5.14.2.3.3  (06-30-2010)
Manage Documents

  1. This task occurs after an audit is conducted, or may occur anytime a document is established or content/information is gathered relating to the audit process. The following activities occur during this task:

    1. The QA Specialist will use the information from the Audit Findings Report and any subsequent meeting minutes, phone conferences, etc., to update the AD QA database with the results of the audit or other record (e.g., the audit results worksheet, AD QA Checklists) of QA audit activities.

    2. All QA documents generated and/or received as a result of an audit are collected and stored in soft and/or hard copy in the appropriate repository.

    3. QA documents (i.e., audit evidence and quality system processes and procedures) are retained for at least six years, after baselined.

    4. Changes to QA process assets (i.e., processes, procedures, templates) shall be controlled in accordance with the AD QA Change Control Procedure located in the QA Quality System shared repository.

  2. As part of this task, when new QA documents are developed and/or change requests have been received to modify existing documents, the following activities are executed:

    1. A sequential Change Request (CR) number is assigned to each CR received and the request in recorded in the QA CR Log.

    2. The CR is forwarded to the Change Control Board (CCB) for disposition. The Chief, AD QA and Senior AD QA Management Analysts serve as the QA CCB.

    3. The AD QA CCB meets monthly and reviews any Change Requests received, in accordance with the AD QA Change Control Procedure. For Process Asset Integration Group assets, the approval process starts with the CCB and then must be approved by the AD, Program Management Office and the AD, Associate Chief Information Officer.

2.5.14.2.3.4  (06-30-2010)
Monitor and Control Audit Findings

  1. Using the data in the QA data repositories and Audit Calendars, the QA Specialist tracks audit findings to closure and, if applicable, schedules a follow-up audit. If noncompliance (corrective) actions are not completed by the resolution date as outlined in the approved corrective action plan, the QA specialist escalates the unresolved findings to senior management.

  2. The Quality Specialist responsibilities are to:

    • Verify resolution of the corrective action.

    • Update AD QA Database to indicate the status of the audit findings.

    • Identify corrective actions that remain unresolved five days after planned resolution date.

    • Prepare AD QA Escalation Reports.

    • Perform trend analysis activities.

    • Prepare audit data and status reports for review activities.

    • Place reports in the appropriate repository.

  3. When all findings are resolved, the auditee is notified that the audit is closed and all data repositories, i.e., AD QA database, project profiles, etc., must be updated.

2.5.14.2.4  (03-05-2008)
Audit Process Management

  1. AD Quality Assurance Program Office will regularly maintain measurements on the status and progress of Quality Assurance tasks for the AD portfolio. Process trends shall be analyzed for efficiency and effectiveness.

  2. Data will be compiled to develop and report trends in performance and compliance. The reporting will occur through performance trends metrics and compliance metrics. Performance and compliance trends metrics will be reported:

    • At the AD portfolio, domain, and project levels; and

    • For all process areas (by AD portfolio, domain, project levels).

2.5.14.2.5  (03-05-2008)
Audit Process Review

  1. On a defined, regular and event-driven basis, the QA Program Office will review the progress and status of activities performed to accomplish the tasks covered by the QA Auditing Process and Procedures.

  2. At least quarterly, Program Management will report to senior management on the overall status of tasks covered by this process.

2.5.14.3  (06-30-2010)
Waiver Justification Procedure

  1. The purpose of the waiver justification is to uniformly track any project team’s deviation from published standards and to have a formal, documented agreement between the process owner, AD QA, and the project team on deviations from accepted standards. The objectives of this procedure are to:

    • Establish a detailed audit trail of what standards have been changed, the justification for the change, and a formal approval process for the change; and

    • Establish a consistent and proactive approach across process areas for deviations from standards.

  2. During the audit process, AD QA will ensure that waivers and tailoring across process areas are properly documented, evaluated, and justified.

  3. Whenever it is determined that the need to deviate from an established standard exists, a Waiver Justification meeting, with the process owner and relevant stakeholders, should be initiated to discuss the proposed waiver. If all parties agree to the waiver, the Project Manager completes the Waiver Justification Form template and submits the form to the Process Owner and to the AD QA Specialist for signature. The signed form is forwarded to the Domain Director for approval and signature.

  4. If the Process Owner disagrees with the need for the waiver, the Process Owner documents the reason(s) in the Waiver Justification Form and returns the form to the Project Manager and the AD QA Specialist. If necessary, the Project Manager can escalate to he Domain Director approval.

  5. If the AD QA Specialist disagrees with the need for the waiver, the AD QA Specialist documents the reason(s) in the Waiver Justification Form, notifies the QA Program Office Chief and obtains concurrence on the decision, and returns the documented waiver form to the Process Owner and the Project Manager. If necessary, the Project Manager can escalate to both the Domain Director for approval to proceed.

  6. The Project Manager communicates the final decision of all waivers to all of the affected stakeholders. The completed waiver justification form is filed in the project notebook (or filed with other project documentation).

Exhibit 2.5.14-1 
Waiver Justification Form

Domain Name:  
Project Name:   Release #:  
Waiver Submit Date:  
Project Manager:  
Release Manager:  
Domain Director:  
Organization Process:  
Proposed Waiver Request: <what waiver is being requested>
Proposed Waiver Justification
 
 
 
 
Project Waiver Approvals
Project Manager
     
Print Name Signature Date
Process Owner Concurrence and Approval
Process Waiver Request □ Approved □Denied
Comments:
 
 
 
 
Process Owner  
       
Print Name Signature Date  
AD QA Process Concurrence
Process Waiver Request □Concur □Disagree
Comments:
 
 
 
 
AD QA Specialist  
       
Print Name Signature Date  
Chief, AD QA Program Office  
       
Print Name Signature Date  
Domain Director Approval
Process Waiver Request □Approved □Denied
Comments:
 
 
 
 
Domain Director  
       
Print Name Signature Date  
       

More Internal Revenue Manual