- 2.5.14 Quality Assurance (QA)
- 220.127.116.11 Introduction
- 18.104.22.168 Quality Assurance Auditing Process
- 22.214.171.124.1 Auditing Process Roles and Skills
- 126.96.36.199.2 Audit Process Control
- 188.8.131.52.3 Auditing Process Tasks and Flow
- 184.108.40.206.3.1 Plan to Audit
- 220.127.116.11.3.2 Perform the Audit
- 18.104.22.168.3.3 Manage Documents
- 22.214.171.124.3.4 Monitor and Control Audit Findings
- 126.96.36.199.4 Audit Process Management
- 188.8.131.52.5 Audit Process Review
- 184.108.40.206 Waiver Justification Procedure
- Exhibit 2.5.14-1 Waiver Justification Form
Part 2. Information Technology
Chapter 5. Systems Development
Section 14. Quality Assurance (QA)
This IRM provides the framework for conducting Quality Assurance (QA) Audit activities within Applications Development (AD). It establishes a standard context for Project Teams, including contractors working for the Applications Development organization, to participate in the QA audit process. This IRM establishes:
the overall approach to Quality Assurance;
the applicable Quality Assurance standards; and
the reporting and control requirements for the QA program as outlined by the AD QA Directive and related processes and procedures.
The Quality Assurance (QA) Program Office supports the delivery of high-quality products and services by ensuring that projects implement a coordinated set of activities that conform to organizational policies, processes and procedures.
Quality Assurance is a systematic, planned set of activities necessary to provide adequate confidence that the product conforms to stated customer requirements. The activities are designed to evaluate the processes (i.e. Project Planning, Project Monitoring and Control, Requirements Management, etc.) by which products are developed.
QA Audit Process and Procedures are used to objectively and independently evaluate adherence of the process and work products to applicable directives, processes, standards, procedures, and guidelines. The objectives of the audit process are to:
identify and track noncompliance instances;
communicate and facilitate the resolution of noncompliance issues;
identify and communicate, to senior management, best practices and opportunities for improvement;
document Quality Assurance activities; and
report quality issues to relevant stakeholders.
Benefits of the audit process are realized through:
consistency in assessing use of organizational processes;
improved compliance with enterprise-wide standards and directives;
facilitation of improvements; and
enhanced planning and resource allocation capability.
To meet the objectives and realize the benefits of QA auditing, the following roles and skill sets are needed to perform QA auditing activities.
Roles Responsibilities AD Quality Assurance Program Office (Quality Specialist) Subject matter expertise on the objectives, procedures, and methods for performing the audit tasks, in order to:
Establish and maintain the AD QA Quality System artifacts (plans, procedures, templates, checklists, project folders, etc.)
Establish and clearly state criteria for audits on QA website
Establish and maintain the AD portfolio project database
Establish and maintain the Quality Assurance Training Materials
Establish and maintain a Master QA Audit Calendar
Compile and report quality information and trends
Use the stated criteria to audit performed processes for adherence
Maintain an audit and/or review calendar
Identify each non-compliance found during the audit/release review
Train and mentor project staff
Conduct audit orientations
Document audit findings
identify quality trends and make recommendations on all organizational levels
Domain Director High level knowledge of the QA objectives in order to ensure Domain compliance with the applicable IRS directives, processes and procedures. Duties include:
Concurrence on Audit Calendar
Review Quarterly Domain Director Report and correct Domain level systemic problems identified
Ensure resolution of escalated of non-compliances
Domain POC Working knowledge of the objectives, procedures, and methods for performing the audit tasks, in order to endure project compliance with the applicable IRS directives, processes and procedures. Duties Include:
Act as liaison between the Domain and AD QA.
Concurrence on Audit Calendar, if delegated by the Domain Director
Coordinate Domain level QA activities, at both the project and Division levels
Correct Domain level systemic problems identified during audits
Reconcile project inventories
Participate in audit opening and closing meetings, as appropriate
Oversee the resolution of audit non-compliances
Coordinate training for project staff to correct trends identified in the Domain Director Report
Attend Domain POC training and quarterly AD QA Domain POC meetings
Project Manager/Branch Chief Overall responsible for quality on the project. They should have a working knowledge of the objectives, procedures, and methods for performing the audit tasks, in order to endure project compliance with the applicable IRS directives, processes and procedures. Duties Include:
Review audit report and provide project resources to correct non-compliances in the required timeframes
Participate in the opening and closing meetings
Identify the Quality Analyst(s) for the project
Attend QA Training, as appropriate.
Project Team (Quality Analyst) Working knowledge of the objectives, procedures, and methods for performing the audit tasks, participating in normal quality control (i.e. development and review of program and project documentation, peer reviews of source code, unit testing, etc.) lifecycle tasks. The designated project Quality Analyst will:
Act as a liaison between the project and AD QA to coordinate the audit and document reviews
Ensure AD QA has access to the project's repository and/or artifacts
Oversee the Correction Action Plan process for the project(s)
Convey trends/problems to the Project Manager and/or Branch Chief
Monitor open non-compliances to ensure that they are closed in the required timeframes
Attend QA training
The audit process is controlled and driven by the Audit Checklists. The checklists are a series of questionnaires used to gather data, for the processes and products being audited, to evaluate the project’s level of compliance. The checklists can be accessed on the Intranet at http://mits.web.irs.gov/AD/PMO/QA/audit_checklist.html. There are seven process areas and three types of audits currently covered by the AD QA audit process. The process areas are:
Project Monitoring and Control
Measurement and Analysis
Supplier Agreement Management
The audit types are:
Process Audit - Evaluates a process area, in depth, based on organizational standards and requirements. This type of audit includes the procedures, requirements, and standards for the development lifecycle, project management, and support processes. Process audits are based on where a project is in the development lifecycle; however, the auditor can look at previous phases for adherence
Work Product Audit - Reviews work products for conformance to the Enterprise Life Cycle (ELC), Data Item Descriptions (DID) and templates. Work Products include all work products and deliverables, as well as the standards and/or procedures used to produce them. This type of audit is either performed as the project completes it during the lifecycle or in conjunction with the QA Release Review.
Release Review Audit - Evaluates a sampling of the processes used and work products produced during a given release. These audits can be seen as a combination of the process and work product audit that is normally performed at the end of a scheduled release for projects that have an accelerated release schedule.
The following tasks constitute the flow of the QA auditing process:
Plan to Audit
Perform the Audit
Monitor and Control Audit Findings
This task is performed by the QA Specialists in conjunction with the overall project level audit planning activities. Project teams and Management support QA Audit planning activities. The following steps describe the audit planning activities:
Prioritize Programs and Projects for Audit Activities. The QA Specialist develops a prioritized audit plan based on the goals of the strategic plan, annual priorities and weighted project risk factors.
Create Domain and Master Audit Calendar. The QA Specialist, in cooperation with the Domain Point of Contact and/or Project Quality Analyst, develops the AD Domain Audit Calendar, which is incorporated into the QA Master Audit Calendar.
Notify stakeholders of final Audit Calendar. The QA Specialist shares the final Calendar with all stakeholders.
This task is performed in accordance with the Master Audit Calendar. The following activities occur during this task:
Projects are formally notified of the audit and access to the project repository is requested.
The QA Specialist reviews the Project Repository and makes a preliminary assessment of the quality status of the project.
The Opening Audit meeting is conducted to establish the scope, objectives and observations from the preliminary assessment.
The Quality Specialist, guided by the appropriate Audit Checklists, will evaluate the level of compliance with standards and processes as well as evaluate associated work products by reviewing instructions and procedures, checking records and through observation.
If necessary, project staff will be interviewed to address questions that arise during the checks and observations.
Subsequent to the observation and interviews a closing meeting is held to discuss audit findings and to allow the auditee to provide factual corrections and explanations, prior to issuance of the final audit report.
The results of the observations, checks and interviews are documented in the Audit Findings Report and delivered to the auditee.
The final Audit Findings Report is issued and stored in the appropriate QA repositories.
When audit findings require corrective action, a Corrective Action Plan is requested, from the auditee, as a part of the Audit Findings Report.
This task occurs after an audit is conducted, or may occur anytime a document is established or content/information is gathered relating to the audit process. The following activities occur during this task:
The QA Specialist will use the information from the Audit Findings Report and any subsequent meeting minutes, phone conferences, etc., to update the AD QA database with the results of the audit or other record (e.g., the audit results worksheet, AD QA Checklists) of QA audit activities.
All QA documents generated and/or received as a result of an audit are collected and stored in soft and/or hard copy in the appropriate repository.
QA documents (i.e., audit evidence and quality system processes and procedures) are retained for at least six years, after baselined.
Changes to QA process assets (i.e., processes, procedures, templates) shall be controlled in accordance with the AD QA Change Control Procedure located in the QA Quality System shared repository.
As part of this task, when new QA documents are developed and/or change requests have been received to modify existing documents, the following activities are executed:
A sequential Change Request (CR) number is assigned to each CR received and the request in recorded in the QA CR Log.
The CR is forwarded to the Change Control Board (CCB) for disposition. The Chief, AD QA and Senior AD QA Management Analysts serve as the QA CCB.
The AD QA CCB meets monthly and reviews any Change Requests received, in accordance with the AD QA Change Control Procedure. For Process Asset Integration Group assets, the approval process starts with the CCB and then must be approved by the AD, Program Management Office and the AD, Associate Chief Information Officer.
Using the data in the QA data repositories and Audit Calendars, the QA Specialist tracks audit findings to closure and, if applicable, schedules a follow-up audit. If noncompliance (corrective) actions are not completed by the resolution date as outlined in the approved corrective action plan, the QA specialist escalates the unresolved findings to senior management.
The Quality Specialist responsibilities are to:
Verify resolution of the corrective action.
Update AD QA Database to indicate the status of the audit findings.
Identify corrective actions that remain unresolved five days after planned resolution date.
Prepare AD QA Escalation Reports.
Perform trend analysis activities.
Prepare audit data and status reports for review activities.
Place reports in the appropriate repository.
When all findings are resolved, the auditee is notified that the audit is closed and all data repositories, i.e., AD QA database, project profiles, etc., must be updated.
AD Quality Assurance Program Office will regularly maintain measurements on the status and progress of Quality Assurance tasks for the AD portfolio. Process trends shall be analyzed for efficiency and effectiveness.
Data will be compiled to develop and report trends in performance and compliance. The reporting will occur through performance trends metrics and compliance metrics. Performance and compliance trends metrics will be reported:
At the AD portfolio, domain, and project levels; and
For all process areas (by AD portfolio, domain, project levels).
On a defined, regular and event-driven basis, the QA Program Office will review the progress and status of activities performed to accomplish the tasks covered by the QA Auditing Process and Procedures.
At least quarterly, Program Management will report to senior management on the overall status of tasks covered by this process.
The purpose of the waiver justification is to uniformly track any project team’s deviation from published standards and to have a formal, documented agreement between the process owner, AD QA, and the project team on deviations from accepted standards. The objectives of this procedure are to:
Establish a detailed audit trail of what standards have been changed, the justification for the change, and a formal approval process for the change; and
Establish a consistent and proactive approach across process areas for deviations from standards.
During the audit process, AD QA will ensure that waivers and tailoring across process areas are properly documented, evaluated, and justified.
Whenever it is determined that the need to deviate from an established standard exists, a Waiver Justification meeting, with the process owner and relevant stakeholders, should be initiated to discuss the proposed waiver. If all parties agree to the waiver, the Project Manager completes the Waiver Justification Form template and submits the form to the Process Owner and to the AD QA Specialist for signature. The signed form is forwarded to the Domain Director for approval and signature.
If the Process Owner disagrees with the need for the waiver, the Process Owner documents the reason(s) in the Waiver Justification Form and returns the form to the Project Manager and the AD QA Specialist. If necessary, the Project Manager can escalate to he Domain Director approval.
If the AD QA Specialist disagrees with the need for the waiver, the AD QA Specialist documents the reason(s) in the Waiver Justification Form, notifies the QA Program Office Chief and obtains concurrence on the decision, and returns the documented waiver form to the Process Owner and the Project Manager. If necessary, the Project Manager can escalate to both the Domain Director for approval to proceed.
The Project Manager communicates the final decision of all waivers to all of the affected stakeholders. The completed waiver justification form is filed in the project notebook (or filed with other project documentation).
|Project Name:||Release #:|
|Waiver Submit Date:|
|Proposed Waiver Request:||>what waiver is being requested<|
|Proposed Waiver Justification|
|Project Waiver Approvals|
|Process Owner Concurrence and Approval|
|Process Waiver Request||□ Approved||□Denied|
|AD QA Process Concurrence|
|Process Waiver Request||□Concur||□Disagree|
|AD QA Specialist|
|Chief, AD QA Program Office|
|Domain Director Approval|
|Process Waiver Request||□Approved||□Denied|