AccessibilitySkip to Top NavigationSkip to Main ContentHome  |  Contact IRS  |  About IRS  |  Site Map  |  Español  |  Help  

Part 7. Rulings and Agreements

Chapter 11. Employee Plans Determination Letter Program

Section 3. Tax Exempt Quality Measurement System (TEQMS)

7.11.3  Tax Exempt Quality Measurement System (TEQMS)

7.11.3.1  (06-30-2004)
Introduction

  1. This IRM describes the TAX EXEMPT QUALITY MEASUREMENT SYSTEM (TEQMS) for management and employees in the Tax Exempt and Government Entities (TE/GE) Division. TEQMS replaced the former EP/EO Quality Measurement System (EQMS).

  2. TEQMS is designed to measure the quality of the work product demonstrated by TE/GE employees in accordance with established quality standards for Employee Plans (EP) determination matters. TEQMS will serve as the quality measurement within the business results component of the Balanced Performance Measurement System.

  3. TEQMS will allow management to use quarterly/yearly reports to identify improvement opportunities. The officials with the primary responsibility for analyzing and taking appropriate action on the TEQMS reports are the managers with responsibility at the corresponding validity level. TEQMS is designed to achieve statistical validity at the national level for determinations.

  4. TEQMS Employee Plans examinations are found at IRM 4.71.15.5, TEQMS Review.

  5. Information provided in this section includes:

    • An overview of TEQMS, including changes resulting from the Balanced Performance Measurement System

    • The Quality Assurance Staff/Role of TEQMS Analyst

    • The case selection process

    • The review/upload process

    • Description of reports

    • Tools for assessing data reliability

    • Case return process

7.11.3.2  (11-01-2002)
Background

  1. The impetus for developing the new quality standards is based on the Services' modernization efforts coupled with the enactment of the IRS Restructuring and Reform Act of 1998 (RRA '98). The Balanced Performance Measurement System, which includes quality, is one of the five key modernization " levers of change" included in RRA '98. The levers are:

    1. Revamped business practices

    2. Four operating divisions

    3. Management roles with clear responsibility

    4. Balanced measures of performance

    5. New technology

  2. Balanced measures of performance is a three-part concept which will be used to assess organizational performance including:

    • Business results - quality and quantity

    • Customer Satisfaction

    • Employee Satisfaction

  3. The effect of the changes to the law enacted by RRA' 98 is the Services' desire to focus on customer service and satisfaction. This increased awareness has been incorporated into TEQMS.

7.11.3.3  (06-30-2004)
Comparing and Contrasting TEQMS with EQMS

  1. What TEQMS is:

    • Important component of Balanced Performance Measurement System

    • Organizational measurements

    • Measure of quality as it relates to cases

    • Indicator of improvement opportunities

    • A tool for assessing various, specific components of merit and non-merit cases

    • Evaluative at the national level

    • A method of providing annual and interim quality reports

  2. What TEQMS is not:

    • It is not evaluative at the group level

    • It is not evaluative at the employee level

    • In general, cases will not be returned

    • Input forms will not be returned

  3. How TEQMS differs from EQMS

    • Attention focused on organizational priorities

    • Focus changed from just the measurements to accomplishing organizational mission

    • TEQMS results used to encourage and tailor meaningful dialogue

    • TEQMS criteria improved; aligned with organizational objectives and balanced approach

    • Success will require a joint effort between management and examiners/specialists

    • UFILL disks (electronic checksheets) instead of paper input forms

    • Case review input forms tailored to types of cases (two separate input forms)

    • TEQMS incorporates customer service and satisfaction into quality standards

    • Quality scores calculated for the TE/GE organization

  4. In addition to serving as an organizational quality measure, TEQMS reviews are expected to result in the following benefits:

    • Improve customer service and satisfaction

    • Reduce the frequency of returned cases from Appeals for additional work

    • Help improve the sustention rate in Appeals

    • Increase consistency in determination reviews

  5. Quality results are reported in a number of high level documents. Examples include the following:

    • IRS Congressional Justification

    • TE/GE Strategy and Program Plan

    • TE/GE Business Performance Review Process

    • IRS Annual Performance Plan

    • Commissioner's Monthly Report

    • IRS Management Board Report

    • Executive Management Support System

7.11.3.4  (11-01-2002)
Quality Assurance Staff (QAS)

  1. Managers and reviewers will support TE/GE's objective to conduct, on a timely basis, quality reviews for each selected determination application.

  2. QAS is staffed by a manager and a centralized determination review team including a TEQMS analyst. All TEQMS reviews will be performed by QAS personnel.

  3. Reviewers are specialists who have demonstrated a comprehensive knowledge of accounting and auditing principles, tax laws, IRS policies, and internal procedures. Reviewers must be objective and exercise independent judgement and individual initiative in the performance of their duties. Reviewers are charged with reviewing determination cases and reporting the results of those reviews via completion of a UFILL disk (electronic checksheet) that contains the appropriate input form. Reviewers are expected to effectively communicate with other technical personnel, management, taxpayers and external customers when providing or obtaining information to timely resolve issues in completing their assignments.

  4. The role of the TEQMS analyst includes oversight of the sample selection process, monitoring receipt of sample cases, preparation of quarterly/yearly reports and implementation of consistency reviews and validity reviews, as appropriate. For EP determinations, the TEQMS analyst serves a dual role as both a senior reviewer with normal Quality Assurance duties and as the TEQMS analyst.

7.11.3.5  (06-30-2004)
TEQMS Case Selection Process

  1. EP determination cases subject to TEQMS review are broken down into the following two classifications:

    • EP Determinations - the population consists of all cases, including mandatory review cases, except: volume submitter specimen plans, master and prototype specimen plans, pooled trusts, non-lead multiple employer plans, and technically screened cases. Each unique combination of "EIN and plan number " is treated as a separate unit in the population. Cases that are not worked, such as returned incomplete, withdrawn by taxpayer, substantially deficient and correction disposal are not included in the population.

    • EP Determinations/Technically Screened Cases - this population consists of all cases closed on merit with or without contact, except for non-lead multiple employer plans. Each unique combination of "EIN and plan number" is treated as a separate unit in the population.

7.11.3.6  (11-01-2002)
Statistical Validity Level/Automated Sample Selection

  1. At the beginning of each fiscal year, the size of each population described above is determined based on the number of cases expected to be closed during the year. An annual sample size is calculated for each population using statistical calculations (95% level of confidence and margin of error of plus/minus 5 percent). Results from determination case reviews are statistically valid at the national level; the actual precision of the quality data is computed at year-end.

  2. Cases are sample selected automatically by the EP/EO Determination System (EDS) during the group closing process. The sample selection rate is monitored during the year and adjustments made as deemed appropriate.

7.11.3.7  (11-01-2002)
TEQMS Review Process/Quality Standards

  1. Cases are reviewed in open status and then closed by QAS after TEQMS review.

7.11.3.7.1  (11-01-2002)
The Input Form

  1. Reviewers will review cases and complete a UFILL disk (electronic checksheet) that contains the appropriate input form. The input form is composed of two sections, an administrative portion and a quality standards/elements portion.

    1. The administrative portion contains fields to capture review information, case return criterion and case information.

    2. The input forms for EP Determinations contain quality standards, elements and aspects. The input forms for EP Technical Screening contains only quality elements.

  2. Specific instructions for completing the input form and guidelines for applying the quality standards, elements and aspects for EP determinations are found in the TEQMS - Tax Exempt Quality Measurement System - EP Determinations (Student Guide), catalog number 86711Q.

7.11.3.7.2  (11-01-2002)
The Upload Process/TEQMS Database

  1. EP determination UFILL disks (electronic checksheets) will be uploaded to the TEQMS database by QAS personnel.

  2. The database performs a series of validity checks and a report is generated on the status of each record. QAS personnel are responsible for resolving all errors.

7.11.3.7.3  (11-01-2002)
TEQMS Reports

  1. The data uploaded to the TEQMS database for EP Determinations and EP Technical Screening will be summarized in the form of various reports that are generated each quarter. These reports provide an indicator of the extent to which completed work meets quality standards. The reports:

    • Provide measures of overall performance on standards.

    • Identify potential improvement areas.

    • Provide feedback on specific root causes.

    • Do not provide evaluative information for individuals.

    • Identify potential training/CPE needs.

  2. The reports include the following:

    • Percentage Yes by Standard/Average Total Score – reflects the percentage of yes answers for each standard as well as the organizational score.

    • Percentage Yes By Standard/Average Total Score By Quarter– reflects the percentage of yes answers for each standard as well as the organizational score for each quarter and cumulative fiscal year.

    • National Quality Standards – reflects the percentage of yes answers for each standard/element/aspect.

    • National Quality Standards Detail - reflects the number of yes, no, and n/a answers for each standard/element/aspect as well as reason codes/process measures.

    • Case Return Criteria – reflects the number of cases returned pursuant to the return criteria.

    • Narrative comments for yes and no answers.

7.11.3.7.4  (11-01-2002)
Report Analysis

  1. The database-generated reports will be analyzed and trend reports prepared. The reports will contain a discussion of where EP determinations have or have not done well, suggestions for improvement and a trend analysis based upon the answers to yes and no questions on the input form. A separate report is prepared for EP Determinations and EP Technical Screening.

7.11.3.7.5  (11-01-2002)
Organizational Quality Score

  1. The Balanced Performance Measurement System requires that organizational case quality be numerically scored. This measure will serve as an indicator of the progress the Service is making in achieving high-level strategic goals. TEQMS provides a rating system that will best measure case quality for the organization.

  2. TEQMS should be distinguished from an individual measure. Manager case reviews should assess individual performance strengths and weaknesses based upon negotiated critical elements. The TEQMS standards are closely associated with critical elements.

  3. The TEQMS database computes the organizational score based on the reviewer's answers concerning whether the case met each standard (yes or no). For EP Determinations there are 6 quality standards. Some standards contain elements and/or aspects. The maximum total score for the organizational equals 100. For EP Technical Screening there are 5 quality elements and the maximum total score for the organizational equals 100. (For a detailed discussion of the quality standards, elements and aspects for EP Determinations and Technical Screening, refer to the TEQMS - Tax Exempt Quality Measurement System - EP Determinations (Student Guide), catalog number 86711Q).

    The organizational score for all EP Determination cases is calculated as:

    Total Number of Standards Passed / Total Number of Standards Rated X 100 = Organizational Score

    The organizational score for all EP Technical Screening cases is calculated as:

    Total Number of Elements Passed / Total Number of Elements Rated X 100 = Organizational Score

    Example 1:

    During the fiscal year, 400 EP Determination cases are sampled for TEQMS. The EP Determination quality input form contains six quality standards. Therefore the total number of standards rated equals 2,400 (400 x 6). Of the 2,400 standards rated, only 1,200 standards are passed (i.e. answered "Yes" ).

    The organizational score is calculated as:

    1,200 / 2,400 X 100 = 50


    Example 2:

    During the fiscal year, 400 EP Technical Screening cases are sampled for TEQMS. The EP Technical Screening quality input form contains five quality elements. Therefore the total number of elements rated equals 2,000 (400 x 5.) Of the 2,000 elements rated, 1,800 elements are passed (i.e. answered " Yes" ).

    The organizational score is calculated as:

    1,800 / 2,000 X 100 = 90

7.11.3.8  (06-30-2004)
Consistency Reviews To Ensure Data Reliability

  1. To encourage greater consistency between reviewers in applying TEQMS standards to cases, the Quality Assurance Manager will be responsible for implementing consistency reviews on a quarterly basis. The TEQMS analyst for EP determinations is responsible for scheduling and administering these consistency reviews. On a quarterly basis, the analyst will select at least one EP Determination case and at least one EP Technical Screening case to be rated independently by each reviewer. The analyst will also review the case(s). Each reviewer and the analyst will prepare the input form. After each reviewer has reviewed the case(s), a copy of the input form will be provided to the analyst. The analyst will tabulate the results from each input form and prepare a summary workpaper of the results to be distributed to each reviewer. The analyst will also schedule a meeting to discuss the conclusions reached by each reviewer with emphasis on the standards, elements and key aspects for which there were differences. It is desired that a consensus will be reached on these differences by all reviewers. If this is not possible, this will be noted in the report with a recommendation for appropriate action, such as additional training or clarification of a procedure. The analyst will prepare a report on the consistency review and forward this report to the Quality Assurance Manager for approval. After approval, this report will be filed by the TEQMS analyst and disseminated as deemed appropriate. The TEQMS analyst will also be the primary reviewer on the cases selected for this review and be responsible for closing the case, including the issuance of a reviewer's memorandum and/or correction of the determination letter, if required. The Quality Assurance Manager is responsible for taking appropriate action if the results of the consistency review indicate that additional training or improved procedures are required.

7.11.3.9  (06-30-2004)
Case Return Criteria Overview

  1. This section provides –

    1. Criteria for returning cases to the specialist for further development.

    2. Instructions to the TEQMS reviewers for returning cases to specialists.

    3. Type of feedback that is prohibited from TEQMS.

    4. Type of feeback that is permitted from TEQMS.

7.11.3.9.1  (06-30-2004)
Procedures for Returning Cases

  1. Cases will be returned to the group when any of the following conditions in section 7.11.3.9.2 or 7.11.3.9.3 exist.

  2. Mandatory review cases are part of the sample universe considered by this system. They will be returned for correction following current procedures. See IRM 7.11.1.12.1 for a list of the Mandatory Review cases for EP determinations.

7.11.3.9.2  (11-01-2002)
Case Return Criteria – EP Determination Cases

  1. There is clear evidence that an incorrect determination has been reached with regard to the qualification of a retirement plan. An underdeveloped case will not be returned unless, in the case file, it is evident that the plan is not qualified.

  2. There is evidence of fraud, malfeasance, collusion, concealment or misrepresentation by the taxpayer or representative.

  3. Other circumstances exist that indicate failure to return would be a serious administrative omission. For example:

    1. Result in serious criticism of the Services' administration of the tax laws;

    2. Establish a precedent that would seriously hamper subsequent attempts by the Service to take corrective action; or

    3. Result in inconsistent treatment of similarly situated taxpayers.

  4. Mandatory review cases will be returned for correction following current procedures.

7.11.3.9.3  (11-01-2002)
Case Return Criteria – EP Technical Screening Cases

  1. Plan qualification not apparent.

  2. Other circumstances exist that indicate failure to return would be a serious administrative omission. For example:

    1. Result in serious criticism of the Services' administration of the tax laws;

    2. Establish a precedent that would seriously hamper subsequent attempts by the Service to take corrective action; or

    3. Result in inconsistent treatment of similarly situated taxpayers.

7.11.3.9.4  (06-30-2004)
Return Criteria Explanatory Notes

  1. EP Quality Assurance reviewers may correct administrative type errors on EDS closing documents. Quality Assurance reviewers may also correct errors on closing letters (e.g., improper caveats, EINs, taxable year, incorrect name or address, plan number, POA letters, advisory paragraphs, etc.).

  2. Generally, technical corrections will not be made by reviewers. The case will be sent back to the specialist.

  3. The Quality Assurance Manager must approve cases that meet the case return criteria, which are returned on a Form 5456, Reviewer's Memorandum - TEGE, Employee Plans Determinations.

7.11.3.9.5  (06-30-2004)
Feedback Prohibited From TEQMS

  1. Except as provided in section 7.11.3.9.6, the following feedback may not be generated on TEQMS because this is an "organizational " measure, not an individual performance review.

    1. Input forms will not be returned to the group manager or specialist. This is true for cases returned to the specialist for further development and for those not returned to the specialist.

    2. Commendatory memos will not be prepared on TEQMS cases.

    3. Advisory memos will not be prepared on TEQMS cases.

    4. For cases that meet the case return criteria, the Reviewer's memorandum will not state the case score.

7.11.3.9.6  (06-30-2004)
Feedback Allowed From TEQMS

  1. When a minor technical or procedural error is found by a reviewer, and the reviewer is able to correct the error, the case will continue to be processed. The case will not be returned to the employee, but effective April 1, 2004 and continuing for a one-year test period, an e-mail will be sent to the specialist detailing the error. No case information will be shared with the specialist. A separate e-mail will be sent to the specialist's manager, but will not reveal the specialist's name or any case information to avoid feedback from being used for evaluative purposes.

  2. During the one-year test, reviewers will also be able to provide positive feedback when they believe commendable actions have been demonstrated on a particular case. Positive feedback notification will follow the same guidelines outlined above in section 7.11.3.9.6(1) to ensure specialist anonymity.

  3. Cases containing significant errors will be processed according to section 7.11.3.9.2, 7.11.3.9.3, & 7.11.3.9.4.


More Internal Revenue Manual