7.11.3  Tax Exempt Quality Measurement System (TEQMS)

Manual Transmittal

July 25, 2012

Purpose

(1) This transmits revised IRM 7.11.3, Employee Plans Determination Letter Program, Tax Exempt Quality Measurement System (TEQMS).

Material Changes

(1) Minor changes including corrections to typographical errors and cite references have been made throughout this IRM.

(2) IRM 7.11.3.5, TEQMS Case Selection Process was updated to provide that classification of a case is determined by the closing code during the initial closing process.

(3) Cites throughout the IRM were updated to provide that electronic checksheets or surveys are completed by EP Determinations Quality Assurance Staff (QAS) reviewers instead of a UFILL disk.

(4) IRM 7.11.3.7.5, Organizational Quality Score was re-written to clarify that employee performance is not assessed using the TEQMS scores and that TEQMS scores are not related to the Critical Job Elements (CJE) Performance Plan.

(5) IRM 7.11.3.9, Case Return Criteria Overview was re-written to improve clarity.

(6) IRM 7.11.3.9.6, Feedback Allowed From TEQMS was moved and renamed to IRM 7.11.3.9.4, Feedback Permitted from TEQMS.

Effect on Other Documents

IRM 7.11.3 dated September 1, 2008 is superseded.

Audience

Employee Plans Personnel

Effective Date

(07-25-2012)

Robert Choi
Director, Employee Plans
Tax Exempt Government Entities Division

7.11.3.1  (09-01-2008)
Introduction

  1. This IRM describes the Tax Exempt Quality Measurement System (TEQMS) and provides guidelines on the process for Employee Plans (EP) Determinations management and employees in the Tax Exempt and Government Entities (TE/GE) Business Unit. TEQMS replaced the former EP/EO Quality Measurement System (EQMS).

  2. TEQMS is designed to measure the quality of the work product demonstrated by TE/GE employees in accordance with established quality criteria for EP Determinations matters. TEQMS serves as the quality measurement within the business results component of the Balanced Performance Measurement System.

  3. TEQMS allows management to use quarterly/yearly reports to identify improvement opportunities. The officials with the primary responsibility for analyzing and taking appropriate action on the TEQMS reports are the managers with responsibility at the corresponding validity level. TEQMS is designed to achieve statistical validity at the national level for EP Determinations.

  4. TEQMS for EP Examinations is found at IRM 4.71.15, Employee Plans Examination of Returns, EP Special Review.

  5. Information provided in this section includes:

    • An overview of TEQMS, including changes resulting from the Balanced Performance Measurement System

    • The Quality Assurance Staff/Role of TEQMS Analyst

    • The case selection process

    • The review and data submission process

    • Description of reports

    • Tools for assessing data reliability

    • Case return process

7.11.3.2  (07-25-2012)
Background

  1. The enactment of the IRS Restructuring and Reform Act of 1998 (RRA '98) required the Service to change its quality standards. The Balanced Performance Measurement System, which includes quality, is one of five key modernization "levers of change" included in RRA '98. The levers are:

    1. Revamped business practices

    2. Four operating divisions

    3. Management roles with clear responsibility

    4. Balanced measures of performance

    5. New technology

  2. The balanced measures of performance lever is broken down into a three-part concept which is used to assess organizational performance.

    • Business Results

    • Customer Satisfaction

    • Employee Satisfaction

  3. RRA '98 increased the Service's awareness of the need to focus on customer service and customer satisfaction. The increased awareness has been incorporated into TEQMS.

7.11.3.3  (07-25-2012)
TEQMS

  1. TEQMS is:

    • An important component of Balanced Performance Measurement System.

    • An organizational measurement.

    • A measure of quality as it relates to cases.

    • An indicator of improvement opportunities.

    • A tool for assessing various, specific components of merit and non-merit cases.

    • A calculator of quality scores for evaluation at the national level.

    • A method of providing annual and interim quarterly quality reports.

  2. TEQMS is not:

    • Evaluative at an area level.

    • Evaluative at the group level.

    • Evaluative at the employee level.

  3. In general, cases will not be returned.

  4. In addition to serving as an organizational quality measure, TEQMS reviews result in the following benefits:

    • Improved customer service and customer satisfaction.

    • Reduction of the frequency of returned cases from Appeals for additional work.

    • Help improve the suspension rate in Appeals.

    • Increased consistency in determination reviews.

    • Creates an avenue to encourage and tailor meaningful dialogue.

  5. Quality results are reported in a number of high level documents. Examples include the following:

    • IRS Congressional Justification

    • TE/GE Strategy and Program Plan

    • TE/GE Business Performance Review Process (BPR)

    • IRS Annual Performance Plan

    • Commissioner's Monthly Report

    • IRS Management Board Report

    • Executive Management Support System

  6. The success of TEQMS requires a joint effort between EP Determinations management and specialists.

7.11.3.4  (09-01-2008)
EP Determinations Quality Assurance Staff (QAS)

  1. EP Determinations QAS is staffed by a manager and reviewers including a TEQMS analyst. All TEQMS reviews are performed by EP Determinations QAS.

  2. The Manager, EP Determinations Quality Assurance and EP Determinations QAS reviewers support TE/GE's objective to conduct, on an independent and timely basis, quality reviews for selected EP Determinations applications.

  3. The Manager, EP Determinations Quality Assurance reports directly to the Director, EP Rulings and Agreements, who also manages the Manager, EP Determinations.

  4. Reviewers are specialists who have demonstrated a comprehensive knowledge of accounting and auditing principles, tax laws, IRS policies, and internal procedures. Reviewers must be objective and exercise independent judgement and individual initiative in the performance of their duties. Reviewers are charged with reviewing EP Determinations cases and reporting the results of those reviews via completion of an electronic checksheet called a "survey" . Reviewers are expected to effectively communicate with other technical personnel, management, taxpayers and external customers when providing or obtaining information to timely resolve issues in completing their assignments.

  5. The role of the TEQMS analyst includes oversight of the TEQMS sample selection process, monitoring receipt of sample cases, preparation of quarterly/yearly reports and implementation of consistency reviews and validity reviews, as appropriate. For EP Determinations, the TEQMS analyst serves a dual role as both a senior reviewer with normal EP Determinations Quality Assurance duties and as the TEQMS analyst.

7.11.3.5  (07-25-2012)
TEQMS Case Selection Process

  1. The following cases are not included in the population for the TEQMS selection process and not eligible to be reviewed:

    • Volume submitter (VS) or master and prototype (M&P) specimen plans.

    • Pooled trusts.

    • Non-lead multiple employer plans.

    • Cases that are not worked to completion (e.g., returned incomplete, withdrawn by taxpayer, substantially deficient, cases re-selected for TEQMS review, and correction disposal).

    • Auto-closure cases designated with closing code 13.

  2. EP Determinations cases subject to TEQMS review are divided into two populations:

    • EP Determinations cases which consist of all eligible cases receiving full reviews. The definition of a full review is determined by EP Determinations management.

    • EP Technically Screened cases which consist of all eligible cases closed on merit, with or without contact. Merit cases are cases that have been subject to treatment that is less than a full review.

      Note:

      The closing code used during the initial group level closing process determines the type of classification.

  3. Each unique combination of Employer Identification Number (EIN) and plan number is treated as a separate unit in the population.

  4. Mandatory review cases are included in the TEQMS selection process.

7.11.3.6  (09-01-2008)
Statistical Validity Level/TEQMS Sample Selection

  1. At the beginning of each fiscal year, the size of each population described above is determined based on the number of cases expected to be closed during the year. An annual sample size is calculated for each population using statistical calculations (95% level of confidence and plus/minus five percent margin of error). Results from EP Determinations case reviews are statistically valid at the national level; the actual precision of the quality data is computed at year-end.

  2. Cases are sample selected automatically by the EP/EO Determination System (EDS) during the group closing process. The sample selection rate is monitored during the year and adjustments are made as deemed appropriate, typically on a quarterly basis.

  3. If the automated sampling process does not function properly, a manual process will be instituted, with the assistance of the Data Analysis Unit (DAU), to ensure the statistical validity of the sample.

7.11.3.7  (09-01-2008)
TEQMS Review Process/Quality Standards

  1. Cases are reviewed in an open status and then closed by EP Determinations QAS after the completion of the TEQMS review process.

7.11.3.7.1  (07-25-2012)
Data Capture Process

  1. Reviewers will evaluate cases and complete a survey (electronic checksheet), that contains the appropriate case review criteria. The survey is composed of two sections:

    1. The administrative portion which contains fields to capture review information, case return criterion and case information.

    2. The quality standards/elements portion which contains quality standards, elements, aspects, and reason codes.

      Note:

      The survey for EP Technical Screening cases contains only quality elements and reason codes.

  2. Specific instructions for completing the survey and guidelines for applying the quality standards, elements, aspects, and reason codes for cases of EP Determinations are found in the TEQMS - Tax Exempt Quality Measurement System - EP Determinations Student Guide, catalog number 86711Q, which is found at the following website:http://core.publish.no.irs.gov/trngpubs/pdf/86711j05.pdf

7.11.3.7.2  (07-25-2012)
Data Submission/TEQMS Database

  1. The surveys are electronically submitted into the TEQMS database by reviewers at the completion of their case review but before the case is closed on EDS.

  2. The database performs a series of validity checks before the survey's data is accepted. Any errors preventing the survey from being submitted will be resolved by EP Determinations QAS.

7.11.3.7.3  (07-25-2012)
TEQMS Reports

  1. Each quarter, the data submitted to the TEQMS database for EP Determinations and EP Technical Screening cases is summarized into various reports. The reports:

    • Provide measures of overall performance on quality standards.

    • Identify potential improvement areas.

    • Provide feedback on specific root causes.

    • Identify potential training/CPE needs.

  2. The various reports do not provide individual, group or area evaluative information.

  3. The reports include the following:

    • Percentage Yes (answers) by Standard/Average Total Score - reflects the percentage of yes answers for each standard (yes answers indicate that the standard was met) as well as the organizational score.

    • Percentage Yes (answers) By Standard/Average Total Score By Quarter - reflects the percentage of yes answers for each standard as well as the organizational score for each quarter and cumulative fiscal year.

    • National Quality Standards - reflects the percentage of yes answers for each standard/element/aspect.

    • National Quality Standards Detail - reflects the number of yes, no, and n/a answers for each standard/element/aspect as well as reason codes/process measures.

    • Case Return Criteria - reflects the number of cases returned pursuant to the return criteria.

    • Narrative comments for yes and no answers.

7.11.3.7.4  (09-01-2008)
Report Analysis

  1. The database generated reports will be analyzed and trend reports prepared. The reports will contain a discussion of where the EP Determinations organization has or has not done well, suggestions for improvement and a trend analysis based upon the answers to yes and no questions on the survey. A separate report is prepared for EP Determinations cases and EP Technical Screening cases.

7.11.3.7.5  (07-25-2012)
Organizational Quality Score

  1. The Balanced Performance Measurement System requires that organizational case quality be numerically scored. This measure serves as an indicator of the progress the Service is making in achieving high-level strategic goals. TEQMS provides a rating system that best measures case quality for the organization.

  2. TEQMS should be distinguished from the Critical Job Elements (CJE) Performance Plan. CJE Performance Plans are the documents that describe for employees the work performance that is expected of them, and the performance standards that are applied to that work performance. Manager case reviews should assess individual performance strengths and weaknesses based upon the CJE's. The TEQMS standards are closely associated with the CJE's, however, performance is not based upon TEQMS scores.

  3. The TEQMS database computes the organizational score based on the reviewer's answers concerning whether the case met each standard (yes or no). For EP Determinations cases there are six quality standards. Some standards contain elements and/or aspects. The maximum total score for the organizational equals 100. For EP Technical Screening there are five quality elements, which may contain reason codes, and the maximum total score for the organizational equals 100. For a detailed discussion of the quality standards, elements and aspects, refer to the TEQMS - Tax Exempt Quality Measurement System - EP Determinations Student Guide, catalog number 86711Q, which is found at the following website: http://core.publish.no.irs.gov/trngpubs/pdf/86711j05.pdf

    The organizational score for all EP Determinations cases is calculated as:

    Total Number of Standards Passed / Total Number of Standards Rated X 100 = Organizational Score

    The organizational score for all EP Technical Screening cases is calculated as:

    Total Number of Elements Passed / Total Number of Elements Rated X 100 = Organizational Score

    Example 1:

    During the fiscal year, 400 EP Determinations cases are sampled for TEQMS. The EP Determinations survey contains six quality standards. Therefore the total number of standards rated equals 2,400 (400 x 6). Of the 2,400 standards rated, only 1,200 standards are passed (i.e., answered "Yes" ).

    The organizational score is calculated as:

    1,200 / 2,400 X 100 = 50


    Example 2:

    During the fiscal year, 400 EP Technical Screening cases are sampled for TEQMS. The EP Technical Screening survey contains five quality elements. Therefore the total number of elements rated equals 2,000 (400 x 5). Of the 2,000 elements rated, 1,800 elements are passed (i.e., answered "Yes" ).

    The organizational score is calculated as:

    1,800 / 2,000 X 100 = 90

7.11.3.8  (07-25-2012)
Consistency Reviews To Ensure Data Reliability

  1. To encourage greater consistency between reviewers in applying TEQMS criteria to cases, the Manager, EP Determinations Quality Assurance, is responsible for implementing consistency reviews as needed, but normally on a quarterly basis. The TEQMS analyst is responsible for scheduling and administering these consistency reviews.

  2. The analyst selects at least one EP Determinations case and at least one EP Technical Screening case to be rated independently by each reviewer. The analyst also reviews the case(s). Each reviewer and the analyst will prepare a checksheet similar to the survey forms. A copy of each reviewer's checksheet will be provided to the analyst for review. The analyst tabulates the results and prepares a summary workpaper of the results to be distributed to each reviewer.

  3. The analyst also schedules a meeting to discuss the conclusions reached by each reviewer with emphasis on the standards, elements and key aspects for which there were differences. It is desired that a consensus will be reached on these differences by all reviewers. If this is not possible, this will be noted in the report with a recommendation for appropriate action, such as additional training or clarification of a procedure.

  4. The analyst may prepare a report on the consistency review and forward this report to the Manager, EP Determinations Quality Assurance for approval. After approval, this report is filed by the TEQMS analyst and disseminated as deemed appropriate. On a rotational basis, at least one reviewer or the TEQMS analyst will be the primary reviewer on the cases selected for the review and they will be responsible for closing the case, including the issuance of a reviewer's memorandum and/or correction of the determination letter, if required. The Manager, EP Determinations Quality Assurance is responsible for taking appropriate action if the results of the consistency review indicate that additional training or improved procedures are required.

7.11.3.9  (07-25-2012)
Case Return Criteria Overview

  1. This section provides:

    1. Criteria for returning cases to the specialist for further development.

    2. Instructions to the TEQMS reviewers for returning cases to specialists.

    3. Type of feedback that is prohibited from TEQMS.

    4. Type of feedback that is permitted from TEQMS.

7.11.3.9.1  (07-25-2012)
Case Return Criteria

  1. EP Quality Assurance reviewers may correct administrative type errors on EDS closing documents. EP Quality Assurance reviewers may also correct errors on closing letters (e.g., improper caveats, EINs, taxable year, incorrect name or address, plan number, POA letters, deficiency paragraphs, etc.).

  2. Generally, technical corrections will not be made by reviewers. The case will be sent back to the specialist.

  3. The following instances will result in cases being returned to the EP specialist for further review:

    1. There is clear evidence that an incorrect determination has been reached with regard to the qualification of a retirement plan.

      Note:

      An underdeveloped case will not be returned unless, in the case file, it is evident that the plan is not qualified.

    2. There is evidence of fraud, malfeasance, collusion, concealment or misrepresentation by the taxpayer or representative.

  4. Cases are also returned when other circumstances exist that indicate failure to return would be a serious administrative omission. For example, cases will be returned which would:

    1. Result in serious criticism of the Services' administration of the tax laws.

    2. Establish a precedent that would seriously hamper subsequent attempts by the Service to take corrective action.

    3. Result in inconsistent treatment of similarly situated taxpayers.

7.11.3.9.2  (07-25-2012)
Procedures for a Returned Case

  1. If a reviewer disagrees with the specialist's recommendation for case closure, the case file may be returned for correction and/or further development along with a completed Form 5456, Reviewer's Memorandum - TEGE, Employee Plans Determinations. Form 5456 must be approved by the Manager, EP Determinations Quality Assurance before the case is returned to the group.

  2. If a case is returned to the specialist, they should complete Form 5457, Response to Reviewer's Memorandum EP/EO with comments addressing all reviewer listed issues. The form will be attached to the case and forwarded to the frontline group manager for approval BEFORE the case is returned to EP Determinations QAS.

7.11.3.9.3  (07-25-2012)
Feedback Prohibited From TEQMS

  1. Except as provided below in IRM 7.11.3.9.4, Feedback Permitted From TEQMS, feedback may not be generated on TEQMS because this is an "organizational" measure, not an individual, group or area performance review. The following rules apply:

    1. The group manager or specialist may not have access to the TEQMS survey.

    2. Commendatory memos will not be prepared on TEQMS cases. This practice was modified in 2005. (See IRM 7.11.3.9.4, Feedback Permitted from TEQMS.)

    3. Advisory memos will not be prepared on TEQMS cases.

    4. For cases that meet the case return criteria, the Reviewer's memorandum will not state the case score.

7.11.3.9.4  (07-25-2012)
Feedback Permitted from TEQMS

  1. As a result of requests from specialists, in 2005, a permanent policy change allowing feedback directly between the specialist and the reviewer was approved and instituted.

  2. After a TEQMS review is complete, an e-mail is sent to the specialist, with a carbon copy to the TEQMS Analyst.

  3. The e-mail details any issues/errors found in the case, if applicable, or may provide positive feedback when commendable actions have been demonstrated. The e-mail will include pertinent case identifying information.

  4. The TEQMS analyst collects the e-mails and, on an as needed bases, prepares management summary reports for each group detailing any identifiable trends that may emerge. The management summaries are not specific to any individual and therefore cannot be used for evaluative purposes. The management summaries may be discussed in a group setting.


More Internal Revenue Manual