7.11.3 Tax Exempt Quality Measurement System (TEQMS)

Manual Transmittal

September 14, 2022

Purpose

(1) This transmits revised IRM 7.11.3, Employee Plans Determinations Letter Program, Tax Exempt Quality Measurement System (TEQMS).

Material Changes

(1) Updated IRM 7.11.3.1.3, Responsibilities.

(2) Updated throughout for editorial and plain language changes.

Effect on Other Documents

This supersedes IRM 7.11.3 dated July 02, 2021.

Audience

Tax Exempt and Government Entities
Employee Plans

Effective Date

(09-14-2022)

Eric D. Slack
Director, Employee Plans
Tax Exempt and Government Entities Division

Program Scope and Objectives

  1. Purpose: This IRM describes the Tax Exempt Quality Measurement System (TEQMS) and the TEQMS process for Tax Exempt and Government Entities (TE/GE) Employee Plans (EP) Determinations specialists and managers.

  2. Audience: EP Determinations and Quality Assurance (QA) staff

  3. Policy Owner: Director, EP

  4. Program Owner: EP

  5. Program Goals: The goal of EP Determinations is to ensure that plans comply with the tax laws by reviewing applications for determination letters (DLs) and opinion letters, and to protect the public interest by applying the tax law with integrity and fairness to all.

Background

  1. TEQMS is designed to measure the quality of the work product TE/GE employees demonstrate using established quality criteria and serves as the quality measurement in the business results part of the Balanced Performance Measurement System (BPMS).

  2. TEQMS is designed to achieve statistical validity at the national level for EP Determinations.

  3. TEQMS for EP Examinations is in IRM 4.70.7, TE/GE Examinations, Special Review (SR) and Tax Exempt Quality Measurement System (TEQMS) Procedures.

  4. This IRM includes:

    1. Overview of TEQMS.

    2. Quality Assurance Staff/Role of TEQMS Analyst.

    3. Case selection process.

    4. Review and data submission process.

    5. Tools for assessing data reliability.

    6. Case return process.

Authority

  1. IRS’s authority to issue favorable determination letters (DLs) on the qualified status of pension, profit sharing, stock bonus, annuity, and employee stock ownership plans under IRC 401, IRC 403(a), IRC 409, and IRC 4975(e)(7) and the status for exemption of any related trusts or custodial accounts under IRC 501(a) is delegated to the Director, Employee Plans, and has been re-delegated to the Director, EP Rulings and Agreements. IRM 1.2.2.8.1. (Delegation Order 7-1).

  2. Find a complete list of delegation orders and policy statements governing EP Rulings and Agreements at https://www.irs.gov/privacy-disclosure/delegation-orders-and-policy-statements-by-process.

Responsibilities

  1. The Director, EP supervises and is responsible for:

    1. EP Rulings and Agreements

    2. EP Examinations

    3. EP Program Management Office

  2. The Director, EP Rulings and Agreements, supervises and is responsible for:

    1. EP Determinations

    2. EP Technical

    3. EP Voluntary Compliance

Acronyms

  1. These acronyms are used in this IRM:

    Acronym Term
    BPMS Balanced Performance Measurement System
    CP&C Compliance Planning and Classification Group
    CJE Critical Job Elements
    DL Determination Letters
    EP Employee Plans
    EIN Employer Identification Number
    QA Quality Assurance
    RRA '98 Restructuring and Reform Act of 1998
    TE/GE Tax Exempt and Government Entities
    TEQMS Tax Exempt Quality Measurement System
    WebETS Web-Based Employee Technical Time System

History

  1. The enactment of the IRS Restructuring and Reform Act of 1998 (RRA '98) required the IRS to change its quality standards. The BPMS, which includes quality, is one of five key RRA' 98 modernization "levers of change." The levers are:

    1. Revamped business practices

    2. Creating operating divisions

    3. Management roles with clear responsibility

    4. Balanced measures of performance

    5. New technology

  2. The balanced measures of performance lever in (d) above are broken into three parts that TE/GE uses to assess organizational performance.

    1. Business results

    2. Customer satisfaction

    3. Employee satisfaction

  3. RRA '98 increased the IRS's awareness of our need to focus on customer service and customer satisfaction. We’ve incorporated this increased awareness into TEQMS.

TEQMS

  1. TEQMS is a/an:

    1. Important component of the BPMS.

    2. Organizational measurement.

    3. Measure of quality as it relates to cases.

    4. Indicator of improvement opportunities.

    5. Tool for assessing components of merit and non-merit cases.

    6. Calculator of quality scores for evaluation at the national level.

    7. Method of providing interim quarterly and annual quality reports.

  2. TEQMS is not evaluative at the:

    1. Area level

    2. Group level

    3. Employee level

  3. In addition to measuring organizational quality, TEQMS reviews benefit TE/GE by:

    1. Improving customer service and customer satisfaction.

    2. Increasing consistency in determination reviews.

    3. Creating a way to increase communication.

  4. The success of TEQMS requires a joint effort between EP Determinations management and specialists.

EP Determinations Quality Assurance (QA)

  1. EP Determinations QA reviewers perform all TEQMS reviews and a TEQMS analyst administers the program.

  2. QA reviewers:

    1. Are senior specialists who have extensive knowledge of tax laws, guidance and policies for determination cases.

    2. Must be objective and exercise independent judgment and individual initiative in performing their duties.

    3. Review EP Determinations cases and report the results in an electronic checksheet called a "survey."

    4. Are expected to communicate effectively with other specialists, and managers, to resolve issues in completing their assignments.

  3. The TEQMS analyst's role includes:

    1. Overseeing the TEQMS sample selection process.

    2. Validating the sample input data from surveys.

    3. Monitoring case receipts for the sample selection.

    4. Preparing interim quarterly and annual reports.

    5. Implementing consistency reviews and validity reviews, as appropriate.

TEQMS Case Selection Process

  1. The following cases aren’t included in the TEQMS sample selection population:

    1. Pre-approved specimen plans.

    2. Pooled/group trusts.

    3. Non-lead multiple employer plans.

    4. Cases that are not worked to completion (for example, returned incomplete, withdrawn by taxpayer, substantially deficient, and correction disposal).

    5. Auto-closure cases (closing code 13).

  2. EP Determination cases subject to TEQMS review are divided into two groups. EP Determinations management determines the type of cases included in each group.

    1. EP Determination cases, also known as non-merit cases, closed with closing code 01. We no longer use closing code 00 to close cases.

    2. EP Merit cases, also known as Technically Screened cases, closed with closing codes 06 and 09.

      Note:

      The group's closing code determines the type of review classification.

  3. Each Employer Identification Number (EIN) and plan number is treated as a separate unit in the population.

  4. Mandatory review cases (IRM 7.11.9.2, Mandatory and TEQMS Case Reviews, paragraph 3) are included in the population of cases that are subject to the TEQMS selection process.

Statistical Validity Level/TEQMS Sample Selection

  1. At the beginning of each fiscal year, the TEQMS analyst determines the approximate sample size of each group described above based on the number of cases TE/GE expects to close during the year based on information from the Compliance, Planning and Classification (CP&C) group. The TEQMS analyst calculates an annual sample size for each sample using statistical calculations with a 95% level of confidence and plus/minus 5% margin of error. Case review results are statistically valid at the national level; the TEQMS analyst computes the actual precision of the quality data at year-end.

  2. The Compliance Planning and Classification (CP&C) group is responsible for selecting the TEQMS samples from cases closed during a WebETS cycle. The TEQMS analyst monitors the sample selection during the year and requests adjustments to the rate, as appropriate, but only once per quarter.

TEQMS Review Process/Quality Standards

  1. QA reviewers review cases in a closed status then complete the survey.

  2. The QA reviewer may return the case to the specialist based on their initial review under IRM 7.11.3.9, Case Return Criteria Overview, after they complete the survey.

Data Capture Process

  1. QA reviewers evaluate cases and complete a survey (electronic checksheet) that contains the case review criteria. The survey captures two types of information:

    1. The administrative items, which contain case information, reviewer information, and case return criteria.

    2. The quality standards/elements items, which contains quality standards, elements, aspects, and reason codes.

      Note:

      The survey for EP Merit cases contains only quality elements and reason codes.

  2. QA reviewers can find the instructions to complete the surveys, and guidelines to apply the quality standards, elements, aspects and reason codes in the TEQMS - Tax Exempt Quality Measurement System - EP Determinations Student Guide, catalog number 86711Q, at: http://core.publish.no.irs.gov/trngpubs/pdf/t4317-002--2005-10-00.pdf. Minor changes have been made to the surveys over time to either expand or supplement some aspects and reason codes for clarification purposes so the current surveys are not the same as in the guide.

Data Submission/TEQMS Database

  1. QA reviewers electronically submit the survey into the CVENT software database when they complete their case review. CVENT is a software licensing vendor used by IRS.

  2. The database performs a series of validity checks before the system accepts the survey's data.

TEQMS Reports

  1. Each quarter, EP Determination and EP Merit surveys are summarized into reports. The reports identify:

    1. Measures of overall performance on quality standards.

    2. Potential improvement areas.

    3. Feedback on specific root causes.

    4. Potential training needs.

  2. The various reports do not provide individual, group or area evaluative information.

Report Analysis

  1. The TEQMS analyst evaluates the database's output and prepares trend reports.

  2. The reports:

    1. Describe areas where EP Determinations has or has not done well.

    2. Provide suggestions for improvement.

    3. Analyze trends based on the answers on the survey.

    Note:

    Typically a separate report is prepared for EP Determination cases and EP Merit cases, but a combined report is acceptable.

Quality Score

  1. The BPMS requires case quality to be scored numerically. This measure is an indicator of the IRS's progress in achieving high-level strategic goals. TEQMS offers a rating system that best measures the organization's case quality.

  2. TEQMS is distinguished from the Critical Job Elements (CJE) Performance Plan. CJE Performance Plans are documents that describe to employees the work performance expected of them and the performance standards applied to their work performance. A manager’s case review should assess an individual's strengths and weaknesses based on the CJE's. Although the TEQMS standards are closely associated with the CJE's, performance evaluation is not based on TEQMS criteria.

  3. The computation of the organizational score is based on the reviewer's answers as to whether the case met each standard (yes or no). EP Determination cases have six quality standards. Some standards contain elements and/or aspects. The maximum possible score is 100 for a standard and/or element. EP Merit has five quality elements, which may contain reason codes. The organization’s maximum possible score is also 100. See the TEQMS - Tax Exempt Quality Measurement System - EP Determinations Student Guide, catalog number 86711Q, for more information about quality standards, elements and aspects at http://core.publish.no.irs.gov/trngpubs/pdf/t4317-002--2005-10-00.pdf.

    The organizational quality score for all EP Determination cases = Total Number of Standards Passed / Total Number of Standards Rated X 100.

    The organizational quality score for all EP Merit cases = Total Number of Elements Passed / Total Number of Elements Rated X 100.

    Example 1:
    During the fiscal year, TEQMS samples 400 EP Determination cases. The EP Determination survey has six quality standards. Therefore, the total number of standards rated equals 2,400 (400 x 6). Of the 2,400 standards rated, only 1,200 standards are passed (answered "Yes" ).
    The organizational quality score is 50 = (1,200 / 2,400 X 100).
    Example 2:
    During the fiscal year, TEQMS samples 400 EP Merit cases. The EP Merit case survey has five quality elements. Therefore, the total number of elements rated equals 2,000 (400 x 5). Of the 2,000 elements rated, 1,800 elements are passed (answered "Yes" ).
    The organizational quality score is 90 = (1,800 / 2,000 X 100).

Consistency Reviews to Ensure Data Reliability

  1. To encourage greater consistency between reviewers in applying TEQMS criteria to cases, the Manager, EP Determinations QA does consistency reviews as needed. The TEQMS analyst schedules and administers these reviews.

  2. The analyst selects at least one EP Determination case and at least one EP Merit case that each reviewer rates independently. Each reviewer prepares a survey with a specific case number and date the analyst gives. The analyst downloads each reviewer's survey and prepares a summary of their answers.

  3. The analyst schedules a meeting with reviewers to discuss the summary document, emphasizing the standards, elements and key aspects where reviewers differ. The goal is that the group will reach a consensus on the differences. If this is not possible, the analyst notes it in a report and recommends appropriate action for reviewers, such as additional training or clarification of a procedure.

  4. The analyst may prepare a report on the consistency review and forward it to the Manager, EP Determinations QA for approval. After approval, the analyst files the report and disseminates it as deemed appropriate. The group rotates reviewers as the primary reviewer on cases selected for the review for each consistency review. The Manager, EP Determinations QA must note if the consistency review results indicate that additional training or improved procedures are required.

Case Return Criteria Overview

  1. This section describes:

    1. Criteria for returning cases to the specialist for further action.

    2. Instructions to the QA reviewers for returning TEQMS cases to specialists.

    3. Type of feedback prohibited from TEQMS.

Case Return Criteria

  1. Reviewers return cases to specialists for further review with approval from QA management, if there is clear evidence that an incorrect determination has been reached. This applies when the qualification of a retirement plan results in a significant negative impact on participants' benefits.

    Note:

    Cases with determination letter or user fee issues will be returned to the group for correction.

Procedures for a Returned Case

  1. Cases are returned to the group under established guidelines, with QA managerial approval. The case is returned for correction and/or further development with a completed Form 5456, Reviewer's Memorandum - TE/GE, Employee Plans Determinations, approved by the QA Manager. An email is sent to the group manager and the specialist, alerting them that the case requires additional action.

  2. If the specialist receives a case from QA, they should correct any issues and complete a Form 5457, Response to Reviewer's Memorandum - TE/GE, explaining corrections or disagreements with all of the reviewer listed issues. Email the form and corrected letters to the group manager for approval and ask that the form, letters and case information be forwarded to the QA Manager.

Feedback Prohibited from TEQMS

  1. Reviewers may no longer give feedback on TEQMS cases because it is an "organizational" measure, not an individual, group or area performance review.