1.5.1 The IRS Balanced Performance Measurement System

Manual Transmittal

May 20, 2019

Purpose

(1) This transmits revised IRM 1.5.1, Managing Statistics in a Balanced Measurement System, The IRS Balanced Performance Measurement System.

Material Changes

(1) The CFO Corporate Budget Office has revised this IRM to reflect new ownership.

(2) IRM 1.5.1.1Program Scope and Objectives, added to conform to the new internal control requirements described in IRM 1.11.2, Internal Revenue Manual (IRM) Process. Also, rearranged and updated existing IRM content to place internal controls information under this subsection. All other subsequent subsections were renumbered accordingly.

(3) Removed Exhibits 1.5.1.6, “The Relationship Between the IRS Mission and a Service Performance Measure,” and 1.5.1.7, “The Relationship Between the IRS Mission and a Compliance Performance Measure,” due to ongoing content changes associated with publishing of the IRS Strategic Plan for Fiscal Years 2018-2022.

(4) Ensured all acronyms were defined the first time they appeared.

Effect on Other Documents

This IRM supersedes IRM 1.5.1, issued October 26, 2016.

Audience

This IRM is directed to all IRS executives and managers.

Effective Date

(05-20-2019)

Ursula S. Gillis
Chief Financial Officer

Program Scope and Objectives

  1. Purpose:

    1. This IRM provides an overview of the IRS Balanced Performance Measurement System and outlines how the IRS uses balanced measures to measure organizational performance. Since it is not possible to prescribe acceptable actions for every situation, this IRM provides general guidance and direction for IRS business units on the appropriate use and application of balanced measures.

    2. The associate CFO for Corporate Budget develops and maintains this IRM.

  2. Audience: IRS executives and managers

  3. Policy Owner: Associate CFO for Corporate Budget

  4. Program Owner: Associate CFO for Corporate Budget

  5. Primary Stakeholders: CFO

  6. Program Goals: Comply with all relevant authorities regarding IRS Enterprise Performance Measures.

Background

  1. The IRS developed the Balanced Performance Measurement System to reflect its priorities consistent with its mission and strategic goals. To help ensure balance, the IRS considers each of the three components of balanced measures - customer satisfaction, employee satisfaction and business results - when setting organizational objectives, establishing targets, assessing progress and results, and evaluating individual performance. Each component should be given equal importance.

  2. The IRS will use balanced measures at both the strategic and the operational level to measure organizational performance. Each business unit supports IRS-wide strategic goals. Business units do not establish additional strategic goals. At the individual level, critical elements and critical performance expectations that support and align with the mission and balanced measures approach will be the basis for evaluating IRS employees.

Authorities

  1. The IRS Restructuring and Reform Act of 1998 (RRA '98) requires the IRS to change its measures to balance customer service with its overall tax administration responsibilities.

  2. 26 CFR 801, Balanced System for Measuring Organizational and Employee Performance, implements the IRS Balanced Performance Measurement System.

Roles and Responsibilities

  1. To integrate and use the Balanced Performance Measurement System IRS-wide, each organization is responsible for establishing a:

    1. Comprehensive approach for measuring and using customer satisfaction, employee satisfaction and business results (quantity and quality) data.

    2. Clear strategy for using all elements of balanced measures in strategic, operational and business planning.

    3. Commitment to ensuring, explaining and demonstrating how it uses and address customer satisfaction, employee satisfaction and business results in its business decisions.

Program Management and Review

  1. Program Reports: Before proposing a new measure or indicator for inclusion in the IRS budget or the internal strategy and program plans, an organization must complete the detailed measures template included in Exhibit 1.5.1–5. The organizations should use the template to facilitate a common understanding across a measure’s definition, formula, data reliability, reporting frequency, etc.. This is essential to meet various oversight requirements.

  2. Program Effectiveness: Each organization is responsible for periodically updating its performance measures according to IRM 1.5.1.10 and this IRM’s exhibits to ensure measure effectiveness. Each organization will provide periodic feedback to the associate CFO for Corporate Budget on the effectiveness of measures from their perspective. The associate CFO also will provide periodic feedback to each organization on outside perspectives on measure effectiveness.

Program Controls

  1. Each organization is responsible for creating and maintaining a Data Dictionary and submitting a copy to the associate CFO for Corporate Budget for each measure that is included in the IRS budget. This document follows the program reports guidance in IRM 1.5.1.1.4, Program Management and Review, and ensures program controls for each measure by documenting the procedure used and facilitating a common understanding across a measure’s definition, formula, data reliability, reporting frequency, etc..

  2. Each organization is responsible for creating and maintaining program controls for its internal performance data, as appropriate. The CFO provides guidance on but does not assess business units’ internal performance measures.

Terms/Definitions

  1. Below are descriptions of terms used throughout this IRM.

    1. Business Performance Reviews - Quarterly reviews conducted by the IRS for each business unit that review a variety of topics including organizational performance, key initiatives, risks, budget, staffing and other considerations as applicable.

    2. Business results measures - Indicators of the quality and quantity of work performed.

    3. Customer - Any internal or external person or entity to whom you provide services.

    4. Customer satisfaction measure - An indicator of the level of overall satisfaction with service provided by the IRS as perceived by internal and external customers.

    5. Cycle time -The total time from the beginning to the end of a process, as defined by the business operating division within the IRS.

    6. Data Dictionary - Document which describes a performance measure in detail, including formal definition, responsible official, data source, data reliability, and program controls.

    7. Diagnostic tools and/or workload indicator - Indicators used to discover the factors impacting changes in the balanced measures.

    8. Efficiency measures - Capture the operation's skillfulness in executing programs, implementing activities and achieving results, while avoiding wasted resources, effort, time and/or money. Efficiency is the ratio of the outcome or output to the input of any program.

    9. Enterprise-wide measures - A select few measures which cross several organizational unit lines that demonstrate the IRS effectiveness in meeting its strategic goals.

    10. IRS balanced measures - Indicators of organizational performance for customer satisfaction, employee satisfaction and business results.

    11. Outcome measures - Describe the intended result from carrying out a program or activity. Define an event or condition external to the program or activity that is directly important to the intended beneficiaries and/or the public.

    12. Outcome neutral - Production or resource data that do not contain information regarding the tax enforcement result reached in any case, such as the number of cases closed, level of service provided and assistance and outreach efforts undertaken.

    13. Output measures - Describe the level of activity that will be provided over a time period, including a description of the characteristics established as standards for the activity, such as timeliness.

    14. Overage - An excess or surplus of resources.

    15. Performance budget - A budget that links performance goals with costs for achieving a target level of performance. A performance budget links strategic goals with related long-term and annual performance goals (outcomes) to the costs of specific activities to influence these outcomes.

    16. Performance budget level measures - Long-term and annual performance measures used to support the annual performance budget.

    17. Performance goal - Sets a target level of performance over time expressed as a tangible, measurable objective, against which actual achievement can be compared, including a goal expressed as a quantitative standard, value or rate. A performance goal is comprised of a performance measure with targets and time frames.

    18. Performance measures - Indicators, statistics or metrics used to gauge program performance.

    19. Program assessment - A determination, through objective measurement and systematic analysis, of the manner and extent to which Federal programs achieve intended objectives.

    20. Quality measure - A numeric indicator of the extent to which completed work meets prescribed standards.

    21. Quantity measure - A numeric indicator of outreach efforts, outcome neutral productivity and resource utilization.

    22. Stakeholders - Groups or individuals who have a vested interest in an organization. They can be internal or external to the IRS.

    23. Strategic goal - A statement of aim or purpose included in a strategic plan. In a performance budget/performance plan, strategic goals should be used to group multiple program outcome goals. Each program outcome goal should relate to and, in the aggregate, be enough to influence the strategic goals or objectives and their performance measures.

    24. Target - Quantifiable or otherwise measurable characteristic that tells how well a program must accomplish a performance measure.

Acronyms

  1. Below are definitions of acronyms used throughout this IRM.

    AFR Agency Financial Report
    BPR Business Performance Review
    BPRS Business Performance Review Summary
    CJ Congressional Justification
    FTE Full-Time Equivalents
    IDRS Integrated Data Retrieval System
    MBPS Monthly Business Performance Summary
    MD&A Management's Discussion and Analysis Report
    MEPS Measured Employee Performance System
    OMB Office of Management and Budget
    PMM Performance Measures Manager
    ROTER Records of Tax Enforcement Results
    RRA 98 The IRS Restructuring and Reform Act of 1998
    SY Staff Years

Related Resources

  1. IRM 1.5.2, Uses of Section 1204 Statistics.

  2. IRM 1.5.3, RRA 98 Section 1204 Self-Certification and the Independent Review Process.

  3. IRM 6.430.3, Performance Management Program for Evaluating Managers, Management Officials and Confidential Management/Program Analysts.

Performance Measures Quick Reference

  1. The following is a quick reference for the IRS balanced performance measurement system.

    1. The IRS must carefully consider each of the three components of balanced measures - customer satisfaction, employee satisfaction and business results - when setting organizational objectives, establishing goals, assessing progress, and results, and evaluating individual performance.

    2. Under the IRS Balanced Performance Measurement System, the approved set of balanced measures is the primary means for assessing organizational performance. The IRS can use other information and data for purposes such as workload planning and analysis.

    3. Each measure should have targets based on the previous year's results, historical patterns and the anticipated mix of resources available; the linkage to organizational priorities and initiatives, planned process improvements or system enhancements; and an assessment of existing and emerging trends, issues and problems.

    4. Targets set for quantity business results must always consider the effects on customer satisfaction, employee satisfaction and quality. The IRS may use quantity measures to evaluate and/or set or suggest goals for organizational units at levels that do not measure customer satisfaction, employee satisfaction or quality, if such measures exist at a higher level in the organization.

    5. The IRS should not use the performance of any one unit as a standard by which the performance of other unit is evaluated because of differences in the types of taxpayers served, specific tax issues addressed and other factors. However, some measures at the enterprise level may combine aspects of multiple business units into one measure. In these cases, individual business units should track their own performance separately from the combined measure.

    6. The primary focus of organizational reviews should be on the effectiveness of actions taken and accomplishments.

    7. The numeric results achieved for any measure will never directly equate to the evaluation of an individual.

    8. Because measures reported by the IRS externally are subject to extensive evaluation and review, measure evaluation should meet an especially rigorous set of standards.

    9. In determining the appropriateness of a specific course of action, managers and employees should exercise sound judgment consistent with 26 CFR Part 801.

The IRS Balanced Performance Measurement System

  1. The IRS Balanced Performance Measurement System provides a means to:

    1. Establish priorities.

    2. Guide and motivate performance and establish a link between performance targets and organizational goals and objectives.

    3. Obtain feedback to identify how well the IRS meets customer and stakeholder expectations and to identify areas for improvement.

    4. Assess overall program effectiveness and communicate results.

Components of the Balanced Performance Measurement System

  1. The components of the balanced performance measurement system are customer satisfaction, employee satisfaction and business results, with business results being comprised of quantity and quality measures.

  2. Each component represents an important aspect of the organization's goals and each should be given equal importance in carrying out the objectives of IRS programs. Some of the components do not change as rapidly as others or require more time for data collection; therefore, the frequency of data availability across the three components may vary. However, differences in the frequency of data availability do not reflect differences in priority.

The Goals of Customer Satisfaction, Employee Satisfaction and Business Results

  1. The IRS’s overall objective is to attain:

    1. a high level of customer satisfaction by providing professional services to both internal and external customers courteously and timely;

    2. a high level of employee satisfaction by achieving a productive and enabling work environment with quality leadership, adequate training and effective support services; and

    3. an adequate level of quality business results, coupled with meaningful outreach to customers.

Measuring Customer Satisfaction

  1. The IRS normally gathers information to measure customer satisfaction from a statistically valid sample of the taxpayers served by a particular organizational unit or who have recently interacted with a particular IRS program. The IRS accomplishes this by using of surveys for the organizational unit overall and transactional surveys for its major programs that:

    1. Affect a significant proportion of the customers served by the organizational unit and/or

    2. Use a significant proportion of the total resources (full-time equivalents and dollars) of the organizational unit.

  2. While the IRS should obtain customer feedback for all programs, a survey instrument or satisfaction score may not be possible or meaningful for every program.

  3. The intent of the customer satisfaction component of the balanced performance measurement system is to ensure the IRS considers and addresses customer satisfaction issues and needs in organizational planning, budgeting and review activities.

    Example:

    When developing strategies to improve business results, the IRS should consider the effect on customer satisfaction outcomes.

  4. Organizations should review survey results on a regular basis and develop plans that include specific strategies aimed at improving customer satisfaction.

    Note:

    The IRS guarantees customer anonymity consistent with 26 CFR Part 801.4.

Measuring Employee Satisfaction

  1. The IRS measures employee satisfaction for organizational units, sub-units and work groups. With some exceptions, it is typically not measurable for programs because the IRS allocates employee time to more than one program and spreads many programs across organizational units.

  2. The intent of the employee satisfaction component of a balanced performance measurement system is to ensure the IRS considers and addresses employee satisfaction issues and needs in organizational planning, budgeting and review activities.

    Example:

    Strategies to improve business results must consider and address the effect on employees, when applicable.

  3. Organizations should review annual employee survey results and develop plans that contain specific strategies aimed at improving employee satisfaction, if applicable.

Measuring Business Results

  1. The IRS assesses operational level business results through output or quantity measures and quality measures. Output or quantity measures consist of outcome-neutral production and resource data like the number of cases closed, work items completed and hours expended. Quality measures consist of data such as case/call review data, accuracy and timeliness. They can also include efficiency measures. The intent of measuring both quantity and quality is to ensure the IRS generates a productive quantity of work in a quality manner.

Performance Measures Objectives

  1. The IRS can establish performance measures to assess progress at three levels:

    1. Budget level measures to support IRS budget requests.

    2. Strategic level measures to assess Servicewide progress in delivering on the mission and goals established by the strategic plan. These measures only apply Servicewide. Since each IRS business unit supports the IRS-wide strategic goals, they do not establish individual strategic goals.

    3. Operational level measures to assess the effectiveness of specific programs.

      Example:

      Submission Processing, Filing and Accounts Management, Field Collection and Field Exam.

  2. The IRS has three types of performance measures:

    1. Outcome measures describe the intended result of carrying out a program or activity.

    2. Output or quantity measures describe the level of activity that will be provided over a period of time.

    3. Efficiency or quality measures capture skillfulness in executing programs, implementing activities and achieving results while avoiding wasted resources, effort, time and/or money.

      Note:

      The IRS establishes a baseline for first-year measures and future targets are based on prior year performance and current budgetary considerations.

Using Performance Information

  1. The IRS collects information about programs and services and uses some of it to establish performance measures. The IRS uses other information and data for workload planning and analysis.

  2. The Balanced Performance Measurement System enables the IRS to use performance results to:

    1. Measure how an organizational unit performs relative to its past performance.

    2. Identify areas for improvement by considering results from measures of customer satisfaction, employee satisfaction and business results.

    3. Align and support various review processes.

    4. Identify the factors that influence performance.

    5. Provide input to manager’s performance appraisals, but not serve as a direct evaluative tool.

Using Workload Indicators and Diagnostic Tools for Planning and Analysis

  1. The IRS uses workload indicators for resource planning during the development of program plans and budgets. Workload indicators project expected levels of activity for an organization or program, identify resource needs and justify the IRS budget requests to the Department of the Treasury, the Office of Management and Budget (OMB) and Congress.

    Example:

    Refunds issued, number of walk-in customers and number of returns filed.

  2. Organizational performance assessments include workload indicators to compare actual workload to planned workload for making future projections and estimating resource requirements.

  3. Diagnostic tools are any data used to understand what influences and affects performance. In some cases, organizations may share data used as a diagnostic tool for their organization (such as cycle time) so that other organizations may use it as a budget level measure. This is permissible if the measure conforms to the guidelines and restrictions set forth in this IRM.

  4. Using diagnostic tools and workload indicators provides a mechanism to study the factors that influence performance and identify opportunities that managers can take to improve customer satisfaction, employee satisfaction and business results.

  5. The business units may establish improvement targets for diagnostic tools and workload indicators but only in direct support of overlying budget or operational level measures.

  6. Using results from diagnostic tools or workload indicators to compare one unit to other units may be appropriate for conducting analysis, exploring best practices or seeking process enhancements to improve the overarching balanced measure(s).

  7. The IRS uses diagnostic tools and/or workload indicators to identify and understand the underlying factors that cause changes in the balanced measures.

  8. Examples of diagnostic tools and workload indicators:

    1. Customer satisfaction survey results (for example: particular survey questions, identified improvement priorities, narrative comments)

    2. Employee survey results

    3. Cycle time

    4. Employee experience/training/skill levels (for example: hours of training per employee, work force mix, average education level of newly-hired employees.)

    5. External factors (for example, tax law macroeconomic cycles)

    6. Employee absenteeism and/or turnover rates

    7. Return closures per unit of effort

    8. Inventory level

    9. Individual Master File (IMF)/Business Master File (BMF) workload mix

    10. Staffing resources (for example, full-time equivalent (FTE) appropriated, FTE realized and resource utilization)

    11. Results for individual quality standards/elements

    12. Wait time/transaction time

    13. Cost information

    14. Number of returns filed

    15. Records of Tax Enforcement Results (ROTERs) (More information on ROTERs can be found in IRM 1.5.2.)

      Note:

      Organizations will set no targets and make no comparisons between units for ROTERs.

  9. Examples on how to apply diagnostic tools and workload indicators when conducting analyses are available in the Reference Guide for Diagnostic Tools.

Setting Targets or Performance Goals

  1. There are two types of targets an organization can use to communicate priorities and guide performance - qualitative and quantitative:

    1. Qualitative targets are general in nature and suggest a desired direction.

      Example:

      "Improve Customer Satisfaction"

    2. Quantitative or numeric targets are specific.

      Example:

      "Improve Customer Satisfaction from 70% to 80%"

  2. The IRS will use actual results against qualitative and numeric targets to report on agency progress in delivering its tax administration responsibilities.

  3. All levels of the organization can establish qualitative targets to support organizational strategies and plans.

  4. Organizations should base numeric targets for any measure on a review of prior year results, historical patterns, the anticipated mix of resources available, the link to organizational priorities and initiatives and an assessment of existing and emerging trends, issues and problems.

    Note:

    Organizations should establish a baseline for first-year measures and base future targets on first-year actuals.

Communicating Targets

  1. The IRS must exercise caution in how numeric targets are shared within the organization to avoid numbers-focused pressures.

  2. Servicewide and organizational unit numeric targets will be included in documents that are distributed broadly both within and outside the organization, including performance budget submissions, Business Performance Review (BPR) documents and internal program plans. These documents and the numeric targets included therein can be shared and discussed with both managerial and non-managerial employees at the discretion of each organizational unit’s management.

  3. The following questions may help guide the decision to provide numerical targets:

    1. What is the business reason for communicating the numeric target?

    2. What is the business risk of not providing the numeric target?

    3. What is the potential undesirable outcome that could come from the misuse of the numeric target and how can that outcome be minimized?

  4. Regarding (3) above, consider:

    1. The degree of organizational knowledge and understanding of the intended recipients.

    2. The organizational climate at the time and place of the communication.

    3. The context in which the communication will to be made.

    4. Any guidance on how the numeric target can or cannot be used.

    5. How the communication is delivered.

    6. The expectation of follow-up with respect to the numeric target and the nature of the expected follow-up.

    7. The probable perception of the communication of the numeric target internally and externally.

Evaluating the Performance of an Organizational Unit

  1. In conducting a formal review of an organization's performance, the IRS will use the balanced performance measures as the indicators used to determine achievement of the targets outlined in the annual program plan.

  2. The primary focus of organizational reviews should be on the actions taken and related accomplishments, not numeric results. Reviews also should be forward-looking, - using the information and results obtained to identify plan revisions and improvement opportunities.

  3. During an organizational review, units can communicate the numeric results achieved for any of the balanced performance measures to provide a point of reference for a more detailed discussion of the actions taken to help achieve the IRS mission and strategic goals.

  4. Organizational units may use workload indicators and diagnostic tools to help explain factors that may affect the balanced performance measures results.

Evaluating Performance of an Individual

  1. For detailed information on evaluating individual performance see IRM 6.430.3, Performance Management Program for Evaluating Managers, Management Officials and Confidential Management/Program Analysts.

  2. The IRS will base non-supervisory employee evaluations on critical elements or performance standards, as appropriate, and the review of work performed. The IRS will not directly tie individual evaluations to achievement of numeric goals.

  3. The IRS will base supervisory evaluations on the actions taken in accordance with an agreed upon plan and performance standards. Evaluations may mention the achievement of numeric goals in the narrative section in manager evaluations as analytical input to determine whether the actions taken achieved the desired performance result. Direct measures of performance may not use numeric measures.

  4. Using the above guidance, when focusing on actions taken to achieve desired performance results, a supervisor's written evaluations or performance discussions may cite the specific goal or result to demonstrate the effect of the supervisor's actions.

    Example:

    "John Smith met the performance goals established in his performance plan. As a direct result of John implementing (cite specific actions taken), his office closed 500 more cases than the goal."

Analyzing Performance Using the Traditional Problem Solving Model

  1. The IRS may use balanced performance measures and corresponding underlying metrics to identify and understand changes in performance and to identify improvement options.

  2. The Traditional Problem Solving Model is an approach the IRS has used effectively and has several steps.

    1. Receive Data

      1. Organizational units obtain data from a variety of sources such as performance results, quality review results and data collected by the Taxpayer Advocate. In other instances, Organizational units may have to develop data locally.

      2. Organizational units may often repeat this step after they define a problem. For example, an initial scan of data, such as survey results, may suggest that a problem exists with customer service and lead to the development of an initial problem statement. This in turn would require additional data collection to accurately define the problem.

    2. Define the Problem

      1. In some cases, the problem may be obvious. In other cases, you will need to analyze the data to identify the problem.

      2. State the problem in objective terms. An accurately worded problem statement is important for the other steps in the process.

      Example:

      "Taxpayers do not have a sufficient understanding of IRS procedures."
      Not: "IRS employees are not explaining our procedures sufficiently to taxpayers."

    3. Determine Potential Causes of the Problem

      1. Determining potential causes requires research. Look at all the data available to you. Talk to employees and peers and search out potential causes.

      2. Assess the possible causes. Don't jump to conclusions about the solution.

      3. Evaluate causes and prioritize them based upon their effect on the problem.

    4. Generate Alternative Solutions

      1. Include all involved parties to generate alternatives.

      2. Specify alternatives consistent with organizational goals.

      3. Specify short and long term alternatives.

      4. Brainstorm using others’ ideas.

    5. Evaluate and Select an Alternative

      1. Evaluate all alternatives without bias.

      2. Evaluate alternatives relative to goals.

      3. Evaluate both proven and possible outcomes.

      4. State the selected alternative explicitly.

    6. Implement and Follow Up on the Solution

      1. Plan and implement a pilot test of the chosen alternative.

      2. Gather feedback from all affected parties.

      3. Seek acceptance or consensus by those affected.

      4. Establish ongoing measures and monitoring.

      5. Evaluate long-term results based on the final solution.

      Note:

      In cases where you have no solutions that work, you will need to repeat step (4) Generate Alternative Solutions. Alternatively, consider re-evaluating the problem again as sometimes you may not find a solution because the problem definition is not well-defined.

Proposing, Reviewing and Updating Performance Budget Measures

  1. OMB Circular A-11 requires the IRS to prepare a performance budget. A performance budget is a presentation that clearly explains the relationship between performance goals and the costs for achieving targeted performance levels. In general, a performance budget links strategic goals with related outcome-oriented, long-term and annual performance goals and the costs of specific activities that contribute to achieving those goals.

  2. While management will need to collect enough measures to capture all aspects of performance, external policy makers will be primarily interested in the success of the program. Therefore, for purposes of the performance budget submission, measures that focus on results and the desired achievement of a program or service are preferable.

  3. Since balanced performance measures reported by the IRS externally are subject to extensive evaluation and review, this set of measures should meet an especially rigorous set of standards. In general, it is best to ensure the validity and integrity of all measures to ensure decision makers can base their decisions on accurate, reliable data.

    1. Meaningful - Is the measure significant and directly related to the mission and goal?

      Example:

      If the goal of a program is to provide service to taxpayers, measures that tell how well the IRS served those taxpayers are most meaningful.

    2. Valid - Does the measure accurately represent the desired output?

      Example:

      A measure of customer satisfaction based only on internal data does not fully reflect the purpose of the measure.

    3. Responsibility linked - Is the measure matched to the organization responsible for achieving it?

      Example:

      A measure of quality aggregated from various programs and services across organizational units is not responsibility linked, since no clear organization has responsibility for improving the level of performance reflected in the aggregate measure.

    4. Customer focused - Does the measure reflect the customers’ and stakeholders’ point of view? Organizational units should measure the quality of their work products or services in part on the customers’ definition of quality.

    5. Credible - Is the measure based on accurate and reliable data? Organizational units should choose cases selected for a quality measure at random to ensure the resulting score accurately reflects the work performed.

    6. Cost effective - Is the measure based upon acceptable data collection and processing costs? Is the measure easy to calculate, interpret and understand? If the costs of collecting statistically valid data at all levels of the organization outweigh the expected benefits of the information obtained, the organization may choose to limit the measure to higher levels or require a lower level of precision.

    7. Comparable - Is the measure useful for making comparisons with other data over time? An organization may consider basing an internal quality score on the same sample of cases/calls used to measure customer satisfaction to compare internal results to external perceptions over time.

    Division commissioners and chiefs are accountable for performance results. Organizational units are responsible for providing the associate CFO for Corporate Budget with accurate and timely progress reports for their organization's performance budget measures to ensure that the IRS meets internal and external reporting deadlines.

  4. Establishing Performance Targets for Budget Submissions

    Organizations should base their performance targets for the annual budget requests to OMB and Congress on current year actual performance and a set of assumptions. They should provide to the associate CFO for Corporate Budget each performance target and explain the basis on which the IRS established the target and known factors that contribute to increases or decreases in performance, such as inflation or new workload. Assumptions must be included for all:

    1. Output/outcome measures (that is, unit count measures).

    2. Efficiency measures (that is, the assumed number of full-time equivalents (FTEs) or staff years (SYs) and their productivity). Fully loaded FTEs or SYs should be used for this purpose unless there are sound business reasons for an exception. Organizational units should note and fully document such exceptions.

    3. Accuracy or quality measures assumptions must lay out the contributing factors and their relative weight to the overall score.

      Example:

      A quality measure might include weights of one-third each on following the correct process, accuracy and speed of response.

    Corporate Budget will submit the list of budget level measures, proposed targets and explanations of known factors influencing planned increases or decreases in performance to the deputy commissioners. The deputy commissioners will approve or disapprove the proposed targets before they are included in the IRS budget submission.

  5. Performance Measures - Review and Reporting

    1. Each operating division conducts the Business Performance Review (BPR) process quarterly. During these reviews, division commissioners and chiefs discuss their progress on meeting their performance targets or goals and new or emerging issues that may impact major programs and/or performance.

    2. The Department of the Treasury requires the IRS to report progress on its suite of performance budget measures quarterly in the Performance Measures Manager (PMM) system. This typically occurs twenty to thirty days following the end of a quarter.

    3. The IRS reports its performance on the budget level measures in the annual Management's Discussion and Analysis Report (MD&A) and the annual Congressional Budget Justification.

  6. Performance Measures - Updating

    As priorities change and new program requirements evolve, the suite of measures included in the budget must adjust. These changes are most likely to occur during strategic assessments, with new initiatives, from changes in funding levels or as a result of new legislation. Generally, the IRS cannot change or drop measures that have been published in the Congressional Justification (CJ) or Agency Financial Report (AFR) in the middle of a year. Division commissioners or chiefs may request revising, adding or dropping performance budget measures by written memorandum to the associate CFO for Corporate Budget or the CFO. Examples of appropriate changes include:

    1. Replacing a quality measure with a new one concurrent with a change in the quality review process.

    2. Adding a measure for a new program or one that is gaining attention and resources.

    3. Eliminating a measure for a program that is being de-emphasized. Organizations should review their suite of measures periodically to ensure alignment with strategic goals and objectives.

    4. Changing a measure when OMB passback occurs and there is an adverse impact on a performance budget measure.

    5. During the development of the CJ.

    The IRS must make any changes to the performance budget measures before the start of the fiscal year; the IRS never drops measures in the middle of a year.

  7. Changes to Historical (Previously Published) Performance Data

    1. The IRS cannot make changes to historical data that has been reported in the IRS CJ, the MD&A or the AFR.

    Note:

    While this section deals exclusively with changes to budget level measures, there may be instances where an organization has requirements to report performance externally for non-budget level measures. The same requirements regarding approval, elimination and timing will apply. All changes must be formally approved by the division commissioner or chief and communicated to the appropriate office of the external organization.

Questions and Answers about the IRS Balanced Performance Measurement System

Q1. How do organizational measures link to individual front-line appraisals?
The IRS bases individual appraisals of front-line employees (non-management officials) on critical elements for their positions. The critical elements in some areas reflect the IRS priorities as reflected in the balanced measures. All employee standards include the retention standard requiring the "fair and equitable treatment of taxpayers." Overall, the IRS must base employee evaluations on a review of actual work performed, judged against elements and standards, with consideration given to the specific facts and circumstances of each case.
Q2. How do these organizational measures link to individual manager appraisals?
The IRS bases managerial evaluations on the actions taken in accordance with an agreed upon plan and performance standards. The IRS uses measures in managerial evaluations as an analytical input to determine whether the actions taken achieved the desired performance result. The IRS shall not use these organizational measures as a stand-alone evaluative tool.
Q3. What guarantees are there in the process to prevent managers from relying on "numbers?"
The IRS expects managers to look not just at the measurement results, but at the facts, circumstances and specific situations in any area the measures warrant attention. Additional steps to reinforce the balanced performance measures approach include changes to the review processes and the individual performance management processes.
Q4. Will the IRS compare numerical results against prior years to assess progress?
The IRS will compare performance results with prior year results to assess progress. This does not mean the IRS will determine a manager’s appraisal by how measures results change over the year. The results can change for numerous reasons, many of which are not under the control of the manager. The actions taken by the manager to improve performance will be what influences the manager’s appraisal.
Q5. How will the IRS measure customer satisfaction?
The IRS bases the customer satisfaction measure on customer perceptions of the service they receive, as obtained by telephone, mail or paper surveys when visiting a Taxpayer Assistance Center. Survey responses provide the basis for identifying areas with the greatest potential for improvement.
Q6. How will the IRS measure employee satisfaction?
The IRS bases employee satisfaction on the information gathered through the annual federal employee viewpoint survey. All employees have an opportunity to take the employee survey under conditions that guarantee them anonymity.
Q7. How will the IRS measure quality in business results?
The IRS generally will base quality on an independent review of closed cases or ongoing case work using systems such as the National Quality Review System.
Q8. How will the IRS measure quantity in business results?
Quantity consists of outcome-neutral measures that count outreach activities or the amount of work completed, such as cases closed. The IRS may use results from quantity measures as an input in evaluating the performance of an organizational unit if the evaluation also considers measures of customer satisfaction, employee satisfaction and quality, when appropriate.
Q9. How will the IRS know when the measures and goals for customer satisfaction, employee satisfaction and business results are in balance?
Each element of the Balanced Performance Measurement System represents an important aspect for assessing progress toward the IRS’s goals. Any activity involving balanced measures, such as setting goals, assessing progress and evaluating results, must consider all three elements. Performance results help shape future plans and strategies for overall performance. The mix of programs and strategies proposed should cover all three elements. The purpose of the IRS balanced performance measures approach is to ensure each element gets due consideration. The IRS must prioritize critical issues and select a mix of strategies and programs aimed at attaining overall progress toward achieving the IRS mission and strategic goals.
Q10. How will the IRS use the results of performance measures to set budget priorities?
The IRS will use balanced measurement results to assess the effectiveness of strategies and actions undertaken to improve performance and those results will serve as a factor in budget development. The IRS will use performance results, detailed analysis of factors affecting program performance, and an assessment of existing and emerging trends to determine the appropriate mix of program strategies and allocation of resources necessary to achieve the IRS’s goals.
Q11. Does the balanced performance measurement system limit organizational units from implementing procedures for prioritizing work or setting standards for certain processes and procedures?
No. The IRS does not intend the Balanced Performance Measurement System to limit organizational units from developing work plans, establishing projections for workload indicators, issuing procedures for performing work or setting standards for the prioritization and timeliness of work to ensure efficient workload management and quality service to customers. The goal of the balanced performance measurement system is to ensure the IRS makes business decisions considering each element of the balanced measures framework.

Example:

When establishing timeliness standards, the organizational unit needs to ensure they have the appropriate resources and are equipped to meet those standards so that they do not affect customer satisfaction and quality.

Q12. Is it permissible to discuss failure to meet aspects of a work plan or business plan with a manager?
Yes. However, the conversation must include a discussion of what factors may have contributed to the manager not being able to meet the plan and what actions the manager took and/or should have taken to rectify the situation. There may be a very good reason the IRS did not meet the plan or there may be other circumstances that require a revision to the plan, for example, a reallocation of resources or a change in priorities. Finally, during the discussion, be sure to consider and relate the effect of the performance issue being discussed, as well as the effect of potential corrective actions to the balanced measures elements of business results, customer satisfaction and employee satisfaction.
Q13. Can the IRS tell employees that a service standard for the telephone assistance program is that the IRS should not leave taxpayers on hold for longer than a set number of minutes?
Yes. In the interests of customer service and business performance, the IRS needs to establish organizational timeliness standards to support improved service to all customers. The IRS should base such standards on the level of resources available and the length of time necessary to provide top quality service to each caller. The agency should not establish standards it cannot meet. Doing so leads to sacrifices in quality and poor service to customers. The focus of any evaluation against timeliness standards should not be on the number, but on the appropriateness of the actions taken. In some instances, it will be necessary to leave taxpayers on hold for longer periods of time.
Q14. Is it permissible to rely on system data or reports to evaluate front-line employees?
No, with the exception of employees performing measured work (for example, Submission Processing uses the Measured Employee Performance System (MEPS) for frontline employee evaluations). System data, such as talk-time, call histories and sign-on and sign-off times, can serve as diagnostic indicators that point to areas where performance issues may exist. The manager needs to address these, and through review and monitoring of the employee's work, determine whether an actual performance problem exists.

Example:

System data alone could suggest a performance problem, but upon further review and monitoring, managers might find a valid or appropriate cause for the system results.

Q15. Why is cycle time or overage considered a quality measure and not a quantity measure?
Under the balanced performance measurement system, quantity measures provide information about the amount of work performed and products or services provided. Quality measures provide information about the extent that work performed met prescribed standards, including accuracy, timeliness and completeness. Therefore, cycle time and overage measures assess the timeliness in which employees complete work. They do not reveal how much work was completed. However, each organizational unit may make its own determination about whether managers may use certain categories of outcome-neutral data in quality or quantity measures.

Checklists for Developing Measures, Targets and Operational Measures (Metrics)

Measures to ensure results achievement

Measures Development Checklist Yes No
Is the measure results-oriented? Does it indicate achievement of outcomes?    
Is the measure specific and meaningful?    
Is this measurable?    
Will someone be accountable for the measure?    
Is the measure time bound? Does it indicate when the measure will realize results?    
 

Targets set milestones for results achievement

Targets Development Checklist Yes No
Have targets been established using baseline data or industry standards?    
Are targets aggressive, yet attainable?    
Do targets make sense to staff?    
Do targets measure positive progress?    
Do targets encompass a reasonable time frame?    
 

Metrics or Operational Measures track short-term progress toward results

Metrics Development Checklist Yes No
Do metrics reflect that which is most important to the customer?    
Will these metrics track progress and activity completion?    
Can the IRS collect and analyze this data?    
Is the data collection burden worth the result?    

Defining Data: What Counts As Positive Performance?

Measure  
Data Elements: What will the IRS collect?  
Data Source: Where is the data located?  
Data Definition/key terms  
Who is accountable for this measure?  
Who is responsible for collecting the data?  
What is the data collection process?  
How will the IRS calculate this measure?  
How often will the IRS collect this data?  
When and where will the IRS report the data?  

Process to Create a Performance Model for New (or Revised) Programs

Identify IRS Strategic Goal Addressed Identify End Outcomes Develop Intermediate Outcomes Identify Activities and Outputs to Achieve Outcomes (1) Develop Metrics and Output Measures to Track Activities Prioritize and Choose Measures (1) Set Targets for Measures Identify and Allocate Resources Needed to Achieve Outcomes Define, Collect, Analyze, and Report on Measures
Review the IRS Strategic Plan. What is the program’s "bottom line?" What needs to change to achieve outcome goals? What will produce changes needed to achieve outcome? What is trackable to assess progress of activities? Identify links to strategic plan. What baseline data exists? Use normal budget process or new initiative process. Use Data Collection Checklist.
Review Goals, Objectives, and Activities in the IRS Strategic Plan. What evidence would you need to defend the program? Who has the capacity to make the change? What specific things will influence the target of change? What are the numerical values of outputs and activities? Link and track with budgets. If none, what baseline data do you need to collect?   Will this be: Budget level? Operational? Oversight Board?
  What is the end benefit? Is the outcome measurable? What products do you need? What efficiency measures concern customers and stakeholders? Assess results.      
  How will you know you have been successful?   What services do you need?   Alter and improve strategies, activities and processes based on data. (1) Refers to Objectives and Activities in the IRS Strategic Plan.

Detailed Measures Template

REQUIRED INFORMATION DESCRIPTION
Operating Division(s) Operating division(s) that owns this measure
Measure Name Name of measure/indicator
Type of Measure Identify whether this is a strategic measure, balanced measure, diagnostic tool, workload indicator or other non-performance statistic. Identify the component of the Balanced Measurement System that this measure applies to: customer satisfaction, employee satisfaction, business results quality or business results quantity.
  1. Program Category

  2. Related Strategic Goal

  1. Identify program category of performance measure (or Budget Activity Level), such as Pre-Filing Services, Filing and Account Services or Compliance Services.

  2. Identify the strategic goal that the measure supports in the current IRS strategic plan.

Responsible Official The IRS official responsible for the accuracy of the measure, such as: SB/SE director, Finance, Research and Strategy.
Definition Provide a clear narrative explanation of the measure that can be easily understood by non-IRS reviewers.
  1. Reporting Level(s)

  2. Report Data Source

  3. Reports

  1. Identify the organizational level(s) for which the measure/indicator is being used

    Example:

    Operating division, area, center, territory, division, branch, office, etc.).

  2. Identify the source of the data

    Example:

    Integrated Data Retrieval System (IDRS), Command Code ISTSR).

  3. Identify where the organization reports the information

    Example:

    Business Performance Review Summary (BPRS), Monthly Business Performance Summary (MBPS) and strategy and program plans.

Formula/Methodology Provide a detailed narrative describing exactly how the IRS calculates the measure/indicator, including overall approach, scoring methodology, reporting period, etc.

Example:

Rolling averages for period ending xx/xx/xxxx), important status codes, if any, special cases (for example, suspense items)

Data Source/Measurement Tools Identify the report or management information system that provides the data and identify the mechanism used to collect the data being reported

Example:

Survey, sample reviews, audit

Reliability of Data Describe the process or procedures used to verify and/or validate the data collected. Then, describe the data quality of the measure by using the appropriate term: "Reasonable Accuracy" or "Questionable or Unknown Accuracy."
Reasonable Accuracy: There is a reliable system or process in place that validates or verifies the accuracy of the data being reported.
Questionable or Unknown Accuracy: A reliable system or process to validate or verify the accuracy of data is lacking. In such cases, the IRS should add a statement regarding efforts underway to improve the reliability of reported data.
Frequency of Data Availability/Reporting Frequency with which the IRS has measure/indicator data available and reports it, (daily, weekly, monthly, etc.). If the IRS reports the measure/indicator for periods different (that is, less often) than when the data is available, explain.
Purpose of Measure Describe how the measure applies to the segment of IRS operations the IRS is measuring.
Data Limitations Describe any limitations for the measure data.

Example:

The data for ACS Taxpayer Delinquent Account Closures does not include the counts for inventory removed systemically.

Calculation Changes Describe any approved changes in the formula/methodology made during the fiscal year and the purpose for change.

Example:

A change to the methodology to back out calls to automated systems and better reflect the number of calls being handled by live assistors.

Complete Description of the process(s) measure originates from: Critical Path:
1.
2.
3.
4. (add additional items when necessary)
Management Controls for items on critical path (supervisory/management reviews of data quality, include description of how supervisor/manager indicates completion of a review). If there are no supervisor/management reviews, describe how the IRS assures data reliability, completeness and accuracy:
1.
2.
3.
4. (add additional items when necessary)