1.5.1  The IRS Balanced Performance Measurement System

Manual Transmittal

September 04, 2014

Purpose

(1) This transmits revised IRM 1.5.1, Managing Statistics in a Balanced Measurement System, The IRS Balanced Performance Measurement System.

Material Changes

(1) This IRM has been restructured to improve comprehension and flow; therefore, new sections were created and some were combined.

(2) IRM 1.5.1.1, Overview. Item (2) was revised for clarity.

(3) IRM 1.5.1.2, Background - new.

(4) IRM 1.5.1.3, Authorities - new.

(5) IRM 1.5.1.4, Related Resources - new.

(6) IRM 1.5.1.5, Definitions - title changed from Glossary of Terms, which was formerly Exhibit 1.5.1-2.

(7) IRM 1.5.1.7.1, Components of the Balanced Performance Measurement System - new.

(8) Exhibit 1.5.1-5, Detailed Measures Template, was revised to include additional information.

(9) Exhibit 1.5.1-6, The Relationship Between the IRS Mission and a Service Performance Measure, was revised to include current measure information.

(10) and 1.5.1-7, The Relationship Between the IRS Mission and a Compliance Performance Measure, was revised to include current measure information.

Effect on Other Documents

This transmittal supersedes IRM 1.5.1, issued November 1, 2011.

Audience

This transmittal is directed to All IRS Executives and Managers.

Effective Date

(09-04-2014)

Robin L. Canady
Chief Financial Officer

1.5.1.1  (08-21-2009)
Overview

  1. This Internal Revenue Manual (IRM) provides an overview of the IRS Balanced Performance Measurement System and outlines how balanced measures are used to measure organizational performance at the IRS. Since it is not possible to prescribe acceptable actions for every situation, this IRM provides general guidance and direction for IRS strategy and performance offices on the appropriate use and application of balanced measures. The Chief Financial Officer (CFO), Corporate Planning and Internal Control Unit, Office of Corporate Planning and Performance, develops and maintains this IRM.

1.5.1.2  (09-04-2014)
Background

  1. The IRS Balanced Performance Measurement System was developed to reflect IRS priorities consistent with the mission and strategic goals. To help ensure balance, each of the three components of balanced measures - customer satisfaction, employee satisfaction, and business results - is considered by the IRS when setting organizational objectives, establishing targets, assessing progress and results, and evaluating individual performance.

  2. The IRS will use balanced measures at both the strategic level and the operational level to measure organizational performance. Each business unit supports IRS-wide strategic goals. Business units do not establish additional strategic goals. At the individual level, critical elements and critical performance expectations that support and align with the mission and balanced measures approach will be the basis by which employees are evaluated. Exhibits 1.5.1-6 and 1.5.1-7 demonstrate the relationship between the IRS mission and performance measures used in the budget to support service and enforcement programs.

1.5.1.3  (09-04-2014)
Authorities

  1. The IRS Restructuring and Reform Act of 1998 (RRA '98) required the IRS to change its measures to balance customer service with overall tax administration responsibilities.

  2. 26 CFR Part 801, Balanced System for Measuring Organizational and Employee Performance, implements the IRS Balanced Performance Measurement System.

1.5.1.4  (09-04-2014)
Related Resources

  1. IRM 1.5.2, Managing Statistics in a Balanced Measurement System, Uses of Section 1204 Statistics.

  2. IRM 1.5.3, Managing Statistics in a Balanced Measurement System, RRA 98 Section 1204 Self-Certification and the Independent Review Process.

1.5.1.5  (09-04-2014)
Definitions

  1. Below are descriptions of terms used throughout this IRM.

    1. Business Results Measures - Indicators of the quality and quantity of work performed.

    2. Customer - Any internal or external person or entity to whom you provide services.

    3. Customer Satisfaction Measure - An indicator of the level of overall satisfaction with service provided by the IRS as perceived by internal and external customers.

    4. Diagnostic Tools and/or Workload Indicator - Indicators that are used to discover the factors impacting changes in the balanced measures.

    5. Efficiency Measures - Capture the operation's skillfulness in executing programs, implementing activities, and achieving results, while avoiding wasted resources, effort, time, and/or money. Efficiency is the ratio of the outcome or output to the input of any program.

    6. Enterprise-wide Measures - A select few measures which cross several organizational unit lines that demonstrate the IRS effectiveness in meeting its strategic goals. Enterprise-wide measures are not used in annual reports or in the performance budget.

    7. IRS Balanced Measures - Indicators of organizational performance for Customer Satisfaction, Employee Satisfaction, and Business Results.

    8. Outcome Measures - Describe the intended result from carrying out a program or activity. Define an event or condition external to the program/activity that is of direct importance to the intended beneficiaries and/or the public.

    9. Outcome Neutral - Production or resource data that do not contain information regarding the tax enforcement result reached in any case, such as the number of cases closed, level of service provided, and assistance and outreach efforts undertaken.

    10. Output Measures - Describe the level of activity that will be provided over a period of time, including a description of the characteristics established as standards for the activity, such as timeliness.

    11. Performance Budget - A budget that links performance goals with costs for achieving a target level of performance. A performance budget links strategic goals with related long-term and annual performance goals (outcomes) with the costs of specific activities to influence these outcomes.

    12. Performance Budget Level Measures - Long-term and annual performance measures used to support the annual performance budget.

    13. Performance Goal - Sets a target level of performance over time expressed as a tangible, measurable objective, against which actual achievement can be compared, including a goal expressed as a quantitative standard, value, or rate. A performance goal is comprised of a performance measure with targets and time frames.

    14. Performance Measures - Indicators, statistics, or metrics used to gauge program performance.

    15. Program Assessment - A determination, through objective measurement and systematic analysis, of the manner and extent to which Federal programs achieve intended objectives.

    16. Quality Measure - A numeric indicator of the extent to which completed work meets prescribed standards.

    17. Quantity Measure - An indicator of outreach efforts, outcome neutral productivity, and resource utilization.

    18. Stakeholders - Groups or individuals who have a vested interest in an organization. They can be internal or external to the IRS.

    19. Strategic Goal - A statement of aim or purpose included in a strategic plan. In a performance budget/performance plan, strategic goals should be used to group multiple program outcome goals. Each program outcome goal should relate to and, in the aggregate, be sufficient to influence the strategic goals or objectives and their performance measures.

    20. Target - Quantifiable or otherwise measurable characteristic that tells how well a program must accomplish a performance measure.

1.5.1.6  (08-21-2009)
Performance Measures Quick Reference

  1. The following are key points of this section.

    1. Each of the three components of balanced measures - customer satisfaction, employee satisfaction, and business results - must be carefully considered when setting organizational objectives, establishing goals, assessing progress and results, and evaluating individual performance.

    2. Under the IRS Balanced Performance Measurement System, the approved set of balanced measures is the primary means for assessing organizational performance. Other information and data can be used for purposes such as workload planning and analysis.

    3. Targets for any measure should be set based on a review of the previous year's results, historical patterns and the anticipated mix of resources available, the linkage to organizational priorities and initiatives, planned process improvements or system enhancements, and an assessment of existing and emerging trends, issues, and problems.

    4. Targets set for business results of quantity must always consider the effects on customer satisfaction, employee satisfaction, and quality. Quantity measures may be used to evaluate and/or set or suggest goals for organizational units at levels which do not measure customer satisfaction, employee satisfaction, or quality, provided such measures exist at a higher level in the organization.

    5. The performance of any one unit should not be used as a standard by which the performance of any other unit is evaluated because of differences in the types of taxpayers served, specific tax issues addressed, and other factors.

    6. The primary focus of organizational reviews should be on the correctness of actions taken and related accomplishments.

    7. The numeric results achieved for any measure will never directly equate to the evaluation of an individual.

    8. Because measures reported by the IRS externally are subject to extensive evaluation and review, an especially rigorous set of standards should be met.

    9. In determining the appropriateness of a specific course of action, managers and employees are encouraged to exercise sound judgment consistent with 26 CFR Part 801.

1.5.1.7  (11-01-2011)
The IRS Balanced Performance Measurement System

  1. The IRS Balanced Performance Measurement System provides a means to:

    1. Establish priorities.

    2. Guide and motivate performance and establish a link between performance targets and organizational goals and objectives.

    3. Obtain feedback to identify how well the IRS meets customer and stakeholder expectations and to identify areas for improvement.

    4. Assess overall program effectiveness and communicate results.

1.5.1.7.1  (09-04-2014)
Components of the Balanced Performance Measurement System

  1. The components of the balanced performance measurement system are customer satisfaction, employee satisfaction, and business results, with business results being comprised of measures of Quantity and Quality.

  2. Each component represents an important aspect of the organization's goals and each should be given equal importance in carrying out the objectives of IRS programs. Some of the components do not change as rapidly as others or require more time for data collection; therefore, the frequency of data availability across the three components may vary. However, differences in the frequency of data availability do not reflect differences in priority.

1.5.1.7.2  (09-04-2014)
The Goals of Customer Satisfaction, Employee Satisfaction, and Business Results

  1. Goals of the Balanced Performance Measurement Components:

    1. Customer Satisfaction - to measure how well the IRS provides accurate and professional services to internal and external customers in a courteous, timely manner.

    2. Employee Satisfaction - to measure how well the IRS creates an enabling work environment for employees by providing quality leadership, adequate training, and effective support services.

    3. Business Results - to measure how much work the IRS produces in a quality manner and to provide meaningful outreach to all customers.

1.5.1.7.3  (11-01-2011)
Measuring Customer Satisfaction

  1. Information to measure customer satisfaction is normally gathered from a census or from a statistically valid sample of the taxpayers served by a particular operating division or who have recently interacted with a particular IRS program. This is most often accomplished through the use of surveys for the organizational unit overall and transactional surveys for its major programs that:

    1. Affect a significant proportion of the customers served by the organizational unit, and/or

    2. Use a significant proportion of the total resources (FTE and dollars) of the organizational unit.

  2. While customer feedback should be obtained, where possible, for all programs, a survey instrument or satisfaction score may not be possible or meaningful for every program.

  3. The intent of the customer satisfaction component of the balanced performance measurement system is to ensure customer satisfaction issues and needs are considered and addressed in organizational planning, budgeting and review activities.

    Example:

    When developing strategies to improve business results, the impact on customer satisfaction outcomes should be considered.

  4. Organizations should review survey results on a regular basis and develop plans that include specific strategies aimed at improving customer satisfaction.

    Note:

    Customers are guaranteed anonymity consistent with 26 CFR Part 801.4.

1.5.1.7.4  (08-21-2009)
Measuring Employee Satisfaction

  1. Employee satisfaction is measured for organizational units, sub-units, and work groups. With some exceptions, it is typically not measurable for programs because employee time is often allocated to more than one program and many programs are spread across organizational units.

  2. The intent of the employee satisfaction component of a balanced performance measurement system is to ensure employee satisfaction issues and needs are considered and addressed in organizational planning, budgeting, and review activities.

    Example:

    Strategies to improve business results must consider and address, when applicable, the impact on employees.

  3. Organizations should review annual employee survey results and develop plans that contain specific strategies aimed at improving employee satisfaction, if applicable.

1.5.1.7.5  (08-21-2009)
Measuring Business Results

  1. Business results are assessed at the operational level through measures of output or quantity and measures of efficiency or quality. Output or quantity measures consist of outcome-neutral production and resource data like the number of cases closed, work items completed, and hours expended. Quality or efficiency measures consist of data such as case/call review data, accuracy, and timeliness. The intent of measuring both quantity and quality is to ensure a productive quantity of work is generated in a quality manner.

1.5.1.7.6  (09-04-2014)
Performance Measures Objectives

  1. Performance measures can be established to assess progress at three levels:

    1. Budget level measures are used by the IRS to support IRS budget requests. They identify expected levels of organizational performance at both the strategic level and the operational level for major budget activities.

    2. Strategic level measures are used to assess overall progress in delivering on the mission and goals established by the Strategic Plan. These measures apply to the organization as a whole. Each IRS business operating division supports the IRS-wide strategic goals; therefore, they do not establish individual strategic goals.

    3. Operational level measures are used to assess the effectiveness of specific programs.

      Example:

      Submission Processing, Filing and Accounts Management, Field Collection, and Field Exam.

  2. The IRS has three types of performance measures:

    1. Outcome Measures describe the intended result of carrying out a program or activity.

    2. Output or Quantity Measures describe the level of activity that will be provided over a period of time.

    3. Efficiency or Quality Measures capture skillfulness in executing programs, implementing activities, and achieving results while avoiding wasted resources, effort, time and/or money.

      Note:

      A baseline is established for first-year measures and future targets are based on first-year actuals.

1.5.1.8  (11-01-2011)
Using Performance Information

  1. The IRS collects information about programs and services, some of which is used as performance measures. Other information and data are used for workload planning and analysis.

  2. The Balanced Performance Measurement System enables the IRS to use performance results to:

    1. Measure how an organizational unit has performed relative to its past performance.

    2. Identify areas for improvement taking into consideration results from measures of customer satisfaction, employee satisfaction, and business results.

    3. Align and support various review processes.

    4. Identify the factors that influence performance.

    5. Provide input to the manager performance appraisal, but not serve as a direct evaluative tool.

1.5.1.9  (11-01-2011)
Using Workload Indicators and Diagnostic Tools for Planning and Analysis

  1. The IRS uses workload indicators for resource planning during the development of program plans and budgets. Workload indicators project expected levels of activity for an organization or program, identify resource needs, and justify the IRS budget requests to the Department of the Treasury, the Office of Management and Budget (OMB), and Congress.

    Example:

    Refunds issued, number of walk-in customers, and number of returns filed.

  2. Workload indicators are included in organizational performance assessments to compare actual workload to the planned workload for making future projections and estimating resource requirements.

  3. Diagnostic tools are any data used to understand what influences and impacts performance. In some cases, data used as a diagnostic tool for one organization (such as cycle time) may be used as a budget level measure for another organization, as long as the measure conforms to the guidelines and restrictions set forth in this IRM.

  4. The use of diagnostic tools and workload indicators provides a mechanism to study the factors that influence performance and identify improvement opportunities that can be taken by managers to improve customer satisfaction, employee satisfaction, and business results.

  5. Do not use diagnostic tools or workload indicators to measure individual performance.

  6. Improvement targets may be established for diagnostic tools and workload indicators but only in direct support of overlying budget or operational level measures.

  7. Using results from diagnostic tools or workload indicators to compare one unit against other units may be appropriate for conducting analysis, exploring best practices, or seeking process enhancements to support improvement of the overarching balanced measure(s).

  8. Use diagnostic tools and/or workload indicators to identify and understand underlying factors that cause changes in the balanced Measures.

  9. Examples of Diagnostic Tools/Workload Indicators:

    1. Customer satisfaction survey results such as: responses to particular survey questions, improvement priorities identified, and narrative comments

    2. Employee survey results

    3. Cycle time

    4. Employee experience/training/skill levels such as: hours of training per employee, work force mix, and average education level of newly-hired employees.

    5. External factors such as: tax law macroeconomic cycles

    6. Employee absenteeism and/or turnover rates

    7. Return closures per unit of effort

    8. Inventory level

    9. IMF/BMF workload mix

    10. Staffing resources such as: FTE appropriated, FTE realized, and resource utilization

    11. Results for individual quality standards/elements

    12. Wait time/transaction time

    13. Cost information

    14. Number of returns filed

    15. Records of Tax Enforcement Results (ROTERs) (More information on ROTERs can be found in IRM 1.5.2, Managing Statistics in a Balanced Measurement System, Uses of Section 1204 Statistics.)

      Note:

      No targets will be set and no comparisons between units will be made for ROTERs.

  10. Examples on how to apply diagnostic tools and workload indicators when conducting analyses are available in the Reference Guide for Diagnostic Tools.

1.5.1.10  (11-01-2011)
Setting Targets or Performance Goals

  1. There are two types of targets an organization can use to communicate priorities and guide performance, qualitative and quantitative:

    1. Qualitative targets are general in nature and suggest a desired direction.

      Example:

      "Improve Customer Satisfaction"

    2. Quantitative or numeric targets are specific.

      Example:

      "Improve Customer Satisfaction from 70% to 80%"

  2. Actual results against qualitative and numeric targets will be used by the IRS to report on agency progress in delivering its tax administration responsibilities.

  3. Qualitative targets can be established at all levels of the organization to support organizational strategies and plans.

  4. Numeric targets for any measure should be set based on a review of prior year results, historical patterns, the anticipated mix of resources available, the link to organizational priorities and initiatives, and an assessment of existing and emerging trends, issues and problems.

    Note:

    A baseline is established for first-year measures and future targets are based on first-year actuals.

1.5.1.10.1  (11-01-2011)
Communicating Targets

  1. The IRS needs to exercise caution in how numeric targets are shared within the organization in order to avoid numbers-focused pressures.

  2. Servicewide and organizational unit numeric targets will be included in organizational documents that are distributed broadly both within and outside the organization such as the performance budget submission, Business Performance Review (BPR) documents, and internal program plans. These documents and the numeric targets included therein can be shared and discussed with both managerial and non-managerial employees at the discretion of management within each organizational unit.

  3. The following questions may help guide the decision to provide numerical targets:

    1. What is the business reason for communicating the numeric target?

    2. What is the business risk of not providing the numeric target?

    3. What is the potential undesirable outcome that could come from the misuse of the numeric target, and how can that outcome be minimized?

  4. Regarding (3) above, consider:

    1. The degree of organizational knowledge and understanding of the intended recipient(s).

    2. The organizational climate at the time and place of the communication.

    3. The context in which the communication is to be made.

    4. Any guidance on how the numeric target can or cannot be used.

    5. The manner in which the communication is delivered.

    6. The expectation of follow-up with respect to the numeric target and the nature of the expected follow-up.

    7. The probable perception of the communication of the numeric target internally and externally.

1.5.1.11  (08-21-2009)
Evaluating the Performance of an Organizational Unit

  1. In conducting a formal review of an organization's performance, the balanced performance measures will be the indicators used to determine achievement of the targets outlined in the annual program plan.

  2. The primary focus of organizational reviews should be on the actions taken and related accomplishments, not numeric results. Reviews should also be forward-looking - using the information and results obtained to identify plan revisions and improvement opportunities.

  3. The numeric results achieved with any of the balanced performance measures can be communicated orally and in writing in an organizational review to provide a point of reference for a more detailed discussion of the actions taken to help achieve the IRS mission and strategic goals.

  4. Workload indicators and diagnostic tools may be used to help explain factors that may have affected the balanced performance measures results.

1.5.1.12  (11-01-2011)
Evaluating Performance of an Individual

  1. Non-supervisory employee evaluations will be based upon critical elements or performance standards, as appropriate, and the review of work performed. Individual evaluations will not be directly tied to achievement of numeric goals.

  2. Supervisory evaluations will be based on the actions taken in accordance with an agreed upon plan and performance standards. Achievement of numeric goals may be mentioned in the narrative section in manager evaluations as an analytical input to determine whether the actions taken achieved the desired performance result. They are not to be used as a direct measure of performance.

  3. Using the above guidance, when focusing on actions taken to achieve desired performance results, a supervisor's written evaluations or performance discussions may cite the specific goal or result to demonstrate the effect of the supervisor's actions.

    Example:

    "John Smith met the performance goals established in his performance plan. As a direct result of John implementing (cite specific actions taken), his office closed 500 more cases than the goal."

1.5.1.13  (11-01-2011)
Analyzing Performance Using the Traditional Problem Solving Model

  1. Balanced performance measures and corresponding underlying metrics can be used to identify and understand changes in performance and identify improvement options.

  2. The Traditional Problem Solving Model is an approach the IRS has used effectively and has several steps.

  3. Receive Data

    1. Data are obtained from a variety of sources such as performance results, quality review results, and data collected by the Taxpayer Advocate. In other instances, data may have to be locally developed.

      Note:

      This step is often repeated again after a problem has been defined.

      Example:

      An initial scan of data, such as survey results, may suggest a problem exists with customer service and lead to the development of an initial problem statement which requires additional data collection in order to more accurately define the actual problem.

  4. Define the Problem

    1. In some cases, the problem may be obvious. In other cases, you will need to analyze the data to identify the problem.

    2. State the problem in objective terms. An accurately worded problem statement is important for the other steps in the process.

      Example:

      "Taxpayers do not have a sufficient understanding of IRS procedures."
      Not: "IRS employees are not explaining our procedures sufficiently to taxpayers."

  5. Determine Potential Causes of the Problem

    1. Determining potential causes requires research. Look at all the data available to you. Talk to employees and peers, and search out potential causes.

    2. Assess the possible causes. Don't jump to conclusions about the solution.

    3. Evaluate causes and prioritize them based upon their impact on the problem.

  6. Generate Alternative Solutions

    1. Include all involved parties to generate alternatives.

    2. Specify alternatives consistent with organizational goals.

    3. Specify short and long term alternatives.

    4. Brainstorm on others’ ideas.

  7. Evaluate and Select an Alternative

    1. Evaluate all alternatives without bias.

    2. Evaluate alternatives relative to goals.

    3. Evaluate both proven and possible outcomes.

    4. State the selected alternative explicitly.

  8. Implement and Follow Up on the Solution

    1. Plan and implement a pilot test of the chosen alternative.

    2. Gather feedback from all affected parties.

    3. Seek acceptance or consensus by all those affected.

    4. Establish ongoing measures and monitoring.

    5. Evaluate long-term results based on the final solution.

    Note:

    In cases where you have no solutions that work, you will need to repeat step (5) Generate Alternative Solutions. Alternatively, consider re-evaluating the problem again as sometimes you may not find a solution because the problem definition is not well-defined.

1.5.1.14  (08-21-2009)
Integrating the Balanced Performance Measurement System

  1. To integrate and use the Balanced Performance Measurement System throughout the IRS, each organization must establish:

    1. A comprehensive approach for measuring and utilizing customer satisfaction, employee satisfaction, and business results (quantity and quality) data.

    2. A clear strategy for using all elements of balanced measures in strategic, operational, and business planning.

    3. A commitment to ensuring, explaining, and demonstrating how customer satisfaction, employee satisfaction, and business results are being used and addressed in the business decisions of the organization.

1.5.1.15  (09-04-2014)
Proposing, Reviewing, and Updating Performance Budget Measures

  1. The IRS is required under OMB Circular A-11 to prepare a performance budget. A performance budget is a presentation that clearly explains the relationship between performance goals and the costs for achieving targeted levels of performance. In general, a performance budget links strategic goals with related outcome-oriented, long-term, and annual performance goals and the costs of specific activities that contribute to the achievement of those goals.

  2. While management will need to collect enough measures to capture all aspects of performance, external policy makers will be primarily interested in the success of the program. Therefore, for purposes of the performance budget submission, measures that focus on results and the desired achievement of a program or service are preferable.

  3. Because balanced performance measures reported by the IRS externally are subject to extensive evaluation and review, an especially rigorous set of standards should be met for this set of measures. In general, it is best to ensure the validity and integrity of all measures to ensure decision making is based on accurate, reliable data.

    1. Meaningful - Is the measure significant and directly related to the mission and goal?

      Example:

      If the goal of a program is to provide service to taxpayers, measures that tell how well those taxpayers were served are most meaningful.

    2. Valid - Does the measure accurately represent what is being measured?

      Example:

      A measure of customer satisfaction based only on internal data does not fully reflect what is purported as being measured.

    3. Responsibility linked - Is the measure matched to an organization responsible for achieving that measure?

      Example:

      A measure of quality aggregated from various programs and services across organizational units is not responsibility linked, as no clear organization has responsibility for improving the level of performance reflected in the aggregate measure.

    4. Customer focused - Does the measure reflect the point of view of customers and stakeholders? Standards used to determine the quality of a work product or service should be based in part on the customers’ definition of quality.

    5. Credible - Is the measure based on accurate and reliable data? Cases selected for a quality measure should be chosen at random to ensure the resulting score accurately reflects the work performed.

    6. Cost effective - Is the measure based upon acceptable data collection and processing costs? Is the measure easy to calculate, interpret and understand? If the costs of collecting statistically valid data at all levels of the organization outweigh the expected benefits of the information obtained, the organization may choose to limit the measure to higher levels or require a lower level of precision.

    7. Comparable - Is the measure useful for making comparisons with other data over time? An organization might consider basing an internal quality score on the same sample of cases/calls used to measure customer satisfaction to compare internal results to external perceptions over time.

    Division Commissioners and Chiefs are accountable for performance results. Organizational units are responsible for providing Corporate Planning and Internal Control (CPIC) with accurate and timely progress reports for their organization's performance budget measures to ensure that internal and external reporting deadlines are met.

  4. Establishing Performance Targets for Budget Submissions

    Performance targets for the fiscal year the budget request is being developed are based on current year actual performance and have a set of assumptions. Divisions shall provide to the Chief Financial Officer (CFO) assumptions for each performance target in the annual budget submission to OMB and Congress. These assumptions explain the basis on which the target was established and include an explanation of known factors that contribute to increases or decreases in performance, such as inflation or new workload. Assumptions must be included for all:

    1. Output/outcome measures (i.e., unit count measures).

    2. Efficiency measures (i.e., the assumed number of full-time equivalents (FTEs) or staff years (SYs) and their productivity). Fully loaded FTEs or SYs should be used for this purpose unless there are sound business reasons for an exception. Such exceptions should be noted and fully documented.

    3. Accuracy or quality measures assumptions must lay out the contributing factors and their relative weight to the overall score.

      Example:

      A quality measure might include weights of one-third each on following the correct process, accuracy and speed of response.

    CPIC will submit the list of budget level measures, proposed targets, and explanations of known factors influencing planned increases or decreases in performance to the Deputy Commissioners. The Deputy Commissioners will approve or disapprove the proposed targets before they are included in the IRS budget submission.

  5. Performance Measures - Review and Reporting

    1. The Business Performance Review (BPR) process is conducted quarterly for each operating division. During these reviews, Division Commissioners and Chiefs discuss their progress on meeting their performance targets or goals and new or emerging issues that may impact major programs and/or performance.

    2. The IRS is required by the Department of the Treasury to report progress on its suite of performance budget measures quarterly in the Budget Formulation and Execution Manager (BFEM) system. Typically this occurs between the 11th and 14th day of the month following the end of a quarter.

    3. The IRS is required to report its performance on the budget level measures in the annual Management's Discussion and Analysis Report (MD&A) to the Government Accountability Office (GAO) and the annual Congressional Budget Justification to OMB and Congress.

  6. Performance Measures - Updating

    As priorities change and new program requirements evolve, the suite of measures included in the budget must adjust. These changes are most likely to occur during strategic assessments, with new initiatives, changes in funding levels, or as a result of new legislation. Generally, measures that have been published in the Congressional Justification (CJ) or Agency Financial Report (AFR) cannot be changed or dropped in the middle of a year. Only Division Commissioners or Chiefs may request revising, adding, or dropping performance budget measures. Such requests must be submitted to the CFO or the CFO designee via official memorandum for approval. Examples of appropriate changes include:

    1. Replacing a quality measure with a new one concurrent with a change in the quality review process.

    2. Adding a measure for a new program or one that is gaining attention and resources.

    3. Eliminating a measure for a program that is being de-emphasized.

    4. When the passback of the current year budget from OMB occurs and there is an adverse impact on a performance budget measure.

    5. During the development of the CJ.

    Any changes to the performance budget measures must be made before the start of the fiscal year; measures are never dropped in the middle of a year.

  7. Changes to Historical (Previously Published) Performance Data.

    1. Changes cannot be made to historical data that has been reported in the IRS Congressional Budget Justification, the MD&A, or the AFR.

    Note:

    While this section deals exclusively with changes to budget level measures, there may be instances where an organization is required to report performance externally for non-budget level measures. The same requirements regarding approval, elimination, and timing will apply. All changes must be formally approved by the Division Commissioner or Chief and communicated to the appropriate office of the external organization.

1.5.1.16  (11-01-2011)
Detailed Measures Template Requirement

  1. Before proposing a new measure or indicator for inclusion in the IRS budget or the internal strategy and program plans, an organization must ensure the detailed measures template included in Exhibit 1.5.1–5 has been completed.

  2. The template facilitates a common understanding across the organization of a measure’s definition, formula, data reliability, reporting frequency, etc. and is essential in meeting various oversight requirements.

Exhibit 1.5.1-1 
Questions and Answers about the IRS Balanced Performance Measurement System

Q1. How do organizational measures link to individual front-line appraisals?
Individual appraisals of front-line employees are based on critical elements for their positions. The critical elements in some areas reflect the IRS priorities as reflected in the balanced measures. All employee standards reflect the retention standard that requires the "fair and equitable treatment of taxpayers." Overall, evaluations of individual employees must be based on a review of actual work performed as judged against elements and standards and with consideration given to the specific facts and circumstances of each case.
Q2. How do these organizational measures link to individual manager appraisals?
Managerial evaluations will be based on the actions taken in accordance with an agreed upon plan and performance standards. Measures are used in managerial evaluations as an analytical input to determine whether the actions taken achieved the desired performance result. They are not to be used as a stand-alone evaluative tool.
Q3. What guarantees are there in the process to prevent managers from relying on "numbers?"
Managers will be expected to look not just at the measurement results, but at the facts, circumstances, and specific situations in any area the measures warrant attention. Managerial expectations support this change and additional steps to reinforce a balanced performance measures approach include changes to the review processes and the individual performance management processes.
Q4. Will the IRS compare numerical results against prior years to assess progress?
Performance results will be compared with prior year results to assess progress. This does not mean a manager’s appraisal is determined by how measures results change over the year. The results can change for numerous reasons, many of which are not under the control of the manager. The actions taken by the manager to improve performance will be what influences the manager’s appraisal.
Q5. How will the IRS measure customer satisfaction?
The customer satisfaction measure is based on customer perceptions of the service they receive, as obtained by telephone, mail, or paper surveys when visiting a Taxpayer Assistance Center. Survey responses provide the basis for identifying areas with the greatest potential for improvement.
Q6. How will the IRS measure employee satisfaction?
Employee satisfaction results are based on the information gathered through the annual Employee Survey. All employees have an opportunity to take the employee survey under conditions that guarantee them anonymity.
Q7. How will the IRS measure quality in business results?
Quality will generally be based on an independent review of closed cases or ongoing case work using systems such as the National Quality Review System.
Q8. How will the IRS measure quantity in business results?
Quantity consists of outcome-neutral measures that count outreach activities or the amount of work completed, such as cases closed. Results from quantity measures may be used as an input in evaluating the performance of an organizational unit as long as the evaluation also considers measures of customer satisfaction, employee satisfaction, and quality, when appropriate.
Q9. How will the IRS know when the measures and goals for customer satisfaction, employee satisfaction, and business results are in balance?
Each element of the Balanced Performance Measurement System represents an important aspect for assessing progress toward the IRS goals. Any activity involving balanced measures, such as setting goals, assessing progress, and evaluating results, must consider all three elements. Performance results help shape future plans and strategies for overall performance. The mix of improvement programs and strategies proposed should cover all three elements. The purpose of the IRS balanced performance measures approach is to ensure each element is given due consideration. The IRS must prioritize critical issues and select a mix of strategies and programs aimed at achieving overall progress toward achievement of the IRS mission and strategic goals.
Q10. How will the IRS use the results of performance measures to set budget priorities?
The IRS will use balanced measurement results to assess the effectiveness of strategies and actions undertaken to improve performance, and those results will serve as a factor in budget development. Performance results, detailed analysis of factors affecting program performance, and an assessment of existing and emerging trends, are all factors that will be used to determine the appropriate mix of program strategies and allocation of resources necessary to achieve the IRS goals.
Q11. Does the balanced performance measurement system limit organizational units from implementing procedures for prioritizing work or setting standards for certain processes and procedures?
No. The Balanced Performance Measurement System is not intended to limit the IRS from developing work plans, establishing projections for workload indicators, issuing procedures for the manner in which work will be performed, or setting standards on the prioritization and timeliness of work in order to ensure efficient workload management and quality service to customers. The goal of the balanced performance measurement system is to ensure business decisions are made in consideration of each element of the balanced measures framework.

Example:

When establishing timeliness standards, the organizational unit needs to ensure the organization has appropriate resources and is equipped to meet those standards so that customer satisfaction and quality are not affected.

Q12. Is it permissible to discuss failure to meet aspects of a work plan or business plan with a manager?
Yes. However, the conversation must include a discussion of what factors may have contributed to the manager not being able to meet the plan and what actions the manager took and/or should have taken to rectify the situation. There may be a very good reason the plan was not met or there may be other issues that require a revision to the plan, a reallocation of resources, or a change in priorities. Finally, during the discussion, be sure to consider and relate the impact of the performance issue being discussed, as well as the impact of potential corrective actions to the balanced measures elements of business results, customer satisfaction, and employee satisfaction.
Q13. Can employees be told that a service standard for the telephone assistance program is taxpayers should not be left on hold for longer than five minutes?
Yes. In the interests of customer service and business performance, the IRS needs to establish organizational timeliness standards in order to support improved service to all customers. Such standards should be based on the level of resources available and the length of time necessary to provide top quality service to each caller. The agency should not establish standards it cannot meet. Doing so leads to sacrifices in quality and poor service to customers. The focus of any evaluation against timeliness standards should not be on the number, but on the appropriateness of the actions taken. Obviously, in some instances, it will be necessary to leave taxpayers on hold for longer periods of time.
Q14. Is it permissible to rely on system data or reports to evaluate front-line employees?
With the exception of employees performing measured work (Submission Processing uses the Measured Employee Performance System (MEPS) for frontline employee evaluations) the answer is No. System data, such as talk-time, call histories, and sign-on and sign-off times, can serve as diagnostic indicators that point to areas where performance issues may exist which the manager needs to address, and through review and monitoring of the employee's work, determine whether an actual performance problem exists.

Example:

System data alone could suggest a performance problem, but upon further review and monitoring, a valid or appropriate cause for the system results might be found.

Q15. Why is cycle time or overage considered a quality measure and not a quantity measure?
Under the balanced performance measurement system, quantity measures provide information about the amount of work performed and products or services provided. Quality measures provide information about the extent to which work performed met prescribed standards, including accuracy, timeliness, and completeness. Therefore, cycle time and overage measures assess the timeliness in which work is completed. They do not tell us how much work was completed. However, each organizational unit may make its own determination as to whether certain categories of outcome-neutral data are to be used in quality or quantity measures.

Exhibit 1.5.1-2 
Checklists for Developing Measures, Targets, and Operational Measures (Metrics)

Measures to ensure results achievement

Measures Development Checklist Yes No
Is the measure results-oriented? Does it indicate achievement of outcomes?    
Is the measure specific and meaningful?    
Can this be measured?    
Will someone be held accountable for the measure?    
Is the measure time bound? Does it indicate when the results will be realized?    

Targets set milestones for results achievement

Targets Development Checklist Yes No
Have targets been established using baseline data or industry standards?    
Are targets aggressive, yet attainable?    
Do targets make sense to staff?    
Do targets measure positive progress?    
Do targets encompass a reasonable time frame?    

Metrics or Operational Measures track short-term progress toward results

Metrics Development Checklist Yes No
Do metrics reflect that which is most important to the customer?    
Will these metrics track progress and activity completion?    
Can this data be collected and analyzed?    
Is the data collection burden worth the result?    

Exhibit 1.5.1-3 
Defining Data: What Counts As Positive Performance?

Measure  
Data Elements: What will be collected?  
Data Source: Where is the data located?  
Data Definition/key terms  
Who is accountable for this measure?  
Who is responsible for collecting the data?  
What is the data collection process?  
How will this measure be calculated?  
How often will the data be collected?  
When and where will the data be reported?  

Exhibit 1.5.1-4 
Process to Create a Performance Model for New (or Revised) Programs

Identify IRS Strategic Goal Addressed Identify End Outcomes Develop Intermediate Outcomes Identify Activities and Outputs to AchieveOutcomes (1) Develop Metrics and Output Measures to Track Activities Prioritize and ChooseMeasures (1) Set Targets for Measures Identify and AllocateResources Needed to AchieveOutcomes Define, Collect, Analyze, and Report on Measures
Review the IRS Strategic Plan. What is the program’s "bottom line?" What needs to change to achieveoutcome goals? What will producechanges needed to achieveoutcome? What can be tracked to assess progress of activities? Identify links to strategic plan. What baseline data exists? Use normal budget process or new initiativeprocess. Use Data Collection Checklist.
Review Goals, Objectives, and Strategies in the IRS Strategic Plan. What evidence would you need to defend the program? Who has the capacity to make the change? What specific things will influence the target of change? What are the numerical values of outputs and activities? Link and track with budgets. If none, what baseline data needs to be collected?   Will this be: Budget level? BEFM?Operational? Oversight Board?
  What is the end benefit? Can the outcome be measured? What products are needed? What efficiency measures concerncustomers and stakeholders? Assess results.      
  How will you know you have been successful?   What services are needed?   Alter and improvestrategies, activities, and processes based on data. (1) Refers to Objectives and Strategies in the IRS Strategic Plan.

Exhibit 1.5.1-5 
Detailed Measures Template

REQUIRED INFORMATION DESCRIPTION
Operating Division(s) Operating Division(s) that owns this measure
Measure Name Name of measure/indicator
Type of Measure Identify whether this is a Strategic Measure, Balanced Measure, Diagnostic Tool, Workload Indicator, or other non-performance statistic. Identify the component of the Balanced Measurement System that this measure applies to: Customer Satisfaction, Employee Satisfaction, Business Results Quality or Business Results Quantity.
  1. Program Category

  2. Related Strategic Goal

  1. Identify program category of performance measure (or Budget Activity Level), such as Pre-Filing Services, Filing and Account Services, or Compliance Services.

  2. Identify the strategic goal that the measure supports in the current IRS Strategic Plan.

Responsible Official The IRS official responsible for the accuracy of the measure, such as: SB/SE Director, Finance, Research and Strategy.
Definition Provide a clear narrative explanation of the measure that can be easily understood by non-IRS reviewers.
  1. Reporting Level(s)

  2. Report Data Source

  3. Reports

  1. Identify the organizational level(s) for which the measure/indicator is being used

    Example:

    Operating Division, Area, Center, Territory, Division, Branch, Office, etc.).

  2. Identify the source of the data

    Example:

    Integrated Data Retrieval System (IDRS), Command Code ISTSR).

  3. Identify where the information is reported

    Example:

    Business Performance Review Summary (BPRS), Monthly Business Performance Summary (MBPS), and Strategy and Program Plans.

Formula/Methodology Provide a detailed narrative describing exactly how the measure/indicator is calculated, including overall approach, scoring methodology, reporting period, etc.

Example:

Rolling averages for period ending . . . ), important status codes, if any, special cases to be considered (suspense items, etc.)

Data Source/Measurement Tools Identify the report or management information system that provides the data and identify the mechanism by which the data being reported is collected

Example:

Survey, sample reviews, audit, etc.

Reliability of Data Describe the process or procedures used to verify and/or validate the data collected. Then, describe the data quality of the measure by using the appropriate term: "Reasonable Accuracy" or "Questionable or Unknown Accuracy."
Reasonable Accuracy: There is a reliable system or process in place that validates or verifies the accuracy of the data being reported.
Questionable/Unknown Accuracy: A reliable system or process to validate or verify the accuracy of data is lacking. In such cases, a statement should be added regarding efforts underway to improve the reliability of reported data.
Frequency of Data Availability/Reporting Frequency with which measure/indicator data is available and reported, (daily, weekly, monthly, etc.). If the measure/indicator is reported for periods different (i.e. less often) than when the data is available, explain.
Purpose of Measure Describe how the measure applies to the segment of IRS operations being measured.
Data Limitations Describe any limitations for the measure data.

Example:

The data for ACS Taxpayer Delinquent Account Closures does not include the counts for inventory removed systemically.

Calculation Changes Describe any approved changes in the formula/methodology made during the fiscal year and the purpose for change.

Example:

A change to the methodology to back out calls to automated systems and better reflect the number of calls being handled by live assistors.

Complete Description of the process(s) measure originates from: Critical Path:
1.
2.
3.
4. (add additional items when necessary)
Management Controls for items on critical path (supervisory/management reviews of data quality, include description of how supervisor/manager indicates completion of a review). If there are no supervisor/management reviews, describe how data reliability, completeness and accuracy is assured:
1.
2.
3.
4. (add additional items when necessary)

Exhibit 1.5.1-6 
The Relationship Between the IRS Mission and a Service Performance Measure

This image is too large to be displayed in the current screen. Please click the link to view the image.

Exhibit 1.5.1-7 
The Relationship Between the IRS Mission and a Compliance Performance Measure

This image is too large to be displayed in the current screen. Please click the link to view the image.

More Internal Revenue Manual