- 1.5.1 The IRS Balanced Performance Measurement System
- 184.108.40.206 Overview
- 220.127.116.11 Background
- 18.104.22.168 Authorities
- 22.214.171.124 Related Resources
- 126.96.36.199 Definitions
- 188.8.131.52 Performance Measures Quick Reference
- 184.108.40.206 The IRS Balanced Performance Measurement System
- 220.127.116.11.1 Components of the Balanced Performance Measurement System
- 18.104.22.168.2 The Goals of Customer Satisfaction, Employee Satisfaction, and Business Results
- 22.214.171.124.3 Measuring Customer Satisfaction
- 126.96.36.199.4 Measuring Employee Satisfaction
- 188.8.131.52.5 Measuring Business Results
- 184.108.40.206.6 Performance Measures Objectives
- 220.127.116.11 Using Performance Information
- 18.104.22.168 Using Workload Indicators and Diagnostic Tools for Planning and Analysis
- 22.214.171.124 Setting Targets or Performance Goals
- 126.96.36.199.1 Communicating Targets
- 188.8.131.52 Evaluating the Performance of an Organizational Unit
- 184.108.40.206 Evaluating Performance of an Individual
- 220.127.116.11 Analyzing Performance Using the Traditional Problem Solving Model
- 18.104.22.168 Integrating the Balanced Performance Measurement System
- 22.214.171.124 Proposing, Reviewing, and Updating Performance Budget Measures
- 126.96.36.199 Detailed Measures Template Requirement
- Exhibit 1.5.1-1 Questions and Answers about the IRS Balanced Performance Measurement System
- Exhibit 1.5.1-2 Checklists for Developing Measures, Targets, and Operational Measures (Metrics)
- Exhibit 1.5.1-3 Defining Data: What Counts As Positive Performance?
- Exhibit 1.5.1-4 Process to Create a Performance Model for New (or Revised) Programs
- Exhibit 1.5.1-5 Detailed Measures Template
- Exhibit 1.5.1-6 The Relationship Between the IRS Mission and a Service Performance Measure
- Exhibit 1.5.1-7 The Relationship Between the IRS Mission and a Compliance Performance Measure
Part 1. Organization, Finance, and Management
Chapter 5. Managing Statistics in a Balanced Measurement System
Section 1. The IRS Balanced Performance Measurement System
October 26, 2016
(1) This transmits revised IRM 1.5.1, Managing Statistics in a Balanced Measurement System, The IRS Balanced Performance Measurement System.
(1) This revision involves an editorial change, only. The Planning, Programming, and Audit Coordination (PPAC) office has revised this IRM to reflect new ownership.
(2) PPAC has also stylistically revised all sections to comply with the Plain Writing Act of 2010.
Chief, Planning, Programming and Audit Coordination
This Internal Revenue Manual (IRM) provides an overview of the IRS Balanced Performance Measurement System and outlines how the IRS uses balanced measures to measure organizational performance. Since it is not possible to prescribe acceptable actions for every situation, this IRM provides general guidance and direction for IRS strategy and performance offices on the appropriate use and application of balanced measures.
The Office of Planning, Programming, and Audit Coordination is the program office which oversees strategic priorities and audit management for the IRS.
The IRS developed the Balanced Performance Measurement System to reflect IRS priorities consistent with the mission and strategic goals. To help ensure balance, the IRS considers each of the three components of balanced measures - customer satisfaction, employee satisfaction, and business results - when setting organizational objectives, establishing targets, assessing progress and results, and evaluating individual performance. Each component should be given equal importance.
The IRS will use balanced measures at both the strategic level and the operational level to measure organizational performance. Each business unit supports IRS-wide strategic goals. Business units do not establish additional strategic goals. At the individual level, critical elements and critical performance expectations that support and align with the mission and balanced measures approach will be the basis by which the IRS evaluates employees. Exhibits 1.5.1-6 and 1.5.1-7 demonstrate the relationship between the IRS mission and performance measures used in the budget to support service and enforcement programs.
The IRS Restructuring and Reform Act of 1998 (RRA '98) requires the IRS to change its measures to balance customer service with overall tax administration responsibilities.
26 CFR 801, Balanced System for Measuring Organizational and Employee Performance, implements the IRS Balanced Performance Measurement System.
IRM 1.5.2, Uses of Section 1204 Statistics.
IRM 1.5.3, RRA 98 Section 1204 Self-Certification and the Independent Review Process.
IRM 6.430.3, Performance Management Program for Evaluating Managers, Management Officials and Confidential Management/Program Analysts.
Below are descriptions of terms used throughout this IRM.
Business Performance Reviews - Quarterly reviews conducted by the IRS for each business unit that review a variety of topics including organizational performance, key initiatives, risks, budget, staffing and other considerations as applicable.
Business Results Measures - Indicators of the quality and quantity of work performed.
Customer - Any internal or external person or entity to whom you provide services.
Customer Satisfaction Measure - An indicator of the level of overall satisfaction with service provided by the IRS as perceived by internal and external customers.
Cycle Time - total time from the beginning to the end of a process, as defined by the business operating division within the IRS.
Diagnostic Tools and/or Workload Indicator - Indicators used to discover the factors impacting changes in the balanced measures.
Efficiency Measures - Capture the operation's skillfulness in executing programs, implementing activities, and achieving results, while avoiding wasted resources, effort, time, and/or money. Efficiency is the ratio of the outcome or output to the input of any program.
Enterprise-wide Measures - A select few measures which cross several organizational unit lines that demonstrate the IRS effectiveness in meeting its strategic goals. Enterprise-wide measures are not used in annual reports or in the performance budget.
IRS Balanced Measures - Indicators of organizational performance for Customer Satisfaction, Employee Satisfaction, and Business Results.
Outcome Measures - Describe the intended result from carrying out a program or activity. Define an event or condition external to the program/activity that is of direct importance to the intended beneficiaries and/or the public.
Outcome Neutral - Production or resource data that do not contain information regarding the tax enforcement result reached in any case, such as the number of cases closed, level of service provided, and assistance and outreach efforts undertaken.
Output Measures - Describe the level of activity that will be provided over a period of time, including a description of the characteristics established as standards for the activity, such as timeliness.
Overage - an excess or surplus of resources.
Performance Budget - A budget that links performance goals with costs for achieving a target level of performance. A performance budget links strategic goals with related long-term and annual performance goals (outcomes) with the costs of specific activities to influence these outcomes.
Performance Budget Level Measures - Long-term and annual performance measures used to support the annual performance budget.
Performance Goal - Sets a target level of performance over time expressed as a tangible, measurable objective, against which actual achievement can be compared, including a goal expressed as a quantitative standard, value, or rate. A performance goal is comprised of a performance measure with targets and time frames.
Performance Measures - Indicators, statistics, or metrics used to gauge program performance.
Program Assessment - A determination, through objective measurement and systematic analysis, of the manner and extent to which Federal programs achieve intended objectives.
Quality Measure - A numeric indicator of the extent to which completed work meets prescribed standards.
Quantity Measure - An numeric indicator of outreach efforts, outcome neutral productivity, and resource utilization.
Stakeholders - Groups or individuals who have a vested interest in an organization. They can be internal or external to the IRS.
Strategic Goal - A statement of aim or purpose included in a strategic plan. In a performance budget/performance plan, strategic goals should be used to group multiple program outcome goals. Each program outcome goal should relate to and, in the aggregate, be sufficient to influence the strategic goals or objectives and their performance measures.
Target - Quantifiable or otherwise measurable characteristic that tells how well a program must accomplish a performance measure.
The following are key points of this section.
The IRS must carefully consider each of the three components of balanced measures - customer satisfaction, employee satisfaction, and business results - when setting organizational objectives, establishing goals, assessing progress and results, and evaluating individual performance.
Under the IRS Balanced Performance Measurement System, the approved set of balanced measures is the primary means for assessing organizational performance. The IRS can use other information and data for purposes such as workload planning and analysis.
Each measure should have targets based on a review of the previous year's results, historical patterns and the anticipated mix of resources available, the linkage to organizational priorities and initiatives, planned process improvements or system enhancements, and an assessment of existing and emerging trends, issues, and problems.
Targets set for business results of quantity must always consider the effects on customer satisfaction, employee satisfaction, and quality. The IRS may use quantity measures to evaluate and/or set or suggest goals for organizational units at levels which do not measure customer satisfaction, employee satisfaction, or quality, provided such measures exist at a higher level in the organization.
The IRS should not use the performance of any one unit as a standard by which the performance of any other unit is evaluated because of differences in the types of taxpayers served, specific tax issues addressed, and other factors. However, some measures at the enterprise level may combine aspects of multiple business units into one measure. In these cases, individual business units should track their own performance separately from the combined measure.
The primary focus of organizational reviews should be on the correctness of actions taken and related accomplishments.
The numeric results achieved for any measure will never directly equate to the evaluation of an individual.
Because measures reported by the IRS externally are subject to extensive evaluation and review, measures should meet an especially rigorous set of standards.
In determining the appropriateness of a specific course of action, managers and employees should exercise sound judgment consistent with 26 CFR Part 801.
The IRS Balanced Performance Measurement System provides a means to:
Guide and motivate performance and establish a link between performance targets and organizational goals and objectives.
Obtain feedback to identify how well the IRS meets customer and stakeholder expectations and to identify areas for improvement.
Assess overall program effectiveness and communicate results.
The components of the balanced performance measurement system are Customer Satisfaction, Employee Satisfaction, and Business Results, with Business Results being comprised of measures of Quantity and Quality.
Each component represents an important aspect of the organization's goals and each should be given equal importance in carrying out the objectives of IRS programs. Some of the components do not change as rapidly as others or require more time for data collection; therefore, the frequency of data availability across the three components may vary. However, differences in the frequency of data availability do not reflect differences in priority.
The IRS’ overall objective is to attain:
a high level of customer satisfaction as indicated by providing accurate professional services to both internal and external customers courteously and timely;
a high level of employee satisfaction as indicated by achieving a productive and enabling work environment with quality leadership, adequate training, and effective support services; and
an adequate level of quality business results, coupled with meaningful outreach to customers.
The IRS normally gathers information to measure customer satisfaction from a statistically valid sample of the taxpayers served by a particular operating division or who have recently interacted with a particular IRS program. The IRS accomplishes this through the use of surveys for the organizational unit overall and transactional surveys for its major programs that:
Affect a significant proportion of the customers served by the organizational unit, and/or
Use a significant proportion of the total resources (FTE and dollars) of the organizational unit.
While the IRS should obtain customer feedback for all programs, a survey instrument or satisfaction score may not be possible or meaningful for every program.
The intent of the customer satisfaction component of the balanced performance measurement system is to ensure the IRS considers and addresses customer satisfaction issues and needs in organizational planning, budgeting and review activities.
Organizations should review survey results on a regular basis and develop plans that include specific strategies aimed at improving customer satisfaction.
The IRS measures employee satisfaction for organizational units, sub-units, and work groups. With some exceptions, it is typically not measurable for programs because the IRS allocates employee time to more than one program and spreads many programs across organizational units.
The intent of the employee satisfaction component of a balanced performance measurement system is to ensure the IRS considers and addresses employee satisfaction issues and needs in organizational planning, budgeting, and review activities.
Organizations should review annual employee survey results and develop plans that contain specific strategies aimed at improving employee satisfaction, if applicable.
The IRS assesses business results at the operational level through measures of output or quantity and measures of efficiency or quality. Output or quantity measures consist of outcome-neutral production and resource data like the number of cases closed, work items completed, and hours expended. Quality or efficiency measures consist of data such as case/call review data, accuracy, and timeliness. The intent of measuring both quantity and quality is to ensure the IRS generates a productive quantity of work in a quality manner.
The IRS can establish performance measures to assess progress at three levels:
The IRS uses budget level measures to support IRS budget requests.
The IRS uses strategic level measures to assess overall progress in delivering on the mission and goals established by the Strategic Plan. These measures apply to the organization as a whole. Each IRS business operating division supports the IRS-wide strategic goals; therefore, they do not establish individual strategic goals.
The IRS uses operational level measures to assess the effectiveness of specific programs.
The IRS has three types of performance measures:
Outcome Measures describe the intended result of carrying out a program or activity.
Output or Quantity Measures describe the level of activity that will be provided over a period of time.
Efficiency or Quality Measures capture skillfulness in executing programs, implementing activities, and achieving results while avoiding wasted resources, effort, time and/or money.
The IRS collects information about programs and services and uses some of them as performance measures. The IRS uses other information and data for workload planning and analysis.
The Balanced Performance Measurement System enables the IRS to use performance results to:
Measure how an organizational unit performs relative to its past performance.
Identify areas for improvement taking into consideration results from measures of customer satisfaction, employee satisfaction, and business results.
Align and support various review processes.
Identify the factors that influence performance.
Provide input to the manager performance appraisal, but not serve as a direct evaluative tool.
The IRS uses workload indicators for resource planning during the development of program plans and budgets. Workload indicators project expected levels of activity for an organization or program, identify resource needs, and justify the IRS budget requests to the Department of the Treasury, the Office of Management and Budget (OMB), and Congress.
Organizational performance assessments include workload indicators to compare actual workload to the planned workload for making future projections and estimating resource requirements.
Diagnostic tools are any data used to understand what influences and impacts performance. In some cases, organizations may share data used as a diagnostic tool for their organization (such as cycle time) so that other organizations may use it as a budget level measure. This is permissible as long as the measure conforms to the guidelines and restrictions set forth in this IRM.
The use of diagnostic tools and workload indicators provide a mechanism to study the factors that influence performance and identify improvement opportunities that managers can take to improve customer satisfaction, employee satisfaction, and business results.
Organizations within the IRS may establish improvement targets for diagnostic tools and workload indicators but only in direct support of overlying budget or operational level measures.
The IRS using results from diagnostic tools or workload indicators to compare one unit against other units may be appropriate for conducting analysis, exploring best practices, or seeking process enhancements to support improvement of the overarching balanced measure(s).
The IRS uses diagnostic tools and/or workload indicators to identify and understand underlying factors that cause changes in the balanced measures.
Examples of Diagnostic Tools/Workload Indicators:
Customer satisfaction survey results such as: responses to particular survey questions, improvement priorities identified, and narrative comments
Employee survey results
Employee experience/training/skill levels such as: hours of training per employee, work force mix, and average education level of newly-hired employees.
External factors such as: tax law macroeconomic cycles
Employee absenteeism and/or turnover rates
Return closures per unit of effort
IMF/BMF workload mix
Staffing resources such as: FTE appropriated, FTE realized, and resource utilization
Results for individual quality standards/elements
Wait time/transaction time
Number of returns filed
Records of Tax Enforcement Results (ROTERs) (More information on ROTERs can be found in IRM 1.5.2.)
Examples on how to apply diagnostic tools and workload indicators when conducting analyses are available in the Reference Guide for Diagnostic Tools.
There are two types of targets an organization can use to communicate priorities and guide performance, qualitative and quantitative:
The IRS will use actual results against qualitative and numeric targets to report on agency progress in delivering its tax administration responsibilities.
All levels of the organization can establish qualitative targets to support organizational strategies and plans.
Organizations should base numeric targets for any measure on a review of prior year results, historical patterns, the anticipated mix of resources available, the link to organizational priorities and initiatives, and an assessment of existing and emerging trends, issues and problems.
The IRS needs to exercise caution in how numeric targets are shared within the organization in order to avoid numbers-focused pressures.
Servicewide and organizational unit numeric targets will be included in organizational documents that are distributed broadly both within and outside the organization such as the performance budget submission, Business Performance Review (BPR) documents, and internal program plans. These documents and the numeric targets included therein can be shared and discussed with both managerial and non-managerial employees at the discretion of management within each organizational unit.
The following questions may help guide the decision to provide numerical targets:
What is the business reason for communicating the numeric target?
What is the business risk of not providing the numeric target?
What is the potential undesirable outcome that could come from the misuse of the numeric target, and how can that outcome be minimized?
Regarding (3) above, consider:
The degree of organizational knowledge and understanding of the intended recipient(s).
The organizational climate at the time and place of the communication.
The context in which the communication is to be made.
Any guidance on how the numeric target can or cannot be used.
The manner in which the communication is delivered.
The expectation of follow-up with respect to the numeric target and the nature of the expected follow-up.
The probable perception of the communication of the numeric target internally and externally.
In conducting a formal review of an organization's performance, the IRS will use the balanced performance measures as the indicators used to determine achievement of the targets outlined in the annual program plan.
The primary focus of organizational reviews should be on the actions taken and related accomplishments, not numeric results. Reviews should also be forward-looking - using the information and results obtained to identify plan revisions and improvement opportunities.
Organizational units can communicate the numeric results achieved with any of the balanced performance measures in an organizational review to provide a point of reference for a more detailed discussion of the actions taken to help achieve the IRS mission and strategic goals.
Organizational units may use workload indicators and diagnostic tools to help explain factors that may affect the balanced performance measures results.
For detailed information on evaluating individual performance see IRM 6.430.3, Performance Management Program for Evaluating Managers, Management Officials and Confidential Management/Program Analysts.
The IRS will base non-supervisory employee evaluations upon critical elements or performance standards, as appropriate, and the review of work performed. The IRS will not directly tie individual evaluations to achievement of numeric goals.
The IRS will base supervisory evaluations on the actions taken in accordance with an agreed upon plan and performance standards. Evaluations may mention the achievement of numeric goals in the narrative section in manager evaluations as an analytical input to determine whether the actions taken achieved the desired performance result. Direct measures of performance may not use numeric measures.
Using the above guidance, when focusing on actions taken to achieve desired performance results, a supervisor's written evaluations or performance discussions may cite the specific goal or result to demonstrate the effect of the supervisor's actions.
The IRS may use balanced performance measures and corresponding underlying metrics to identify and understand changes in performance and identify improvement options.
The Traditional Problem Solving Model is an approach the IRS has used effectively and has several steps.
Organizational Units obtain data from a variety of sources such as performance results, quality review results, and data collected by the Taxpayer Advocate. In other instances, Organizational Units may have to locally develop data.
Define the Problem
In some cases, the problem may be obvious. In other cases, you will need to analyze the data to identify the problem.
State the problem in objective terms. An accurately worded problem statement is important for the other steps in the process.
Determine Potential Causes of the Problem
Determining potential causes requires research. Look at all the data available to you. Talk to employees and peers, and search out potential causes.
Assess the possible causes. Don't jump to conclusions about the solution.
Evaluate causes and prioritize them based upon their impact on the problem.
Generate Alternative Solutions
Include all involved parties to generate alternatives.
Specify alternatives consistent with organizational goals.
Specify short and long term alternatives.
Brainstorm on others’ ideas.
Evaluate and Select an Alternative
Evaluate all alternatives without bias.
Evaluate alternatives relative to goals.
Evaluate both proven and possible outcomes.
State the selected alternative explicitly.
Implement and Follow Up on the Solution
Plan and implement a pilot test of the chosen alternative.
Gather feedback from all affected parties.
Seek acceptance or consensus by all those affected.
Establish ongoing measures and monitoring.
Evaluate long-term results based on the final solution.
To integrate and use the Balanced Performance Measurement System throughout the IRS, each organization must establish:
A comprehensive approach for measuring and utilizing customer satisfaction, employee satisfaction, and business results (quantity and quality) data.
A clear strategy for using all elements of balanced measures in strategic, operational, and business planning.
A commitment to ensuring, explaining, and demonstrating how it uses and address customer satisfaction, employee satisfaction, and business results in the business decisions of the organization.
OMB Circular A-11 requires the IRS to prepare a performance budget. A performance budget is a presentation that clearly explains the relationship between performance goals and the costs for achieving targeted levels of performance. In general, a performance budget links strategic goals with related outcome-oriented, long-term, and annual performance goals and the costs of specific activities that contribute to the achievement of those goals.
While management will need to collect enough measures to capture all aspects of performance, external policy makers will be primarily interested in the success of the program. Therefore, for purposes of the performance budget submission, measures that focus on results and the desired achievement of a program or service are preferable.
Because balanced performance measures reported by the IRS externally are subject to extensive evaluation and review, this set of measures should meet an especially rigorous set of standards. In general, it is best to ensure the validity and integrity of all measures to ensure decision makers can base their decisions on accurate, reliable data.
Meaningful - Is the measure significant and directly related to the mission and goal?
Valid - Does the measure accurately represent the desired output?
Responsibility linked - Is the measure matched to an organization responsible for achieving that measure?
Customer focused - Does the measure reflect the point of view of customers and stakeholders? Organizational units should base standards used to determine the quality of a work product or service in part on the customers’ definition of quality.
Credible - Is the measure based on accurate and reliable data? Organizational units should choose cases selected for a quality measure at random to ensure the resulting score accurately reflects the work performed.
Cost effective - Is the measure based upon acceptable data collection and processing costs? Is the measure easy to calculate, interpret and understand? If the costs of collecting statistically valid data at all levels of the organization outweigh the expected benefits of the information obtained, the organization may choose to limit the measure to higher levels or require a lower level of precision.
Comparable - Is the measure useful for making comparisons with other data over time? An organization might consider basing an internal quality score on the same sample of cases/calls used to measure customer satisfaction to compare internal results to external perceptions over time.
Division Commissioners and Chiefs are accountable for performance results. Organizational units are responsible for providing Planning, Programming, and Audit Coordination (PPAC) with accurate and timely progress reports for their organization's performance budget measures to ensure that the IRS meets internal and external reporting deadlines.
Establishing Performance Targets for Budget Submissions
Organizations should base performance targets for the fiscal year they are developing the budget request for on current year actual performance and have a set of assumptions. Divisions shall provide to the Chief of Planning, Programming and Audit Coordination (PPAC) assumptions for each performance target in the annual budget submission to OMB and Congress. These assumptions explain the basis on which the IRS established the target and include an explanation of known factors that contribute to increases or decreases in performance, such as inflation or new workload. Assumptions must be included for all:
Output/outcome measures (i.e., unit count measures).
Efficiency measures (i.e., the assumed number of full-time equivalents (FTEs) or staff years (SYs) and their productivity). Fully loaded FTEs or SYs should be used for this purpose unless there are sound business reasons for an exception. Organizational Units should note and fully document such exceptions.
Accuracy or quality measures assumptions must lay out the contributing factors and their relative weight to the overall score.
PPAC will submit the list of budget level measures, proposed targets, and explanations of known factors influencing planned increases or decreases in performance to the Deputy Commissioners. The Deputy Commissioners will approve or disapprove the proposed targets before they are included in the IRS budget submission.
Performance Measures - Review and Reporting
Each operating division conducts the Business Performance Review (BPR) process quarterly. During these reviews, Division Commissioners and Chiefs discuss their progress on meeting their performance targets or goals and new or emerging issues that may impact major programs and/or performance.
The Department of the Treasury requires the IRS to report progress on its suite of performance budget measures quarterly in the Performance Measures Manager (PMM) system. Typically this occurs twenty to thirty days following the end of a quarter.
The IRS reports its performance on the budget level measures in the annual Management's Discussion and Analysis Report (MD&A) and the annual Congressional Budget Justification.
Performance Measures - Updating
As priorities change and new program requirements evolve, the suite of measures included in the budget must adjust. These changes are most likely to occur during strategic assessments, with new initiatives, changes in funding levels, or as a result of new legislation. Generally, the IRS cannot change or drop measures that have been published in the Congressional Justification (CJ) or Agency Financial Report (AFR) in the middle of a year. Only Division Commissioners or Chiefs may request revising, adding, or dropping performance budget measures. They must submit such requests to the Chief of PPAC or the PPAC designee via official memorandum for approval. Examples of appropriate changes include:
Replacing a quality measure with a new one concurrent with a change in the quality review process.
Adding a measure for a new program or one that is gaining attention and resources.
Eliminating a measure for a program that is being de-emphasized. Organizations should periodically review their suite of measures to ensure alignment with strategic goals and objectives.
When the passback of the current year budget from OMB occurs and there is an adverse impact on a performance budget measure.
During the development of the CJ.
The IRS must make any changes to the performance budget measures before the start of the fiscal year; the IRS never drops measures in the middle of a year.
Changes to Historical (Previously Published) Performance Data.
The IRS cannot make changes to historical data that has been reported in the IRS Congressional Budget Justification, the MD&A, or the AFR.
While this section deals exclusively with changes to budget level measures, there may be instances where an organization has requirements to report performance externally for non-budget level measures. The same requirements regarding approval, elimination, and timing will apply. All changes must be formally approved by the Division Commissioner or Chief and communicated to the appropriate office of the external organization.
Before proposing a new measure or indicator for inclusion in the IRS budget or the internal strategy and program plans, an organization must complete the detailed measures template included in Exhibit 1.5.1–5.
The template facilitates a common understanding across the organization of a measure’s definition, formula, data reliability, reporting frequency, etc. and is essential in meeting various oversight requirements.
|Q1. How do organizational measures link to individual front-line appraisals?|
|The IRS bases individual appraisals of front-line employees (non-management officials) on critical elements for their positions. The critical elements in some areas reflect the IRS priorities as reflected in the balanced measures. All employee standards reflect the retention standard that requires the "fair and equitable treatment of taxpayers." Overall, the IRS must base evaluations of individual employees on a review of actual work performed as judged against elements and standards and with consideration given to the specific facts and circumstances of each case.|
|Q2. How do these organizational measures link to individual manager appraisals?|
|The IRS will base managerial evaluations on the actions taken in accordance with an agreed upon plan and performance standards. The IRS uses measures in managerial evaluations as an analytical input to determine whether the actions taken achieved the desired performance result. The IRS shall not use these organizational measures as a stand-alone evaluative tool.|
|Q3. What guarantees are there in the process to prevent managers from relying on "numbers?"|
|The IRS expects managers to look not just at the measurement results, but at the facts, circumstances, and specific situations in any area the measures warrant attention. Managerial expectations support this change and additional steps to reinforce a balanced performance measures approach include changes to the review processes and the individual performance management processes.|
|Q4. Will the IRS compare numerical results against prior years to assess progress?|
|The IRS will compare performance results with prior year results to assess progress. This does not mean the IRS will determine a manager’s appraisal by how measures’ results change over the year. The results can change for numerous reasons, many of which are not under the control of the manager. The actions taken by the manager to improve performance will be what influences the manager’s appraisal.|
|Q5. How will the IRS measure customer satisfaction?|
|The IRS bases the customer satisfaction measure on customer perceptions of the service they receive, as obtained by telephone, mail, or paper surveys when visiting a Taxpayer Assistance Center. Survey responses provide the basis for identifying areas with the greatest potential for improvement.|
|Q6. How will the IRS measure employee satisfaction?|
|The IRS bases employee satisfaction results on the information gathered through the annual Employee Survey. All employees have an opportunity to take the employee survey under conditions that guarantee them anonymity.|
|Q7. How will the IRS measure quality in business results?|
|The IRS will generally base quality on an independent review of closed cases or ongoing case work using systems such as the National Quality Review System.|
|Q8. How will the IRS measure quantity in business results?|
|Quantity consists of outcome-neutral measures that count outreach activities or the amount of work completed, such as cases closed. The IRS may use results from quantity measures as an input in evaluating the performance of an organizational unit as long as the evaluation also considers measures of customer satisfaction, employee satisfaction, and quality, when appropriate.|
|Q9. How will the IRS know when the measures and goals for customer satisfaction, employee satisfaction, and business results are in balance?|
|Each element of the Balanced Performance Measurement System represents an important aspect for assessing progress toward the IRS goals. Any activity involving balanced measures, such as setting goals, assessing progress, and evaluating results, must consider all three elements. Performance results help shape future plans and strategies for overall performance. The mix of improvement programs and strategies proposed should cover all three elements. The purpose of the IRS balanced performance measures approach is to ensure each element gets due consideration. The IRS must prioritize critical issues and select a mix of strategies and programs aimed at achieving overall progress toward achievement of the IRS mission and strategic goals.|
|Q10. How will the IRS use the results of performance measures to set budget priorities?|
|The IRS will use balanced measurement results to assess the effectiveness of strategies and actions undertaken to improve performance, and those results will serve as a factor in budget development. The IRS will use performance results, detailed analysis of factors affecting program performance, and an assessment of existing and emerging trends to determine the appropriate mix of program strategies and allocation of resources necessary to achieve the IRS goals.|
|Q11. Does the balanced performance measurement system limit organizational units from implementing procedures for prioritizing work or setting standards for certain processes and procedures?|
|No. The IRS does not intend the Balanced Performance Measurement System to limit organizational units from developing work plans, establishing projections for workload indicators, issuing procedures for the manner in which work will be performed, or setting standards on the prioritization and timeliness of work in order to ensure efficient workload management and quality service to customers. The goal of the balanced performance measurement system is to ensure the IRS makes business decisions in consideration of each element of the balanced measures framework.|
|Q12. Is it permissible to discuss failure to meet aspects of a work plan or business plan with a manager?|
|Yes. However, the conversation must include a discussion of what factors may have contributed to the manager not being able to meet the plan and what actions the manager took and/or should have taken to rectify the situation. There may be a very good reason the IRS did not meet the plan or there may be other issues that require a revision to the plan, a reallocation of resources, or a change in priorities. Finally, during the discussion, be sure to consider and relate the impact of the performance issue being discussed, as well as the impact of potential corrective actions to the balanced measures elements of business results, customer satisfaction, and employee satisfaction.|
|Q13. Can the IRS tell employees that a service standard for the telephone assistance program is that the IRS should not leave taxpayers on hold for longer than a set number of minutes?|
|Yes. In the interests of customer service and business performance, the IRS needs to establish organizational timeliness standards in order to support improved service to all customers. The IRS should base such standards on the level of resources available and the length of time necessary to provide top quality service to each caller. The agency should not establish standards it cannot meet. Doing so leads to sacrifices in quality and poor service to customers. The focus of any evaluation against timeliness standards should not be on the number, but on the appropriateness of the actions taken. Obviously, in some instances, it will be necessary to leave taxpayers on hold for longer periods of time.|
|Q14. Is it permissible to rely on system data or reports to evaluate front-line employees?|
|With the exception of employees performing measured work (Submission Processing uses the Measured Employee Performance System (MEPS) for frontline employee evaluations) the answer is No. System data, such as talk-time, call histories, and sign-on and sign-off times, can serve as diagnostic indicators that point to areas where performance issues may exist which the manager needs to address, and through review and monitoring of the employee's work, determine whether an actual performance problem exists.|
|Q15. Why is cycle time or overage considered a quality measure and not a quantity measure?|
|Under the balanced performance measurement system, quantity measures provide information about the amount of work performed and products or services provided. Quality measures provide information about the extent to which work performed met prescribed standards, including accuracy, timeliness, and completeness. Therefore, cycle time and overage measures assess the timeliness in which employees complete work. They do not tell us how much work was completed. However, each organizational unit may make its own determination as to whether managers may use certain categories of outcome-neutral data in quality or quantity measures.|
Measures to ensure results achievement
|Measures Development Checklist||Yes||No|
|Is the measure results-oriented? Does it indicate achievement of outcomes?|
|Is the measure specific and meaningful?|
|Is this measurable?|
|Will someone be accountable for the measure?|
|Is the measure time bound? Does it indicate when the measure will realize results?|
Targets set milestones for results achievement
|Targets Development Checklist||Yes||No|
|Have targets been established using baseline data or industry standards?|
|Are targets aggressive, yet attainable?|
|Do targets make sense to staff?|
|Do targets measure positive progress?|
|Do targets encompass a reasonable time frame?|
Metrics or Operational Measures track short-term progress toward results
|Metrics Development Checklist||Yes||No|
|Do metrics reflect that which is most important to the customer?|
|Will these metrics track progress and activity completion?|
|Can the IRS collect and analyze this data?|
|Is the data collection burden worth the result?|
|Data Elements: What will the IRS collect?|
|Data Source: Where is the data located?|
|Data Definition/key terms|
|Who is accountable for this measure?|
|Who is responsible for collecting the data?|
|What is the data collection process?|
|How will the IRS calculate this measure?|
|How often will the IRS collect this data?|
|When and where will the IRS report the data?|
|Identify IRS Strategic Goal Addressed||Identify End Outcomes||Develop Intermediate Outcomes||Identify Activities and Outputs to AchieveOutcomes (1)||Develop Metrics and Output Measures to Track Activities||Prioritize and ChooseMeasures (1)||Set Targets for Measures||Identify and AllocateResources Needed to AchieveOutcomes||Define, Collect, Analyze, and Report on Measures|
|Review the IRS Strategic Plan.||What is the program’s "bottom line?"||What needs to change to achieveoutcome goals?||What will producechanges needed to achieveoutcome?||What is trackable to assess progress of activities?||Identify links to strategic plan.||What baseline data exists?||Use normal budget process or new initiativeprocess.||Use Data Collection Checklist.|
|Review Goals, Objectives, and Strategies in the IRS Strategic Plan.||What evidence would you need to defend the program?||Who has the capacity to make the change?||What specific things will influence the target of change?||What are the numerical values of outputs and activities?||Link and track with budgets.||If none, what baseline data do you need to collect?||Will this be: Budget level? BEFM?Operational? Oversight Board?|
|What is the end benefit?||Is the outcome measurable?||What products do you need?||What efficiency measures concerncustomers and stakeholders?||Assess results.|
|How will you know you have been successful?||What services do you need?||Alter and improvestrategies, activities, and processes based on data.||(1) Refers to Objectives and Strategies in the IRS Strategic Plan.|
|Operating Division(s)||Operating Division(s) that owns this measure|
|Measure Name||Name of measure/indicator|
|Type of Measure||Identify whether this is a Strategic Measure, Balanced Measure, Diagnostic Tool, Workload Indicator, or other non-performance statistic. Identify the component of the Balanced Measurement System that this measure applies to: Customer Satisfaction, Employee Satisfaction, Business Results Quality or Business Results Quantity.|
| || |
|Responsible Official||The IRS official responsible for the accuracy of the measure, such as: SB/SE Director, Finance, Research and Strategy.|
|Definition||Provide a clear narrative explanation of the measure that can be easily understood by non-IRS reviewers.|
| || |
|Formula/Methodology||Provide a detailed narrative describing exactly how the IRS calculates the measure/indicator, including overall approach, scoring methodology, reporting period, etc.|
|Data Source/Measurement Tools||Identify the report or management information system that provides the data and identify the mechanism by which the IRS collects the data being reported|
|Reliability of Data||Describe the process or procedures used to verify and/or validate the data collected. Then, describe the data quality of the measure by using the appropriate term: "Reasonable Accuracy" or "Questionable or Unknown Accuracy." |
Reasonable Accuracy: There is a reliable system or process in place that validates or verifies the accuracy of the data being reported.
Questionable/Unknown Accuracy: A reliable system or process to validate or verify the accuracy of data is lacking. In such cases, the IRS should add a statement regarding efforts underway to improve the reliability of reported data.
|Frequency of Data Availability/Reporting||Frequency with which the IRS has measure/indicator data available and reports it, (daily, weekly, monthly, etc.). If the IRS reports the measure/indicator for periods different (i.e. less often) than when the data is available, explain.|
|Purpose of Measure||Describe how the measure applies to the segment of IRS operations the IRS is measuring.|
|Data Limitations||Describe any limitations for the measure data.|
|Calculation Changes||Describe any approved changes in the formula/methodology made during the fiscal year and the purpose for change.|
|Complete Description of the process(s) measure originates from: Critical Path:|
4. (add additional items when necessary)
|Management Controls for items on critical path (supervisory/management reviews of data quality, include description of how supervisor/manager indicates completion of a review). If there are no supervisor/management reviews, describe how the IRs assures data reliability, completeness and accuracy:|
4. (add additional items when necessary)