U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Https

Secure .gov websites use HTTPS
A lock () or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

2 Year Modification

Plan: Massachusetts PYs 2018-2019
Combined Plan C

Section: WIOA State Plan Common Elements

Narrative: II. b. 4.

Published
Located in:
  • II. Strategic Elements

    The Unified or Combined State Plan must include a Strategic Planning Elements section that analyzes the State’s current economic environment and identifies the State’s overall vision for its workforce development system. The required elements in this section allow the State to develop data-driven goals for preparing an educated and skilled workforce and to identify successful strategies for aligning workforce development programs. Unless otherwise noted, all Strategic Planning Elements apply to Combined State Plan partner programs included in the plan as well as to core programs.

    • b. State Strategic Vision and Goals

      The Unified or Combined State Plan must include the State’s strategic vision and goals for developing its workforce and meeting employer needs in order to support economic growth and economic self-sufficiency. This must include—

II. b. 4. Assessment

Describe how the State will assess the overall effectiveness of the workforce development system in the State in relation to the strategic vision and goals stated above in sections (b)(1), (2), and (3) and how it will use the results of this assessment and other feedback to make continuous or quality improvements.

Current Narrative:

The Baker—Polito Administration supports enterprise—wide performance management as a tool for both process and outcome improvement. As such the Administration embraced the new federal Indicators of Performance, as well as the latitude afforded states to create their own distinct indicators stemming from WIOA. A Performance Measurements Workgroup was established as part of the MWIB WIOA Steering Committee structure. The group consisted of internal and external members of the workforce development system and core partners under WIOA.

The Performance Measurements workgroup is currently exploring:

  • Data each measure is meant to capture and what it would communicate;
  • Available data sources / agency capacity to capture the data;
  • Balance between exhaustive list and meaningful list;
  • Whether the performance measures will apply to all core WIOA partners
  • How the measures would be compared (i.e. by area, statewide, etc.);
  • Developing baseline data on program performance;
  • Future targets for performance measures.

Federally Required WIOA Measures

The federal indicators of performance focus on employment outcomes in the second and fourth quarter after exit, median earnings in the second quarter after exit, credential attainment, and measurable skill gains for job seekers and adult education participants. For the first time, all partners and their federally funded programs will be measured in the same way for these same outcomes. While they are only a slight departure from the indicators captured and reported on by Workforce Investment Boards and One—Stop Career Centers, they represent a wholesale change for the other core partners. Moreover, a penalty could be applied to WIOA discretionary funds for repeated missed performance targets, which may create a new sense of shared accountability for shared customers and their outcomes. There is also a new federal indicator focused on “effectiveness in service to employers” that all partners would be measured on and required to report. As of the writing of this draft, this indicator is still under development with our federal regulators.

REQUIRED FEDERAL MEASURES (WIOA Legislation)

Included in the Measures:

All measures applied to participants in each WIOA funding stream (reports separate by agency)

  • All One—Stop Career Center Customers (Specific One—Stop Career Center report or statewide rollup)
  • Title II participants (adult education)
  • Title IV vocational rehabilitation participants (MRC and MCB)

Targets:

  • Federal Government sets target for federal measures
  • First report period begins July 1, 2016
  • The first year will serve as the baseline period as it is the first time any partner will be collecting this information in this format

Federal Measures:

  • Employed 2nd Quarter after Exit (federal measure)
  • Employed 4th Quarter after Exit (federal measure)
  • Median Earnings 2nd Quarter After Exit (federal measure)
  • Credential Attainment Rate (federal measure)
  • Measureable Skill Gains (based on training/education gains) — (federal measure)
  • Effectiveness in Serving Employers — (federal measure to be defined)

Given the significant change to performance measurement for several of the WIOA partners, there were efforts internal to each system (adult education, vocational rehabilitation, etc.) to better understand the impact of the required WIOA measures for each unique system as well. For example, in addition to the cross—agency WIOA Workgroup on Performance Measurement, the Adult and Community Learning Services (ACLS) department in Massachusetts created a stakeholder engagement process for providers in the field to better understand the implications and impact of the federal measures on adult education.

The WIOA performance measures provide one strategy for assessing the quality of the adult literacy services provided. In FY16, ACLS convened a WIOA Performance Measures Task Force comprised of five Massachusetts adult education directors and ACLS staff to:

  • Review the WIOA performance measures and program performance data;
  • Decide whether the federal measures should be part of a new state performance accountability system for awarding past performance points to local programs;
  • Decide whether additional measures should be added;
  • Weight and rank the measures;
  • Address implications; and
  • Create a plan for rolling out the new system including a performance measures pilot for FY17.

ACLS continues to integrate the feedback and work with the adult education field into the cross—agency Performance Workgroup and the WIOA Steering Committee.

State—Designed Performance Measures

Although these federal indicators represent a significant step toward systems alignment and shared accountability, the various workgroups and the Steering Committee felt it was important to consider additional indicators that would specifically speak to our achievement of the goals previously outlined.

As such, the Steering Committee and Performance Measurements Workgroup developed measurement concepts that reflect the vision, goals and strategies outlined in prior sections. In particular, the Performance Measurements Workgroup identified gaps in the federal measures related to the results for specific job seeker populations, business customers and the long—term impact of education, training and supports on an individual’s career pathway toward self—sustaining wages.

Currently, the Performance Measurements Workgroup is reviewing the following types of state—designed measures.

RESULTS FOR KEY POPULATIONS

A significant focus for WIOA and for the Baker Administration is to ensure that individuals who face barriers to employment benefit from public education, training and workforce programs in order to shift patterns of chronic unemployment for specific demographic populations and job seekers. While the WIOA federal measures look at job placement, states are not required to further review job placement data by population type. Massachusetts is considering state—designed measures that track job placement and results for specific populations WITHIN the required, federal data sets for each funding stream. Currently, the group is reviewing the proposed measures as shown in the table below.

Evaluation Question: Are programs achieving results for program participants?

Results for Key Populations State Designed Job Seeker Performance Measures

Vocational Rehabilitation Participants(WIOA Title IV)

All Career Center Participants(WIOA Title I, III, Vets, TRADE, etc.)Adult Education Participants(WIOA Title II)
Number and Percent of customers enrolled in training and employed in a training related job 2nd Quarter After ExitOnly Career Center Customers enrolled in a training program.Only Title II individuals enrolled in a training program.Only Title IV individuals enrolled in a training program.
Number and Percent of Veterans Employed 2nd Quarter After ExitOnly Career Center Customers who self—declare Veterans status included in measure.Only Title II individuals who self—declare Veterans status included in measure. Only Title IV individuals who self—declare Veterans status included in measure.
Number and Percent of Individuals with Language Barriers at Registration Employed 2nd Quarter After ExitOnly Career Center Customers who self—declare ESOL status included in measure.Title II individuals with ESOL status included in measure.Only Title IV individuals who self—declare ESOL status included in measure.
Number and Percent of Individuals without High School Equivalency at enrollment that obtained a HS equivalency and who are Employed at 2nd Quarter After ExitAll Career Center Participants without HS Equivalency at intake (WIOA Title I, III, Vets, TRADE, etc.)All Adult Education Participants (WIOA Title II)All Vocational Rehabilitation Participants without HS Equivalency at intake (WIOA Title IV)
Number and Percent of Individuals with a Disability Employed 2nd Quarter After ExitOnly Career Center Customers who self—declare included in measure.Only Title II individuals who self—declare included in measure.Only Title IV individuals who self—declare included in measure.
Number and Percent of Individuals receiving TANF or SNAP Employed at 2nd Quarter After ExitAll Career Center Participants receiving TANF or SNAP required to enroll in job assistance services (WIOA Title I, III, Vets, TRADE, etc.)All Adult Education Participants receiving TANF or SNAP (WIOA Title II)All Vocational Rehabilitation Participants receiving TANF or SNAP (WIOA Title IV)

CAREER PATHWAY RESULTS FOR INDIVIDUALS

The individual completion or job placement results for individuals by funding stream stop short of providing information on how a person progresses across systems or resources as they (hopefully) make progress along a career pathway. As the Commonwealth builds new IT systems to track cross—agency referrals and to match individuals to wage record data from different systems, the state will move toward more in—depth analysis of the impact of education, training and support systems on an individual’s long—term success.

A handful of leading states have developed longitudinal data systems and practices of answering important questions about public services and investments. These states work with the Workforce Data Quality Campaign. The Campaign has identified a group of 10 states working together as “The Alliance for Quality Career Pathways” (Arkansas, California, Illinois, Kentucky, Massachusetts, Minnesota, Oregon, Virginia, Washington and Wisconsin) that is developing shared metrics for career pathway systems. Career pathways reorient existing education and workforce services into a structure that focuses on the workforce needs of employers and on the education and training needs of individuals as they pursue their career paths. The effort is led by the ten states and facilitated by the Center for Law and Social Policy (CLASP), a WDQC national partner. States are currently testing a set of metrics that include educational outcomes (e.g. credit accumulation, certificate attainment) and labor market outcomes (e.g. employment placement rates, initial earnings). These metrics would give multiple programs shared outcome measures and similar definitions, such as measuring “initial earnings” as the median earnings in the second and third quarters after career pathway exit. Several of the states are integrating the metrics testing into state data system development projects.

Massachusetts leadership across the Administration, the State Workforce Development Board, WIOA Partners, the WIOA Steering Committee, and stakeholders hopes to learn from how these states have developed IT systems across existing infrastructure and the key drivers to building this capacity.

The Steering Committee has discussed ideas to follow individuals (not funding streams) across programs/services to look at longer term impact. The concepts and vision to develop career pathways measures in Massachusetts are captured below.

Evaluation Question: Are programs assisting individuals to move along a career pathway toward self—sustaining wages?

Included in the Measures:

  • Cohort—based, longitudinal report.
  • Build cohort based upon individuals found across data systems with from agency referrals.

Targets:

  • Target goals to be determined.
  • These measures will entail additional IT system development and costs to collect data for the measures.

Career Pathway Results Measures (State Designed):

  • Track an individual across programs over multiple years:
  • Report on pattern of “bundled” services or service use across Career Centers, adult education, vocational rehab, higher education etc. service
  • Number and percentage of individuals who complete a Career Action Plan (CAP) with assessments (education, TORQ, CRI, inventory)

BUSINESS CUSTOMER RESULTS

The required WIOA measure to look at the “effectiveness in service to employers” needs additional definition and refinement. The Steering Committee, Performance Workforce and Business Strategies Workgroups also identified a need to greatly improve the data collection and ability to report out on results for business customers, in particular those business customers working with One—Stop Career Centers. The WIOA partners outside of the One—Stop Career Center system do not serve the same volume of business customers and therefore need to continue to identify ways in which these types of measures would be meaningful and doable.

Evaluation Question: Are programs meeting the needs of Massachusetts’ businesses?

Included in Measures:

  • Business customers of One—Stop Career Centers (OSCC)

Target:

  • Create a Business Engagement/OSCC Dashboard; these measures will entail additional IT systems development and costs to collect data for the measures.

Business Service Measures — State Designed(Measures reported by Career Centers)
Number of businesses served
Number of new businesses served
Number of repeat businesses served
Number of businesses served by industry type
Number of Business Visits Weekly by OSCC Business Service Representatives
Number (and percent) of businesses registered with OSCC (JobQuest) who hire customers
Number of hires per industry
Retention rate for individuals hired by businesses registered with OSCC (JobQuest) who hire customers

WAGE RECORD DATA

All of the federal and proposed state—designed measures require broad access to wage record data. One critical step to implementing WIOA performance measurements is the ability to share individual—level information on the employment status and earnings of participants in our newly expanded workforce system. The Administration has filed legislation to facilitate sharing of information in the new ways that WIOA requires with expected passaged prior to July 1, 2016.

  • Individual wage record data (collected by DUA) is needed by all partner agencies (DCS, ACLS, MRC and MCB) for the required federal reports on the new federal measures.
  • This type of information will also be needed for the state—designed measures outlined in the State Plan, which will focus on longitudinal, individual achievement over time.