Narrative: III. b. 4. B.

Representatives from the Workforce Innovation and Opportunity Act (WIOA) Steering Committee report results of Section 116 performance data to the State Workforce Development Board as do the local workforce development boards (WDB). These results are used to assess program effectiveness of core, one-stop partners. The WIOA Steering Committee uses this data to review trends, look for improvement opportunities, and set program and policy direction. The primary responsibilities of the WIOA Steering Committee are to oversee the execution of the combined state plan, oversee and direct subcommittees focused on plan strategies, and to evaluate partner performance on a quarterly basis to ensure the quality and effectiveness of service delivery to workforce system participants. The committee also utilizes performance outcomes and quality assessment information to implement strategies and best practices to improve any deficiencies.

For PY 2021, OWD achieved the following performance outcomes:

Adult Program

Employment Rate 2^nd Quarter After Exit:  78.82% (Goal:  72.50%)

Employment Rate, 4^th Quarter After Exit:  75.02% (Goal:  68.00%)

Credential attainment:  66.16% (Goal:  67.00%)

Measurable Skill Gains:  55.80% (Goal:  48.00%)

Dislocated Worker Program

Employment Rate 2^nd Quarter After Exit:  76.61% (Goal:  77.00%)

Employment Rate 4^th Quarter After Exit:  77.49% (Goal:  74.00%

Credential Attainment:  74.05% (Goal:  68.00%)

Measurable Skill Gains: 63.18% (Goal:  59.00%)

Youth Program

Employment Rate 2^nd Quarter After Exit:  81.77% (Goal:  73.00%)

Employment Rate 4^th Quarter After Exit:  80.35% (Goal:  70.00%)

Credential Attainment:  58.86% (Goal:  63.00%)

Measurable Skill Gains:  53.08% (Goal:  38.50%)

Wagner-Peyser

Employment Rate 2^nd Quarter After Exit:  63.49% (Goal:  69.00%)

Employment Rate 4^th Quarter After Exit: 64.78% (Goal:  71.00%)

For PY 2022, OWD achieved the following performance outcomes:

Adult Program

Employment Rate 2^nd Quarter After Exit:  78.17% (Goal:  74.00%)

Employment Rate, 4^th Quarter After Exit:  79.02% (Goal:  70.00%)

Credential attainment:  61.12% (Goal:  66.50%)

Measurable Skill Gains:  65.00% (Goal:  51.50%)

Dislocated Worker Program

Employment Rate 2^nd Quarter After Exit:  75.25% (Goal:  76.00%)

Employment Rate 4^th Quarter After Exit:  74.37% (Goal:  74.00%

Credential Attainment:  65.85% (Goal:  71.00%)

Measurable Skill Gains: 64.30% (Goal:  60.00%)

Youth Program

Employment Rate 2^nd Quarter After Exit:  81.70% (Goal:  77.00%)

Employment Rate 4^th Quarter After Exit:  79.66% (Goal:  74.50%)

Credential Attainment:  65.84% (Goal:  63.00%)

Measurable Skill Gains:  51.89% (Goal:  44.50%)

Wagner-Peyser

Employment Rate 2^nd Quarter After Exit:  72.58% (Goal:  67.00%)

Employment Rate 4^th Quarter After Exit: 69.30% (Goal:  69.50%)

While some performance has seen a slight downturn, OWD continues to meet or exceed all negotiated common measures.

At the beginning of PY 2023, OWD moved to an automated process for surveying customers. At the end of each week, reports identifying customers that received a staff-assisted service during the week are queried from MoJobs. The reports are securely transferred to Qualtrics, and the survey is emailed to customers at the beginning of each work week.

The new version of the survey captures customer sentiments on satisfaction with services provided through the job center as well as the customer experience. Open-ended questions aid in understanding the interaction of customers and staff and help gain information on other services that the customer wishes were offered in the job center.

Data compiled from the surveys is displayed in dashboards. Information is displayed in graphic form; for measures, word clouds with key words from the comments complement the graphics.  Full comments are displayed for key questions of services the customer wishes were available and for the customer to provide any feedback on the interaction. Dashboards are available on-demand to supervisors and partners designated by the local workforce board director.

The new process has achieved several goals. Since staff no longer send the survey, OWD ensures that all customers with an email that received a staff-assisted service will have the opportunity to provide feedback. The process is more efficient since staff no longer spends time sending messages to customers, allowing more focus on the customer. 

For PY 2021, the JVSG program achieved the following performance:

Employment Rate 2nd Quarter After Exit:  53.29% (Goal:  59.0%)

Employment Rate 4th Quarter After Exit:  54.13% (Goal:  57.00%)

Median Earnings 2nd Quarter After Exit:  $7,392 (Goal:  $6,000)

For PY 2022, the JVSG program achieved the following performance:

Employment Rate 2nd Quarter After Exit:  58.32% (Goal:  59.00%)

 Employment Rate 4th Quarter After Exit:  58.84% (Goal:  57.00%)

Median Earnings 2nd Quarter After Exit:  $8,158 (Goal:  $6,000)

An assessment of JVSG performance indicated that Missouri missed the employment 2nd and 4th quarter goals by 3 – 4% each of the last two quarters of 2023. They have, however, exceeded the median earnings goal by approximately $2,000 in both quarters.

CSBG, SNAP, and TANF meet the federal reporting requirements to their federal partners, but do not report similar WIOA assessments as the core partners. OWCI is currently in the process of drafting measurements, determining program specific outcomes, and attempting to align assessments with the core partners to meet similar results. This process is under internal review and will need an analysis period to evaluate effectiveness.

Missouri Vocational Rehabilitation (MVR) and Rehabilitation Services for the Blind (RSB) previously used federal standards and indicators to evaluate their performance. With the passage of WIOA, MVR and RSB are now utilizing the required performance indicators found in Section 116 of the Act for program evaluation along with the other core partners in the MPWS. The review and analysis of this quarterly data is a valuable assessment tool and aids MVR and RSB in determining goal attainment as well as providing the opportunity for comparison across core partner agencies. MVR and RSB have adjusted their case management systems to support the common performance measures.

Until the last modification plan period, MVR and RSB were only required to report baseline data for the first five performance indicators as performance data collection was not mandatory until PY 2017. For the PY 2020 – 2023 state plan, MVR and RSB were required to only report an expected level of performance and negotiate a performance level for measurable skill gains (MSG).

MVR and RSB use MSG to evaluate how well clients and employers are being served as MSG is a good measure of program effectiveness and success. The agencies negotiated 20.0% for PY 2020 and 20.5% for PY 2021 with Rehabilitation Services Administration (RSA).

MVR’s Performance for MSG:

Negotiated Levels:                 PY 2020: 20.0%         PY 2021: 20.5%

MVR Actual Levels:               PY 2020: 78%            PY 2021: 74.8%

RSB’s Performance for MSG

Negotiated Levels:                   PY 2020: 20.0%           PY 2021: 20.5%

RSB Actual Levels:                 PY 2020: 73.3%           PY 2021: 67.7%

MVR and RSB more than tripled their negotiated levels in both PY 2020 and 2021.

In 2022, agencies were required to submit expected levels of performance and negotiate performance levels for the first five indicators for PY 2022 and PY 2023. These levels are reported below along with the agencies’ actual levels of performance for PY 2022. Because PY 2023 has not ended, actual levels are not available at this time.

  Employment Rate 2nd Quarter After Exit

Negotiated Levels:                 PY 2022: 58.8%

Combined Actual Levels:      PY 2022: 65.6%

MVR Actual Levels:               PY 2022: 65.96%

Employment Rate 4th Quarter After Exit

Negotiated Levels:                 PY 2022: 54.5%

Combined Actual Levels:      PY 2022: 63.3%

MVR Actual Levels:               PY 2022: 63.54%

Median Earnings 2nd Quarter After Exit

Negotiated Levels:                 PY 2022: $5,450

Combined Actual Levels:      PY 2022: $5620%

MVR Actual Levels:               PY 2022: $4,679

Credential Attainment Rate

Negotiated Levels:                 PY 2022: 28.0%

Combined Actual Levels:      PY 2022: 61.6%

MVR Actual Levels:               PY 2022: 64%

Measurable Skill Gains

Negotiated Levels:                 PY 2022: 77.9%

Combined Actual Levels:      PY 2022: 73.6%

MVR Actual Levels:               PY 2022: 73.26%

MVR will continue to work toward increasing the levels of MSG and credential attainment for individuals with disabilities focusing its efforts on staff training and data entry.

During PY 2021 and 2022, MVR’s customer satisfaction results continue to reflect positive performance with rates at 94% and higher for the following five areas of service: staff availability, courteous and respectful treatment by staff, purpose of MVR services explained, counselor helped customers plan services, and service choices were explained.

Upon assessment of enrollment data since the 2020 State Plan, the MPWS noticed decreased enrollment in all programs. From 2018-2019 to 2020-2021, AEL enrollment decreased by 50%. The MPWS evaluated student barriers and demographics identifying trends. That assessment prompted the MPWS to innovate AEL service delivery, meeting adult students where they are. Trends include barriers to employment, specifically transportation and childcare, and dramatic growth in the proportion of students seeking English language proficiency.

To address access barriers to AEL programming, the MPWS updated its distance education delivery, funding all local service providers to offer virtual education for the first time in 2022. With careful monitoring to ensure equitable opportunities for educational growth, all AEL programs in Missouri now have a virtual option with comparable measurable skill gains to in-person classes. The MPWS also expanded AEL class sites in employer facilities to assist adults with educational attainment and English acquisition. Seeing an increase of 11% from 2020-2021 in the proportion of adult students needing English proficiency, the MPWS expanded Integrated English Literacy and Civics Education providers and created targeted professional development for English language educators. Since implementing these initiatives to address assessed changes in Missouri’s workforce needs, the MPWS has increased total AEL enrollment by 38% since the low in 2020-2021 and expects to recover to pre-pandemic rates by 2026.

Moving forward, progress toward these outcomes will be evaluated by the partners through the WIOA Steering committee.