Narrative: III. b. 4. B.

The WIOA partnership, using data stewards from the core programs, fully intends to reach out and collaborate with data stewards for each of the other twelve combined plan programs to assess their programs in the context of WOIA integration. These include such programs as Job Corps, Youth Build, Senior Community Service Employment Program, Carl Perkins Career and Technical Education, and Housing and Urban Development programs

This partnership will assess the progress of meeting its performance goals by looking at performance on a regular basis. While we have the ability to look at most of these on a monthly basis, we intend to view a subset of the following monthly and all of them on a quarterly basis. This includes resulting if we are on target for meeting the following:

• The WIOA common measures: Employment (Q2 and Q4), Earnings, Credential attainment, Measurable Skills gain

• Total customers to be served;

• Rate of achieving performance targets (percent of goal attained) and any anomalies in the data;

• We will also look at whether or not the actual data are meeting accuracy of forecasted data;

• Additional outcomes beyond the reported performance measure are also reviewed and will be developed with One-Stop partners.

Missouri is strongly committed to helping local programs meet success in producing quality student outcomes based on core indicators of performance. MOAEL staff provides continuous monitoring, evaluation and assistance in activities that help improve outcomes, share resources, provides technical assistance and support, both directly and indirectly.

Missouri Adult Education has a two component monitoring system to assess the progress of adult education and literacy activities under title II.

1. Desk Review - a monthly review of AEL grants/contracts that includes review of financial reports and program data. The data will include numbers served, testing data, performance data that includes testing, outcomes, persistence, progression, and enrollment data. Additionally, data quality and professional development are monitored. Based upon monthly reviews, technical assistance is provided to programs.

2. Self-Assessment - for a select number of districts/programs, a self-monitoring report will be submitted to the Department for review to determine compliance with grant/contract requirements.

MOAEL also regularly evaluates the quality of providers. A number of programs are chosen for on-site review each year based on a revolving schedule and local risk factors. The reviews ensure that state and federal guidelines are followed, review program performance, and provide ongoing technical assistance to support improvement and continued compliance. These reviews include assessment of the following areas:

a) Fiscal/program management and reporting

b) Certification and professional development requirements

c) Data entry and management

d) Assessment procedures e) Intake and orientation processes

f) Teaching/classroom observation

g) Instruction

h) Recruitment and retention

i) Facilities

j) Program policies

k) Documentation requirements

Corrective Action: If any of the monitoring components reveal areas of noncompliance, the local provider will be required to submit a corrective action plan with the Missouri Adult Education office. Ongoing technical assistance is provided by the state staff.

While a complete assessment approach integrating the resources of all core partners has not yet been finalized, the expected WIOA measures beyond federal reporting will include:

• Consideration of pre- and post- service characteristics;

• New emphasis on credential attainment;

• Training success and training related employment;

• In addition, new measures for Effectiveness in Serving Employers will be determined, approved and these will also be assessed.

Missouri VR will assess its performance based upon the WIOA common performance measures as well as agency goals and objectives. The VR executive leadership team along with district office management reviews performance data on a monthly basis to determine progress toward goals and opportunities for improvement. VR’s case services and financial services work in tandem evaluating performance data for short- and long-term decision making. In addition, VR’s statewide quality assurance team evaluates case management providing guidance on program performance.

The State Plan herein explains elsewhere the capacity of Missouri to conduct One-Stop (Job Center) certification process. As this certification process is implemented, it will both engage and incorporate how assessment will compare local and regional planning goals as well as learn and refine the input of measures used by the other twelve One-Stop partners not previously involved in the State Plan process.

For evaluation, the WIOA Steering Committee is developing a proposal for considering the potential to perform an effectiveness evaluation to benchmark combined partner data so as to enable an assessment of outcomes after WIOA. We will have deliberation, direction and approval from the State Board before proceeding with such an evaluation. The partnership will also accommodate Vocational Rehabilitation, Rehabilitation Services for the Blind and the TANF programs in their assessment efforts. The partnership will assess progress in meeting established goals, forecasting and trending data, developing strategies to remediate any identified deficiencies in progress toward the goals.

Workforce Data Quality Initiative - Round 6

Assessment and evaluation is a core purpose for a dedicated group of program outcome subject matter experts that have been assembled around Missouri’s Workforce Data Quality Initiative (WDQI-Round 6) grant program.

Missouri was a recipient of the WDQI -1 in 2010 as one of only 12 states involved in building a State Longitudinal Data Warehouse. After having produced effectively matched outcome data, the warehouse is still in place and is governed by a comprehensive data matching agreement.

On July 1, 2017, these staff began the deployment of the WDQI-6 for the purposes of research, modification of the data matching agreement and conclusively determining how data matching may enable both integrated performance reporting and back up an evaluation and research plan.

Each WIOA core partner has been asked to assign a subject matter expert. Over the course of the three-year grant period the WDQI has the potential to be a powerful solution to the integrated analysis and evaluation of WIOA programs.