Narrative: III. b. 4. A.

Each core and combined state partner is required by its federal agency to provide performance data annually. Each core partner’s respective federal agency sets the target and timeline for WIOA measures. The partners work with their federal agency to meet or exceed Missouri’s targets, while working with applicable local and regional entities.

Missouri implemented a job center certification process for comprehensive job centers and affiliates under WIOA. Each site that makes available one or more of the one-stop partners’ programs, services, or activities is evaluated based on standards and expectations for service coordination as defined by the SWDB. Certification teams are comprised of one representative from each of the four core partner programs and are charged with conducting independent and objective evaluations. The certification process includes onsite evaluations that incorporate interviews with job center leaders and evaluation of customer satisfaction surveys, documentation required for state and federal compliance, and physical and programmatic accessibility.

The one-stop certification process is conducted at least once every three years and review of certification criteria occurs every two years as part of the review and modification of the WIOA State Plan. The team charged with reviewing the certification criteria is comprised of core and combined state plan partner representatives who examine the certification criteria to ensure it is a comprehensive and pragmatic approach to determining customer satisfaction, physical and programmatic accessibility, and continuous improvement.  

The WIOA partnership assesses the progress of meeting its performance goals by looking at performance on a regular basis. While the partners have the ability to look at most of these on a monthly basis, the intention is to view a subset of the following measures monthly and all of them on a quarterly basis. Additionally, updates on performance will be presented to the State on a quarterly basis. This includes whether the targets are met for the following:

•     the WIOA common measures: Employment (Q2 and Q4), Earnings, Credential attainment, Measurable Skill Gains, and Employer Effectiveness;

•     total customers to be served; 

•     rate of achieving performance targets (percent of goal attained) and any anomalies in the data; and

•     whether or not the actual data are meeting accuracy of forecasted data.

Additional outcomes beyond the reported performance measures are also reviewed and developed with one-stop partners.

Adult Education and Literacy aligns its goals with those of the MPWS through the WIOA Steering Committee, sub-committees, and the local workforce planning process. The MPWS firmly commits to helping local adult education programs provide the highest quality student services. In addition to required annual performance reporting, AEL staff provide continuous monitoring, evaluation, and technical assistance to help improve outcomes, share resources, and grow adult education opportunities based on the needs of the state.

Using a four-component monitoring system, AEL conducts ongoing real-time assessments of local AEL service providers and statewide impact.

1.  Desk Review: State staff conducts monthly desk reviews analyzing each local program’s performance data to see areas of achievement and risk. With growing access to virtual meetings, the MPWS hosts one-on-one discussions of desk review findings with local AEL service providers. These conversations lead to a deeper understanding of individual program difficulties, provide personalized technical assistance based on performance numbers, and allow for sharing of promising practices.

2.  Self-Assessment: Based upon a risk assessment, local programs must submit a self-assessment to state staff, ensuring compliance with grant/contract requirements.

3.  Onsite Monitoring: AEL regularly evaluates the quality of providers onsite each year based on a revolving schedule and risk assessment. The reviews ensure that local programs follow state and federal guidelines, performance measures are on target, program operations are efficient, and instruction is effective.

In addition to mentoring from visiting local program staff, onsite reviews include:

1. 1. fiscal/program management and reporting;
   2. teacher certification and professional development requirements;
   3. data entry and management;
   4. student assessment procedures;
   5. student intake and orientation process;
   6. teaching/classroom observation;
   7. instruction;
   8. recruitment and retention;
   9. facilities;
   10. local program policies; and
   11. documentation and record retention requirements.

4.  Corrective Action: If monitoring reveals areas of noncompliance or concern, the local service provider must submit a corrective action plan for approval by AEL.

Missouri Vocational Rehabilitation (MVR) assesses its performance based upon the WIOA common performance measures as well as agency goals and objectives. The MVR executive leadership team along with district office management reviews performance data on a monthly basis to determine progress toward goals and opportunities for improvement. MVR’s case services and financial services work in tandem evaluating performance data for short- and long-term decision making. In addition, MVR’s statewide quality assurance team evaluates case management providing guidance on program performance. MVR utilizes the results of its monthly customer satisfaction surveys to evaluate its program performance. The feedback is shared with the executive leadership team, management, supervisors, and staff. It is used as a tool to improve services, evaluate staff performance, and determine training needs.

Rehabilitation Services for the Blind’s Training and Quality Assurance Manager conducts annual case reviews to monitor performance, compliance, and case management services.

Department of Social Services Office of Workforce and Community Initiatives’ Performance Monitoring Unit is responsible for monitoring employment and training providers to ensure providers are utilizing federal funds for their intended purpose and outcomes. In addition, programmatic monitoring is completed to ensure providers are adhering to their contracts and other program requirements. Programs such as SkillUP and Jobs League, conduct self-monitoring on 20% of their cases, while other programs like Missouri Work Assistance (MWA), are required to self-monitor two cases per case manager per month.

Along with program monitoring, programs report measures through federal reporting and legislative funding requests that are gathered in the DSS Budget Book.

Community Services Block Grant eligible entities are monitored triennially with a full onsite review and annually for organizational standards. Follow-up reviews, including return visits, are performed for those entities that failed to meet state goals, standards, and requirements, and other reviews as appropriate. 

The Jobs for Veterans State Grant Program (JVSG) works primarily from referrals from WP staff to provide intensive services to qualified individuals. This is either done through direct referral from the State Labor Exchange MOJobs, or through a warm hand off at the job center. The agency utilizes a Veterans Triage Form to help identify qualified individuals for referrals to JVSG staff at the intake point. Once a strategy has been developed to overcome the barriers of the qualified individual, referrals to supportive programs can be made through MOJobs or a warm hand off to other one-stop partners.

The JVSG program undergoes an audit of the case management system annually by the DOL-DVET to ensure programmatic compliance with regulations to include services to non-veterans, identification of barriers, and a plan to overcome those barriers. They also perform site visits to job centers to interview staff in order to confirm compliance with Priority of Service. DVOPs and LVERs are interviewed during the site visits to ensure role adherence. Quarterly, Missouri sends a Managers Report to the DVET that outlines the responsibilities of the individual roles and what has been done to meet performance metrics.

Quarterly, a Technical Performance Narrative to DOL with performance metrics is submitted in comparison to the negotiated rate.  

Missouri has also established an internal auditing process to occur monthly by the Intensive Services Coordinator as part of the Compliance and Monitoring Team. It uses the same metrics established by DOL-VETS to ensure compliance with regulations.