Narrative: III. b. 4. A.

Each of the core programs will be assessed based on the performance accountability measures and other means to assess the quality, effectiveness and improvements of the programs by local area or provider. The performance accountability measures will continue to follow federal requirements which are pending within the approval of final regulations from the Department of Labor for WIOA Titles I and III and pending legislative changes from the Department of Education for WIOA Titles II and IV. All of these efforts will be modified to take into consideration the local and regional planning goals once the local and regional plans are approved. At the time of this State Combined Plan submission, this process has not been completed.

Additionally, programs are assessed in various ways and through different methods. Each WIOA title has existing approaches to gauge the quality, effectiveness and improvements of programs that can be broken down by local area or provider. For the title I and III programs, the following methods are used.

• Mandated Reports to the Legislature for those programs funded by state workforce, general fund, and other special state funds
• Annual Program Results/Reports for both federal and state programs, which often provide the federal negotiated performance goals and outcomes
• Customer Satisfaction for job seekers and employers
• A Uniform Report Card for specific workforce development programs
• For many programs, solicitations for services (requests for proposals) are posted on DEED’s website and through the State Register in an effort to attract the best possible proposals

Not only are programs assessed, sub–grantees and partners are analyzed in several ways, and the agency has standardized and consistent processes in place for conducting assessments.

Prior to awarding a grant, a DEED conducts a risk analysis. In addition, in the early part of each year sub–grantees complete an electronic assessment and a fiscal monitoring guide. These assessment tools provide a sweeping array of information, in such important areas as:

• How sub–grantees will serve and provide priority services to veterans
• Policies regarding sub–grantee conflict–of–interest processes
• The longevity of key directors and/or stakeholders
• Sub–grantee customer complaint processes and physical and program accessibility
• Sub–grantee accounting procedures, positive cash flow, third–party audit reports, review of debt ratio, including federal and state tax debt

For the duration of the each grant, sub–grantees are monitored to ensure they’re complying with the provisions of the grant and meeting performance goals. Monitoring, which usually takes place on–site, consists of assessing appropriate expenditures, conducting financial reconciliations, assuring proper eligibility of customers, case management, equal opportunity, reviewing options of co–enrollment with other partners and programs, and reviewing monthly and quarterly financial reports. Technical assistance may be provided through monitoring, on–site visits, training of case managers and fiscal staff to ensure that state and federal regulations are being met.

Workforce One (WF1) is an important tool for monitoring and technical assistance, specifically case management. Most sub–grantees and programs are required to use WF1 to ensure a consistent method of tracking participants. In addition, many small, locally–operated employment and training grants use WF1. Nearly 2,000 state, city, county and nonprofit employment and training providers use the web–based system to track employment and training services to more than 100,000 customers across Minnesota’s one–stop network.

The WIOA Title II program is operated by the Minnesota Department of Education, which prioritizes and measures program quality through accountability expectations and activities that focus on both compliance and best practice according to research and experience. The expectations and activities include:

1. The state ABE Management Information System (MIS) data: Minnesota’s MIS balances both program quality and program compliance with state and federal expectations. In addition to required reports, local providers can use the state MIS to analyze student persistence, program outcomes, and other measures at the consortium, site, class and student level.
2. A state AEFLA grantee report card: The report card notes actual performance and ranks providers by key program data measures, including measurable skill gain by population, cost of programming per individual served, intensity of service provided, and percentage of population served.
3. A program improvement process: Programs will be identified for review based on rankings in the state AEFLA report card and will undergo a comprehensive review and monitored implementation of a program improvement plan that requires local providers to complete site visits, workshop activities, reports, and plans to strengthen program performance and quality. (For more information on Program Improvement, see the response to (D) in the WIOA Title II program information.)
4. Intensive program application cycle: On a five–year basis, ABE providers are required to complete an in–depth application as a method to adhere to Minnesota statute governing local ABE programs. In this intensive application, local providers describe their:
   1. Consortium
   2. Program accountability procedures
   3. Professional development priorities and activities
   4. Proposed programming
   5. Local program governance
   6. Program collaboration with key stakeholders and partners
   7. Technology resources and integration
   8. Future plans
   9. Performance

These in–depth applications are reviewed and scored by a team of state staff from the Minnesota Department of Education and local AEFLA providers.

5. Local program monitoring site visits: All grantees are subject to monitoring site visits that focus on policy compliance and program quality through multiple methods:

1. Providers with low performance participate in site visits through the program improvement process;
2. All AEFLA grantees receive site visits on a rotating basis, at least once every five years, when the program participates in in–depth application process to continue receiving state ABE funding to comply with Minnesota statute;
3. Monitoring visits based on periodic desk review of provider data;
4. Local ABE providers that participate in state initiatives and/or receive special grants through state or federal funding typically receive site visits as part of their participation; and
5. Local providers request special technical assistance site visits as they face issues.

WIOA Title IV also has an extensive approach to measuring quality, effectiveness and improvement in services. In the area of quality there is a strong focus on staff development. The Comprehensive System of Personnel Development is the planning framework for recruiting, developing and retaining qualified staff to carry out the state’s vocational rehabilitation mission. Pursuant to Section 101 of the Rehabilitation Act, the plan addresses the following topics:

• The Need for Qualified Personnel
• Regional Higher Education for the Preparation of Vocational Rehabilitation Professionals
• Recruitment and Retention of Qualified Personnel
• Staff Training and Development
• Personnel to Address Individual Communication Needs
• Coordination with Personnel Development of the State’s Secondary Education System under the Individuals with Disabilities Education Act

Specific to service provision, Title IV reviews and monitors how the state strategies will be implemented, including the following:

• Methods to expand and improve services
• Assistive technology availability and services
• Methods used to address and improve services and coordination to transition age youth
• Plans for establishing, developing or improving community rehabilitation programs

Specific to supported employment, Title IV reviews and monitors how the state strategies will be implemented, including the following:

• Identifying strategies that contribute to goal achievement
• Identifying the factors that impede achievement of goals and priorities
• Measuring the timing of transitions to extended services

These efforts are expressed in greater detail within the Title IV Program Requirements section of the State Combined Plan.