Narrative: III. b. 4. D.

The Arkansas Workforce Development Board (AWDB) has the responsibility for ensuring public accountability by evaluating the effectiveness of the overall workforce development system. For the federal WIOA performance accountability measures and state measures, each core partner will have a monitoring and corrective action process at the state level that assures identified deficiencies, to the extent they occur, are addressed. This process is intended to be based primarily on data analysis but shall also include monitoring tools at the discretion of the core program. If annual performance targets are not met by any core program, the AWDB may request that the core program submit a corrective action plan.

In the event of a corrective action plan being implemented, the partner will provide regular progress reports to ADWS and other parties.

The AWDB will evaluate its core programs through on-going continuous improvement strategies including:

1. Program performance effectiveness utilizing the State’s Information Management System at the state and local level

2. Review of state and local level data presented on a performance dashboard

3. Core Program Monitoring and Compliance reviews annually

4. Local Workforce Development Area annual reports

5. State and local risk assessment evaluations to review:

Governance

Administrative, program/grant, and financial management systems o Program services, delivery systems, and documentation, and o Performance accountability.

Statewide core partner meetings held bi-annually to discuss and review program effectiveness, best practices and provide technical assistance.

As previously mentioned, we plan to enhance our evaluation protocols beyond the standard program performance matrix with the re-introduction of customer satisfaction surveys for both program participants and employers. We believe that this will provide WIOA core program administrators and Local Workforce Boards with sufficient data to assess the quality of our state's workforce development system.

Additionally, we plan to seek guidance from and coordinate with the Employment and Training Administration - Office of Policy Development and Research to ensure that our efforts are in line with the increased focus upon program evaluations under WIOA.

Title II, AEFLA: Adult Education Section

Adult education state staff evaluates providers quarterly through the review of each program’s quarterly report submission.  In it, providers review and report progress on average attendance hours, educational functioning level (EFL) gains, number and percentage of students post-tested, etc. Quarterly reports also allow providers to review operational aspects of the program such as the percentage of funds expended, completion of required professional development, and completed personnel activity reports.

ADWS/AES also performs an annual site visit for each adult education and literacy provider to ensure that state and federal laws and policies are followed, to review program performance, and to connect with Local Education Agency Administrators. Based upon the findings during the site visit, regional program advisors provide technical assistance to program directors and staff to ensure continuous improvement.

Additionally, each year 25% of the current 36 adult education and literacy providers receive a program review, which is a three-day intensive program monitoring and technical site visit by a team of program advisors in the state office. Each provider can expect to have a program review every four years provided there are no major risks or violations identified during an annual site visit, which would increase review frequency. Providers are assessed in areas including, but not limited to, Program Management and Reporting, Financials, Technology, and Collaboration. This program review process allows program advisors and the State Disabilities Project Manager to thoroughly assess program operations, observe classroom instruction, and review state and federal laws and policies. Once each program is reviewed, ADWS/AES requests a full financial audit review by the agency’s state finance department.

ADWS/AES evaluates each program annually through an E&E (Effective and Efficient) calculation, currently based upon the programs’ progress on the federally negotiated benchmarks in Table 4. Programs that do not meet the determined percentage of the benchmarks are provided additional technical assistance in order to prepare a performance improvement plan. Plan implementation is monitored throughout the year. Upon not meeting the requirements of E&E for three consecutive years, ADWS/AES makes a recommendation to the Arkansas Workforce Development Board as to whether or not the program should continue to be funded.

Lastly, mandatory Administrators' Meetings are held twice a year in which policies and procedures are discussed and information is disseminated.  During the administrators’ meeting, providers share best practices with one another as well as gain additional professional development training from staff or guest speakers.

New Evaluation

As required by 20 CFR § 682.200(d), States must use funds reserved by the Governor for statewide activities to conduct evaluations of activities under the WIOA title I core programs in order to promote continuous improvement, research and test innovative services and strategies, and achieve high levels of performance and outcomes. 

ADWS, following input from the AWDB during the July 16, 2019, meeting negotiated a contract for an evaluation with the University of Arkansas.  The following requirements are the basis of the negotiation. 

The following evaluations using designs that employ the most rigorous analytical and statistical methods that are reasonably feasible, such as the use of control groups (20 CFR § 682.220) as appropriate: 

1. 1. Local Integration Study- Sample Workforce Innovation and Opportunity Act Title I or Title III participants for 3 Local Workforce Development Areas.   This part of the evaluation will focus on two primary areas: 1. Co-enrollment and Co-funding of REQUIRED WIOA Partners and 2. Leveraging of Local Resources available in the Local Workforce Development Area.  
2. Sample Workforce Innovation and Opportunity Act Title I and Title III participants for 3 Local Workforce Development Areas-to determine if, based on the participants information in the Case File, the participant might have been eligible to receive services under the required partners listed in 20 CFR § 678.400. 
3. For each local area selected, interview leadership to identify partner programs and non-profits that collaborate with the Workforce Centers beyond the list of required partners above.  For all partners, required in number 1 and other partners identified in 2, report whether there exists:
   1. Memorandums of Understanding between the programs/entities
      • Referral Procedures
        • Training for partners regarding the services provided or alternative print materials to assist in collaboration OR
          • An alternative method to refer clients between the partners effectively 
4. Identify and report entities in the selected local areas that provide services to the following targeted populations:

• Displaced Homemaker
• Low Income
• Limited English Proficiency
• Migrant Worker
• Disabled
• Veteran
• Older Worker
• Out of School Youth
• Foster Youth
• Basic Skill Deficient
• Ex-Offender
• Single Parent
• TANF Recipient
• SNAP Recipient
• SSDI Recipient
• UI Claimant
• Long-Term Unemployed

 For all programs and services identified in steps 1, 2, and 3, report whether sampled participants were: referred to or co-enrolled in other programs or services.

0. 1. In the Comprehensive Centers for the 3 Local Areas selected for sampling, report case studies on effectiveness of identifying and closing Skill-Gaps
       
      Interview staff for examples of:
       
1. Skills-gap identification on either the occupation level, based on demand, or the skill-level based on interaction with an employer.
2. Determine the services that were provided to address the skills-gap
3. Provide an assessment of the effectiveness of the services provided in closing the skills-gap, if possible.

 Review a sample of Arkansas Workforce Centers against the State’s certification policy and applicable laws/regulations for four comprehensive centers and two affiliate centers.  Each of the six centers selected should be located in different Local Workforce Development Areas and may be selected from Local Workforce Development Areas that overlap the 3 selected above

1.  Conduct a statewide customer service survey for business customers of WIOA Title I and Title III receiving services over the most recent 12 months available.
2.  Provide evaluation status reports on a monthly basis on overall project goals.  These reports will be issued at the end of February, March, April, May, June, and July of 2020.
3.  Provide a preliminary evaluation report by August 15, 2020.
4.  Provide a final evaluation, with results, best practices, and recommendations by September 30, 2020.