Narrative: III. b. 4. A.

The State will assess its core programs through on-going continuous improvement strategies including:

• State and local level fiscal and programmatic monitoring (on-site and/or desk reviews) of all programs within our combined plan
• Customer Satisfaction Surveys (for both the jobseeker and employer) are conducted annually
• State and local level data validation of all programs within our combined plan
• The Program and Performance Evaluation Committee, a subcommittee of Arkansas Workforce Development Board, will meet quarterly to make recommendations on performance standards for the forthcoming year and to review actual outcomes across programs, make recommendation for improvement and/or corrective action to ensure attainment of performance standards for all partners
• As a state agency, there are annual audits of programs conducted and specific, as needed, audits conducted by a third party

The state also ensures that local workforce development areas meet performance accountability measures and will provide technical assistance when needed.

Each agency will be responsible for conducting an evaluation of their programs and reporting results to the PPE Committee of the State Board.

WIOA System Evaluation

The Arkansas Workforce Development Board commissioned the University of Arkansas' Center for Business and Economic Research to complete a comprehensive WIOA Systems Evaluation, Skills Gap Analysis, and Customer Service Surveys. This evaluation is currently underway.

Adult Education monitors local programs through quarterly reports, annual technical site visits, and intensive three-day program reviews performed on each program every four years. A program’s review may be scheduled in fewer than four years if the program exhibits substantially increased risk and has excessive findings. The adult education providers are evaluated and monitored on the following in Program Reviews: Program Management and Operations, Instruction, Workforce Preparation and Training, Career Coach, Technology, Enrollment/Recruitment/Retention, Data Analysis, Advisory Committee, WIOA Partnerships and Collaboration, Financials, Facilities, Data Entry Staff Interview, Instructor Staff Interview, and ADA Coordinator Interview/Observation.

The site visit form is based on the top frequent findings of the Program Review and, therefore, may be modified annually. During site visits, Program Advisors review and check student folders to ensure that items such as a Student Learning Plan, Public School Waiver/Home School Letter/Court Order Letter, and Student Test documentation match the data management system. Program Advisors also check if student attendance matches what is entered into the data management system.

Program Advisors also check for Administration Compliance. Some of the areas reviewed include the completion of an Annual Self-Study, which includes strengths, weaknesses, and measurable goals with a plan showing how the findings are utilized in the program's future planning, Analysis of Program Data, and Integrated Education and Training (IET) compliance. Program Advisors review whether the program is following the Assessment Policy, which includes, but not limited to pre-testing students with the first 12 hours of instruction, following post-testing guidelines, data is entered by the deadline, etc.  The state staff also checks to ensure Personnel Activity Reports and Certifications for faculty/staff paid with Adult Education funds are completed and accurate. Equipment purchased with Adult Education funds is checked, and the Inventory Disposal Policy and Inventory Rotation Schedule are also reviewed for accuracy and compliance.

Adult Education also evaluates each program annually through an E&E (Effective and Efficient) calculation, currently based upon the programs’ progress on the federally negotiated benchmarks in NRS Tables 4 and 5.  Once a program year has ended, the Adult Education Section generates an Effective & Efficient (E&E) report based on each local provider's benchmarks of core performance indicators. For the 2022-23 fiscal E&E report, local providers must meet the state's MSG benchmark of 50%. Monthly Director's Reports, bi-weekly staff meeting data read-outs, quarterly reports, and desk audits are data analytical mechanisms used to evaluate and assess performance. These reports are used with prior reports to help identify trends and rates of performance for a myriad of performing cohorts. Along with these reports, aspects of demographic information are shared for local programs to analyze state data versus local program data.

Programs that do not meet the E&E benchmarks are provided additional technical assistance to prepare a performance improvement plan. The performance improvement plan implementation is monitored throughout the year to assist programs in progressing toward efficient and successful performance. For programs that have not met benchmarks for two consecutive years, meetings occur bi-weekly with technical assistance from a team of program advisors, where goal-setting and crucial data analysis elements are identified and planned. Program advisors and program directors set small, achievable goals for impacting performance bi-weekly. Technical assistance is provided to all programs, including those that meet or exceed the performance benchmarks, to address deficiencies in any specific EFL and to set and reach goals for performance improvement. Through E&E team assistance and targeted training, 34 of the 37 programs met or exceeded the MSG goal of 50% at an overall state average of 60% MSG attainment. This matched the previous year's performance and the number of programs that succeeded in meeting the negotiated benchmarks.

State and national performance data and negotiated targets were shared with local providers at the Fall Administrators' meeting, through advisory committee meetings, specialized training, and the Adult Education committees that identified the professional development needs and requests to promote continued improvement in program performance. In the analysis of performance data, the Administrators' Meetings also highlighted training pieces that addressed recruitment and retention strategies, distance education software usage guidance and strategies, addressing ESL needs, and incorporating IET programs into the services offered by each program to increase the credential attainment rate and help participants prepare for the workforce.

Data was analyzed and shared after the performance data was entered into the National Reporting System. Notes were made on distance learning performance rates for ABE, ESL, and Distance Learning participation groups. Also noted was the trending difference in female to male participation and a trending Distance Learning performance rate change in ESL participants that showed lower functioning ESL participants outperforming the ESL Levels 3 and 4. As other Performance and enrollment data was analyzed, results were shared throughout the year at training, conferences, and Administrators’ Meetings. Additionally, monthly program performance data on a state level was shared with local programs to monitor rates and trends at their centers compared to state numbers.

In analyzing the data, statewide training needs were addressed. Targeted training was held multiple times throughout the year to address enrollment and recruitment, targeted instruction and classroom management, retention strategies, curriculum development in addressing a wide range of contextualized and multi-level focuses, and test anxiety as needed. The importance of clean data was addressed by analyzing the data, focusing on professional development for the importance of the follow-up process, and entering accurate data correctly.

Data analysis guided the state's year-long process of collecting, entering, and improving the data in the state's data management system. This propelled the initiative for discovery and sharing of best practices for developing data entry processes and procedures, monitoring data at the program and class levels, understanding federal tables, utilizing state reports, detailing participant progress by assessment and subject area, developing IETs, quarterly reporting and analysis, and training for diversity and test anxiety. Each of these processes and procedures saw accuracy and efficient progress in reporting. Professional development training will continue to be developed as Adult Education moves to improve and meet targets for performance in measurable skill gains and follow-up measures. 

Identified performance measures are assessed and reported quarterly. ARS and the SRC conducts consumer satisfaction surveys. ARS has developed and is piloting a consumer satisfaction evaluation system that occurs at four pre-determined points. Results are reported in the SRC annual report. Staff are evaluated yearly on position specific metrics and performance improvement plans are developed if necessary.

DSB monitors performance measures quarterly to ensure progress is being made and works to correct deficiencies. DSB also conducts consumer satisfaction surveys as a metric of local performance. Staff are evaluated yearly on position specific metrics with performance improvement plans put in place as necessary to target improvements.