Narrative: f.

The ODHE uses monitoring tools to assess the quality of Ohio’s adult education and literacy providers including program reviews, pre- and post-award risk assessments, technical site visits, desk reviews, and quarterly desk monitoring.

The program review is a comprehensive, on–site review of all components of the local Aspire program including, but not limited to, program administration, student instruction, and fiscal compliance. During the program review, local Aspire providers must demonstrate adherence to all Aspire policies and procedures, the Student Experience Model, and data entry protocols. The goals of the program review are to do the following:

• Ensure that programs meet Adult Education and Family Literacy Act (AEFLA) and state requirements;
• Continuously improve the quality of federally and state funded activities;
• Provide technical assistance in identifying and resolving compliance/accountability issues;
• Ensure the accuracy, validity, and reliability of data collection and reporting as well as adherence to current policies and procedures;
• Interact with program administration, staff, students and stakeholders to get other perspectives of the program; and
• For local programs, provide an opportunity for professional growth and continuous improvement.

Local Aspire programs are identified for on-site monitoring based, in part, on identified risk factors, such as low performance data, key personnel changes, newly approved program, change in organization of the program, and time since the last review. 

Similar to the program review, the technical site visit is on-site monitoring that is used to assist local Aspire programs with a specific, pre-determined problem area. Both the program review and the technical site visit result in a formal follow-up report addressed to the Aspire administrator and the program chief executive officer, president, or superintendent citing any issues of non-compliance (findings) and program improvement recommendations. Local Aspire programs respond in writing to the formal report with action items illustrating how they plan to correct the identified findings and implement the program improvement recommendations. The on-site review includes a 90-day follow-up visit, wherein the program will show, in writing and verified by state staff, implementation of the response to the formal report and correction of the issue(s).

The ODHE Aspire office will continue to use a Risk Monitoring Tool to evaluate the local programs’ level of risk for managing a federal grant. The risk assessment is one tool used each year to determine the amount and type of intervention needed for the provider. In the competitive application, the ODHE includes a Financial Management Survey to assess, pre-award, the capacity of the applicant to manage federal grant funds. Information from this risk assessment is used to assess an organization’s structure and capacity-building needs and identify appropriate technical assistance and/or resources to strengthen operations if funded. The Risk Monitoring Tool has been updated to help hold local grantees accountable and takes into consideration different pieces mentioned during trainings and TA offered through OCTAE, noting the State Directors Training in Spring of 2023. 

The ODHE utilizes an annual desk review to evaluate program success through analysis of data. As the annual report card, the desk review provides a comparison between the local program’s final fiscal year actual data to the negotiated targets from the federal government relative to skills gain and performance outcomes. To assist with data validation, the Aspire office conducts quarterly desk monitoring with each of the local programs. Desk monitoring allows the local program to see the “state” of their data, in both progress toward meeting targets and the condition of the data (error clean-up). Desk monitoring complements the preparation of the quarterly PICP. The primary objectives of the PICP are for the state staff and local providers to work together to:

• Identify program goals, strategies, resources and tools to support successful program improvement; and
• Assist programs in analyzing data and performing a gap analysis to identify program improvement efforts.

The ODHE uses the monitoring tools mentioned above to assess the quality of local providers of adult education and literacy activities. The PDN provides professional development activities to promote continuous improvement of these activities with a goal of impacting student learning outcomes. Data is used for refining and adjusting PD activities. This is done by:

• Collecting participant evaluations/surveys for all PD activities;
• Gathering evaluations and surveys used to determine quality of training (adjustments are made based on the results); and
• Conducting discussions with local providers regarding PD activities as part of the quarterly PICP check-in.

A combination of methods is used to assess effectiveness of PD and dissemination of promising practices and models including classroom observations, student educational gains by teacher, focus groups, and self-reporting tools. Data from the data management system is used to determine if PD activities impact student learning outcomes including priority areas such as reading instruction and other specific needs of adult learners. PD effectiveness may also be measured by examining student learning gains by teacher.