Located in:
- Program-specific Requirements for Adult Education and Family Literacy Act Programs
The Unified or Combined State Plan must include a description of the following as it pertains to adult education and literacy programs and activities under title II of WIOA, the Adult Education and Family Literacy Act (AEFLA).
f. Assessing Quality
Describe how the eligible agency will assess the quality of providers of adult education and literacy activities under title II. Describe how the eligible agency will take actions to improve such quality, including providing the activities described in section 223(a)(1)(B) of WIOA. Describe how the eligible agency will use evaluations, data analysis, program monitoring, or other methods to improve the quality of adult education activities in the State.
Current Narrative:
Ongoing Program Monitoring and Oversight
Texas is in the process of implementing a program quality model to support student success that links the evaluation of monthly program performance, information, and findings from the agency’s monitoring department to just-in-time technical assistance and the deployment of targeted PD to support continuous program improvement.
The AEL department provides a Monthly Performance Report (MPR), also known as the report card, to local programs. The MPR includes a snapshot of both fiscal and programmatic performance of the grant. This is only one of nine non-NRS reports generated by the state’s Division of Information, Innovation, and Insight (I|3). The reports provide data sets for all state and federal imposed performance indicators, also referred to as contracted measures in the Texas RFA. The current contracted measures are as follows:
- Enrollment Targets, which include the following:
- Total enrollment
- Integrated Education and Training
- Intensive Services (Workplace Literacy, Services for Internationally Trained Professionals, and Transition to Post-Release Services)
- English Literacy and Civics—Total
- English Literacy and Civics—Integrated Education and Training
- Measurable Skills Gains (MSGs)
- Credentials
- Employed/Enrolled Quarter 2 After Exit
- Employed/Enrolled Quarter 2–4 After Exit
The MPR shows a monthly snapshot of the above indicators and their corresponding targets by individual grantee. The MPR uses timely data that is gathered from the Texas Educating Adults Management System (TEAMS) to measure program performance against contract deliverables as well as state and federal performance targets. TEAMS is Texas’ state-of- the-art, web-enabled system that maintains student-level data, including demographic, assessment, and outcome data, as well as class, site, program, and provider information. Aside from the state-generated reports, TEAMS users may analyze class, site, or program information by using the system- generated standard reports, which are often referred to as canned reports. TEAMS provides a wide range of information about adult education, including snapshots of student and program performance, personnel qualifications, and staff development activities.
Additionally, AEL program staff members meet to evaluate the overall effectiveness of each program, which is measured against contract deliverables, progress toward measures, state performance measures, and expenditures. Program staff members discuss trends in data and issues of concern and develop support plans to address such issues. This provides a structure under which staff may engage TWC’s Statewide Professional Development Center and other Section 223–funded PD and technical assistance support projects to address concerns.
Responsive PD efforts that are funded under section 223(a)(1)(B) of WIOA is a central component of the quality model. AEL assesses the quality and responsiveness of PD through multiple approaches, including the review of local program improvement and the remediation of deficiencies that may occur due to PD and technical assistance efforts.
For example, through the quality model, a provider that consistently shows poor performance in reading or oral language scores will be advised by their AEL program support specialist or an assigned PD specialist to schedule PD in these areas. Once the program receives PD services, the AEL department monitors program improvement, which is reported in the MPR. PD is only one remedy to address program improvement and is used along with other forms of technical assistance.
Adverse Actions
If a program misses required performance levels in any one area for four consecutive months, the program is placed on a Technical Assistance Plan (TAP). TAPs use a timeline of deliverables to improve and support program function. TAP actions may include additional targeted professional development support, one-on-one training in specific areas of weakness or concern, and technical assistance visits by TWC staff to meet and address issues. Members must meet specific benchmarks to have a TAP lifted. Programs that consistently fail to meet contract deliverables once placed on a TAP are elevated to a Corrective Action Plan (CAP). Programs that fail to meet objectives beyond this point are subject to sanctions and the deobligation of program funds. TWC Chapter 800 General Administration rules on the deobligation and reallocation of AEL funds outline the deobligation criteria and the time frame in which TWC may deobligate funds from an AEL grantee that is not expending their funds or meeting performance requirements, as well as the criteria grantees must meet in order to receive reallocated funds.
Subrecipient Monitoring
TWC’s Subrecipient Monitoring (SRM) department works with AEL staff to ensure that all thirty-seven grantees are compliant and serve as good stewards of federal funds.
TWC monitors for compliance with federal and state requirements related to fiscal and programmatic accountability through the following methods:
- Desk Monitoring—Regular review and analysis of data that is submitted and validated monthly through the state’s data system, TEAMS
- On-site Monitoring—Field based fiscal and programmatic audits that are based on agency risk assessment results
- Data Validation—Virtual reviews of source documents that are used to support performance accountability, per OCTAE Memo 19-1
On-Site Reviews
SRM hosts a preplanning meeting with TWC AEL staff to discuss any potential issues before the on-site visit, which gives SRM the opportunity to ask questions about specific items of interest identified during the desk review. It also provides SRM with any technical assistance items that AEL staff have reviewed and discussed with the grantee, including performance issues. The process ensures that SRM obtains an accurate review of program and financial activity and any ongoing issues. Before a review, SRM requests that the grantee submit documents for review and analysis to determine each area will be tested during on-site reviews.
Several grantees are selected annually for on-site reviews based on a risk analysis that includes both objective criteria and input from AEL. All grantees are monitored either in person or through a desk review, with all programs receiving on-site monitoring once every three or four years. Data validation is conducted with each grantee at least once during the grant cycle.
While on-site, SRM reviews expenditures that include, but are not limited to disbursements, cash management, financial reporting, and records management (including eligibility and assessment documentation). To ensure that the grantee has adequate internal controls, staff conducts interviews with key staff to confirm that internal processes for participant intake, screening, and placement comply with TWC’s AEL policies. Testing samples are identified using TEAMS and are compared to on-site records. Testing includes the review of student test scores, credentials, and MSGs.
At the end of each on-site review, SRM conducts an exit conference with the grantee, in which staff identifies any findings or areas of concern that will be identified later in an official report. The results of the SRM reviews improve the technical assistance targeted at local projects.
Data Validation
Additionally, SRM conducts an annual data validation of select grantees, as coordinated with AEL staff and required by OCTAE PM 19-1. Such data validation enables the joint reporting requirements, as established in OCTAE Memo 17-2, to be addressed. SRM also monitors other WIOA titles, which allows for consistent processes and implementation of requirements across WIOA partner programs. This coordinated effort ensures that reported performance maintains TWC’s integrity standards.
Deobligation of Funds
TWC focuses on performance to ensure that funds are used effectively when serving customers to produce positive results. Contracted performance benchmarks and performance standards highlight TWC’s expectation that grantees maintain or exceed performance standards through effective service delivery and innovation. Section 800.78 allows funds to be deobiligated if they are not being used effectively and at expected levels during specified time frames to serve individuals in need of AEL services. However, TWC staff works diligently with grantees to provide technical support and assistance in developing strategies to ensure that funds are used to serve customers efficiently and effectively, thereby mitigating risks. TWC reviews these rules and considers amendments that may provide the agency with greater flexibility to ensure that AEFLA funds are being used efficiently during the statewide grant cycle.
Self-Evaluation and Monitoring of Local Programs
As part of their deliverables, all TWC grant recipients (sole providers and lead grant recipients for consortia of providers) are required to:
- submit a detailed project plan and expenditure projections for AEL-funded activities;
- submit a detailed local professional development plan;
- develop and maintain a procedure that outlines a process for continuous monitoring of subrecipients; and
- if applicable, use TEAMS to regularly monitor program performance across all providers.
Programs use the detailed project plan as an opportunity to analyze their performance, determine the cause of areas of weakness or concern, and develop strategies for improvement in the coming year. Programs are required to tie their professional development activities to this plan to ensure adequate training and support for program improvements.
TWC AEL staff members also hold regular conference calls and biannual meetings for program members to share challenges and best practices. These forums build on what is working in the field and provide peer-mentoring opportunities.
Additionally, programs are required to regularly solicit student evaluations to assess satisfaction with curricula, instructors, and institutions.