U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Https

Secure .gov websites use HTTPS
A lock () or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Plan: Texas PYs 2016-2017
Combined Plan C

Section: Adult Education and Family Literacy Act Program

Narrative: f.

Published
Located in:

f. Assessing Quality

Describe how the eligible agency will assess the quality of providers of adult education and literacy activities under title II and take actions to improve such quality, including providing the activities described in section 223(a)(1)(B) of WIOA.

Current Narrative:

Ongoing Program Monitoring and Oversight

Evaluation of local provider services is a key priority for the AEL program. TWC staff has worked to develop a strategic evaluation and monitoring process that draws from best practices developed across TWC’s divisions. Texas is implementing of a program quality model to support student success that links evaluation of monthly program performance and information and findings from the agency’s monitoring department to just-in-time technical assistance and the deployment of targeted professional development to support continuous program improvement.

The AEL Department provides a Monthly Performance Report (MPR) to the director of the Workforce Development division. The MPR meeting is facilitated by AEL program and contract management staff, led by the AEL Director and Coordinator for Strategic Planning and Performance Accountability and includes a monthly snapshot of both program and financial performance across all AEFLA programs, including section 233 projects.

The MPR is the result of a multiphased process that uses timely data gathered from the Texas Educating Adults Management System (TEAMS) to measure program performance against contract deliverables as well as state and federal performance targets. TEAMS is Texas’ state-of-the-art, web-enabled system that maintains student-level data, including demographic, assessment, and outcome data, as well as class, site, program, and provider information. TEAMS users can analyze class, site, or program information through use of standard reports generated at the user level. TEAMS provides a wide range of information about adult education, including snapshots of student and program performance, personnel qualifications, and staff development activities.

AEL programs are required to validate their data monthly and submit quarterly Data Sign-off Reports through TEAMS. The Data Sign-off Report requires programs to describe areas of strength as well as concern. Programs also are required to submit monthly expenditure reports in TWC’s Cash Draw and Expenditure Reporting (CDER) system and quarterly narrative reports using a predetermined template that requests information on specific TWC objectives and overall program objectives.

AEL contract deliverables reflect TWC’s commitment to strong performance and accountability. These deliverables include a combination of program and financial accountability and encourage programs to exhibit strong planning through the development of project plans, continuous improvement plans, and standard operating procedures.

To prepare for the MPR report, AEL program staff meet, weekly, at a minimum, to evaluate the overall health of each program, which is measured against contract deliverables; progress toward measures; state performance measures; and expenditures. Program staff discuss trends in data and issues of particular concern, and develops support plans to address those issues. Because services are aligned to workforce areas, staff can see where specific issues are isolated throughout the state. This provides a structure under which staff can engage TWC’s Professional Development Center and other Section 223 funded professional development and technical assistance support projects to address concerns.

If a program does not show improvement after being placed on a support plan, TWC places it on a Technical Assistance Plan (TAP). TAPs use a timeline of deliverables to improve and support program function. TAP actions may include additional targeted professional development support, one-on-one training in specific areas of weakness or concern, and technical assistance visits by TWC staff to meet and address particular issues. Members must meet specific benchmarks for a period of four consecutive months in order to have a TAP lifted. Programs that consistently fail to meet contract deliverables once placed on a TAP are placed on a Corrective Action Plan. Programs that fail to meet objectives beyond this point are subject to sanctions and deobligation of program funds.

Responsive professional development efforts funded under section 223(a)(1)(B) of WIOA is a central component of the quality model. TWC AEL assesses the quality and responsiveness of professional development through multiple approaches, including the review of local program improvement and remediation of deficiencies that should result after professional development and technical assistance efforts.

For example, through the quality model, a provider that month to month shows poor performance in reading or oral language scores will be advised by their TWC AEL Program Support Specialist or assigned Professional Development Specialist to schedule professional development in these areas. Once the program receives professional development services, the AEL department monitors program improvement to report in the MPR. Professional development is only one remedy to addressing program improvement, and is deployed with other forms of technical assistance.

While changes in local program improvement is perhaps the most direct means of assessing the quality and effectiveness of professional development, it is not the only method of quality review in place. Professional development trainers are required to meet established quality and professional criteria developed and assessed by the state an PD Center and AEL staff; training is reviewed by AEL staff to assure alignment with agency objectives and best practices based on the most rigorous or scientifically valid research available Specific professional development efforts funded under section 233 focus on developing curricula and training, including training-of-trainer models on ensuring support for instruction in the essential components of reading instruction and instruction related to the specific needs of adult learners. The Literacy Institute at the state PD Center and the Focus on the Basics project are focused on professional development in the essential components of reading instruction and instruction related to the specific needs of adult learners. TWC AEL review and both training curricula deliverables and training for these projects against best practices based on the most rigorous or scientifically valid research available. A Section 233 funded project, Literacy Texas, focuses specifically on training volunteers in the content areas, including reading. These projects all have deliverables supporting the dissemination of information about relevant models and promising practices

Additionally, all professional development events include session evaluations and these evaluations are provided to the AEL office which reviews these monthly.

Subrecipient Monitoring

TWC’s Subrecipient Monitoring (SRM) department will conduct on-site reviews of 29 percent (10 programs) of AEL grant recipients over the first two years of service under new contracts and a desk review of 57 percent (20 programs). Risk is determined based on an analysis of factors, including size of grant award amount, timeliness of data entry, “newness” of the program or key program staff, and expenditure amounts.

In the second year of new contracts, SRM will conduct on-site reviews of the programs deemed high risk and not reviewed in year one.

While on-site, SRM reviews expenditures and records management (including eligibility documentation) and conducts staff interviews to confirm that internal processes for participant intake, screening, and placement comply with TWC’s AEL policy. SRM identifies records in TEAMS to compare with on-site records, including student test scores, student attendance records, and staff professional development hours.

SRM hosts a preplanning meeting with TWC AEL staff to discuss any potential issues prior to the on-site visit. This gives SRM the opportunity to ask questions about specific items of interest uncovered in the risk assessment. It also provides SRM with any technical assistance items that have been discussed and reviewed with the grant recipient by AEL staff, including performance. The process ensures that SRM gains an accurate review of program activity and any ongoing issues.

SRM also conducts a preplanning conference call to help grant recipients prepare and assemble necessary documentation preceding each on-site review. During the visit, program staff is interviewed about program processes. At the conclusion of the interview, SRM conducts an exit interview, in which staff reviews any findings, which are later distributed in an official report. Grant recipients have the opportunity to dispute any findings.

Deobligation of Funds

TWC is keenly focused on performance to ensure that funds are used effectively to serve customers and produce positive results. Contracted performance benchmarks and performance standards highlight TWC’s expectation that grantees will maintain or exceed performance standards through effective service delivery and innovation. TWC rule §800.78 allows funds to be deobligated, including voluntarily, if they are not being used effectively and at expected levels to serve individuals needing AEL services; however, TWC staff works diligently with grantees to provide technical support and assistance in developing strategies to ensure that funds are used to serve customers efficiently and effectively, thereby mitigating risks.

Under current budgetary conditions—as in the past—the need for AEL services exceeds Texas’ available resources, and no workforce area is funded to fully meet its needs. TWC expects that AEL grant recipients will identify, and coordinate with, all available organizations and programs in their workforce areas to expand and leverage deliverable services beyond those provided solely through AEL funding.

Self-Evaluation and Monitoring of Local Programs

As part of their deliverables, all TWC grant recipients (sole providers and lead grant recipients for consortia of providers) are required to:

  • submit a plan outlining their process for monitoring performance and expenditures;
  • use TEAMS to regularly monitor program performance across all providers; and
  • develop a yearly continuous improvement plan (CIP).

Programs use the CIP as an opportunity to analyze their performance, determine the cause of areas of weakness, and develop strategies for improvement in the coming year. Programs are required to tie their professional development activities to this plan to ensure adequate training and support for program improvements.

TWC AEL staff also holds regular conference calls and biyearly meetings for program members to share challenges as well as best practices. These forums build on what is working in the field and provide peer-mentoring opportunities.

Additionally, programs are required to regularly solicit student evaluations to assess satisfaction with curricula, instructors, and institutions.