Narrative: f.

The Colorado Department of Education Office of Adult Education Initiatives (AEI) will assess the quality of sub-grant recipients through ongoing program monitoring and oversight. In order to empower adult education and family literacy providers to effectively serve individuals and communities, the AEI office will develop a matrix of quality indicators, based on evidence-based best practices. This tool will provide a framework to determine strengths as well as the level and type of support each sub-grant recipient needs from the AEI office. To assist in building stronger adult education and family literacy programs in Colorado, the AEI office will conduct ongoing programmatic and data reviews, plus quarterly and annual evaluations with sub-grant recipients based on this matrix. Quality will be evaluated through monitoring visits, performance reports, and on-site program reviews. Additionally, assessment will measure the effectiveness of the state and local providers in achievement of core indicators and levels of student improvement as negotiated with the U.S. Department of Education Office of Career, Technical, and Adult Education (OCTAE).

Programs must provide student progress measures obtained from all students who have attended at least 12 hours of instruction. Documented progress of student performance measures must include at a minimum:

• Literacy skill level improvements in reading, writing, and speaking the English language, problem solving, numeracy, and other literacy skills
• Placement in, retention in, or transition into postsecondary education, training, unsubsidized employment, or career advancement
• Attainment of a secondary school diploma or its equivalent.

All programs are required to maintain individual student records for all students who have attended 12 hours of instruction. Each record must contain the following:

1. Student identification and demographic information;

2. Attendance rates

3. Years of schooling and placement level at program entry

4. Initial learning goals

5. Specified pre- and post-testing student information

6. Entry and update records

7. Other specified information necessary.

Additionally, all sub-grant recipients are responsible for maintaining the privacy of the data for learners of all ages who participate in their programs.

The AEI team will meet quarterly, at a minimum, to evaluate the overall viability of all sub-grant recipients - both from fiscal and programmatic perspectives. The AEI team will compare trends in data and determine necessary technical assistance and next steps. The AEI team will also develop assistance plans for programs struggling to meet goals and objectives, and those plans will be used for service and support to providers.

If a sub-grant recipient fails to meet performance goals or other programmatic requirements, the AEI office will take specific action to assist the program in improving the quality of the program. AEI will use the following two plans to take action to improve the quality of the adult education and family literacy activities.

1. Performance Improvement Plan (PIP) - A PIP will be required for programs which are identified as low-performing when compared to the State performance on federal or State benchmarks. The PIP will include specific action steps related to performance. AEI will provide support and assistance to programs, which may include targeted professional development, one-on-one training in specific areas of challenge, and technical assistance visits by AEI staff to address particular issues.

2. Corrective Action Plan (CAP) - A CAP will be implemented with programs that are out of compliance with State and/or federal policies. AEI will provide technical assistance throughout the corrective process, and by the end of a designated timeframe, programs should be able to correct the identified issues and end their respective CAP.

As a part of both plans, the AEI office will provide ongoing technical assistance, professional development, and other support until the required steps of the plans are completed. The type of technical assistance, professional development, and other support will be based upon the specific area(s) of deficiency or need for the individual program.

Quality Assessment Review Activities

Through the evaluation and monitoring process, AEI will, through its staff and/or representatives, pay close attention to the goals and objectives for achieving adult learner growth, transitions within the Colorado talent development system, advancement into post-secondary programs or training, and/or employment. The following activities are part of the AEI Quality Assessment Review process:

1. Ongoing Desktop Monitoring - Ongoing monitoring will include monthly data validations through the information management system conducted by regional specialists within the AEI team.

2. Quarterly Data Review - Programs will submit quarterly data review reports whereby programs reflect on progress to date on performance measures and program goals, identify areas of strength and challenge, and note areas of needed technical assistance.

3. Annual Performance Review - Programs will submit an annual performance report on the effectiveness of adult education and family literacy activities of all sub-grant recipients based on the performance indicators, goals and objectives of individual programs.

4. Self-Evaluation - Programs will self-evaluate the effectiveness of adult education and family literacy activities at least once annually by using a uniform self-evaluation. The program must use data collected in the self-evaluation process to develop goals and objectives for improvement in achieving the goals set forth on learner outcomes, professional development plans as well as any other areas of weakness as may be found in the evaluation process. This must be developed and implemented in consultation with AEI.

5. On-Site Review - AEI will conduct on-site reviews of 25% of sub-grant recipients each year. Program order for On-Site Review will be based on levels of risk. Programs determined to be higher risk will receive on-site reviews first. Risk is determined based on an analysis of factors, including size of award amount, timeliness of data entry, “newness” of the program or key program staff, and expenditure amounts.

6. Peer Review - AEI will coordinate and conduct a peer review process focused on quality improvement. The Peer Review process will happen in conjunction with the On-Site Review.

Professional Development Quality Assessment

Participants will be surveyed after all workshops, conference presentations, and webinars to evaluate and provide feedback on the activity or event. Further, CDE will conduct bi-annual grantee support surveys to identify opportunities for improvement. Feedback from the field will be utilized to inform creation of future training and modify existing training, if need be. CDE will encourage professional learning communities on a local level, as well as involvement in State-run, online, communities of practice for new initiatives focused on methods of delivery and learner engagement. CDE will develop an annual professional development plan based on feedback shared from sub-grantees, data collected at the state level to elevate local instructional delivery, and current research-based instructional practices. Finally, CDE will analyze learner performance data collected at the state level compared with local participation in professional development opportunities for correlations with effectiveness.

Additionally, CDE will continue to require the Adult Basic Education Authorization for all instructors at sub-grantee sites. This authorization will require instructors, including volunteers, to document knowledge and competency in the following areas: Introduction to Adult Education, Planning and Delivering Instruction to Adult Learners, Teaching Adult Basic Education/Adult Secondary Education, and Teaching English as a Second Language to Adults.