U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Https

Secure .gov websites use HTTPS
A lock () or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Plan: Rhode Island PYs 2016-2017
Combined Plan C

Section: Adult Education and Family Literacy Act Program

Narrative: f.

Published
Located in:

f. Assessing Quality

Describe how the eligible agency will assess the quality of providers of adult education and literacy activities under title II and take actions to improve such quality, including providing the activities described in section 223(a)(1)(B) of WIOA.

Current Narrative:

RIDE evaluates programs quarterly through a desk audit by collecting data from its web-based database, the Comprehensive Adult Literacy Information System (CALIS). The data which is analyzed includes student demographics, attendance hours, posttest rates, educational functioning level gains, measurable skill gains, employment outcomes, high school credentials, and transitions to postsecondary education or training. RIDE has a data sharing memorandum of agreement with the RI Department of Labor and Training for employment outcomes. RIDE also conducts data matching with the National Student Clearinghouse database for the postsecondary education outcome and with the GED Testing Services database for GED attainment. The RIDE finance office provides the adult education office with a financial analysis of the amount and percentage of funds expended.

The CALIS database has the full array of National Reporting System (NRS) tables for adult education as well as several customized reports that both the state office as well as local program providers can run to check performance at the state level, program level, teacher level, and student level. These reports can determine areas of both strengths and weaknesses. The state office uses an Educational Functioning Level (EFL) Performance Review spreadsheet as a program-level report card that captures the real time EFL data from local providers in relation to the federally negotiated EFL targets. This spreadsheet allows the state office to determine how programs are doing at any moment in meeting and exceeding the state’s negotiated core indicator of EFL performance. Results for educational gains and follow-up outcome measures are calculated and then weighted by the number of students in each level, or cohort group, and converted into a final percentage expressing the provider’s ability in meeting or exceeding the core indicator targets.

The state has set a minimum of 80% achievement on the EFL Performance Review on the program’s final annual performance results. The EFL Performance Review is also checked in the middle of the fiscal year after the first semester posttests have been administered. Programs that fall below 80% receive technical assistance from the state office to ensure quality improvement. The lowest performing programs are targeted for two-day on-site program reviews, which include a review for compliance with federal and state laws and policies, classroom observations, interviews with program staff and students, and a full financial review. Following the site visit, the programs receive a report from the state office that indicates findings, recommendations, and commendations. Programs that receive any findings are required to submit a Corrective Action Plan on their process for correcting the findings. The Corrective Action Plan is tracked by the state office on a regular basis until all findings are addressed and corrected, the quality of the program improves, and the Corrective Action Plan is closed. The state offers high quality professional development opportunities to assist program directors to ensure program quality improvement and data analysis. Quarterly Leadership Institutes (all directors meetings) and monthly CALIS Users Group meetings are both facilitated by the state office.

The office of adult education also supports a statewide literacy resource center, the RI Adult Education Professional Development Center (the PDC), established to improve instruction at all of the AEFLA-funded agencies in the state. The PDC has developed, in collaboration with the state office, a statewide system of professional development to support local activities required under 231 (b), including specific focus points on reading instruction, contextualized curriculum as a component of a statewide Career Pathways system, the development of literacy volunteer training, and the increasing ability to virtually share promising practices and research based models. The PDC conducts several Professional Learning Community opportunities for program directors, instructors, and volunteers. Particular attention is given to program quality, proficiency based teaching and learning, transitions to college, learning disabilities, reading strategies and English language teaching strategies.

The PDC uses various evaluation methodologies, both formatively and summatively, depending on the activity type, to determine the effectiveness of programming. Use of these methodologies also ensures that continuous improvement remains a priority.

Activity Types and Evaluation Methods

Activity TypeFormative Assessment ToolsSummative Assessment ToolsGuskey/ AALPD alignment
Professional Learning Communities (PLC’s)Ongoing feedback through direct questioning throughout activitiesAnnual Survey to evaluate the past year’s curriculum, and to plan for following year’s agenda and curriculumGuskey: 1 & 2 AALPD Standards: 1, 2, 3, 5, 6, 8, 9, 10 (and 11 for the Directors’)
One Time Workshops & Annual ConferenceOngoing feedback through direct questioning throughout activities
  • PDCenter evaluation forms to assess reactions, learning, plans for implementing, and future needs for PD
Guskey: 1, 2, 3 AALPD Standards: 1, 3, 5, 7, 8
Institutes (STAR, LEA*, etc.)
  • Ongoing feedback through direct questioning throughout activities
  • Evaluation after each f2f session
  • Minutes from follow-up meetings
  • Questionnaires
  • Structured interviews with participants and directors/managers
  • Participant portfolios
  • Video or audio tapes
Guskey: 1, 2, 3, 4 (and 5, if participants track the classroom outcomes of students through CALIS) AALPD Standards: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
Agency-based TA/ PD
  • Setting agenda beforehand to identify targeted needs
  • Direct questioning of participants throughout
  • PDCenter evaluation forms to assess reactions, learning, plans for implementing, and future needs for PD
Guskey: 1, 2, 3 (and 4 if follow-up sessions occur- implementation can then be identified and assessed) AALPD Standards: 1, 2, 3, 4, 5, 7, 8, 9 (and 6 if follow-up sessions occur)
On-line surveys/ reports:
  • Annual Needs Assessment
  • Annual Survey of Deeper Learning by participants
  • Questions on Quarterly Reports to RIDE
  • Using self-assessment tool aligned with Practitioner Standards, agencies develop priorities for PD for the coming year and submit
Guskey: 1, 2, 3, 4 AALPD Standards: 3, 8, 10, 11

*STudent Achievement in Reading, Leadership Excellence Academy

The PDC website has been developed to promote and support the dissemination of information on effective models, both national and local as well as according to best practices in adult literacy instruction. Links to Digital Literacy content and curriculum aligned with Minnesota’s North Star assessment, and best practices in blended learning and the use of Open Educational Resources are examples of resources that support current state priorities. This website also aligns with other new, required activities under WIOA, including the Tutor Ready Learning Plans resources which OCTAE has made available for the training of volunteers, as well as references to the local expertise of the Literacy Volunteer agencies that have experience implementing the effective research-based professional development from ProLiteracy. Also available here is the robust online state and national resources for developing career pathways and partnerships with employers, as well as with the nine RI Industry Partners representing and working with identified high wage and high demand industries in the state.

As a means of sharing high quality online resources and on-demand professional development, the PDC is selective in locating only vetted, research-based resources on its website. PDC researches and curates instructional and programmatic practices from around the country, identifying and disseminating those that are the most rigorous and scientifically researched and validated. Two locally developed additions are a pilot implementation of standards based curriculum project and the eight-year long RI Transitions to College and Careers Initiative. There are eight programs who work with the PDC and with the Transitions Coordinator located at the Community College of RI (CCRI) to develop and share curriculum and program design that were developed to assist learner success and persistence in higher education and in postsecondary training opportunities. Based on research from LaGuardia Community College, Christine Smith, and the analysis of TTC graduates at CCRI, these programs have continued to improve outcomes through sharing best practices and research in a strong Professional Learning Community (PLC) that meets every 6 weeks, and communicates virtually in between.

There have also been three cohorts of Administrators completing the Leadership Excellence Academy, which highlight the use of data and research in continuous program improvement.

The PDC website has become an important source for the field during this time of transition, as well as locating all the resources and tools used by PDC staff and high quality consultants while providing trainings, work-embedded coaching and professional development. Practitioners are pointed to these resources and tools in all professional development sessions as well as at program specific technical assistance sessions, and all other statewide meetings. Reminders of the resources are also incorporated into messaging from the state office and PDC where possible.

The current plan on resource sharing includes an annual dissemination and continual curation of proven models - primarily through an annual calendar which includes standards based initiatives, trainings on required standardized assessments and contextualized curriculum, New Practitioner Orientations, and professional learning communities among many other offerings.