Located in:
- III. Operational Planning ElementsThe Unified or Combined State Plan must include an Operational Planning Elements section that support the State’s strategy and the system-wide vision described in Section II.(c) above. Unless otherwise noted, all Operational Planning Elements apply to Combined State Plan partner programs included in the plan as well as to core programs. This section must include—
- b. State Operating Systems and PoliciesThe Unified or Combined State Plan must include a description of the State operating systems and policies that will support the implementation of the State strategy described in Section II Strategic Elements . This includes—
- b. State Operating Systems and Policies
III. b. 4. D. Evaluation
Current Narrative:
Ultimately, the three most important resources available to the workforce system and really any organization are its customers, its people (including Boards and other partners), and its data. TWC’s Division of Operational Insight was created to help better leverage that data to help the system become more effective and efficient, and the division has primary responsibility for conducting evaluations of the workforce system in Texas.
TWC uses a variety of statistical techniques to conduct evaluations but primarily uses quasi-experimental evaluation techniques rather than random assignment trials. While less “academic” in approach than random assignment trials, quasi-experimental evaluation provides a balance between evaluation rigor and the desire to move quickly to implement process changes or new initiatives.
In addition, TWC uses a blend of Six Sigma, Lean, and Theory of Constraints called Rapid Process Improvement (RPI) to evaluate system processes and identify opportunities for improvement and test the results of changes. RPI is a core part of TWC’s approach to continuous improvement and has recently been rolled out to our Boards.
TWC works with local Boards and other partners and academic researchers on system evaluations and has participated in numerous federal studies such as the WIA Gold Standard Study from several years ago. TWC will look for opportunities to coordinate and cooperate with the Secretaries of Education and Labor on studies that are aligned with the research priorities of TWC and its Boards and partners. In addition, TWC believes that the regular webinars and regional calls that the Departments of Education and Labor hold provide an excellent avenue for coordinating such work and sharing results between the states and the Departments.
TWC has developed a new Data Warehouse system to support advanced reporting and analytics. The system will streamline standard reporting to improve efficiency and will provide more resources for analysis and evaluation work. The system will also allow TWC to better connect data on participants, services, and outcomes across programs to enable TWC to monitor the highest level of data (that is, systemwide) as well as monitor at the program level (WIOA Titles I-IV and beyond) or the characteristic level (that is, subpopulations across all programs) or a combination thereof (individuals in a program who meet specific conditions).
Over the past year, many of TWC’s analytic resources have been heavily focused on WOIA implementation activities, such as conducting an updated Comprehensive Statewide Needs Assessment and developing Participant Individual Record Layout (PIRL) and RSA-911 logic and testing. However, as that work begins to wind down over the next year, the Division of Operational Insight (DOI) has begun looking at several evaluation projects.
Regarding the Career and Training program, DOI is developing a new model to identify UI claimants who need additional assistance to have a timely return to work. This will be the seventh such model that TWC has developed and the sixth in the last 12 years. Additionally, TWC is interested in leveraging similar techniques to identify individuals who are not claimants but who might also benefit from additional assistance.
TWC has also invested heavily in providing professional development to teachers in the AEL program. DOI will be working with AEL to see whether TWC can measure the effectiveness of professional development by looking at student engagement and outcomes, before and after the teachers participate in professional development. It is not clear yet whether TWC will have a sufficient number of students to evaluate before and after the professional development. However, at some point, TWC expects to have enough data; therefore, even if the project is delayed, TWC’s initial analysis and planning should ultimately prove useful.
TWC also has a number of evaluation projects planned for the VR program. The VR program has operated several specialized programs that serve specific groups of participants. DOI is working with the VR division to evaluate several of the specialized programs, including the following:
- Project Search - a program that uses a rotation of internships to improve competitive employment in an integrated setting for youth who have intellectual disabilities
- Project Hire - a program that helps individuals with severe developmental disabilities to complete postsecondary education
- Criss Cole Rehabilitation Center - a residential VR training facility that serves adults who are legally blind by helping them learn alternative techniques to prepare for, find, and retain employment and/or attend postsecondary education, and/or live independently in the community
Additionally, Texas is participating in the University of Richmond’s Vocational Rehabilitation Return of Investment project. TWC is providing data to the study as well as serving in advisory and steering roles for the project.