Narrative: II. b. 4.

ASSESSMENT The Steering Committee will conduct an annual assessment regarding the state’s progress in implementing this plan. The assessment will review progress, obstacles and necessary changes in course of action. In addition, the State will assess the overall effectiveness of the workforce development system as described below. Maine’s Workforce Longitudinal Data System may play an integral role in measuring long–term participant outcomes and in providing information that drives program evaluation and policy decisions. The system is designed to measure wage and employment outcomes of program participants by credential and program. A range of programs are currently part of the data sharing partnership, and additional partners will be added. Current partners include Maine’s Community Colleges, the University of Maine System, Adult Education, Vocational Rehabilitation programs and MDOL. Defining Industry Recognized Credentials and measuring the attainment of these credentials will also be an important aspect for assessing progress. Once credentials are defined and data sharing partnerships arranged, the Maine Workforce Longitudinal Data System may be used to measure the employment outcomes of participants earning these awards. Maine will also utilize control group studies to assess performance, as described below. Maine’s economic analysis states that the majority of jobs that are projected to grow in Maine will require postsecondary education and training. Maine has utilized Individual Training Accounts (ITAs) to support WIA participant attainment of traditional two and four year college degrees. However, WIOA strongly emphasizes employer–based workforce preparation and promotes new education alliances intended to limit duplication of services and to increase the engagement of employers in developing standards and articulating pathways workers can take to attain entry to upper level skills. Selecting a pool of individuals who successfully completed training during PY15, the state will analyze and compare the long–term employment and earnings outcomes of participants enrolled in employer–based training, (On–the–Job Training, Registered Apprenticeship, Customized Training, or employer–funded certificate training) with the outcomes of participants enrolled in traditional two or four year college degree or certificate programs. Information used to evaluate the outcomes of each of the participant groups will include the costs borne by the WIOA programs, as follows: • Full–cost of the formal component of participant training: o For college bound trainees: tuition, fees, books, remedial or pre–requisite courses and other charges of the educational institution; o For employer–based trainees: OJT wages, cost of any related classroom training or instruction, cost of customized training funded by WIOA; o Full cost of supportive services funded for the full term of the training program. o Percentage of staff time directly supporting the participant from the point of program entry to successful completion of training and including any follow–up services; o Full amount and type of non–WIOA funds leveraged, such as: PELL, scholarships, individual contributions and employer wage and classroom cost match, including costs of employee benefits paid to participants while in training, if any. Additional aspects that will be evaluated for each participant will include: • Industry for which trainee was prepared; • Length of time in formal training (program/OJT start to program/OJT end date); • Length of time from completion of training to attainment of employment; • Whether or not the training included both an experiential and academic component; • Whether or not trainee attained a portable credential, degree, diploma, license or will attain a portable credential, such as a certificate of apprenticeship; • Wages at second quarter after exit; • Wages at sixth quarter after exit; and • Wages at tenth quarter after exit. In addition to participant analysis, employers offering the work–based training will be surveyed regarding their perspectives on the value and effectiveness of this approach. Employers will be asked to provide their input regarding the overall success of employees they hire just out of college with those they have hired and trained through WIOA supported work–based training. This input will be used to inform service providers who are developing work–based training programs for employers and participants. Participants with similar socio–economic backgrounds and academic levels at entry will be randomly selected from across the state, using data captured through formal intake and assessment. Because it is assumed that the number of employer–based training approaches will increase under WIOA, additional comparison cohort groups will be selected each year through PY17. Data accumulated will be analyzed at the end of each program year and in total at the end of PY20. A report comparing the long–term employment and earnings outcomes of each of the two participant cohorts of the study will be produced. Results of the evaluation will be used to inform workforce development service providers and employers about the benefits and/or pitfalls of each of the approaches and to improve the practices involved with each of the approaches, including the following: • Initial and ongoing involvement of the employer or industry for which employees from both cohorts are being educated and trained; • Processes used to prepare and support participants throughout their preparation for training, training, and subsequent employment; • Formulas for greatest return on investment and cost efficiencies identified; • Level of relevance to the employer for each training approach; • Level of skill quality produced as identified in earnings and employment of each approach • Amount of non–WIOA funds leveraged with each approach; • Overall lessons learned based on the findings including strengths and weaknesses of each approach for both the employer and the worker.