Abstract
E-training is one of the advanced training concepts that have emerged in recent years. In this regard, learning institutions and corporations have adopted e-training programs to achieve efficiency in several aspects. However, evaluating the effectiveness of e-training programs can be done using Kirkpatrick’s evaluation model. This paper discusses how Kirkpatrick’s evaluation model can be used to evaluate an e-training program. From this perspective, the effectiveness of e-training programs is subjected to Kirkpatrick’s four levels of evaluation. The evaluation levels are referred to as reaction, learning, behavior, and results. Each level of evaluation is discussed in detail, and how it can be essential in assessing the effectiveness of an e-training program.
Assessing Effectiveness Of E-Training Programs Based on Kirkpatrick’s Model
Literature review
It is critical to evaluate the effectiveness of e-training programs either in schools or corporations. Establishing an evaluation mechanism that is consistent with the objectives of an e-training program is essential. Kirkpatrick’s four levels of evaluation are the most effective for this assessment. From Kirkpatrick’s model of evaluation, it is evident that a progressive evaluation of e-learning programs is assessed by levels. In this regard, it is easy to detect any error or malpractices exhibited in the e-training programs. As indicated earlier, it is important to achieve an overall goal of efficiency at the end of e-training programs.
Effectiveness of e-training programs based on Kirkpatrick’s model
It is crucial to understand that e-training programs are established to achieve efficiency in terms of functionality and cost. Strother (2002) uses Kirkpatrick’s evaluation model in assessing the effectiveness of corporate e-training programs. Nonetheless, the evaluation model can be replicated in schools’ e-learning programs (Galloway, 2005). Most of the learning institutions are now enrolling students into e-learning courses (Galloway, 2005). Kirkpatrick’s model uses reaction, learning, behavior, and results as the major levels of the evaluation process. Unlike other evaluation models, Kirkpatrick’s model uses scientific methodologies to achieve optimal results.
Level 1: Reaction
This level is used to determine how participants of an e-training program feel. In addition, this level evaluates trainees’ satisfaction with the e-training program. Perception towards the e-training material is to be assessed among the trainees.
Why measure reaction
The reaction level is significant during the initial stages of implementing an e-training program. For example, school administrators can use Kirkpatrick’s model to determine students’ reactions to methods of teaching, the curriculum, and the training material used in the program.
How to measure reaction
In a research conducted among corporations establishing e-training programs, Strother (2002) alleges that the corporations identified 87% of the trainees preferred digital training during office hours. Another 385 of the participants that were interviewed preferred to undertake e-training lessons rather than classroom sessions. The reaction was measured using a questionnaire provided to the trainees.
Effectiveness
Unlike other models, Kirkpatrick’s model identifies both positive and negative reactions. In most cases, a positive reaction among the trainees increases the possibility of e-training success and effectiveness. On the other hand, a negative reaction reduces the effectiveness of the program. A negative reaction requires attention and an immediate correction mechanism. From this perspective, the model assesses why the trainees’ reaction affects the next level of evaluation.
Level 2: Learning
This level describes learning as a process where trainees acquire and understand facts and techniques. This level measures the change of attitude among the trainees. In addition, this level evaluates by how much the trainees have gained new skills and knowledge.
Why measure learning
Some corporations take great interest in their trainees’ progress at this level of evaluation. Therefore, corporations are required to keep records of their employees’ performance at this level as a point of reference. Sun Corporation Network Academy is known to keep records of employees’ performance at the learning level of evaluation (Bylinsky, 2000). A comparison between e-training and class-based lessons suggests that the former is efficient and economically advantageous.
How to measure learning
Kirkpatrick’s model advises trainers to use pretests in determining trainees’ level of understanding before completing the e-training process. To complete the learning level, a posttest can be issued to trainees to determine their qualification to the next level of evaluation. However, the evaluating personnel needs to use a control group to determine the practicability and effectiveness of the learning process. Results derived from the evaluation at this learning level should be used to improve e-training programs. A survey conducted at California State University Northridge reveals that at least 20% of students undergoing e-learning perform better than class-based learners (TeleEducation, 2000). Nelson (2001) performed a similar survey from 406 university students and established that e-learning students acquired better grades than class-based students. Serrano and Alford’s (2000) research on infusing technology into the school curriculum suggests that e-learning empowers students. From the same research, it was evident that students’ ability in critical thinking and literacy skills improved significantly.
Effectiveness
From the model, trainees can apply acquired skills and knowledge in real-life situations. Developing critical thinking and literacy skills are essential goals in any learning environment. Kirkpatrick’s model is effective in assessing the net change in knowledge, behavior, and improved skills.
Level 3: Behavior
The effectiveness of the e-training program is evaluated through a behavioral change among the trainees (Hamtini, 2008). Behavioral change among the trainees is termed as a quantitative learning objective.
Why measure behavior
Incorporations, a direct link between a change of behavior and an improved performance is perceived as the desired results of a training program. The evaluation of behavior does not entirely entail measuring trainees learning results. In any case, behavioral change is measured by the overall organizational improvement. However, the trainees’ performance is also measured against newly acquired behaviors. The performance of the trainees must be tangible for it to qualify as an improved behavioral change. At this level, the effectiveness of an e-training program is perceived by how trainees improve on their previous skills.
How to measure behavior
Bregman and Jacobson (2000) argue that an improvement in customer satisfaction is a simple way of measuring behavioral improvement among employees. However, this improvement must be preceded by a previous e-training program on customer service and behavioral management. Unilever is an example of a corporation that improved its sales volume by more than $20 million after subjecting its sales staff to an e-training program (Hoekstra, 2001). However, Unilever’s e-training program supervisors argue that it is difficult to measure behavioral change. Nonetheless, it is easy to notice how trainees improve on their previous skills as evidenced by the excellent customer relations techniques.
Effectiveness
Kirkpatrick’s model ensures that behavior evaluation is successful through reinforcement methods and follow-up programs. The model involves managers and other stakeholders such as supervisors. This level involves managers and supervisors who are entirely entitled to evaluate the trainee’s desired behaviors. Therefore, e-training instructors are required to prepare administrators, managers, and supervisors for the behavioral evaluation role. For trainees, a behavior change is preceded by a desire to change. In addition, trainees must work towards behavioral change and get a reward upon completing the program successfully. Kirkpatrick’s model advocates for the use of a control group during the behavioral evaluation process. The model entails a continuous evaluation of behavioral change to establish a 100% achievement.
Level 4: Results
It is critical to assess the results of the e-training program against its objectives. This is because results are deemed to affect the overall performance of an organization.
Why measure results
Although it is difficult to measure the direct impact of an e-training program in corporations and other institutions, an assessment of a program’s desired results is possible. For example, reduction of operating costs improved productivity and customer satisfaction are desired results of an effective e-training program (Zimmerman. 2001). On the other hand, learning institutions desire an improved grade score among students, as well as an excellent discipline record.
How to measure results
Unilever Company measures the results of its e-training program in terms of sales volume. The same criterion is used by Etera Nursery Supply Company in measuring its sales volume. Evaluating results requires a control group that is manageable in terms of cost and time. It is important to allow the achievement of results to be revealed with time before evaluating the same. Evaluation of estimated results should be conducted before the commencement of the e-training program. This makes it easier to understand the impact of results after completion of the program. Experts in Kirkpatrick’s model argue that measuring each result at an appropriate time is necessary. Moreover, the evaluation of results should include a cost-benefit analysis as a critical element of the process. In addition, stakeholders of e-training programs are advised to seek satisfaction with the results.
Effectiveness
The model assesses the trainees’ ability to apply theoretical knowledge. Therefore, it is critical to review the e-training program results against the predetermined objectives. In addition, assessing whether a program’s results can be applied in employment or an academic setting is of the essence. The model assesses whether trainees’ change of behavior had an impact on organization performance is critical. Evaluating and comparing the impact on cost and time before and after the introduction of the e-training program is another important element supported by the model. From the model, trainers and trainees may deem it necessary to improve on critical areas of failure. Finally, Kirkpatrick’s model supports a continuous assessment of the program functionality in a regular basis.
References
Bregman, P. & Jacobson, H. (2000). Searching for answers: Yes you can measure the business results of training. Training, 38(8), 68-72.
Bylinsky, G. (2000). Hot new technologies for American factories. Fortune, 142(1), 288A.
Galloway, D. L. (2005). Evaluating distance delivery and e‐learning is Kirkpatrick’s model relevant?. Performance Improvement, 44(4), 21-27.
Hamtini, T. M. (2008). Evaluating e-learning programs: An adaptation of Kirkpatrick’s model to accommodate e-learning environments. Journal of Computer Science, 4(8), 693.
Hoekstra, J. (2001). Three in one. Online Learning, 5(10), 28-32.
Nelson, G. (2001). Do no harm: A first measure of effectiveness in small distance education programs. Proceedings of ED-MEDIA 2001: World Conference on Educational Multimedia, Hypermedia and Telecommunications, 2001. Tampere, Finland.
Serrano, C. & Alford, R. L. (2000). Virtual languages: An innovative approach to teaching EFL/ESL English as a foreign language on the World Wide Web. In Lloyd, L. (Ed.). Teaching with Technology: Rethinking Tradition (pp. 195-205). Medford, NJ: Information Today, Inc.
Strother, J. (2002). An assessment of the effectiveness of e-learning in corporate training programs. International Review of Research in Open and Distance learning, 3(1), 1-17.
TeleEducation. (2000). Is distance education any good? Web.
Zimmerman, E. (2001). A competitive edge. Online Learning, 5(10). E2-E7.