E-learning Evaluation and Organizational Performance

Introduction

The traditional form of imparting training and education is changing phenomenally. The concept of e-learning has gained a foothold all across the world. The use of technology as an aid for imparting education and training has become a radical form of innovation and has provided a challenge to both the educators, institutions and learners (Rossiter, 2007). The e-learning process is highly dependent on information technology. Corporations today have adopted this form of knowledge convey, as they need to transfer knowledge at a lightening fast speed. Training is being imparted to employees through e-learning solutions in many companies like Dell Learning, CISCO e-learning, HP virtual classroom (Wang, Wang, & Shee, 2007). Google has opened an “open-source cloud-based” e-learning platform called CloudCourse (ASTD, 2010). This will help in course management and further the possibilities of e-learning. E-learning to workers is imparted through online databases and tools available in the company intranet or Internet. Therefore, training using e-learning tools has helped corporate to solve the problem of imparting training in geographically spaced multinational companies.

An effective method of training is of the great demand for companies, and especially for multinational companies who are situated in geographically dispersed locations. As e-learning is considered to be a valued process of knowledge sharing, its importance for academicians as an effective method of knowledge transfer has increased. Therefore, there has been a need to understand, for both academicians and practitioners, the efficacy of the tool. However, limited research is found on the effectiveness of e-learning on organizational performance (Wang, Wang, & Shee, 2007). Little research has been conducted to ascertain the effectiveness of e-learning as a tool for delivering knowledge in the corporate training process. However, for efficient and effective utilization of e-learning tools in an organization, the magnitude of its effectiveness must be ascertained. However, little research suggests any relation between e-learning, its effectiveness, and organizational performance. Therefore, it is important to find out the effectiveness of e-learning in an organizational setting. This paper addresses this issue, wherein, the effectiveness of e-learning is measured empirically.

Problem Statement

Many argue that e-learning provides avenues for cost-saving for corporate as compared to traditional training (Strother, 2002). E-learning provides an opportunity for reduced cost of training, and to some extent, it has been reported to have reduced training cost by 35 percent (Strother, 2002). Technology has been an important part of the implementation decision of e-learning. However, many corporate prefer using a mixed approach to training using both e-learning and traditional training, even though it increases cost. However, ASTD identifies e-learning training can be evaluated using traditional forms of training evaluation techniques (Strother, 2002). Therefore, in the following research, the traditional model of training evaluation will be used to evaluate training effectiveness and organizational performance.

A lot of research has been conducted on the effectiveness and evaluation of e-learning incorporate. On such surveys had been conducted by the American Society for Training and Development (ASTD) in 2008 and 2009. The study showed that almost one-third of training content is disseminated electronically (Rossett & Marshall, 2010). The survey shows that by mid of 2009, “E-learning, from their reports, is mostly about measuring and delivering through familiar instructional strategies such as tutorials and scenarios.” (Rossett & Marshall, 2010) Further, the barriers to e-learning identified in the research were found to be investment and financial constraints, resistance to change by learners and trainers, and shortfall of technologies in providing the proper structure to the learning procedure.

The emphasis that companies stress on e-learning is clear from the research done by Online Learning News in 2001 that shows: “Sixty percent had an e-learning initiative. Eighty-six percent had a priority of converting current instructor-led sessions to e-learning. Eighty percent will set up or expand knowledge-management programs. Seventy-eight percent were developing or enhancing electronic performance support.” (Strother, 2002, p. 2). Therefore, there was a huge surge towards adopting e-learning. A lot of research has been conducted on the effectiveness and evaluation of e-learning incorporate. On such surveys had been conducted by the American Society for Training and Development (ASTD) in 2008 and 2009. The study showed that almost one-third of training content is disseminated electronically (Rossett & Marshall, 2010). The survey shows that by mid of 2009, “E-learning, from their reports, is mostly about measuring and delivering through familiar instructional strategies such as tutorials and scenarios.” (Rossett & Marshall, 2010) Further, the barriers to e-learning identified in the research were found to be investment and financial constraints, resistance to change by learners and trainers, and shortfall of technologies in providing the proper structure to the learning procedure. Therefore, companies show interest in e-learning much less than the expected response should be. The reason for this may be unsatisfactory results from e-learning. Does this imply that e-learning is not as effective as is expected out of it? This paper will try to ascertain and evaluate e-learning programs to find an answer to the above query.

E-learning is a method that is being widely used for delivering corporate training. They use a wide range of technology and traditional learning tools to deliver knowledge. According to the estimates made by Meryl Lynch, the company expedited $2 trillion for training purposes (Gunasekaran, McNeil, & Shaul, 2002). When companies spend such a large sum on training, and on using a new method of delivery, they expect it to work and give them effective results. However, many believe the effectiveness of e-learning is a “mixed bag” as many early adopters of the system have faced failure and there is demand for the traditional method of training delivery (Gunasekaran, McNeil, & Shaul, 2002). Therefore, it is believed that an improved instructional facility is required for better delivery and the usage of the medium.

As the effectiveness of e-learning on organizational performance has not been substantially established, it is important to understand the relationship between the two. The main intention of this research is to generate the reasons for the success or failure of e-learning and how it can be improved as a delivery tool.

Various problems are faced while doing evaluation research of e-learning methods of training. The first problem that is faced is if the traditional training evaluation method can be adopted in evaluating the e-learning process. On the other hand, should the e-learning process be evaluated using the traditional IS evaluation process? Alternatively, a unique model that evaluates both must be developed to assess its effectiveness on organization performance. The effectiveness of e-learning training cannot be determined using a simple learner’s satisfaction method or an evaluation of the overall success using a single scale (Wang, Wang, & Shee, 2007). It is important to understand the utilization of all the e-learning tools and systems used for the assessment of e-learning. Therefore, understanding the e-learning process and its evaluation must be important. Further, the effectiveness of the e-learning process on organizational performance is essential. The purpose of the study is to assess the effectiveness of the e-learning technique in an organization and its effect on organizational performance.

The e-learning evaluation process will entail a four-level evaluation method of Kirkpatrick. The evaluation will be done at various stages of the pre-and post-training process. A control group will be used to estimate the change in performance that can be solely attributed to online training. Candidates will be interviewed in four levels and others like the superiors and peers of the candidates will be surveyed for level 3 and level 4 of the evaluation process. Following this research method, the purpose of the research will finally be to ascertain the effectiveness e-learning has on organizational performance.

The purpose of this experimental study will be to test the theory of training evaluation developed by Donald Kirkpatrick that compares training outcome to individual and organizational performance, controlling for training for employees in the organization under study. The dependent variable training effectiveness will be generally defined as behavioral and productivity change, and the control and intervening variables motivation, knowledge, experience, familiarity, and behavior will be statistically controlled in the study.

Why is e-learning evaluation required?

E-learning has assumed a lot of importance among educational, corporate, and other institutions due to its supposedly low-cost model and flexibility. These projects have received a high success rate and acceptance level over the past years. However, it does not mean that the reported success rates should be all-inclusive. Rather, these are indicative of the possibilities, but the application again depends on independent cases. Further, it should be noted that only successful cases are reported. Therefore, success stories are not a measure of your success. ASTD states, “Only your evaluation can measure your success.” (Horton, 2001, p. 2) Therefore the reason that stands for evaluating e-learning and its effect on organizational performance is to justify the investment cost that is incurred in switching from traditional training to e-learning model. Further, it will also help in undertaking better decisions for the training process. In addition, it will help in demonstrating the financial responsibility of the training process. The reason for this research is, therefore, to open a new leaf in the debate on the effectiveness of e-learning on organizational performance.

Another reason for conducting this study is to ascertain the difference in the evaluation process of traditional and e-learning models of training. Though both e-learning and distance learning follows a similar format of knowledge dissemination, there is a distinct difference between the two. E-learning has often been hailed as the intelligent way of delivering training in the modern world. However, the distinct differences it has with traditional learning and the elements that make it a great option are ambiguous in literature. Therefore, there have been several views on the pros and cons of e-learning and theories regarding its success or failure. Thus, this research strives to give a unilateral direction to the debate of the efficacy of e-learning.

Research Questions

A thorough understanding of the literature on e-learning evaluation suggests that there is a potential gap in e-learning evaluation research. E-learning evaluation literature provides a basic understanding of the evaluation models developed so far. However, there arises little consistency in the method, approach, and results of these researches. Further, most of the evaluation studies of e-learning so far have been qualitative, except for those that evaluated the system software. Although various studies have tried to ascertain qualitatively the mode of evaluation of e-learning, few studies have tried to evaluate e-learning’s effect on organizational performance. This paper aims to fill in the gap that is evident in e-learning literature.

The research aims to assess the effectiveness of the e-learning technique and the effect it has on organizational performance. E-learning training is a non-traditional mode of knowledge transfer, and therefore, a conventional model of assessment of training be utilized to understand the effectiveness of the technique. E-learning as a training process undergoes various stages and the effect it has in improving employee performance is the same as that of traditional evaluation processes. In addition, how can research be conducted to understand the effect e-learning has on organizational performance. This is done by estimating the effect of e-learning on the increase or positive effect of e-learning on individual and group performance as viewed by the candidates as well as their supervisor and peers. However, in trying to ascertain this, there are a few questions that arise –

1. What effect e-learning has on individual performance?
2. How effective e-learning technique is to impart knowledge has on organizational performance?

Literature Review

Introduction

E-learning has become a popular tool for imparting training. However, there exists a lot of confusion regarding the method of using this model and the way this can be evaluated. Further, the question is raised on the potential difference of e-learning technique with that of a traditional form of teaching, and the difference it may cause to the evaluation method of the former. This section presents the review of previous researches and papers on e-learning in the corporate training process and the evaluation tools used. E-learning exponents believe that this is one of the best modes of transferring knowledge and has far greater capability in reaching a wider range of audiences (Macpherson, Elliot, Harris, & Homan, 2004). Companies like Ernst and Young had reduced as much as 35 percent of their cost of training by using a blend of 80 percent e-learning and 20 percent classroom training (Strother, 2002). However, e-learning has faced many initial hurdles as is observed in the case of SmartForce, DigitalThink, and SkillSoft who faced reduced revenues after adopting e-learning (Bersin, 2002). It is still believed that e-learning is more than a fad and has the capability that “truly can transform the way organizations communicate, train their employees and increase productivity, unlike nearly any other Web-based application.” (Bersin, 2002, p. 26) Given the expectation that the industry and academicians have from e-learning, it is important to understand if the results and effectiveness of the process are quantifiable and if the effect can be related to business results. However, there has been limited research that shows this relation. Research has overlooked the effect e-learning has on the corporate environment. Most have been based on the pedagogical issue of e-learning and issues related to the delivery or implementation of e-learning. Those that have been done, are mostly based on evaluation of the e-learning information system, rather than on the training outcome. Many generally believe that e-learning is expected to bring positive economic outcomes. This has made e-learning an issue of high priority among corporate. However, most of the discussion on e-learning is based on technology, which is a wrong way to evaluate the process, as Strother (2002) e-learning comprises both technology and the human factor. Therefore, in undertaking a literature review on e-learning the first imperative is to understand what e-learning is and what is the purpose to undertake a scholarly understanding of e-learning and the effect it has on organizational performance. The following literature review answers three distinct questions – what, how, and why. First, it deals with what is e-learning? Second, it shows if e-learning is an effective tool for learning and how effective or ineffective it is for organizations? Does the third section assess why an evaluation of the e-learning training process is required?

Definition of e-learning

There are various definitions of e-learning available in the literature. E-learning is learning using the Internet as a medium (Gunasekaran, McNeil, & Shaul, 2002). They further the definition by stating that e-learning entails “Components [that] can include content delivery in multiple formats, management of the learning experience, and a networked community of learners, content developers and experts.” (Gunasekaran, McNeil, & Shaul, 2002, p. 46) Therefore, they imply that e-learning is enabled by the Internet that includes components of knowledge delivery in various formats, management of the content, and the way of delivery through a networked community. This definition of e-learning limits the definition to only delivery of training through the Internet, Lima, Ripley, & O’Steen extended the concept of e-learning to training delivery through the Internet, intranet, audio, video, television, CD-ROM, satellite, broadcast, etc. (Lim, Ripley, & O’Steen, 2009). In another definition provided from the point of view of the information technology base of e-learning, Wang, Wang, & Shee (2007, p. 1794) has defined e-learning process or system as a “special type of IS”.

Therefore, from these definitions, it is clear that e-learning has two specific characters – traditional pedagogical content and information technology base. Rossiter presented three dimensions of e-learning – “use of communication and information technologies (CITs) to mediate and enhance teaching and learning, contextually derived conceptions of quality, infusion of creative and innovative elements into teaching and learning.” (2007, p. 95) He implied that the use of information and communication technology has assisted e-learning in improving its capability beyond the simple usage of technologies like the Internet or television as a medium of transferring education. Therefore, the technologies that enable or enhance the possibilities of communication have a greater role to play in e-learning than others (Rossiter, 2007). Second, Rossiter shows that the quality of education can be increased and/or standardized with the use of technology and specifically e-learning that helps in maintaining the quality of education imparted. This essence of quality is based on the concept of excellence, organizational values, and standard of education. Further, e-learning has provided scope to organizations to think out of the box and become inventive. Thus, the dimensions presented by Rossiter show that e-learning is more than just knowledge imparting beyond using mere technology. Therefore, e-learning is found to be more than just delivery through the Internet, but the whole process.

Theory of E-learning

Traditionally e-learning is viewed as a process of providing live training as the only means of providing training for sales and management training (Gunasekaran, McNeil, & Shaul, 2002). However, the problem with these firms of the traditional view of e-learning, as perceived by Gunasekaran, McNeil, and Shaul is that “Companies fail to consider distance learning as an alternative-or-additional-to so that management and staff development programs can be even more effective.” (2002, p. 47). Organizations fail to accept distance learning as a form of possible cost saver and more effective in terms of management and staff training. The main advantage that e-learning provides compared to the traditional form of educating is flexibility, efficiency, motivation, and cost reduction (Gunasekaran, McNeil, & Shaul, 2002).

Innovation arises out of a process of interaction between various people, technologies, and cultures. It is through the transmission of knowledge and technology that the product, process, and services for an organization are developed. It is believed that ICT provides the most ground for the creation of teaching and learning. Further, it is argued that the success of implementing e-learning requires serious management involvement and support (Gunasekaran, McNeil, & Shaul, 2002). As many believe e-learning to be the framework that helps them to face global business change, it has gained importance among organizations. However, it is found that most e-learning classes face a maximum rate of attrition compared to traditional classroom teaching. Given this high importance of e-learning and high rate of dropout, it is believed that a series of measures could make the technique more effective: “When considering best practices in e-learning, critical factors would include whether selected methods are both efficient in using the organizational resources and effective in achieving educational goals and values.” (Lim, Ripley, & O’Steen, 2009, p. 212) this demonstrates that e-learning is an important part of today’s corporate training and is a potential strategic opportunity.

Difference between e-learning and traditional form of training

Though both e-learning and distance learning follows a similar format of knowledge dissemination, there is a distinct difference between the two. E-learning has often been hailed as the intelligent way of delivering training in the modern world. However, the distinct differences it has with traditional learning and what elements make it a great option are pretty blurred in literature. Therefore, there have been several views on the pros and cons of e-learning and theories regarding its success or failure. Initially, it was believed that e-learning as a concept made learners conceited and self-limiting (Ellet & Naiman, 2003). A second view related that e-learning was the only way that could be used to deliver training cost-effectively and effectively. Therefore, both the first and second view was too radical to be accepted. Therefore, a third view emerged that stressed the effectiveness and efficiency of the delivery of knowledge through e-learning. However, a fourth view showed a more realistic view that told that e-learning is a new concept and must be evaluated and evolve continuously to reach its full potential (Ellet & Naiman, 2003).

Is e-learning an effective tool of training?

Much of the literature on e-learning demonstrates the opportunities that e-learning poses for corporate. The discussion of advantages in the literature is based on two arguments – cost advantage and flexible delivery (Macpherson, Elliot, Harris, & Homan, 2004). The first advantage shows how training time can be reduced and cost related to traveling and time away from the office is evident. Other than this, e-learning poses the possibility of addressing a larger number of students with the same capacity and infrastructure. This increases the possibility of the companies reaching a larger number of people at a shorter time limit, thus reducing extra cost (Koprowski, 2000; Macpherson, Elliot, Harris, & Homan, 2004). Further, e-learning is found to provide a strategic advantage and provide a competitive edge to companies. Macpherson et al. show that these believes “raises expectations in organizations that introduce e-learning in terms of both the extent of the return on investment (ROI) and the period over which the payback will take place” (2004, p. 297). Further study has shown that 80 percent of the Fortune500 companies are using e-learning or are expecting to implement it (Macpherson, Elliot, Harris, & Homan, 2004). This demonstrates that e-learning is being adopted by companies readily, however, no study or model has demonstrated the pros and cons of using e-learning.

Mandinach (2005) presents a few points that make e-learning a unique proposition different from the traditional training format. He points out that presently there is a lot of importance given to the technological inclination of e-learning. However, it must be noted that e-learning is just another tool of knowledge and education dissemination. Therefore, he states, “The technology becomes a necessary but not a sufficient condition.” (Mandinach, 2005, p. 1815) Other variables that affect the delivery of e-learning effectively are “institutional infrastructure, pedagogical or teacher processes, and student learning processes” (2005, p. 1816). Therefore, in implementing e-learning, the greatest mistake that organizations do is to translate traditional lessons to computer-based training. However, this is not an effective measure of e-learning. Thus, e-learning “must capitalize on the new capabilities that are made available by the technology” (2005, p. 1816). This makes e-learning distinct from the traditional training process.

Hurdles to e-learning

In using e-learning as a tool for studying and imparting training various problems might occur. Yuan (2007) showed that three main problems are faced by the e-learning process in an organization. The three problems are – administrative, attitude towards e-learning by instructors and students, and the traditional methods’ interaction with computer technology. E-learning practitioners have to find the way e-learning can be used to influence policymaking and administration. This imperative is necessary as the cost of installation of the systems required for establishing the e-learning systems in situations and administering them assumes priority importance. Proper administration of the computer systems is required as without which the whole process of imparting education becomes futile (Yuan, 2007). Therefore, Yuan state: “… without careful planning, the integration of the technology will also lead to unanticipated systemic effects, such as a change in the way the organizations define itself and its objectives.” (2007, p. 422). Apart from this, the attitude of the new discourse process users must be accommodating. If the students or learners and the teachers are unwilling or incapable of adopting the new technology-based teaching method, then the whole effectiveness of the system will be lost. Further researches have also been done on the attitude of the teachers and learners towards technology-based teaching, and they feel that it does not affect teaching largely (Yuan, 2007). Another problem that is observed in the case of e-learning is the fast technological innovation has made the student/teacher–computer interaction process complex. Further technology has advanced considerably but has not obliterated the use of teachers or instructors. However, these teachers are not always adept in handling this interactive technology that aid in training, and therefore reducing the level of educational discourse. Therefore, these problems are often observed in the case of electronic-based instructional procedures.

Another problem observed in the case of e-learning is that they hamper the relationship between the people who are undergoing training within the e-learning framework (Oiry, 2009). Oirwy conducted interviews in French banks and found that e-learning failed or was obstructed as the simulations or roles given to the trainees during the e-learning process did not correspond with their regular socialization roles, and therefore the learning process was hampered. The solution to this problem that the organizations came up with was ‘blended learning. The exploratory study, therefore, concluded: “… the spread of e-learning might be slowed because of the deterioration it causes in relationships between trainees and their direct managers or colleagues. In the four banks we studied, we notice the recurrence of the phenomenon.” (Oiry, 2009, p. 121) Oiry states that in the case of the blended learning process, a problem of role conflict arises among the participants. His interview results demonstrated the following: “In all four banks interviewed, our training department informants mentioned the fact that direct managers did not really like employees training in the workplace, particularly because most of the time they used the same computers usually reserved for work.” (Oiry, 2009, p. 119) Therefore, the role conflict that arises in the case of e-learning from the workplace is that learners know that they are in midst of the training process, however, the managers are confused if the employees are working or training. Therefore, a direct manager may feel that the employee is still working and not training and thus think that he is “subject to productivity targets.” (Oiry, 2009, p. 119) Their research showed that in many cases e-learning’s main interest in its being a flexible process of training is erased, as has been explained by Oiry:

Bank D Training Manager explains: ‘[It is a] place located away from the branch with a computer reserved for training. Employees can go there when they wish or when they have work to do in an e-learning session. Most often this is what we do for office skills training and English training’. This computer is different from employees’ workstations and is out of sight of direct managers, colleagues, and clients. … This solution means that e-learning loses part of that flexibility which appeared initially to be its primary interest (for example, enabling an employee to do 20 min of training per day [depending on workload] without leaving his/her workstation). (2009, p. 120).

Therefore this way of e-learning takes away the flexibility character posed by e-learning, however, it does reduce the problem of role conflict. The role conflict situation is so acute that direct managers tend to harbor a paradoxical attitude towards the employee. This problem is solved by changing the training venue from the employees’ workstation to a separate training location where a computer is available, close to the office. However, this takes away the main plus point of e-learning i.e. flexibility.

Research on E-learning evaluation

Evaluation Research

The studies that evaluated the effectiveness of e-learning have been based on higher education, government, corporate, medical, and military institutions. In this section, the literature review will show how e-learning has been evaluated in all these institutional backgrounds. E-learning as a training strategy has been employed in various areas.

Reviews of educational models have been important to find their suitability for the present business environment. McAleavy (2000) believes that the electronic education system has the scope and capacity to spread education among a greater number of people than what has access to it right now.

Research conducted by Mandinach on the effectiveness evaluation methodologies applicable in e-learning in higher education shows that e-learning evaluation should consider the “conative processes, student satisfaction, role changes, and other noncognitive variables” to understand the effect e-learning has on the students or the participants in post-training period (2005, p. 1826). Therefore, the first area of evaluation of e-learning should concentrate on the effect the training has on the students. The second effect that must be measured is on the teachers or instructors. The second stage should concentrate on the effect the process has on “teachers, professors, and pedagogical processes” (2005, p. 1826). As teachers are considered to be the primary instrument in the education system, their pedagogical skills should be considered first before considering the student’s skill development (Mandinach, 2005). In other words, if the instructor is unable to deliver the lecture properly through an electronic medium, it is the fault of the teacher that effective knowledge is not imparted, which would lead to low organizational performance. First, an understanding of the instructional techniques of teachers must be undertaken to understand the “pedagogical effectiveness in e-learning and then conduct comparative studies among various implementations of e-learning and between e-learning and traditional delivery.” (Mandinach, 2005, p. 1826). The third level of evaluation should focus on the effect of e-learning on the organization. The main reason for conducting this level of evaluation is to gather information on the service improvement of e-learning. Therefore the third step should try to evaluate the effect e-learning has on the organizational outcome. Mandinach mentions that all these three processes of evaluation must be considered together and not independently.

In most of the studies irrespective of the institutional background, three main areas of e-learning have been evaluated – utility of the product or the e-learning service, its return on investment or cost-effectiveness, and the satisfaction of the students or candidates. Here utility is the content, and process of delivery of e-learning. Many studies have shown that e-learning is as good as if not better than the traditional form of training. Further, it is believed a combined method of instruction is better than only the traditional or only e-learning method (Ruiz, Mintzer, & Leipzig, 2006).

Gibbons and Fairweather (2000) mention many studies that had been conducted before the Internet was available, that studied and compared the teaching technique using computers and traditional methods of teaching. The studies adopted various research designs for both training and research environments. However, their studies provided inconsistent results. However, their studies demonstrated that the knowledge of the students or candidates improved in the post-training periods. Further, their research also showed that students using a computer-based learning method learned more than the others did. Therefore, their study showed that electronically discoursed training demonstrated a higher rate of retention among students than the traditional form of teaching.

Evaluation of e-learning in the case of the business environment has mostly dealt with the evaluation of the e-learning system rather than the process. Hildrum (2009)has added to the ongoing debate on the effect ICTs have on e-learning or tacit knowledge. The study was conducted on the philosophies of Michael Polanyi. The study is a case study analysis of Cisco Systems and therefore argues against the view that ICTs or technology-based teaching are incapable of spreading tacit knowledge. On the contrary, their case study analysis showed that the e-learning process helps in the dissemination of tacit knowledge: “e-learning activities can facilitate the interpersonal sharing of tacit knowledge.” (Hildrum, 2009, p. 212) Further, the study also concluded that the motivation is a key factor that affects a learner’s ability to acquire knowledge online: “successful e-learning performance depends primarily on the degree to which the users are motivated to acquire new knowledge online.” (Hildrum, 2009, p. 213) Further, the study also concludes that there is a close relationship between the knowledge sharing performance of learners:

… there is a mutually reinforcing relationship between knowledge sharing performance and motivation, meaning that past successful performance is likely to inspire more e-learning in the future. These final relationships are important because they point to opportunities for encouraging virtuous circles of motivation and knowledge-sharing performance; initiatives to increase motivation through community and network participation can lead to more e-learning and better knowledge-sharing performance, which in turn can boost motivation and so on. (Hildrum, 2009, p. 213)

Therefore, the study shows that ICTs are capable of dispersing tacit knowledge against popular views.

Another study conducted by Zaharias & Poylymenakou (2009) is questionnaire-based research that evaluates e-learning applications. The method of the study is based on the e-learning research practice that focuses on cognitive as well as affective considerations that have a strong influence on the usability of the e-learning process. the method employed in the research was based on the HCI research and the framework used for the Web-based instructional designs and correlated them with important e-learning dimensions. They conducted two empirical studies were conducted to evaluate e-learning training in a corporate environment. The research conducted a first-factor analysis to identify the different dimensions of e-learning usability and factor analysis showed existing internal consistency between the factors that had been identified in the previous analysis. These were conducted on the pilot survey results. Then multiple regressions were conducted to see if there exists any relationship between e-learning usability and the motivation of the students to learn. Their results of the study showed “a strong relationship between e-learning usability and motivation to earn.

The latter is proposed as a new type of e-learning usability measurement and the questionnaire is proposed as a proper instrument toward this goal.” (Zaharias & Poylymenakou, 2009, p. 89) They indicate there is further research required in the field of e-learning evaluation.

A study of evaluation of e-learning through WebCT using Bloom’s taxonomy is done by Halawi, Mccarthy, & Pires (2009). The exploratory research is done to study the effectiveness of e-learning and evaluate the process using loom’s taxonomy. The questionnaire research was done through a questionnaire survey to find out if institutional factors affected the e-learning process. the survey was conducted on undergraduate students and the study showed that individual and instructional factors affect the learning process in the case of e-learning. They performed a regression analysis on the results of the questionnaire survey. The regression was done between the individual and the instructional factors. Their study showed that individual factors like “gender, age, educational level familiarity, time dedicated to studying, and learning style” and institutional factors that were considered were “effectiveness of tools used, interaction with the professor, and ease of use of technology” (Halawi, Mccarthy, & Pires, 2009, p. 378). The study demonstrates that the use of ICTs are an essential part of e-learning as they fund that the “Student success in an MIS course ultimately requires them to demonstrate their ability to use the software and obtain feedback on their use of it.” (2009, p. 378) Their study shows that there exists no: “… significant differences among the constructs of individual factors, instructional factors, and learning through WebCT. Thus, results indicate that individual and instructional factors do not have a significant effect on e-learning.” (2009, p. 378) Research on e-learning effectiveness is huge and the results of these researches are varied, however, they provide an empirical understanding of e-learning effectiveness using Bloom’s taxonomy.

Strother (2002) studies the effect e-learning has on organizational performance, therefore indicating if firms face a cost-advantage by adopting e-learning practices. He uses Kirkpatrick’s training evaluation model to estimate the effect training has on organizational and individual performance. Kirkpatrick’s model is divided into four levels – reaction, learning, transfer, and results. He uses the quality measure to review the content of the training program. Strother states that analysis of the content of e-learning programs is important, as this will provide an overview of the quality of the online learning program. He points out that if the quality of the e-learning is established to be good, the program will essentially be good. However, little stress is played on the delivering and acceptance capacity of the instructor and the learner who remain on the delivery and the receiving end of the process. This research on the content of the e-learning program will only provide a partial overview of the over-learning process and therefore will not help in e-learning the training process properly.

Another study on the effectiveness evaluation of e-learning was conducted by Tzeng, Chiang, & Li (2007). Their research was an effort to identify and make a generalized quantitative model for e-learning evaluation. Their paper was based on the MCDC model and addressed the evaluation criteria were analyzed by factor analysis. The empirical study pointed out that the MCDC model was capable of evaluating e-learning facilities. The study concluded that there is a deficiency in e-learning evaluation. The reason for the lack of an evaluation model is due to the availability of different procedures that could help in the evaluation of e-learning using a quantitative model. Their research proposes a hybrid form of the MCDC model based on their empirical study. They propose:

According to the results of the empirical study, the hybrid MCDM model should be a workable and useful model to evaluate e-learning effectiveness and to display the interrelations of intertwined criteria. As a result, if the effectiveness of an e-learning program is deficient we could find out the problem based on AHP weights and interrelation based on the impact-digraph-map of each factor. After using this e-learning effectiveness evaluation model, the evaluators could find the aspects needing improvement, so that e-learning program effectiveness could increase. (Tzeng, Chiang, & Li, 2007, p. 1040)

This model, as compared with the traditional model of evaluation, considers more aspects of e-learning evaluation criteria. Therefore, they present great weight on e-learning environment, expectation, and reward as important factors that affect e-learning outcomes. Another factor that has been discussed is the fuzziness factor that affects the perception of e-learning outcome of the employee (learner): “This means improving individual criterion performance by considering the effect from the others if the synthetic effect exists.” (Tzeng, Chiang, & Li, 2007, p. 1040)

There is a conflict in the methodology and results in the qualitative and quantitative evaluation fields of e-learning. Further, there is also a difference between the evaluation community based on erudition, theory, and proof. The theoretical movement maintains that e-learning evaluation must be done within the mores of the theory. The learning community believes that stakeholders must gain knowledge from the evaluation process, and the evidence-based movement stresses experimental methods that should be employed to determine the workable method. Though there is a lot of difference among evaluation researchers, Mandinach (2005) believes that there is an overlap of ideas:

There are at least three major dimensions by which evaluations can be characterized: (a) internal versus external validity, (b) formative versus summative, and (c) qualitative versus quantitative. Evaluators stress different perspectives and different dimensions. Although there is overlap within the dichotomies, there certainly is no consensus. (2005, p. 1819)

The literature on e-learning effectiveness evaluation provides broad and sometimes conflicting results. There is no consistency in the research results of researchers that demonstrate the right evaluative process or result. Therefore, further research in the area is essential.

Training and Organization Performance

Through various researches have evaluated the effectiveness of e-learning, none have correlated it with organizational performance. Therefore no comprehensive understanding of the effect e-learning had vis-à-vis traditional learning is unavailable in e-learning evaluation literature.

In developing the methodology, the initial axiom that is taken is that the quality of management represents the performance of the firm. This can be extended that an effective training session can produce a good performing employee. Therefore, various aspects of training can be related to the performance of the organization. There have been numerous studies that have tried to establish a relationship between training and organizational performance (Russell, Terborg, & Powers, 1985; Delaney & Huselid, 1996; Nikandrou, Apospori, Panayotopoulou, Stavrou, & Papalexandris, 2008). Studies have suggested that there exists a direct relationship between increased investment in training and better employee performance and productivity, the profitability of firms, and shareholder value. Research has shown the percentage of trained employees affects the performance of the organization (Russell, Terborg, & Powers, 1985). However, other studies have failed to establish any such relation (Cunha, Cunha, Morgado, & Brewster, 2003). It is agreed upon that training evaluation provides the information to demonstrate the benefits that the organization enjoys due to the training. Therefore, organizations “invest in strategically oriented T&D, then some effort should be made to assess the results of T&D interventions.” (Nikandrou, et al., 2008, p. 2059).

Conclusion

The above literature review points out that there is a dearth of academic research. Earlier researchers had presented a similar view: “utility of content, problems with the technology, learner support, and evaluation” of e-learning (Macpherson, Elliot, Harris, & Homan, 2004, p. 297). Thus, a potential gap in e-learning research is the evaluation of e-learning training. Therefore e-learning evaluation literature provides a basic understanding of the evaluation models developed so far. However, there arises little consistency in the method, approach, and results of these researches. Further, most of the evaluation studies of e-learning so far have been qualitative, except for those that evaluated the system software. Therefore a comprehensive empirical quantitative study needs to be developed that can be utilized to demonstrate the effectiveness of e-learning and its relation to organizational performance.

Methodology

The literature review shows that there exist various proponents on the evaluation of e-learning and not two researchers agree on method and design in evaluation literature. Evaluation literature has been based on a few juxtaposing models – internal and external validity, qualitative and quantitative, and summative and formative (Mandinach, 2005). However, it must also be noted that design decisions are personal choices of the researcher in connection with the pertinent stakeholders (Mandinach, 2005). As has been pointed out by Cronbach (1982) the main goal of evaluating a training process is to understand the program and areas of improvement of the program. In such cases, exploratory research methodologies are suggested methodology. This will help in identifying more controlled assumptions or hypotheses while testing a theory. Nevertheless, if the goal of the evaluation is to provide more constructive feedback, then the focus should be on formative evaluation that helps to concentrate on multiple components. This establishes the need for an open-ended form of inquiry. This is essential in the case of studies where the researcher expects unexpected results (Cronbach, 1982).

The other method of evaluating a program is to completely clinch the program and examine the way it is effective. In this case, both qualitative and quantitative techniques are possible methods that can be adopted for this type of research design. Qualitative study is used to understand the various processes and methods in the training process, and quantitative analysis is conducted to validate the findings of the qualitative research. This type of hybrid method of research helps researchers determine the appropriate method that should be used to answer a specific question (Mandinach, 2005).

Various methodologies can be utilized for e-learning evaluation research. Mandinach suggests that as “e-learning is a relatively young and emerging instructional medium, it would be premature to look only at its outcomes” (2005, p. 1821). Being a young field, a lot of information regarding the process and implementation of e-learning can be obtained. One methodology for e-learning study has been mentioned to be implementation analysis (Mandinach, 2005). Another form of evaluation of e-learning is based on the formative method that poses that the process of implementation and conduct of the e-learning process must be evaluated. This process helps in understanding the process of interaction of the technology with the pedagogy. However, this is not suited for our research as the objective of the research is to find the organizational performance that of affected by e-learning. A form of evaluation method of e-learning that is used is systems learning. This process allows the amalgamation of the different components for e-learning i.e. learners, instructors, computers, etc. therefore this process helps “purely as a heuristic device to explore relationships among the components of a program” (Mandinach, 2005, p. 1822). Then the quantification is done, which sometimes may prove to be difficult.

The literature on the research of evaluation of e-learning suggests the acknowledgment of the different perspectives of e-learning. Therefore the goals for evaluation of e-learning must be – effects of learning on the cognitive ability of the learner, the impact it has on teachers, and the benefit it provides to the institution and organizations. The third area must be evaluated as this area has been neglected in the evaluation literature of e-learning. Given this background on the methodological processes available for e-learning, the following section shows how e-learning evaluation study must be conducted. This will show the proposed research method that is adopted for the research conducted in the proposed e-learning evaluation.

Research Design

The research design for evaluating e-learning effectiveness has been under debate however, no constructive model for evaluation of e-learning has been provided. This research is therefore divided into two particular parts – evaluation of the training and its effectiveness and then links it to the organizational performance. Kirkpatrick’s training evaluation tool is used to determine the effectiveness of the e-learning training process.

Kirkpatrick’s Training Evaluation Tool

Kirkpatrick’s training evaluation tool has been used in the paper to evaluate the e-learning training process. Kirkpatrick describes four levels of training evaluation broadly termed as a reaction, learning, behavior, and results (Kirckpatrick, 1996). There had been alternative methods that had been attempted to be developed (Kaufman & Keller, 1994), Kirkpatrick remains a standard for training evaluation in business. Before accepting the Kirkpatrick model, it is important to endorse the method as suitable for e-learning evaluation. This method has been widely used and accepted as a suitable method for e-learning training evaluation in the literature and has been used in the evaluation of the effectiveness of online training programs (Welber, 2002; Strother, 2002; Mayberry, 2005; Wang, Wang, & Shee, 2007). The other reason for considering Kirkpatrick’s evaluation method is that it provides greater flexibility for evaluating a more technology-oriented interactive process of teaching as is observed in the case of e-learning.

The main imperative that is taken to understand here is Level 3 of the Kirkpatrick model that determines if the training outcomes are transcended to organizational performance. Mayberry (2005) believes, “successful implementation of Level 3 evaluations affects learners, manager, project stakeholders, subject matter experts (SMEs), and line supervisors.” The assumption behind level three is that the first two levels of evaluation of training have been met, and the students have learned and absorbed the training content. Thus, this level of interrogation intends to ascertain if the transfer of knowledge through online training has been transferred to the workplace. Therefore, the following process will be followed in interrogating the training and its effect on organizational performance.

The first level of intervention will be a questionnaire survey that will be given to the managers and supervisors of the candidates who had taken part in the online training process. This coincides with the third level of evaluation as suggested by Kirkpatrick (Kirckpatrick, 1996) where he states that to capture a “systematic appraisal” of the improvement of the employee’s on-the-job performance after the training, it is important to gather information regarding his performance from his superiors, peers, and subordinates (1996, p. 133). The statistical analysis should entail a comparison of the organizational performance before and after the training process. Further, the use of a control group is also suggested by Kirkpatrick to demonstrate the effect of training has on organizational performance. This will demonstrate the effectiveness of the training and its effect on organizational performance.

This survey will try to ascertain the outcome, according to the managers and supervisors, of the training. It will demonstrate if knowledge transfer has occurred, the skills and attitude the employees have developed after the training, etc. therefore, the survey will aim at understanding the changes in the performance, productivity, and attitude of the employees in the post-training period.

Research Plan

For this research, the design that is evolved is an experimental model using a control group. The research will be conducted on a particular department in an organization where a group of employees 80 to 90 employees will be randomly selected from the same department. This group of employees will be divided into three groups – group A and group B. Group A will be subjected to an online training program and group B will not be given any kind of training. The evaluation will be done according to Kirkpatrick’s Four-Level training evaluation model (Kirkpatrick, 1994). Therefore, the four levels can be summarized as follows:

• Level 1: What was the reaction of Group A and Group B participants and their perception of the training process and content?
• Level 2: What did they learn out of the training?
• Level 3: Was the learning gained from the training was being used practically?
• Level 4: Was there a positive effect of the learning on the organizational performance? (Kirkpatrick, 1994)

One digression from the traditional Kirkpatrick model is done in this case where level 3 of the training evaluation process as mentioned by Kirkpatrick is conducted before the beginning of the training and the experiment process. In this case, both the groups are evaluated on the behavioral changes that may have occurred. Here the questionnaire is given to the candidates and their supervisors. This process is done following Mayberry (2005) to understand the effect of “pre-and post-assessments that will help you measure the degree of comprehension regarding newly gained KSAs.”

The training that is given to group A will be on cost reduction. The training program for group A will be developed based on the online learning procedure. Group A will be evaluated on levels 1 and 2 of the training evaluation process as directed in the Kirkpatrick model. The level 1 evaluation is stated by Kirkpatrick to be “Positive reaction may not ensure learning, but negative reaction almost certainly reduces the possibility of its [learning] occurring” (Kirkpatrick, 1994, p. 22). Therefore, this level of understanding of the training programs is important, as this will help in ascertaining the reaction of the participants in the e-learning process. Further, the results of the reaction from the two experimental groups will enable us to ascertain the difference in reaction of the participants of the e-learning and traditional training processes.

The second level aims at understanding the learning that has been done by the participants. Here learning is defined as the “extent to which participants change attitude, improve knowledge, and/or increase skill as a result of attending the program.” (Kirkpatrick, 1994, p. 22) This level will show if learning has been acquired due to the training and the level of individual learning. This process will help us understand if there has been an increase in skills or motivation or commitment of the employee due to the training. Here too the evaluation will be done for groups A and B. Kirkpatrick (1994) points out that one or more changes must occur to bring about a change in behavior of the candidate that is transcended to better group performance. The first two levels of training evaluation are important even though it is not an essential part of the main experiment because Kirkpatrick believes that bypassing the first two levels of evaluation may give erroneous results (Kirkpatrick, 1994).

Level 3 of the evaluation will be done on all three groups under study. This level evaluates the change in behavior that is brought about due to training. The behavior here is defined as “the extent to which change in behavior has occurred because the participant attended the training program.” (Kirkpatrick, 1994, p. 22) The control group (group B) is included in this level of evaluation to ascertain what level of performance on an individual level the candidates in the control group that had no training and the other two groups that had training pose.

Level 4 of analysis is done to identify the effectiveness of the training. Here a comparison is made on the performance outcome of the two groups. The survey is done of candidates and managers. This level aims at understanding the benefits that the training has given to the business. The difference between level 3 and level 4 is that in the former, micro-level analysis of the effect of training is done, and in level 4 it is seen how the individual performance is transgressed to increase organizational performance. In this case, we will follow Kirkpatrick’s belief that ROI is a part of the level 4 (Kirckpatrick, 1996; ASTD, 2008) assessment and will not be segregated into level 5 assessment. As ROI is not completely assessed as most training activities directly affect performance but not organization finances, the research will not consider the effect e-learning has on the ROI of the company.

Here once the results are gained, they are compared for pre-and post- assessment results for each group to see the degree of improvement each candidate had. A comparison is done between the groups in case of post-training assessment to see the level of change that has occurred to group A and group B that gained electronic training and the group that received no training. The goal of this experiment is to determine if the control group performed better, worse, or the same as group A. Therefore a summary of the research design is as follows –

• Two groups (A and B) are randomly selected and each is assessed on level 3 of the evaluation process.
• Group A is subjected to online training and then is evaluated from levels 1 and 2 of Kirkpatrick’s evaluation to ascertain if reaction and learning have transgressed to behavioral change. At least two conditions of learning must satisfy for behavioral change to take place (Kirkpatrick, 1994).
• Both groups A and B are evaluated on the survey for level 3. Here the participants of the survey are the candidates themselves and their managers. This level is the most important level of the whole experiment as it establishes the difference in the performance outcome of the candidates in pre-and post-training assessment. Further, it also helps in ascertaining if performance has been increased, reduced, or remained unaltered for the control group and group A.
• The fourth level of analysis will determine the effect training had on organizational performance in comparison to the control group. This will demonstrate how the organizations benefit from e-learning.

It must be noted, that the whole of the Kirkpatrick model is not used as a process of training evaluation, rather the model is adapted to resolve some of the issues in evaluating e-learning training program evaluation.

Research Method

Before the training on group A was done, a pre-assessment survey was conducted on groups A and B. Then the training was conducted. Immediately after the training was over, candidates were questioned regarding their views about the course content and their experience in the training process. The questions that are asked during this time are related to the protocol established by Kirkpatrick’s evaluation model. On completion of the course, all the participant’s responses to the questionnaire survey results were analyzed using quantitative statistical tools. In level 3, analysis is done on the pre-and post- analysis of group A, and comparison of the learning outcome of groups A, and control group, group B.

The level 4 analysis is done to ascertain the organizational outcome that can be generated due to the training. This problem may face the issue of isolating the effect of e-learning. The ideal method of analysis of the effect of training on organizational performance is using control groups (ASTD, 2008). Therefore, the problem of separating the effect of e-learning is done by introducing the control group (Group B) for comparing the group performance of A and B that would allow us to ascertain the effectiveness of e-learning. According to ASTD, the process of involving control groups can be done in the following process:

This approach requires that you study two or more groups of individuals of similar composition under similar circumstances. The goal is to ensure that everything is the same between the groups, except for the introduction of a new condition to the experimental group (that is, e-learning). You can compare the results of the two groups to see what change, if any, resulted from the new variable (ASTD, 2008, p. 27).

Data

The data for the research is collected from a questionnaire survey following the set of questions developed by Kirkpatrick (1996) for the four levels of analysis. The scale of study followed is a five-point Likert scale. Likert scale is a valid scale used in many organizational and behavioral studies (Kirkpatrick, 1994) where 1 demonstrated strongly disagree and 5 stands for strongly agree. The organizational impact of the training was determined using the turnover of the employees for the given periods (Kirckpatrick, 1996). The measure for turnover was kept for those who participated in e-learning and also for the members of the control group. This data is evaluated for the control group and the experiment group.

Participants

The participants for the research are employees of one organization, preferably from the same division. These randomly selected employees are divided into two equal groups. One group is made to go through an experimental electronic training process, while the other group is set aside as a control group. The participants will be tested before the beginning of the training. During the training group, A will be surveyed on their reaction and learning outcome. Once the training is completed, both groups A and B are asked to fill another survey that measures level 3 and level 4 of the evaluation model. Then a comparison is done to establish the difference in the individual and organizational performance outcomes for the experiment and control groups.

Variables

The main aim of the research is to estimate the organizational performance and individual productivity after the training and to ascertain how much has been due to the training only. Therefore training is expected to increase the productivity of the individuals and therefore of the organization. Thus, the variable of measurement would be knowledge, experience, and familiarity in the first two levels. This will show what behavioral change has come about due to the training. Once this is ascertained, the managers and the candidates will be would be questioned on their ability to do the work with greater or lesser ease after the training process. This will again try to ascertain three variables familiarity, knowledge, attitude, skill, and experience of the candidates from self-report as well as through the assessment of their managers. Here the managers are questioned regarding both the groups and the results are compared to ascertain which showed greater effectiveness and increased performance. Therefore the primary focus variables that will be dealt with in the research are – motivation, knowledge, familiarity, experience, and behavior (Kirkpatrick, 1994). Motivation implies the reason why people are inclined to do a particular activity – be it cost reduction or leadership activities. This is an important variable as this allows the researcher to ascertain if the candidate is doing it of his own volition or is being forced to do it. This also demonstrates the interest level in the activity of the candidate. Here another aspect that must be added to the variable of motivation is that if the candidates in group A were leaving the course and if they were doing so due to lack of motivation (Welber, 2002).

The second variable that is discussed is knowledge. It evaluates the skill and knowledge specific to the topic of training. This measures the “technical, formal aspects of the educational process.” (Kirkpatrick, 1994, p. 171) Here it has been observed that many students taking an e-learning course adopted and learned only those sections of the course that they required in their job, avoiding the rest (Welber, 2002). Therefore, knowledge should also ascertain the areas that are required by the students.

Experience determines the capability of the individual to put the knowledge into practice. Therefore, the skills acquired from the training must be transformed into practical work to gain experience. Without this, training effectiveness will not be transformed into organizational performance.

Familiarity with the training content and then using it at work is essential. This shows the success of the e-learning training process and therefore is considered to be a variable. As Kirkpatrick mentions: “Our model suggests that the ingredients of knowledge and experience mix together in varying amounts to produce familiarity, and only when familiarity reaches a critical level can we expect the behavior “bulb” to go on.” (Kirkpatrick, 1994, p. 172)

Behavior is measured from the candidate’s involvement in the practical work process and how they can utilize the skills derived from training in real action. In this case, the survey must be conducted for both the experiment and control group and this should measure repeatedly (Kirkpatrick, 1994). Therefore, the behavior variable will be compared for both the experiment and control group.

Hypothesis

The research aims at finding the effectiveness of e-learning training in an organizational setting and its effect on organizational performance. From the literature review, it is deducible that many researchers believe that training is inclined to increase the effectiveness of individual and therefore organizational performance (Russell, Terborg, & Powers, 1985; Delaney & Huselid, 1996; Nikandrou, Apospori, Panayotopoulou, Stavrou, & Papalexandris, 2008). In the case of the control group experiment, the behavioral and organizational results are compared between the experiment and control group to receive meaningful results. Therefore, from this assertion, we derive our first assumption.

Hypothesis 1: The level of motivation, knowledge, and motivation for the job will be greater among group A as compared to group B.

Hypothesis 2: The candidates undergoing e-learning training will have a positive outcome for at least two of the variables, indicating that there behavioral outcome for the candidates in Group A.

Hypothesis 3: The behavioral outcome of Group A will be greater and more positive than the control group results.

Hypothesis 4: Organizational performance reported for group A candidates will be greater than group B candidates.

Timeline

This section provides an expected timeline for the research. This will demonstrate the main areas that need to be done for the research in a step-by-step manner and the routine that will be followed for its completion. Table 1 demonstrates the timeline that will be followed for the research.

Conclusion

The research proposal aims to undertake a study that evaluates the e-learning process and the effect it has on organizational performance. This area of the evaluation research has been widely neglected and no consistent result has been found yet. Therefore, for e-learning evaluation, Kirkpatrick’s training evaluation method is used to ascertain the effect electronic training has on organizational performance. The main essence of the experiment is a control group experiment that allows the researcher to ascertain the level of performance that is attributable solely to the training process. Data for the research is collected from the employees of an organization. The selection of participants will depend on the design of the experiment. The participants for the research are employees of one organization, preferably from the same division. These randomly selected employees are divided into two equal groups. One group is made to go through an experimental electronic training process, while the other group is set aside as a control group. This control group and the experiment group will have similar characteristics and will be evaluated and compared for the individual and organizational performance through self-reported and superior reported survey results. The variables that are used for the research are motivation, knowledge, familiarity, experience, and behavior. These variables will demonstrate how training has affected these crucial variables and will demonstrate the effect it will have on organizational performance. This research will allow us to identify the effect e-learning has on organizational performance. In addition, it will help to see if there are any implementation issues with the process. As many e-learning programs are wrought with the problem of high learner attrition, an evaluation process of the training will help to identify the reason for the dropout. Further, the controlled experiment method of research will identify the increase in performance solely attributable to e-learning and not to any other cause. This will help us identify the degree of effectiveness of e-learning to organizational performance.

References

1. ASTD. (2010). Headlines.
2. ASTD. (2008). Measurement & Evaluation: Essentials for Measuring Training Success. Denver, MA: American Society for Training and Development.
3. Bersin, J. (2002). Measure the Metrics. e-learning , pp. 26-28.
4. Cronbach, L. J. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass.
5. Cunha, R., Cunha, M., Morgado, A., & Brewster, C. (2003). Market Forces, Strategic Management HRM Practices and Organizational Performance, A Model Based in a European Sample. Management Research, 1(1) , 79–91.
6. Delaney, J. T., & Huselid, M. A. (1996). The impact of human resource management practices on perceptions of organizatioanl performance. Academy of Management Journal; 39(4) , 949-969.
7. Ellet, B., & Naiman, A. (2003). Introduction: Is e-learning better than… ? In G. M. Piskurich, The AMA handbook of e-learning: effective design, implementation, and (pp. 11-36). New York: AMACOM a division of American Management Association.
8. Gibbons, A., & Fairweather, P. (2000). Computer-based instruction. In T. S, & F. J, Training & Retraining: A Handbook for Business, Industry, Government, and the Military.: (pp. 410–42). New York: Macmillan Reference.
9. Gunasekaran, A., McNeil, R. D., & Shaul, D. (2002). E-learning: research and Application. Industrial and Commercial Training, 34(2) , 44-53.
10. Halawi, L. A., Mccarthy, R. V., & Pires, S. (2009). An Evaluation of E-Learning on the Basis of Bloom’s Taxonomy: An Exploratory Study. Journal of Education for Business , 374-380.
11. Hildrum, J. M. (2009). Sharing Tacit Knowledge Online: A Case Study of e-Learning in Cisco’s Network of System Integrator Partner Firms. Industry and Innovation, 16(2) , 197–218.
12. Horton, W. (2001). Evaluating e-learning. Alexandria, VA: American Society for Training and Development.
13. Kaufman, R., & Keller, J. (1994). Levels of evaluation: beyond Kirkpatrick, , pp. Human Resource Development Quarterly, 5(4) , 371 – 80.
14. Kirkpatrick, D. (1994). Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler.
15. Kirckpatrick, D. (1996). Techniques for evalutaing training programs. In D. P. Ely, & T. Plomp, Classic writings on instructional technology, Volume 1 By (pp. 119-142). Englewood, CO: Libraries Unlimited.
16. Koprowski, G. (2000). Online learning: the competitive edge. Information Week , pp. 124 – 128.
17. Lim, D. H., Ripley, D., & O’Steen, B. (2009). E-learning methodologies in practice: similarities and differences between North American countries and New Zealand. Human Resource Development International, 12(2) , 209–224.
18. Macpherson, A., Elliot, M., Harris, I., & Homan, G. (2004). E-learning: reflections and evaluation of corporate programmes. HRDI, (7)3 , 295–313.
19. Mandinach, E. B. (2005). The Development of Effective Evaluation Methods for E-Learning: A Concept Paper and Action Plan. Teachers College Record, 107(8) , 1814–1835.
20. Mayberry, E. (2005). Kirkpatrick’s Level 3: Improving the Evaluation of E-Learning.
21. McAleavy, G. (2000). Virtual and flexible learning methods. Virtual University Journal, 3 , 11-18.
22. Nikandrou, I., Apospori, E., Panayotopoulou, L., Stavrou, E. T., & Papalexandris, N. (2008). Training and firm performance in Europe: the impact of national and organizational characteristics. The International Journal of Human Resource Management, 19(11) , 2057–2078.
23. Oiry, E. (2009). Electronic human resource management: organizational responses to role conflicts created by e-learning. International Journal of Training and Development, 13(2) , 111-123.
24. Rossett, A., & Marshall, J. (2010). E-Learning: What’s Old Is New Again.
25. Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The Impact of E-Learning in Medical Education. Academic Medicine, 81(3) , 207-212.
26. Russell, J. S., Terborg, J. R., & Powers, M. L. (1985). Organizational Performance And Organizational Level Training And Support. Personnel Psychology, 38 , 849-863.
27. Strother, J. (2002). An Assessment of the e-learning in Corporate Training Programs.
28. Tzeng, G.-H., Chiang, C.-H., & Li, C.-W. (2007). Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL. Expert Systems with Applications, 32 , 1028–1044.
29. Wang, Y.S., Wang, H.Y., & Shee, D. Y. (2007). Measuring e-learning systems success in an organizational context: Scale development and validation. Computers in Human Behavior, 23 , 1792–1808.
30. Welber, M. (2002, June). How AT&T adopted Kirckpatrick’s evaluation tools to e-learning and then applied the same rigor to selecting vendors. e-learning , pp. 24-25.
31. Yuan, Z. (2007). Problems in Researching E-learning: The Cae of Conputer-assisted language learning. In T. S. e-learning, Richard Andrews; Caroline A. Haythornthwaite (pp. 416-436). Thousand Oaks, CA: SAGE.
32. Zaharias, P., & Poylymenakou, A. (2009). Developing a Usability Evaluation Method for e-Learning Applications: Beyond Functional Usability. International Journal Of Human–Computer Interaction, 25(1) , 75–98.