The age of value-free inquiry for the human disciplines is over [Denzin & Lincoln 2003].
The vocabulary entry defines ‘value-free’ as ‘free from criteria imposed by subjective values or standards; purely objective’. Hence, with the above mentioned statement Denzin & Lincoln convey qualitative research, opposite to pure objectivity and monism in general.
Firstly, it has to be highlighted that, as qualitative methodology itself states, one may acquire a much deeper understanding of a phenomenon only through analyzing it in context. Let us observe in a large scale, why “The age of value-free inquiry is over” [Denzin & Lincoln 2003].
Globally, the processes occurring within the society have direct connection to other processes and phenomena – this is a natural way how a general picture is drawn. Therefore, not being an exception, the outset of extensive qualitative research application is conditioned by several factors. Firstly, it is a system of government, which the majority of countries are ideally striving for – democracy, the ‘power of people’. Democracy, unlike other forms of government, appreciates individualism, pluralism, and the right of every person is no reason to discuss for this is widely spread and known information.
Hence, with the democracy ‘settling’, the value of purely objective, sole information is being replaced by the value of various approaches to this information, producing several points of view. The latter, however, is democratically being accepted and considered for the single correct answer does not exist – the positivist postulates are being totally disproved.
Indeed, it is already well established that pluralism has an overweighing potential than monism; the diversity of approaches enables to achieve considerable progress in data analyses. However, just like the true democracy has predisposition to spread chaos in the society, it is either the qualitative research. The variety of ‘correct’ definitions concerning a particular subject under consideration may lead humanity to confusion and misinterpretation.
Due to interpretive, naturalistic approach to the world, which is used by qualitative research, “The researcher studies things in their natural setting, attempting to make sense of, or to interpret, phenomena in terms of the meanings people bring to them. The observer/researcher is inseparable from the study. He/She deploys a wide range of interpretive practices, hoping to get a better understanding of the subject matter. It is understood that each practice makes the world visible in a different way.” [Denzin & Lincoln 2003]
Undoubtedly, we may not diminish the value of qualitative research and the uprising perspectives of its application, but there is an increasing danger of oversaturation with options, ‘correct’ points of view, and additional information. The latter may become the explosive rupture of the paving stones, on which the whole science is steadily situated. The ‘general truth’ notion will no longer be topical for there have appeared a vast variety of the ‘truths’, defined by subjective factor.
Therefore, there is an urgent need to highlight the importance of balanced qualitative research application. By this we understand the appropriate adoption of its methods only to the appropriate fields of study, not making it a universal answer for all questions of research.
Hence, within global consideration, there is no possibility to overestimate the qualitative research for the accomplishments are immeasurable; however, it is important to control the balance between qualitative and quantitative methods application as far as democratic freedom in qualitative research may lead to critical consequences.
Switching from general perceptions and analyses of the qualitative research, one should take into account the complex combination of its different characteristics, both positive and negative ones. In order to hold a detailed examination of qualitative research, there is again a need in consulting the work by Denzin & Lincoln “The Landscape of Qualitative Research: Theories and Issues”. The authors state, that the essence of qualitative research is twofold:
- a commitment to some version of the naturalistic interpretive approach to its subject matter;
- an ongoing critique of the politics and methods of postpositivism.
[Denzin & Lincoln 2003]
In other words, qualitative research is investigation in which the researcher makes effort to comprehend some greater reality by analyzing it as the whole or by examining constituents of that reality within the particular context. Denzin & Lincoln have a bright metaphor concerning the holistic way of perceiving the world in qualitative research, which does not tolerate single part analysis. Hence, the researcher is compared to a ‘brocoleur’ and the research that he/she carries out a ‘bricolage’ or a ‘montage’.
“The product of the interpretive bricoleur’s labor is a complex, quiltlike bricolage, a reflexive collage or montage – a set of fluid, interconnected images and representations. This interpretive structure is like a quilt, a performance text, a sequence of representations connecting the parts to the whole” [Denzin & Lincoln 2003].
Indeed, qualitative research is conducted with respect to variety of different factors, which may influence the variable considered, different points of view, that reflect the subject’s properties and may provide the wholly new outlook on the subject considered. That is why qualitative research can be called really ‘qualitative’, which, not being unilateral in its result, supplies with profound, deep understanding of the subject under consideration.
“The solution [bricolage] which is the result of the bricoleur’s method is an [emergent] construction” (Weinstein & Weinstein, 1991 :161) that changes and takes new forms as different tools, methods, and techniques of representation and interpretation are added to the puzzle” [Denzin & Lincoln 2003].
One of the features of qualitative research is the simultaneous perception, due to the fact that the subject interpretation takes place in a “montage sequence not sequentially, or one at a time, but rather simultaneously” [Cook, 1981]. “In texts based on the metaphors of montage, quilt making and jazz improvisation, many different things are going on at the same time, – different voices, different perspectives, points of view, angles of vision. Like performance texts, works that use montage simultaneously create and enact moral meaning.” [Denzin and Lincoln, 2003]
The perfect visual understanding of the qualitative research value represents Richardson in the work of Denzin and Lincoln:
“A montage, an improvisation, a mixed-genre text “combines symmetry and substance with an infinite variety of shapes, substances, transmutations …. Crystals grow, change, alter … Crystals are prisms that reflect externalities and refract within themselves, creating different colors, patterns and arrays, casting off in different directions” [Richardson quoted in Denzin and Lincoln, 2003]
Indeed, similar to looking through a crystal appears the qualitative investigation, which uncovers the spectrum of meanings, encloses the deeper sense of a particular phenomenon, which is, undoubtedly, the greatest advantage of it.
In the work by Matusov In Search of ‘the Appropriate’ Unit of Analysis for Sociocultural Research, the author highlights: “Each study (beam) is providing ‘weak evidence’ with a lot of uncertainty of possible alternative explanations due to the incompleteness of their units of analysis. But together they may provide a rather convincing story, making alternative explanations implausible. Due to its incomplete nature, the unit of analysis cannot be fully known before the research is started”. [Matusov, 2007]
However, this ‘all-embracing’ feature of qualitative research frequently relies on a pure theoretical basis. The scope of the data gathered still has particular limits and the researcher-bricoleur is still able to miss some of the points. Especially, when the researcher is dealing with a complex phenomenon, or with their correlation, and the subject under consideration is difficult to model. The latter means that the replicability of the subjects of qualitative research is usually quite challenging.
Subjectivity and Bias
‘’Subjective forms of cognition, social conventions, politics, ideology, power, and narration all, in a complicated way, permeate scientific activity” [Hammersley, 1992]
Naturally, science is developed by the human mind, which, from one person to another, greatly varies. At this moment the researcher should uncover the personal definition of subjectivity.
Due to the fact that the epistemological basis for qualitative research is idealism, the general view outlines that “Social world differs from natural world. Intersubjective world of culture, consciousness and purposive actions in which relationships are organized through the ideas, values and interests of community members, producing human action and interaction.” [Markus, 1986]
Hence, contradicting the philosophical grounds of positivism, which claim about the existence of objective reality the researcher is able to investigate, subjectivist epistemology states, that “the knower and the known interact and shape one another; there is a multiple truth based on the diversity of opinions”. [Weber, 1946] That is why one may consider the epistemological basis of qualitative research to be a vector stemming from a researcher to the reality/world, where the researcher through prism of personal visions and preconceptions shapes the understanding of it.
“The biographically situated researcher enters the process from inside an interpretive community
- With history and research traditions;
- With conceptions of self and the other ;
- With ethics and politics of research.
The gaze is always filtered through the lenses of language, class, race, gender, ethnicity”. [Silverman, 2004 (a)]
Therefore, it is always very difficult to define the extent of influence, which the researcher makes upon the results of investigation for he/she is analyzing the data through his/her own bias. While taking to qualitative research, it is important, that the potential researcher manages some preparations concerning personal outlook.
First of all, the researcher should identify towards which points his/her personal bias may be directed. In order to make investigation as objective and bias-free as possible, the researcher should be totally aware of his/her stereotypes and prejudices; he/she should consider, how natural (race, age), social (social status), and gender (sex) factors influence these prejudices. Being aware of personal preconceptions, the researcher may attempt to eliminate them from the investigation process so that “the researcher’s bias would bias neither the design of a study, nor the data collection process”. [Denzin, 1970] Undoubtedly, researcher’s bias is “built in and unavoidable”, but there is a possibility to diminish its influence on the investigation process and avoid procedural problems. [Denzin and Lincoln, 2000]
Due to some degree of subjectivity, which is, moreover, very difficult to define, the scientists involved in qualitative research experience some obstacles in shaping the results of the investigation for it is almost impossible to check them for validity as, for example, in quantitative research. Subjectivity keeps the scientists from summing up the results of an investigation and drawing a general conclusion needed for science.
Therefore, it would be vital here to remind the properties of the researchers’ results so that they could be accepted to the scientific achievements. Lincoln and Guba “posit that trustworthiness of a research study is important to evaluating its worth.” [Lincoln and Guba, 1985] Trustworthiness involves establishing:
The complexity of carrying out qualitative research consists in following these four aspects that define the validity of the process and the outcomes of particular investigation. Constant attention to keeping these standards, naturally, requires effort and thoroughness from the side of a researcher, and, due to these reasons, is often quite time-consuming and expensive.
In order to achieve credibility, transferability, dependability, and confirmability of the research’s results, the researcher-bricoleur is introduced to a “set of criteria for judging the quality and rigour of inquiry within naturalistic inquiry”, which were developed by Lincoln and Guba. The scientists define a range of techniques that a researcher should apply in order to succeed in holding a qualitative research. The explanations to each technique mentioned and the difficulties that a researcher face are described further.
[Denzin and Lincoln, 2003]
Hence, to acquire credibility of the qualitative research, the scientist is expected to employ the following techniques:
- prolonged engagement is aimed at building trustworthy relationships and acquiring relatively accurate data in wide scopes. As it may be inferred, this technique, however, demands a considerable time length in a particular sphere, which is quite difficult for the researcher to maintain;
- persistent observation is either directed to obtain accurate and, which is quite important, relevant information through intentional, assertive research. This technique forces a scientist to adapt a large scope of information in a limited time period through quite stressful data analyses;
- triangulation has a purpose of verifying data through usage of diverse sources, which, with enabling the researcher to acquire more angles of vision, may also constitute the distracting factor;
- referential adequacy introduces the ‘living record’ through the use of brochures, photos, etc.
- peer debriefing conveys different kind of interaction with a peer, including discussion, which are majorly being applied to investigate appearing hypotheses. This technique can become strongly influenced by the researcher’s bias, however, because, while analyzing the data he/she tends to choose information corresponding to his/her own preconceptions;
- member checking implies checking the information by the people interested, which would enable to uncover the peculiarities of the researcher’s interpretation;
- negative/deviant case analysis gives a possibility to broaden the investigation results through representation of some contradicting materials, that would negate the assumed theory or hypotheses.
Thus, as one may observe, there is a great variety of techniques a researcher may use in order to acquire credibility of the results of his/her investigation. However, the appropriate application still depends solely on a researcher for he/she is the one to collect and analyze data, and it is a researcher who defines how effective one or another technique would be.
Transferability, or the ability to apply the results in other contexts, is also achieved by several methods, which are proposed to a researcher’s attention:
- thick description with the use of descriptive data accomplishes a profound interpretation of the subject in different contexts, inducing deeper understanding of the subject under consideration. However, the data introduced may be irrelevant for, again, this is a persona choice of a researcher;
- reflexive journal implies making records in a written form, like a diary, during a particular time period, which may be weekly or daily;
- purposive sampling is aimed at producing information about appearing hypotheses through providing a variety of samples in order to make the results of the investigation more broad.
Dependability, or the proof that the findings are firm, also has a couple of techniques to present, which are as follows:
- inquiry audit, managed by another person, gives an opportunity for the researcher to check the accuracy and the validity of his/her findings through summarizing preliminary investigation and checking the adequacy of data. The external audit is directed to the interaction of a researcher and the outsider, which may produce the whole new outlook and attitude to the investigation;
- adequacy of an audit trail occurs in case of letting the auditor define the adequacy of the study through notes, journal, etc.
Finally, confirmability, the naturalistic term for ‘objectivity’, also represents a few techniques to be applied in order to accomplish the desirable ‘neutrality’ of the research. They include confirmability audit, triangulation, audit trail, and reflexivity, which are all aimed at checking the trustworthiness of the information; assess the accuracy of data collection and analyses.
Lincoln and Guba offer a set of additional criteria for judging the quality of case reports, whereby “the writing about an inquiry ought to reflect the values and frameworks inherent in the conducting of the inquiry”. [Lincoln and Guba, 1985]
- Resonance criteria: a degree of fit, overlap or reinforcement between the case study report as written and the basic belief system underpinning the paradigm the researcher has chosen to follow.
- Rhetorical criteria: It should display unity:
- be well-organised, contain some central idea easily discernible to the reader;
- should display simplicity or clarity;
- and should display craftmanship (craftmanship has a series of dimensions as follows: has power and elegance; is creative; is open and problematic; shows awareness of writer’s own constructions; displays courage; shows emotional and intellectual commitment to constructions advanced; displays egalitarian stance towards others.
- Empowerment criteria: The extent to which the findings enable readers to take action;
- Applicability criteria: The extent to which readers can apply the findings to their own contexts. This elaborates upon transferability criteria.
Thus, the list of criteria mentioned above enables the researcher to check whether his/her research corresponds to set standards. The successful qualitative research accomplishes value through four aspects of credibility, transferability, dependability and confirmability, which are aimed at eliminating irrelevance and subjectivity in the investigation. These aspects are achieved through application of a series of techniques, described above. However. the majority of these techniques is quite time-consuming and expensive, requires a lot of effort and dedication.
Researcher
“The question is not whether we should take sides, since we inevitably will, but rather whose side are we on” [Becker, 1967]
In qualitative research there exists the eternal problem of the role of the researcher. Generally, they define two different positions that a researcher may occupy – a position of a scholar or the position of a partisan (supported by critical paradigm). Due to the explanation by Weber, a researcher-scholar “is a partisan for truth and rejects political partisanship in academic work”. Denzin either expresses that a scholar-researcher takes a liberal position: the position of “the ultimate arbiter of proper conduct remains the conscience of the researcher”. [Denzin and Lincoln, 2000]
However, the position of a researcher solely as a scholar avoids considering some vital factors, such as social organization, the processes in society, which, define the practice of investigation.
Unlike the researcher-scholar, the researcher-partisan is carrying out a particular investigation in order to provide some support to the political movement, either to prove or disprove some of the assumptions, taking part in political games.
The researcher-partisan is looking for a particular data to support his/her theory and frequently ignores the contradicting factors, which do not ‘suit’ the investigation. That is why a researcher-partisan holds a research, which is quite unilateral, does not consider the variety of opinions and interpretations. This is a case, when the research is held in an artificial way, when the conclusion is not inferred from the investigation process, which is the first to take place, but the factual material and the investigation is ‘matched’ to the results of the research, which are defined beforehand.
Generally, both positions of a researcher have some dangerous preconceptions and illustrate that, in carrying out social research, it is not possible for the researcher to occupy a neutral position.
“Research is contaminated to some extent by the values of the researcher”. [Silverman 2004]
Silverman “advocates a pragmatic position:
- research can reveal surprising facts;
- research has the possibility for influencing… practice”. [Silverman 2004]
The connection between experience of the researcher and the text is difficult to distinguish. It is hard for the researcher to embrace all the details and aspects of lived experience. As Lincoln and Denzin claim, “The qualitative researcher is not an objective, authoritative, politically neutral observer standing outside and above the text”. [Denzin and Lincoln, 2000] The scientists mention the dialogical unity of the researcher and the subject of investigation, where qualitative research is structured as a public, involved activity.
“Any of the disputes that exist at the level of methodological debate are simply not resolvable by further discussion, but are a matter of preference…. The personal biographical situation and local circumstances of researchers and their likely audiences are the main influences on how projects proceed and quality is judged”. [Seale, 2004] This is the way Seale explicates the position of a researcher in the investigation process. ( Indeed the theme of investigation is the result of personal interest of a researcher, which is either the representative of a particular nationality, has a particular skin color, gender, age, social status, and the variety if other factors which influence the researcher’s nature. Due to personal preferences of a researcher, particular methods and techniques are decided to be applied during the research.
Naturally, it either depends on the researcher how to carry out the investigation process. Therefore, the research results, so as the data collection, experience a great possibility of being biased by the preconceptions and stereotypes of the researcher. The latter may manipulate the data if it does not correspond to the desirable results of the investigation. However, the researcher may unintentionally miss some of the aspects which need to be considered for qualitative research anticipates analyzing large scopes of information, taking into consideration various points of view, diverse approaches. There lies the complexity of the qualitative research for the researcher: to be able to adapt a variety of relevant data, remaining unbiased throughout the whole investigation process, which either requires thoroughness and decency.
That is why qualitative research demands quite a long period of time for checking the appropriateness of the results of the investigation and assessing their validity. “The quality of the data alone is problematic” as Lauer and Asher claim, and, therefore, there is a need in ‘secondary’ person intrusion, who would bring some new outlook on the issue [Lauer and Asher from Denzin and Lincoln, 2003]. Either demands time and patience the development of trust between the researcher and the participants of the investigation. Moreover in such qualitative research method as interview, the questions always have an open-ended formulation, which is quite complicated for further data analyses.
Apart from the interview, there are some other methods of managing qualitative research, such as ethnographic studies, narrative inquiries, observations and focus groups. As far as qualitative research, unlike quantitative, deals with the scope of linguistic data, not numerical, it is natural that the research methods would bear a particular degree of subjectivity. However, qualitative research outweighs in getting ‘live’ information, which comes straightaway from the subject under investigation.
Further, some methods of qualitative research will be analyzed in accordance with their relevance to the investigation goal, the problems they induce both for the researcher and the investigation process in general.
“The interview is a particularly effective method for gathering data about individuals. The data gathered through an interview includes subjective thoughts and reasoning, emotions and past experiences”. [Filmer, 2004] With the help of interview a researcher is able to get ‘fresh’ and immediate information about the subject. However, this technique is quite time-consuming for the both sides. What is more, to make the process of interviewing more effective, the researcher is expected to model the interview, building as relevant questions as possible.
Observation is another method of carrying out a qualitative investigation. This particular method enables a researcher to view the behavioral patterns of a target group, identify the peculiarities of collaboration and communication within a particular group. However, this method sets a range of complications. The latter include the possibility of active influence of the researcher’s bias on the design of study and data collection process for, unlike in the interview, the researcher does not majorly rely on the text, but on personal interpretation of the visual material, what he or she is able to watch, not put down. Naturally, the outlook of the researcher shapes the process of data collection. Further, what should be also paid attention to is the fact, that during observation not all subjects of the investigation may be believed in an equal manner. The difficulty represents also the way of defining, which representative of the group observed is more or less credible. Moreover, it is important that the researcher would choose the study group that could represent the larger group of the society so that the results of the research could correspond to transferability standards described above.
Another point of concern in observation method lies in Heisenburg Uncertainty Principle. “Any group that is studied is altered to some degree by the very presence of the researcher. Therefore, any data collected is somewhat skewed”. [] Besides, while managing the observation, it is important to dedicate a particular amount of time for the results of the investigation held in a short period of time may lack validity.
Similar to the method of observation, the method of ethnographic studies either has some points of difficulty, which lead to disadvantages of the whole investigation. First, to manage a credible qualitative research, the researcher has to balance between the amounts of data under consideration. On the one hand, too little information may not appear to give valid results. On the other hand, however, too much information is hard to process, and the researcher may be confused according to which direction he should analyze it. Undoubtedly, the researcher’s bias is also present in the investigation process applying the method of ethnographic studies.
The narrative inquiry method is also considered to have a range of drawbacks. “Narrative inquiries do not lend themselves well to replicability and are not generalizable”. [Guba, 1990] Due to its ‘indefiniteness’, the narrative inquiry is frequently regarded as unreliable, the results of which can hardly predict the behaviors in future.
The focus groups method is usually used by the researchers to analyze the differences between the representatives of other nationalities, people who belong to diverse cultural groups. That is why the researcher (more artificially than naturally) forms some focus groups and holds a series of different discussions. This is a suitable way of determining cultural differences and exposing some new ways of solving problems, for diversity is always rich on ideas, but this method is either considerably influenced by the researcher’s bias and the researcher’s prejudices towards a particular culture at the first place.
To conclude, it would be reasonable to admit the importance of the qualitative research application. This kind of investigation encloses the whole spectrum of world interpretation, representing the results of the research in the general picture of knowledge perception. Due to considering all the phenomena of research in context, the results of the investigation are easy to understand and accept as the indivisible part of the science in general. In spite of the fact, that qualitative research has considerable drawbacks in application, which concern subjectivity-bias, irrelevance, and overall inconveniences of carrying out, qualitative research is vital to today’s science. The researcher should only balance in application of this research method and keep to all the standards described in this essay.
Reference List
Becker, H. 1967, “Whose side are we on?” Social Problems.
Creswell, J 2003, Research design: Qualitative, quantitative, and mixed method approaches. Thousand Oaks, CA: Sage Publications.
Denzin, N 1970, The Research Act. Chicago: Aldine.
Denzin, N, Lincoln, Y, (eds.) 1994, The Handbook of Qualitative Research. London: Sage.
2000, The Handbook of Qualitative Research, 2nd edition. London: Sage.
2003, Collecting and Interpreting Qualitative Materials. London: Sage.
Engeström, Y 1987, Learning by Expanding. Helsinki: Orienta-Konsultit.
2001, Expansive Learning at Work: Toward an activity-theoretical reconceptualization. Journal of Education and Work, Vol.14, N°1.
Filmer, P., Jenks, Ch., Seale, C., Hoburn, N., Walsh, D. 2004, Developments in social theory. In:Seale, C. (Ed.). Researching Society and Culture, 2nd edition, 35-46. London: Sage.
Giddens, A 1990, The consequences of modernity. Stanford, CA: Stanford University Press.
Guba, E.G. 1990, The alternative paradigm dialogue. In: Guba, E.G. (Ed.). The Paradigm Dialogue, 17-30. London: Sage.
Guba, E.G., Lincoln, Y.S. 1989, Fourth Generation Evaluation. London: Sage.
1990, ‘Can there be a Human Science?; constructivism as an alternative.’ Person Centred Review, vol.5, 2, 130-154.
Hammersley, M. 1992, What’s wrong with Ethnography: Methodological Explorations. London: Sage.
Holliday, A 2007, Doing and Writing Qualitative Research, 2nd Edition. London: Sage Publications.
Huber, J. 1995, Centennial Essay: Institutional perspectives on sociology. American Journal of Sociology, 101, 194-216.
Kaminski, M 2004, Games Prisoners Play. Princeton University Press.
Lincoln, Y.S., Guba, E.G. 1985, Naturalistic Inquiry. London: Sage.
Mahoney, J & Goertz, G. 2006 A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research, Political Analysis
Malinowski, B 1922/1961, Argonauts of the Western Pacific. New York: E. P. Dutton.
Markus, G.E., Fischer, M.M.J. 1986, Anthropology as cultural critique: An experimental moment in the human sciences. Chicago: University of Chicago Press.
Matusov, E 2007 In Search of ‘the Appropriate’ Unit of Analysis for Sociocultural Research.
Maykut, P, Morehouse, R 1994, Beginning Qualitative Research. Falmer Press.
Miles, M & Huberman, A 1994, Qualitative Data Analysis. Thousand Oaks, CA: Sage.
Hammersley, M. 1992, What’s wrong with Ethnography: Methodological Explorations. London: Sage.
Patton, M 2002, Qualitative research & evaluation methods ( 3rd ed.). Thousand Oaks, CA: Sage Publications.
Pawluch D. & Shaffir W. & Miall C. (2005). Doing Ethnography: Studying Everyday Life. Toronto, ON Canada: Canadian Scholars’ Press.
Rex, L., Steadman, S., & Graciano, M.K. (in press). Researching the complexity of classroom interaction. In: Green, J.L., Camilli, G., Elmore, P.B. (Eds.), Complementary methods for research in education (3rd edition). Cidade: AERA.
Seale, C. 2004 (a), Researching Society and Culture, 2nd edition. London: Sage.
2004 (b). Validity, reliability and the quality of research. In: Seale, C. (Ed.). Researching Society and Culture, 2nd edition,71-83. London: Sage.
Silverman, D 2004 (a), Research and social policy. In: Seale, C. (Ed.). Researching Society and Culture, 2nd edition,59-69 London: Sage.
2004 (b), Research and social theory. In: Seale, C. (Ed.). Researching Society and Culture, 2nd edition,47-58. London: Sage.
2006, Interpreting qualitative data. Third edition, p.276. London : Sage.
Weber, M 1946, “Science as a vocation” and “Politics as a vocation”. In: Gerth H., Mills C.W. (Eds.). From Max Weber. Oxford: Oxford University Press.
Wolcott, H 1999, Ethnography: A way of seeing. Walnut Creek, CA: AltaMira Press.
Ziman, J 2000, Real Science: what it is, and what it means. Cambridge, Uk: Cambridge University Press.