Understanding Research. A critical, comparative analysis of the methods and methodologies employed in the course case studies

Learner: Oscar Vallazza – Linköping University

Course: Understanding Research (Linköping University)

Instructor: Martin Lundberg, Emilia Fägerstam and Fredrik Sandberg

Tutor: Fredrik Sandberg

Group: Indian Ocean

Essay 2 Date: December 5, 2009 Words:  2139

Understanding Research.  A critical, comparative analysis of the methods and methodologies employed in the course case studies.

TABLE OF CONTENTS: Each heading has a hyperlink to the relevant section


SECTION ONE : Structural analysis of the four studies

1.1 Kalman

1.2 Wiers-Jenssen

1.3 Aiken et al.

1.4 Chen & Barnett

SECTION TWO : Cross-sectional comments

2.1 Generalizability

2.2 Methods and analysis

2.3 Researcher’s role

2.4 Meaning






In this essay I will present a critical analysis of the research methods and methodologies employed in the case studies examined during the first part of the course. To help the reader with the relevant epistemological and ontological placement of each article, salient academic terms and definitions are shown in bold.

In SECTION ONE, I will examine the epistemology, ontology, methodology, and methods of each research paper. Additional comments may be included at the end of each sub-section.

In SECTION TWO, I will outline issues of generalizability, methods and analysis, the researcher’s role, and meaning as they emerged from my cross-sectional reading of the research papers.


Structural analysis of the four studies

1.1      Kalman, J. (2000)


Kalman’s research is anchored in interpretivist epistemology; it recognizes that reality is not an independent observable context, but is instead the result of social interaction that cannot be researched using the standard scientific approach typical of natural sciences. Kalman’s interactive methodology allows her to move beyond the role of a detached observer in order to investigate participants’ responses and gain insight into their perspective.


This is a study of social entities (mecanografos) approached from a constructivist perspective. Kalman recognizes that the participants in her research are not only a source of information, but also contribute to shaping their own reality.


In line with her epistemological approach, Kalman does a descriptive qualitative research. Stemming from her intention to describe the experience of the scribes through their own eyes, her research has a hermeneutic slant that includes face-to-face interaction and considers events and the social world from the perspective of the people under study (Bryman, 385). True to her qualitative methodology, Kalman emphasizes contexts and process as they emerge from the thick descriptions of her interviews and field notes.


Her methods include semi-structured interviews and a form of oral history interviews that she later examines using narrative analysis. She selects participants following a purposive sampling approach (Bryman, 415) and augments the interviews with field notes to triangulate the inductive findings from the interviews. Unfortunately, she does not clearly explain haw that occurs.

1.2      Wiers-Jenssen, J. (2008)


This study is grounded in positivist epistemology, which sees theory as the precursor of hypotheses that are then tested through the collection of data and a deductive form of analysis. Positivism explores issues of causality using a purported value-free approach. The focus is therefore on finding explanations rather than exploring a deeper understanding of human behavior. The issue concerning the researcher’s detachment from the population object of the study is particularly evident in this case.


This research suggests an objectivist approach, whereby the researcher sees himself as a neutral data collector and the phenomena under scrutiny are viewed as independent objects (Bryman, 2008, p.19).


This study develops from a quantitative research strategy on study abroad students. At first I thought that it aimed at helping students to make informed choices for their future, but later realized that the author does not explicitly address the personal and social significance of his research. As a result, this study provides analytical data but does not suggest how and by whom the findings could be used. I believe that researchers may overcome this kind of shortcoming in quantitative studies by injecting some personal narrative considerations into the dry presentation of data.


The researcher carries out a secondary analysis of cross-sectional data from existing surveys. The literature review provides the background for the research, is grounded in Human Capital Theory, and informs the formulation of a set of research questions to guide the researcher. The questions constitute the basis for the research hypotheses, which are tested against the analyzed data.

This research employs an “unobtrusive method”, defined by Denzin (1970) as “any method of observation that directly removes the observer from the set of interactions or events being studied” (cited in Bryman, 309). Furthermore, I believe that in this study one can see an example of what Alan Bryman (p.307) calls ecological fallacy, i.e. “the error of assuming that inferences about individuals can be made from findings relating to aggregate data.”

1.3      Aiken, L. C., Cervero, R. M., & Johnson-Bailey, J. (2001)


Aiken et al. (2001) define the scope of their study as follows: “To explain factors that encouraged and discouraged the participation of Black women in RN completion programs” (p.306-307) aimed at contributing to “to a more complete understanding of the college experience for all adults” (p.319). They anchor their research within a Black feminist theoretical epistemological framework (Aiken et al. 306, 309).


Like in Kalman’s (2000), this study stems from a constructivist perspective that emphasizes the collaborative role of researchers and interviewees in shaping the reality and context of the relevant phenomena. Furthermore, researchers claim an empathic role and assume cultural affinity with interviewees. i.e. it has an emic slant (see “additional comments” below).


As seen above, in the researchers’ words, this is “a qualitative design” within a Black feminist theoretical framework (Aiken et al. 306, 309).


An extensive literature review informs the investigation, although it is not clear whether the relevant literature is quantitative or qualitative, or whether it instead derives from mixed approaches. The methods are clearly described on page 310, which makes it easier for the reader to understand the steps of this research, compared for example with the lack of such guidance in Kalman’s work. The researchers use semi-structured interviews to collect qualitative data using Grounded Theory as a way to guide them through a process of evaluation and refinement of the categories used in the research. Grounded Theory, as conceptualized by Glaser & Strauss (1967), is used in this study as a framework for both data collection and analysis (Aiken et al, 310). The issue of internal validity, a term borrowed from quantitative research (Bryman, pp. 377-379) was approached through peer examination.

Additional comments

In general, I believe that the findings should have been presented in a less deterministic fashion. The way they were presented raises the issue of transferability (Bryman, pp. 377-379), even in the case of a qualitative strategy.

Another issue concerning the role of the researchers relates to their “belongingness” to the cultural context of the group they are investigating. It is a contested, long-standing discussion whether a research carried out by a cultural insider would produce more accurate findings than those emerging from an outsider’s participation. This is known as the etic and emic debate. In a nut shell, emic refers to the perspective of an insider, i.e. of people belonging to a specific culture, whereas etic reflects the perspective of an outside observer who does not belong to that specific culture. The issue is very important, as it relates to how we see other cultures and is directly linked to issues of identity, representation and otherization. It is particularly relevant to ethnographic research. For more on this topic, I have included a link in the webliography.

With regard to Grounded Theory, Bryman (pp.548-549), citing other researchers, points at its objectivist nature. This affirms that even qualitative-oriented researchers find themselves observing reality from the outside and engage in a process of indexing and coding that stems from their own experience and “personal ontology”. This is similar to critical realism (Bryman, pp. 14, 590). Furthermore, Grounded Theory researchers’ consistent re-evaluation of their findings is imbued with an enactivist/ecological perspective as discussed in Fenwick (2001).[i]

1.4      Chen, T., & Barnett, G. A. (2000)

Epistemology and ontology

This is another case of positivist epistemology applied to objectivist ontology. The researchers analyze international student flows from a macro perspective and set out to test the validity of World System Theory. Their questions (p. 438-438) are formulated accordingly.


Like Wier-Jenssen, the researchers in this study do not collect new data, but instead focus on existing data to carry out a quantitative secondary analysis of official statistics provided by UNESCO and the World Bank.


The secondary data are analyzed using NOGOPY, a computer application for “communication network analysis.” The researchers admit that the data are partly biased, as they exclude certain parameters that could have altered the research outcome. This study, though nominally systems-oriented, is actually centered on and informed by one approach, namely by World System Theory (p.435), and emphasizes its relevant economic perspective.

Additional comments

I believe that large scale quantitative analysis such as this can provide useful findings to understand certain trends, although it may overlook certain factors that go undetected and that would most likely become apparent in a careful qualitative research strategy. Therefore, this study seems to raise issues of unobtrusive method and ecological fallacy as defined above.

It also seems that the researchers are influencing the outcomes by introducing biased sampling criteria on how certain countries remain excluded from the study. In qualitative analysis, data interpretation depends entirely on the researchers, who extrapolate meanings based on a set of categories that they themselves have identified. Chen & Barnett’s study is a good example of how even in quantitative research it is possible to interfere with the assumed separation between researchers and subjects.  This seems to contradict the standard notion of positivist epistemology applied to quantitative research as a guarantee for impartiality.


Cross-sectional comments

2.1 Generalizability

Broadly, the two qualitative studies by Kalman and Aiken et al. are limited to a small population sample. This confirms Bryman’s assertion (2008) that “the people who are interviewed in qualitative research are not meant to be representative of a population” (p. 391). To me, this signifies a fundamental difference between the two methodologies. However, this obvious issue of broader generalizability may not affect the solidity of qualitative research, if we consider its ontological and epistemological foundations in its own merit, which emphasizes processes and context over the static view of a phenomenon.

2.2 Methods and analysis

The four studies differ in their respective data analysis and how that is presented to the readers. For example, Kalman’s paper is not clear on how she approached such task. Her final findings and conclusions are based on direct quotes, her consideration on juxtaposed case scenarios and categories, but lack the structural clarity that typifies quantitative research.

2.3 Researcher’s role

The researchers’ role is widely different in the four studies. Beyond the expected epistemological and ontological differences that reflect on the respective research strategy, the individual researchers’ behavior varies considerably.

Wiers-Jenssen and Chen & Barnett fulfill their roles of “detached” researchers, seemingly independent from the object of their study, but in reality unable to escape issues of unobtrusive method and ecological fallacy as defined earlier. Aiken et al and Kalman, given their preference for a qualitative research strategy, are understandably more personally involved with their participants, albeit in very different ways. Kalman acts as a participating outside observer and informal interviewer from an etic perspective. Aiken et al. instead act from an emic cultural assumption within an explicit feminist orientation.

In all the above cases, however, one can easily recognize that unbiased and value-free research remains an elusive chimera. Bryman (2008) writes that “The social researcher is always providing his or her own ‘spin’ on the text that are analysed. The same is true of all social science data: the conclusions you derive from your questionnaire or ethnographic data are always going to be a reflection of your own personal interpretation” (p. 526). This applies to both qualitative and quantitative methodologies, as also emerged from the discussion within our cohort.

2.4 Meaning

One last theme that I would like to bring up in this essay is meaning. It relates directly to the differences between qualitative and quantitative strategies. Bryman (2008, p.211) writes that phenomenologically, “meaning is something that is worked at and achieved – it is not simply pre-given. […] the problem of meaning implies that interviewer and respondent may not be sharing the same meaning systems and hence imply different things in their use of words is simply sidestepped in structured interview research. The problem of meaning is resolved by ignoring it” (Italics added). This last citation raises the question whether in quantitative research the nuances of the differentiated human experiences are for the most part lost, as it has been pointed out in our forum discussions.


Although it would be easy to juxtapose qualitative to quantitative research strategy and create a dichotomous picture of opposing epistemological and ontological traditions, I believe that through the close examination of the case studies I have developed a less polarized view of current research approaches.

This essay has attempted a preliminary critical analysis of four different research papers and their underlying epistemological, ontological and methodological perspectives. My observations reflect my current appreciation of research-related issues and are by no means to be considered final. I expect my understanding and practice will evolve, as I continue my reflective learning experience in this course.


Aiken, L. C., Cervero, R. M., & Johnson-Bailey, J. (2001) Black women in nursing education completion programs: Issues affecting participation. Adult Education Quarterly, 51(4), 306-321

Bryman, A. (2008). Social Research methods (3rd ed). Oxford; Oxford university press

Chen, T., & Barnett, G. A. (2000). Research on international student flows from macro perspective: A network analysis of 1985, 1989 and 1995. Higher Education, 39(4), 435 – 453

Denzin, N. K. (1970). The Research Act in Sociology. Chicago: Aldine

Fenwick, T. (2001). “Experiential Learning: A Theoretical Critique from Five Perspectives” Information Series No 385, ERIC Clearinghouse on Adult, Career and Vocational Education now located at the Centre for Education & Training for Employment at Ohio State University, accessed on June 2, 2009 at http://www.uni-koeln.de/hf/konstrukt/didaktik/situierteslernen/fenwick1.pdf

Kalman, J. (2000). Learning to write in the street. International Journal of Qualitative Studies in Education, 13(3), 187 – 203

Wiers-Jenssen, J. (2008). Does Higher Education Attained Abroad Lead to International Jobs? Journal of Studies in International Education, 12(2), 101-130


William M.K. Trochim’s website:


Emic and etic perspectives:


On Grounded theory:



[i] The enactivist perspective insists that learning cannot be understood except in terms of co-emergence: each participant’s understandings are entwined with those of other participants, and individual knowledge co-emerges with collective knowledge. Educational theory also must examine the subtle particularities of “context” created through the learning of complex systems, embedded in their constantly shifting interactional dynamics, and the relations among these particularities. Educators need to become alert to a “complexified awareness…of how one [individual] exists simultaneously in and across these levels, and of how part and whole co-emerge and co-specify one another. (Davis and Sumara 1997, p. 120, in Fenwick, 2001)


One Response

  1. Evaluation by Fredrik Sandberg
    Hi Oscar
    It has been a pleasure having you on the course so far – your activity has been remarkable. It was fun reading your essay – besides some minor remarks it was excellent. ( for instance I wonder if Aiken et.al actually uses grounded theory per se, even if they do have a reference to Glaser? However, they do talk about constant comparative analysis. In connection to their theoretical framwork, which is highly normative, I really dont think grounded theory would work. . .) .
    Hope all is well and happy holidays.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: