LSI 580-70
Fall 1999
CONTENTS:
|
|
5. ANALYZING ACTION RESEARCH DATA
This unit is divided into two
parts. Part I is a general overview of the stages of anlaysis. Part II looks
more closely at the process of analysis.
Before we look at ways to analyze data,
we might want to ask why it is we collect data in the first place. Data are
props which aid memories in reconstructing more vividly the situation which is
to be analyzed. The analytic proces takes place during the entire research
process. The decisions made in each phase of anlaysis have consequences for what
follows in the research process. During the critical analysis we need to ask: Do
the data bring the observed event to mind? Has the data selection focused on the
central issues? Does the data presented clarify the relationships between events
and stimulate further analysis? Does the interpretation explain the data
satisfactorily?
ANALYZING ACTION RESEARCH DATA - Part I
Analysis of research data consists of four general stages:
- Processing the evidence
Editing; coding; sampling: Conceptual and
theoretical
- Mapping the data
Noting the frequency of recurrence of issues, themes,
and units
- Interpreting the evidence
Interpreting data; building a model
- Presenting the results
Reporting evidence; drawing conclusions
The most essential factor in analysis is the reflexivity of the
researcher; that is, the ability of the researcher to think and reflect
critically about what has been athered and what is still required.
Following are some general guidelines for each stage of the process.
PROCESSING THE EVIDENCE
Editing data
- Check that you have all the questionnaires, interview schedules, etc., and
that all coded values are entered for all items.
- Check that each response has been interpreted uniformly (treated with the
same criteria).
Coding
- Classify evidence and place the data into net categories so that patterns
may be coherently established.
- Set up coding frames.
Sampling
- Become immersed in the data by comparing and contrasting findings and by
ordering themes and components.
- Examine the data creatively and reflexively. Develop grounded theories.
MAPPING THE DATA
It is now important to get some grasp of the frequency of occurrence of specific
units or themes. It is also important to try to chart of map these relationships
to determine clusters of topics within themes, categories, or components. A good
part of the analysis is spent constructing tables and frequencies.
During mapping,we are not concerned with inference, but with organizing and
describing the collected data. Percentages, frequencies, and simple descriptive
tables will suffice for even the most sophisticated studies.
INTERPRETING THE EVIDENCE
Interpretation of the data occurs when we move beyond description and try to
make some statement about what various responses mean and to suggest
relationships among data. Finally, we build a model of the research data by
trying to get the larger picture in focus by assembling the various indicators
and themes into a more self-explonatory set of relationships.
Theories result from continuously looking at the collected data, posing
questions and seeing how these hang together. An incubation period* is often
needed before ideas and theories begin to surface.
PRESENTING THE RESULTS
After the analysis is complete, we must assemble the major findings and present
them for others. The presentation will depend on the audience of the study. The
presentation must adhere to the language of reporting, which includes clear
exposition and ensuring conclusions are fair and based upon careful selection of
concepts and indicators.
First, summarize the problem studied and present summary tables of the main
findings. Then interpret what those findings mean within the context of the
study. Finally, the report should describe how the actions taken have improved
or not improved the problem and pose new lines of research or new proposals for
curriculum inquiry.
SUMMARY
- After a cycle of data collection, review the general proposal and plan of
the study. If necessary, reformulate the problem definition.
- Read through all the data, records, field notes, diaries, etc. in a linear
(narrative) routine: What was the problem; What plan was developed; How was data
collected; How was the data processed and analyzed; What has been found out?
- Keep your audience in mind, reporting the study in a linear, narrative
manner. The narrative should record the paradigm for work in the field of
curriculum inquiry.
- Keep a balance between the inquiry (field research) and the reporting of the
inquiry (written and oral communication of the process).
ANALYZING ACTION RESEARCH DATA - Part II
Making sense of the data
Sense-making is one of our most important abilities. It helps people see meaning
in the world around them and it helps us see our environment as a coherent and
predictable network in interrelationships.
In the analytic (or sense-making) process, observations are selected, put
into relations with each other, and interpreted. A purpose of analysis is to
find explanations which Tfit our understanding and, therefore, seem emotionally
plausible. The analytic process consists of:
- observing events
- selecting observations within the event on which to focus
- interpreting (drawing conclusion about) the perceived situation
- critically examining the constructed theory (conclusion)
The essential elements of the analytic process are:
- Reading data involves closely scrutinizing data in order to
recall the events and experiences that they represent: What was done? What was
said? What really happened?
- Selecting data involves separating important factors from
unimportant factors; grouping similar factors; sorting and, where possible,
simplifying complex details
- Presenting data involves reducing selected data to a form that
is easy to take in at a glance, such as a written outline or a diagram
- Interpreting data and drawing conclusions involves explaining
relationships and constructing a practical theory (or model) to fit the
situation being studied and relating to the research focus
Critical examination of data includes:
- questioning during your observation
- observing what seems to contradict your interpretation
- taking a definite action to test your interpretation
A basic tool kit
of elementary methods
for the constructive and critical stages of data analysis
We will look at two methods of data analysis: constructive methods and critical
methods. Constructive methods of data analysis include:
- Making data summaries
- Developing categories and coding data
- Writing theoretical notes
- Quantification
- Shaping metaphors
Critical methods of data analysis include:
- Testing the findings
- Communicative validation
Constructive methods
Making data summaries
Review data immediately after they have been collected. Write a summary. The
purpose of a summary is to provide easy access to the data later and to get an
overview of what the data offer concerning the research question
The summary might contain answers to the following questions:
- What is the context in which the data were collected?
- Why were they collected?
- Why is this particular situation?
- Why using this method of collection?
- What are the most important facts in the data?
- Is anything surprising?
- About which research issue are the data most informative?
- Do the data give rise to any new questions, points of view, suggestions,
ideas?
- Do the data suggest what should be done next, in terms of further data
collection, analysis, or action?
Be sure to cross-reference each answer to the relevant passages in the data (use
a number counter for tape-recordings; use the number of the line in
transcriptions, etc.)
The summary should be no more than 2 pages in length.
Developing categories and coding data
We organize data into categories to gain "conceptual leverage" in presenting
observations and conclusions. The categories need to be chosen from concepts
which are relevant to the research question and which express the contents of
the data
Two well recognized methods of coding data are 1) the deductive method and 2)
the inductive method.
Deductive method:
In the deductive method, categories are chosen from the researcherUs theoretical
knowledge and the data is then searched for relevant passages. Development of
categories are independent of the data.
Inductive method:
In the inductive method, categories are chosen during and after scrutinizes the
data. Categories are TderivedU from the data from interesting, surprising or
unexpected events--in relation to your research question.
In practice
- it is most helpful to use a mixture of both methods
- it is helpful to write definitions for each category
- data should be coded as soon as possible (recall, the data may suggest next
steps to take in conducting the research)
- categories are key concepts which form the nuclei of ideas for possible
actions
- the practitioner-research has relevant background information which aids in
constructing categories; but this background may also yield "blind spots;"
therefore, a critical friend should review categories
Writing theoretical notes
Writing theoretical notes is appropriate at any stage in the research process.
In writing, note any ideas or theories that come to mind relating to research
question: what certain data mean, how facts could be explained, how an important
concept could be defined, etc.
Always date each note and label it with suitable catchword or keyword. For
each entry, make a brief note of the data or event that prompted or gave rise
to the idea.
Quantification
Some elements of quantification are of great importance in people's thinking.
Quantification can be used in analyzing action research to carry out a
preliminary survey and get some data quickly (e.g. number of student who
participate--and demographics on students), to reveal researcher bias, or o
explore the generalizability of findings through statistics.
Shaping metaphors
Metaphors transfer meaning (from one field of experience to another), generate
meaning (e.g. label directs interaction), enrich the research process looking
for metaphors widens the researcher's horizons and enables the researcher to
better understand the task at hand), provide alternative approaches to reality,
are good at communicating complex matters, and come to mind naturally during
conversation.
Critical methods of data analysis
Two activities should make up the critical methods: 1) Testing the findings and
2) Communicative validation. These activities should be conducted only after
findings are clearly formulated.
In conducting critical analysis, the researcher must be open to data which
question the theories upon which the research is based--and not just confirm
them.
Testing the findings
Write a series of sentences on cards, each expressing one important result of
the analysis.
Sort the sentences into sets according to issues to which they refer. Lay out
each set of cards. Using photocopies, cut out any data which seem to relate to
the sentence and place them beside the card.
Compare data and sentence; then expand, modify, illustrate each sentence
either by writing additional sentences and adding these to the layout of cards,
or by rewriting the original card.
This creates the "backbone" of a written report that will be rich in detail
and grounded in the data.
Communicative validation
Communicate interpretations to the participants (or to a critical friend who is
familiar with the issues) and see if they agree. The amount of agreement
indicates the validity of the results of the analysis.
Developing Action Strrategies
and
Putting them into Practice
Action strategies and practice further test theories developed by the
practitioner-researcher. Action strategies test whether the researcher's
practical theory about a situation work when put into practice. Practical action
is an integral part of the action research cycle.
Characteristics of Action Strategies
Action strategies:
... are actions planned and put into practice by the teacher-researcher in order
to improve a situation or it context
... are connected to educational aims and are considered successful if the
desired effects are achieved without unexpected negative side-effects
... are typically closely linked to theories developed from action research and
tested in practice
... can be thought of as preliminary answers to the researcher's questions; that
is, they are experimental solutions to the problem being investigated
... may aim to make profound or subtle changes.
Provisos in understanding action strategies
... Complex social situations are not changed by a single action
... Change is usually a long-term process rather than an immediate solution
... In any social situation, actions usually have unforeseen side-effects. These
must be judged in terms of the set educational goals.
Finding Action Strategies
Sources of action strategies: 1) new understandings gained from analysis of
data; 2) the process of data collection; 3) our own aims, objectives and values
as educators; and 4) external sources (colleagues, books, etc.).
Suggestions to aid in finding action strategies:
- seek several possible strategies rather than be content with just one idea
- focus on potential opportunities, rather than worry about feasibility at the
beginning
- remember to consider existing strengths
- a group is usually better than an individual at finding action strategies
- individual brainstorming can prove helpful if a group is not accessible
Criteria for Evaluating Alternative Action Strategies
- Usefulness
- How useful is this action strategy?
- Will it solve the problem? For how long?
- Might there be additional positive effects?
- Might there be any negative side-effects?
- Practicality
- How practical and feasible is this action strategy?
- What room for flexibility will there be when implementing this strategy?
- Can this be done alone or does it require the goodwill, support and
cooperation of others?
- Acceptability
- Will this action strategy be acceptable to the teacher(s), pupils and others
concerned?
Planning Concrete Steps to take with the Action Strategy
Before an action strategy can be put into practice, it is necessary to feel
comfortable with the idea and to have confidence that you can carry it out:
- Imagine the situation and play it through in your mind.
- Try out the action strategy in advance at home or at school.
- Visit a colleague's classroom where this action strategy is already in use
and talk it over with this colleague.
- Try to arrange an opportunity to become more familiar with certain action
strategies with the school's in-service training program.
Generally, it is not possible to feel completely comfortable with an action
strategy until it has been put into practice.
Checking the Results of Action Research
We want to learn as much as possible from trying an action strategy; therefore,
it is important to consider in advance:
- what data we need to collect and
- the purpose for which we are collecting that data
A time plan may help you avoid
- unrealistic expectations
- running out of time
while helping you to
- think through complex tasks
- coordinate research activities with different action strategies
When an action strategy does not bring about the expected results:
Were you really comfortable with the strategy? Did you carry out the strategy in
a restrained or different way from what had been planned?
Was too little time allowed for the action strategy to make an impact? Did
you misjudge how much preparation the pupils would need before the new approach
was implemented?
Did you fail to take alternative interpretations into account or jumped to
premature conclusions about the data?
Were important sources of data overlooked?
Was the problem you investigated the 'real' problem? Did the problem change
since planning began?
Criteria for Success
[Evaluating the results of action research]
Has the action strategy resulted in an improvement* of the situation?
Has it improved the situation in a way that it has not also caused
unintended, negative side-effects which detract from the main, positive
effects?
Is the improvement more than shot-term (vanishing after only a short
time)?
* An improvement may be defined or viewed differently by researchers,
participants, and other interested persons. An improvement may be in terms of
processes or products; may refer to emotional well-being, performance, or
insights.
Where does it end?
Action research concentrates on issues that need close scrutiny and this
concentration is for a specific period of time.
In practice, pragmatic reasons, rather than theoretical ones, tend to
determine closure on a given cycle of investigation, even if some questions are
unanswered and further investigation is needed.
Examples of pragmatic reasons for concluding a cycle of research include:
- ...the practitioner-researcher is reasonably satisfied with the outcome
- ...the practitioner-researcher has to cope with another task time and
energy intensive
- ...the practitioner-researcher needs rest from extra demands
|
