Collaborative Electronic Journal
on Evaluation and Research in Information Science
Department of Library Science and Instructional Technology
Southern Connecticut State University, New Haven CT

LSI 580-70
Fall 1999

CONTENTS:
Week 1 Reviews
Week 2 Reviews
Week 3 Reviews
Week 4 Reviews

Week 1 Reviews of Research Articles

• Worthy, Moorman, Turner: What Johnny like to read is hard to find in school. [by Christine Bird ]
• Radford: The Role of Nonverbal Communication in the Academic Library User's Decision to Initiate a Reference Encounter. [by Beth Crowley]
• Vaughan: Considerations in the choice of an Internet search tool. [by Eileen Dunnack]
• Williams: Library services to the blind in the United Kingdom and Ireland: a comparative study five years on. [by Eileen Hackett]
• Drabenstott, Simcox, & Fenton: End-User Understanding of Subject Headings in Library Catalogs. [by Paige Collins]
• Mckinney & Mckinney: Prayer in the lives of late adolescents. [by Mona Panaitisor]
• Hirsh: How do children find information on different types of tasks? Childrenıs use of the science library catalog. [by Carol Downs]

Worthy, J. Moorman, M. Turner, M. (1999). What Johnny likes to read is hard to find in school. Reading Research Quarterly, 34, 12-26.

Reviewed by: Christine Bird

This study examines what kinds of materials middle school students like to read for pleasure, where they get the materials, and how the items available in their school and classroom libraries compare to their preferences. The study found that the majority of students prefer to read scary books, popular magazines, cartoons and comics, and drawing books. Research showed that a majority of these materials are either not included at all in school-based libraries or are available in such low numbers that access is difficult, so students often end up purchasing items themselves rather than choosing from what is available.

To conduct their study, researchers used both open-ended (short answer or fill in the blank) and closed (check-off boxes) surveys to determine student reading material preferences and how they go about accessing them. Teacher and school librarian interviews were conducted and observations were made of the items in the libraries as a means of determining material availability in the school.

This methodology could be used in library science studies in a variety of ways. Surveys and interviews could be conducted which note patron reading interests, what sections (both fiction and nonfiction) they browse most, what kinds of materials they have trouble finding in their libraries, etc. Observational studies could be done to see how much time patrons spend in different areas of the library, whether or not materials in a display attract attention, or what age groups use the OPACs most.

I thought that the research methods seemed sound. One possible flaw concerns the observational methods used to examine the availability of materials. Obviously, if an item is popular, it may not be on the shelf when the observation is done, thus affecting the results of the study, especially if the teacher or librarian is unaware of the exact materials available or if materials are rotated at all. This may or may not be an issue if observations took place over a certain amount of time, and would be completely resolved if there were a list on file of the items made available to the students. Another problem which is actually mentioned in the article is the fact that some books cross genres and wouldn't be easily classified by students taking a survey: there are science fiction books that are funny, biographies about sports figures, and mysteries that are scary. I don't think this flaw could be eradicated, however, especially with middle school students whose categorical definitions may differ from those of the researchers.

Radford, M. L., (1998, Spring). Approach or Avoidance? The Role of Nonverbal Communication in the Academic Library User's Decision to Initiate a Reference Encounter. Library Trends, 46, (4), p699, 19p.

Reviewed by: Beth Crowley

Abstract

This study by Marie Radford examined how nonverbal behaviors of reference librarians impacted the library user's decision to approach and initiate a reference interaction. Through observation and interviews with library users in two academic libraries, Radford identified five categories of nonverbal reasons why users chose to approach one librarian over another: 1) initiation, 2) availability, 3) familiarity, 4) proximity, and 5) gender. This study showed that nonverbal behaviors do play a role in determining which librarian a user will approach. Users most frequently reported eye contact as the behavior that indicated whether or not a librarian was available. Talking on the phone was the behavior most frequently cited as negatively influencing a user's decision to approach. Physical factors such as attractiveness or the appearance of a disability also seemed to impact a user's decision.

Methodology

Radford used a naturalistic approach in which reference transactions were observed as they naturally occurred in two academic libraries over a period of thirty-seven hours. During the observation period, users always had a choice of two librarians to approach. The nonverbal behaviors of the librarian and user were recorded on a form for each interaction. All 155 users who approached thirty-four librarian volunteers during this time period were interviewed after the reference encounter. They were asked "Remember when you approached the reference desk with a question, there were two librarians on duty?" and "How did you decide which librarian you were going to ask for help?" The user's answers were written on the data collection form word for word. User's responses were analyzed for content. and like answers were grouped together into categories.

This methodology could be used to analyze user satisfaction with reference service. An observer could watch a number of reference transactions, noting both verbal and nonverbal behavior on the part of the librarian and user. The user could then be interviewed to determine how satisfied they were with the outcome of the transaction (i.e. did they get the information they needed, did they feel the librarian was interested in their question, etc.). By having the observer witness the entire transaction, they would be better able to determine if reference transactions reported as unsuccessful by the user were due to actions on the part of the librarian or due in part to the behavior of the user. Interviewing the librarians after the transaction would also help clarify this.

Another study that could use this methodology would be one in which library users were observed using the OPAC. They could be timed to see how long they spent searching the catalog. As they left the terminal they could be interviewed about their experience using the OPAC to discover how satisfied they were with the user interface and whether or not they found what they needed. It might also be interesting to observe how many users asked for help and how many just gave up.

Limitations

The author reports a number of limitations to this study. She states that researcher bias may have been a factor during the interviews and that utilizing multiple observers might help minimize this problem. In some cases there was a contradiction between what the observer saw and what the user reported. The author suggests videotaping the reference encounters to clear up any ambiguity and to better catch the often subtle and quick nonverbal signals. Videotape could also help solve the bias problem. The author indicates future studies should include a larger random sample in a variety of library settings. She suggests interviewing the librarians as well as the users and including additional variables such as physical appearance, modes of dress, stance, motion and eye contact.

Vaughan, J. (1999). Considerations in the choice of an Internet search tool. Library Hi Tech, 17, 89-106.

Reviewed by: Eileen Dunnack

Abstract

A study was conducted at the University of North Carolina to determine whether utility factors or convenience factors are more important when using a search tool. The survey found that 72.6% of graduate students in the library and information science program, as well as academic librarians/staff, would choose a search tool that returned the most comprehensive and relevant results. Utility was clearly the greatest concern. This compares to 22.6% who would use the tool that was easiest to use, and 4.7% who were undecided. While subject catalogs help produce more relevant results quicker, most information professionals would rather use search engines. They want the most information, and are willing to sift through it themselves.

Methodology

A survey was given to a potential pool of 215 subjects. The subjects were academic reference librarians/staff and currently enrolled students in the library and information science graduate program. There were 84 responses from the 215 pool. The survey consisted of 26 major questions with subparts for a total of 41 questions. The survey itself was mounted on the Web-server. Only the pool of people previously described received the URL for the location of the survey on the Web. There were two stipulations. First of all, the subject must have used a search tool on the Internet to look for information. Secondly, they must have felt they had a search tool they used most often. If they met the criteria, a hyperlink took them to the online survey. The results were then returned anonymously to the researcherıs e-mail.

One LIS/IT study that could use this methodology would be to survey students in the MLS program to find out their opinions on online courses. This could be sent to students who have taken a course already. I donıt know the details of how one would be able to respond anonymously, I assume eCollege could do that.

Another study that could be done is to survey libraries in your region to find out the preferred method for limiting patron time on the Internet. You would be collecting information from libraries in your geographic region. This may be helpful for comparisons, as they would hopefully have some similarities as far as socioeconomic characteristics of the population.

Flaws and Ambiguities in the Article

I thought this article was very well written and organized. It was a good article for learning the structure of a written research report because it had categories clearly spelled out, such as Background, Research Design and Definitions.

One of the possible flaws that I could see was the length of the questionnaire. I think a 41 question survey is long, even if that includes subparts. Secondly, if I were being asked some of the questions, Iıd have to go back to my favorite site and look up the answer. Some of these I just wouldnıt know off the bat. For an example, does my search tool index include Usenet (Newsgroups)? I donıt study my search tool, I use it. Finally, I would be curious to see how many of the 84 responses were students, and how many were librarians. This was not mentioned and I think this could be important.

Williams, Sinead. Library services to the blind in the United Kingdom and Ireland: a comparative study five years on. Journal of Librarianship and Information Science, 1996, 28(3), 133-140.

Reviewed by: Eileen Hackett

Abstract

This article documents a follow-up survey to one Ms. Williams did in 1990 to find out what kind of services are available for the blind in Ireland and the United Kingdom. In the first survey the author found little in the way of services in Ireland, and as of 1995 these services had not improved. In the case of one library, services had actually deteriorated. The only area of real improvement has been in the establishment of regional talking newspapers, which have become very popular. In contrast, the United Kingdom offers its blind citizens much more organized and useful services. Its government run Talking Book Service serves almost 20% of the U.K.ıs blind population, whereas the national service in Ireland serves only 10% in that country. Ms. Williams also surveyed the administrators of several new programs for the blind in the U.K. to see how well they were meeting their goals. These programs were all doing well at the time of the survey.

Methodology

Ms. Williams used a survey to gather her information. Unfortunately, no detail was given about this survey or the way it was filled out. I gathered from the article that she probably did these surveys herself, and may have done interviews with various librarians and program directors.

Surveys can be used in many areas of LIS/IT research. A survey could be given to library patrons to see what improvements they would like to see in the services provided, in the collection, and to the library building itself. Surveys can also be used to see what changes library staff would like to see made to make their jobs more useful and satisfying.

Shortcomings

As stated previously, this article gave no detail on the contents of the survey or how it was actually carried out. I think this is important information that should be part of a good research article. It would seem that a confident researcher would be willing to have this information published along with the findings. I was intrigued that Ireland has such poor services for the blind, given its literate population and growing economy. While it may not be the proper place for opinions, I would like to know what the author thinks is the reason for the poor services there.

Drabenstott, K. M., Simcox, S., & Fenton, E.G. (1999). End-User Understanding of Subject Headings in Library Catalogs. Library Resources & Technical Services, 43, 140-160.

Reviewed by: Paige Collins

Summary of findings

This article evaluated the extent to which library users understand subject headings, based in part on the 1991 Library of Congress Subject Subdivisions Conference suggestion to standardize the order of subject subdivisions. Overall, 31% of children and 39% of adults surveyed correctly understood the meanings of the subject headings presented. No significant changes in levels of understanding resulted when comparing subject heading subdivisions in their original order to the suggested revised order. No significant changes in the levels of user understanding were found when subject heading context changed (subject headings were presented by themselves, as part of a bibliographic citation, or in an alphabetical browse list). The researchers suggest standardizing the order of subject heading subdivisions would have little effect on end-user understanding, and would simplify cataloging procedures. The researchers also suggested examining more drastic changes to increase overall end-user subject heading understanding. Tables included.

Methodology

Through self-administered questionnaires distributed to 144 children and 144 adult users of three public libraries in Michigan, participants assigned meanings to eight subject headings and rated the certainty of their definitions. The responses were analyzed against the definitions provided by two expert catalogers and placed into several different categories of correct or incorrect answers. These categories were tabulated to look for trends in levels of understanding among user groups (adults and children), in subject heading subdivision order, and in different contexts.

Two studies which could use a questionnaire methodology would be a study designed to look at library patron satisfaction with library services provided by a public library, and a study designed to gauge end-user comfort with online searching techniques.

Flaws and Ambiguities

This research study exhibited the traits of good research given by Hernon in our textbook. However, the extensive comprehensiveness in some ways gave too much information. The several categories of correct and incorrect definitions was confusing and hard to follow. Coders initially assigned individual responses into one of two main categories and then continued within each category by assigning one or more of four sub-categories. A simple categorization would have been easier to keep track of and would not have compromised the results, which were presented so clearly and simply themselves.

The method for deciding if the participant correctly understood the subject headings seemed to be based more on semantics and syntax, than actual comprehension. This was due, in part, to the need to absolutely define the parameters of correct definitions, but more care could have been taken to ascertain whether or not the participant really understood the subject heading if their definitions were in the "grey zone." This could be accomplished by asking respondents the questions in a short interview so that on-the-spot clarification could be obtained by the researchers when a definition was potentially correct. Participants in this survey wrote down their responses, and people often do not write exactly what they mean. Obtaining verbal clarification would help narrow in on a potentially correct definition.

Several coders were used to assign the definitions into categories and sub-categories, allowing for subjective decisions on the part of the coders. Since several coders were used, the potential for inconsistency increased. One coder may have assigned a definition into the "correct" category which another coder would have assigned into the "incorrect" category. This might have been avoided by having one person code all of the responses (which would take a long time and could open the potential for mistakes due to fatigue), or by having any response which was questionable be decided by a consensus among all of the coders.

Despite these ambiguities, the research was carried out in a very professional and complete manner, and the results were carefully tallied and presented.

Mckinney, John P. & Mckinney Kathleen G. (1999, April 1). Prayer in the lives of late adolescents. Journal of Adolescence, 22, 279-290.

Reviewed by: Mona Panaitisor

College students were interviewed and asked to keep 7-day diaries about prayers activities.

Analysis revealed that there is a ralationship between identity status and frequency of praying, as well as between identity status and commitment to religion. Prayer may be a revealing approach to the psychological lives of late adolescents: their temporal orientation and the social bounds of their definition of self.

What the research study brought new in the theory building on the late adolescents' lives is that the moratorium identity status correlated negatively for college students with frequency of church attendance, and with frequency in private prayer, and stated commitment to religion. The researchers used document study, interviews, and questionnaires as research methods. I found the mathematical formulas they used for the data interpretation confusing because there was no explanation of the symbols used, or of the way the formulas were working.

I think that this methodology could very well work for LIS/IT studies as: "The impacts and implications of technological changes in the library" or "The library's mission and its values".

Hirsh, Sandra G. (1997, Spring).How do children find information on different types of tasks? Childrenıs use of the science library catalog. Library Trends, 45, 725-745.

by Carol Downs

Abstract

While children are making increasing use of electronic information, Hirsh states, they are most often using search tools designed for adults. Hirsh alludes to previous research which indicated that children enjoy using online catalogs but frequently find it difficult to locate specific information. This study examines how children located different kinds of information using the Science Library Catalog, a system designed for elementary students, questioning the influence of specific factors. The studyıs findings suggest that childrenıs success in locating information on an automated catalog is influenced by the degree of prior knowledge of the topic searched, the complexity of the search task, and the search methods employed.

Methodology

Sixty-four fifth graders from two public elementary schools took part in the study. The study sample was balanced by gender, knowledge level in science and school, with one school having a library but no computers, the other, comprehensive computer facilities but no library. During two sessions, children were assigned a total of eight science and technology topics (four per session). Search topics were balanced for topic and browsing task complexity but not for topic and keyword complexity. Individual search sessions of forty-five to sixty minutes in length were conducted approximately one week apart. The same database was used at each school, and initial sessions were preceded by a short automated tutorial on searching the Science Library Catalog. Following the tutorial, children were instructed to find book records on four questions intended to simulate research assignments.

Selected search topics were drawn from the fifth-grade science curriculum and identified by teachers as being of interest to students of this age. During the initial session, children were instructed to indicate to the interviewer when they had found the book they would need to write a paper on the given topic, and told that they had the option of stopping the search if they could not locate an appropriate book. No restrictions were placed on search methods or length of time spent searching. Each search task was presented both orally and in written form. During the second session, children were instructed to locate book records for a new set of four questions. Children were also asked to explain search methods used (browsing, keyword, or a combination) and then interviewed about their previous experience with online catalogs, computers and video games. A monitoring program on the Science Library Catalog automatically collected information on user and system actions during both sessions.

This studyıs methods of task assignment and evaluation could be applied to other LIS/IT studies. One possibility would be a study of usersı preferences for electronic or print resources for information gathering, accomplished through assignment of separate search topics specified for retrieval in print, electronic, and either print or electronic formats. Another study might entail a comparison of childrenıs rate of success in retrieving information using different online search engines specifically designed for children.

Comments

Research methods employed in this study appeared to be organized and carefully designed, and Hirshıs article included extensive information on the structure of the Science Library Catalog, relevant research, the methodology of the study, results, conclusions, and discussion of the original research questions.

Regarding findings on the influence of search task complexity on retrieval results, Hirsh states that some of the vocabulary used in the Science Library Catalog database (modified from Dewey Decimal Classification terms) may have exceeded the reading levels of children in the study and, further, that some of these children were not reading at grade level. Hirsh acknowledges the added challenge represented by these factors.

The study also addressed the question of the influence of prior knowledge on information retrieval, concluding that domain knowledge affected success in searching for all task types. However, Hirsh states that since grades were used as a determiner of childrenıs science knowledge, variation in search behavior and success may have been affected by motivation and attitudes about science, which were not considered in the study. Aside from Hirshıs observation, I questioned whether, in selecting search topics from the fifth grade science curriculum, it could be accurately determined which curriculum topics had already been covered by children taking part in the study.

Correlating study vocabulary to grade level, selecting participants reading at grade level, further assessment of prior science knowledge and measuring attitude and motivation, perhaps by means of interviews or inventories administered during the participant selection process, would help to address these concerns.