Back to StaffWeb Index | Contents | Previous section | Next section
Library Gateway Focus Groups Report
January 1999
Prepared by Karen Calhoun and Zsuzsa Koltay
Assisted by Diane Cellentani of Marketing Backup
Research supported by a 1998 CUL Internal Grant
Context of Study
CUL
On January 5, 1998 CUL introduced the Library Gateway (Figure 1). This high-visibility system was built based on librarians’ understanding of user needs and behavior. The research reported here, which was funded by a CUL internal grant, provides a user-centered report card for the Gateway and a roadmap for future development. We, the principal investigators, played instrumental roles in the development of the Gateway. Familiar with the Gateway’s features and limitations, we knew what needed to be examined from a library user’s point of view, but we lacked the expertise to conduct a methodologically sound, impartial user study. To assist us in planning, designing, and carrying out the study, we engaged Marketing Backup, a research consulting firm specializing in library-related projects and user studies.
Other Institutions
The ubiquity of the Internet and widespread access to personal computers have dramatically altered library users’ options and preferences for obtaining information. All major libraries have been affected by these trends, and much has appeared about them in the library and information science literature. For example, Zorana Ercegovac has written a helpful essay on the fundamental changes that affect the very definition of "library use."1 However, while a great deal has appeared in the literature about the role of the catalog in the digital age, digital library design, the user interface, and user searching behavior, little has been published outside of Cornell that is directly applicable to the present study.2
Therefore, to place our research into the context of what is happening at other institutions, we evaluated the library Web sites of the nine largest U.S. ARL libraries and three additional peer libraries (Penn, Dartmouth and Brown). As a follow up, we queried the libraries about user studies they have conducted or plan to conduct of Web-based access to their collections.
Table 1 (appended to this report) provides a summary description of each library’s Web presence.
Our evaluation of library Web sites suggests that the largest ARLs and our peer libraries have striven to:
Most sites treat "databases and reference tools" differently than they treat "Internet resources" despite the artificiality of this distinction. Most sites maintain an elaborate system of static pages to present their electronic resources separate from the library catalog. Only a few libraries use a searchable database of electronic resources, and these sites tend to be the most effective in other ways as well. Among the twelve sites we explored, we believe that Penn, Berkeley and Michigan have the best, most progressive library Web sites.
It has proved difficult to gather information about whether user studies have been conducted by these twelve libraries. We have received only four responses to date, three negative. While no formal user study has been done, staff at the University of Texas at Austin regularly monitor usage logs for individual pages as well as the IP addresses of users accessing the library’s site.
Research Objectives
We had four objectives in mind when designing and carrying out this research project:
Six focus groups were conducted to collect the information we needed.
Methodology
Focus groups are a purposive sampling method—that is, participants are deliberately selected based on a set of criteria. They are not randomly or blindly chosen, as is often the case with other user research methodologies, such as transaction log analyses, intercept interviews, questionnaires or phone surveys. A tool for collecting in-depth, thoughtful feedback, focus groups represent a qualitative rather than a quantitative methodology. Focus group interviewing is well suited to the study of needs, perceptions, satisfaction, and user expectations, as it overcomes the limitations of pre-determined, closed-ended questions. While the results are not statistically representative (that is, they may or may not reflect the attitudes of all users), they provide useful information for planning and evaluation.
Focus groups involve face-to-face interviews. A group of eight to twelve participants are chosen for similarity of background (e.g., undergraduates who are Gateway users), but possible dissimilarity of attitudes (e.g., use different CUL libraries; come from different disciplines).3 A trained facilitator, who remains neutral throughout the session, provides the framework (called an "interview guide") for the session and serves as the catalyst for the group discussions.
The consultant who owns and manages Marketing Backup (Diane Cellentani) is an experienced focus group designer and moderator. She led six focus groups for us—two each of faculty/graduate students, two each of undergraduate students, and two each of library staff members. We worked closely with the consultant on the design of the interview guide; canvassed for, screened and selected the participants; and made the local arrangements for the focus group sessions. Each session was audio- and videotaped. We and Kathy Chiang observed and took notes (as a back up to the tapes) at the undergraduate and faculty/graduate student sessions. Pam Stansbury took notes at the library staff sessions (fearing our presence might be a damper on the conversation, we avoided attending the library staff sessions).4
The undergraduate student and faculty/graduate student sessions, for which the interview guide had been designed, were highly successful. Participants were candid, the atmosphere informal and lively, and participant remarks had the air of a round-table discussion. A review of the notes and tapes of library staff sessions, plus direct feedback we heard from participants, indicated that the consultant was less successful at establishing rapport and drawing out participants in the library staff focus groups. In the two library staff sessions, participants seemed to feel inhibited.
2. However, several studies of the Mann Library Gateway have been published. The most recent is Payette, Sandra D. and Oya Y. Rieger, "Supporting scholarly inquiry: incorporating users in the design of the digital library." Journal of academic librarianship (March 1998): 121-129. Back
3. The information presented in this section is based on several useful sources: Rao, Vithala R. "Identifying unmet needs." In Analysis for Strategic Marketing (Reading, Mass., Addison-Wesley, 1998.), p. 83-86; Paley, Norton. "Getting in focus." Sales & marketing management (March 1995): 92-95; and Connaway, Lyunn Silipigni, Debra Wilcox Johnson and Susan E. Searing. "Online catalogs from the users' perspective: the use of focus group interviews." College & research libraries (September 1997): 403-420. Back
4. The following paragraphs, excerpted from the consultant's report, describe the methodology and research design in more detail:
Marketing Backup, an independent marketing research firm, conducted the focus groups on November 17 and 18, 1998. The attendance was good for each of the groups. The two undergraduate student sessions consisted of eleven and nine participants. The two faculty and graduate student sessions included eleven and nine participants. And finally, the two librarian sessions consisted of twelve and eleven participants.
This study focused on current users' perceptions of the Library Gateway to identify potential enhancements. Non-users were not part of this study because their inclusion would have required a different research design. Identifying non-users to participate would be difficult. Also, non-users may not be cognizant enough of the Library Gateway to comment on it.
In order to investigate opinions among non-users, Cornell University Library should investigate conducting an awareness study among its patrons. A mail or e-mail survey should investigate which library services (i.e., Library Gateway) are used and why or why not. This study would identify Library Gateway users and non-users who could be used to test future enhancements for the Gateway. Back
Back to StaffWeb Index | Contents | Previous section | Next section
rev. 1/27/99 peo