MyLibrary User Observation Session Summary

See sections below:

• Introduction
• Results
• Appendix A: Details of Participants Actions
• Appendix B: Pre-Test Questionnaire

Introduction

Background: The MyLibrary@CDL prototype was made available in July 2000. During the test phase (July-December 2000) we gathered feedback through a variety of venues, including focus groups and user observation sessions.

Goal: To get detailed interface and functionality feedback in order to release a site that is easy to understand and use.

Methodology: Seven UC Berkeley students were observed while performing a set of tasks using the MyLibrary prototype. Each one-hour session was conducted by an experimenter and one observer (with the exception of two sessions when there was an additional observer). Each task was given to the participants in writing to ensure that, for each participant, the task was described in the same way. While the participants were completing the tasks the experimenter encouraged them to think out loud (verbalize their thoughts as they completed each task). The observer recorded the participants "click trail" and any comments about the system. After the session, participants were asked to share overall impressions of the system. Prior to the test, each participant was asked to fill out a demographic questionnaire (see Appendix B: Pre-Test Questionnaire).

Results

General Comments Made by Participants

Most participants expressed an interest in using MyLibrary, primarily because they saw it as a way to access a variety of resources from one location. Although there seemed to be some confusion regarding remote access, participants saw MyLibrary as an authentication mechanism - once logged into their MyLibrary account they would be able to access all resources remotely.

An interface suggestion that came up several times was to organize resources into a graphical folders metaphor rather than in lists. Regarding the categories (i.e., database, ejournal), there was no consensus of whether participants preferred to have the categories on one page or on separate pages, as it is now. Several participants suggested that they would like to create their own categories (e.g., based on research papers). One suggestion raised by two participants was to integrate MyLibrary with a bibliographic tool such as EndNote.

Task Summary

Creating an account and later changing one of the preferences (in this case discipline choice) proved to be the easiest tasks to complete. However, several participants did not understand the term "cookie" or the implications of choosing whether or not to accept cookies. Similarly, when the participants were asked to sign out and remove the cookie, several were unclear as to what was meant by "remove cookie."

The "Add/Remove" task was the most confusing for the most participants. When adding new resources, participants thought that the "Add/Remove" page was a search page (they expected that once they had entered the resource name, they would be able to conduct a search from that page, or the URL would automatically be filled in). Only one participant saw the "Add Resources via the CDL Directory" link the first time around. In addition, once at the Directory, three participants had difficulty finding the specified resource. When deleting a resource, participants did not initially see the delete resources options because they did not scroll down; they expected to find the delete options at the top of the page.

Most participants were able to add a personal link (generally by typing in a known URL) without too much difficulty, but several suggested that a "search the Internet" button should be added to the "Add/Remove" page to facilitate searching for URLs.

Questionnaire

All the participants were UC Berkeley students (six undergraduates and one graduate). The undergraduate students appeared to have somewhat more knowledge of the library than most undergraduates; however, none of them indicated extensive experience with electronic resources and none had previously used MyLibrary.

Five of the students including the graduate student used the library six or more times a month, one used it three to five times a month and one used the library one to two times a month. Most were very comfortable with Netscape Browser (the program used during the test); only two were somewhat comfortable. Only one participant had prior experience with personalization tools such as MyYahoo!.

Summary for Each Task

Task 1: Set up a MyLibrary account.

All seven participants were able to complete this task with little difficulty, although two participants entered their name and password before they saw the "Create a MyLibrary Account" link.

Figure 1: Create an account

Once at the preferences page (see Figure 6: Preferences Page), the only category that made participants hesitate was the "Use cookies?" question. Four of the participants either skipped it or were unclear on what to choose. Most participants did not change the default settings for "Default Page" and "New Browser Window."

Task 2: Open one of the pre-selected/default electronic journals.

Six of the participants completed this task. Two participants took longer to complete it as they looked around the site for the ejournal link. Of those who did choose and follow a link, two were unclear that they had in fact left MyLibrary and were at the publisher's site.

Figure 2: Electronic Journals Page

Task 3: Add the Oxford English Dictionary.

This task was the most difficult for all of the participants. The primary difficulty was that six of the participants did not see the "Add Resources via the CDL Directory" link on the "Add/Remove" page. The overall impression was that the "Title" input box was a search input box. Two participants thought that the "Category" pull-down menu was part of the search criteria. Once at the Directory, three participants had difficulty conducting the search. Appendix A: See Task 3 for a detailed description of the paths followed by the participants for this task.

Figure 3: Add/Remove Page

Task 4: Remove the Magazine and Journal Articles database.

Most participants had problems with this task primarily because they did not scroll down on the "Add/Remove" page to find the "Delete/Edit" table. Once they found the table they had no difficulty deleting the resource; several commented that two confirmation screens were unnecessary. See Appendix A:Task 4 for a detailed description of the paths followed by the participants for this task.

Figure 4: Add/Remove Page

Task 5: Add a Personal Link that you like to use.

All participants were able to complete this task without too much difficulty. Some commented that they would have liked to have a link to an Internet search engine on this page. See Appendix A: Task 5 for a detailed description of the paths followed by the participants for this task.

Figure 5: Add Personal Link Page

Task 6: Change your discipline choice.

Five of the participants were able to complete this task with no problem. They chose the preferences link in the top left-hand corner, scrolled and selected another discipline from the pull-down menu. One participant had a little difficulty but eventually found the Preferences link on the MyLibrary Home page in the introductory text.

The participant who could not complete the task was unclear on what was meant by discipline choice; this participant went to the ejournal home page and made a selection from the "narrow topic" pull-down menu; she thought by choosing a different "narrow topic" that the default resource choices would change.

Figure 6: Preferences Page

Comment: When asked if they would like the system to have removed the resources they were given from their original discipline choice or keep them along with the new ones, most preferred the latter; two suggested giving the user a choice and one commented that it would be helpful if the resource could be separated by discipline on one page.

Task 7: Sign out and remove cookie.

Five of the participants completed this task with no problem; one of them wasn't sure what remove cookie meant, but by choosing "Sign out" she then easily found the "Remove Cookie" link. The other two went first to Preferences, chose "Cookies No" then chose "Sign out."

Figure 7: Remove Cookie Page

Appendix A: Details of Participants' Actions

Task 3: Add the Oxford English Dictionary

Paths followed by participants:

Category

• Three participants were already at the ejournal page from there they chose "add/remove"; one chose "personal links" and three chose "reference shelf" then the "add/remove" link.

Add/Remove Page

First Pass

• All participants typed in the "Oxford English Dictionary" in the Title box and expected to then find a search button. They thought that by clicking the "Add to MyLibrary" button the URL would be added; two participants thought that that by choosing something from the "Category" pull-down they could conduct a search.

• Only one participant found the "Add Resources via the CDL Directory" link, but only after much searching around.

Second Pass (after having added the OED without the URL)

• Two participants tried to "edit" it through the "Add/Remove Edit" page and were unsuccessful. (Note: there is no link to the Directory from the edit page.)
• Three participants saw the "Add Resources via the CDL Directory." One got to the Directory by clicking the "All UCB Biological and Medical Sciences Electronic Journals" link on the electronic journals home page. One went to Yahoo, did a search and cut and paste the URL.

Directory

• Once at the Directory three chose the "Search Tab" and searched "Oxford English Dictionary"; one browsed, and one chose the "Search" link in the top navigation bar and was at first "overwhelmed" by the search page options, but was able to find the Directory search box.
• For two participants their first attempt to find "Oxford English Dictionary" was unsuccessful - one chose "general public" for the "available to" option (she thought it would give her more resources) and another one limited Browse to "electronic journals."

Technical note: The test computer was not properly connected to the network so the "Add to MyLibrary@CDL" button did not appear in the Directory. When participants got to the Directory results page the experimenter told them what they should have seen and how it would have worked.

Task 4: Remove the database Magazine and Journal Articles

Paths followed by participants:

Category:

• Five participants were either already at the database page or chose it then clicked "add/remove"
• Two participants chose "add/remove" from the pages they were on (one electronic journals and one reference shelf).
  • One then chose database once he/she did not see Magazine and Journal Articles on the list of electronic journals
  • From the reference shelf the participant then chose electronic journals because of the name Magazine and Journal Articles she/he thought it was related to electronic journals. At this point the experimenter intervened and explained that Magazine and Journal Articles was a database.

Add/Remove Page

• Only two participants got to the "Add/Remove" page and scrolled down to find the "Delete/Edit" table.
• The others tried a variety of ways to remove the resource
  • One typed in "Magazine and Journal Articles" in the Title input box, scrolled down and then found the "delete" link
  • One clicked back to the database and contemplated clicking the "Magazine and Journal Articles" link, then went back to the "Add/Remove" page, scrolled down and found the the "delete" link
  • One clicked the "Magazine and Journal Articles" link and once at the cdl-hosted database home page, they looked for a "remove" button
  • One chose "Add Resources via the CDL Directory" link (the participant was going to search for the Magazine and Journal Articles, but the experimenter intervened)

Delete Resource

• Once the participants found the "Delete/Edit" table they could easily delete the resources.
• Many commented that there was no need for two delete confirmations

Task 5: Add a Personal Link that you like to use

Paths followed by participants:

Three participants chose to add a site for which they did know the URL (rather than type in one they had memorized)

• One participant chose the "Add Resources via the CDL Directory" link thinking that the site they wanted (Chemical course home page) could be searched through the Directory.
• One participant had a problem with the URL - the participant typed in the URL without the http:// preface and the URL did not work.
• One participant opened a new browser window and searched on the web using their favorite search engine.

Appendix B: Pre-Test Questionnaire

MyLibrary Prototype Pre-Test Questionnaire

Are you: (check one)

• ________________ UCB undergraduate student
• ________________ UCB graduate student
• ________________ UCB faculty
• ________________ UCB staff
• ________________ Not affiliated with UCB

Approximately how often do you use the UCB Libraries services including walk-in visits to any library, use of any library web site, telephone calls to the libraries, etc? (check one)

• ________________ Less than once per month
• ________________ 1-2 times per month
• ________________ 3-5 times per month
• ________________ 6-10 times per month
• ________________ More than 10 times per month

How comfortable are you using Netscape Internet Browser?

• ________________ Very comfortable
• ________________ Somewhat comfortable
• ________________ I've never used Netscape before

Have you used personalization tools such as MyYahoo?

• ________________ YES
• ________________ NO

If yes, which services?____________________________________________