Usability Testing: Week of October 20, 2008



Usability Testing Report

October 29, 2008

Library Website, Search360, Library Catalog

Purpose:

The User Assessment Group (UAG) gathered user feedback to address remaining questions about the overall design, functionality, clarity of terminology and users’ expectations for the library website, including specific feedback as to user access to Search360, and initial user response to expanding searches in the library catalog.

Methodology:

We user tested seven undergraduates in the class of 2012, using the list of volunteers from the Class of 2012 Library Open House to draw upon. Further testing of other undergraduates, graduate students and faculty could be done in the future, as needed.

Introduction:

We explained to each person being tested that the Library had recently redesigned its website, and we were asking for feedback from students to assist us in further refining the site to better meet user needs. We created seven questions to address issues suggested by the Library Web Redesign Team, the WebFeat Implementation Team, and the Library Catalog Advisory Committee.

Results:

As of this date we have tested seven undergraduate students from the class of 1012. Specific questions and answers are listed in the Appendix. Here is a summary of the results for each question:

1. Quick links/ top page visibility

*Students were unclear, as were we, as to what was meant by quick links on the top page, so no clear results here for that part of the question.

*All seven students were attuned to the need to scroll down to see the entire top page of the library website, and were not bothered by that.

2. Readability of the top page

*All seven students thought that the page size, font, and overall color, contrast, clarity and resolution were fine.

3. LibGuides access

*Five of the seven students would go to “Resources by subject” to approach getting information by subject (and thereby eventually getting to LibGuides). Many had not seen LibGuides before, but thought they would be helpful.

*Two students did not discover accessing LibGuides via “Resources by subject”, but would simply use the catalog, Search360, or eJournals for doing research in a particular subject area.

4. Use of Library top webpage categories/functions

*Most frequently used: Search360, Hours, eJournals, Library catalog (each 3 mentions); Locating/Borrowing Materials, Social Sciences, RWIT (each 2 mentions); Paddock Music Library (under the Libraries), Get Research Help, Library instruction for courses (each mentioned once).

*Least likely to use: Digital Library for Alumni (4 mentions); Friends, About our collections, Visitor information, DCAL, Blackboard (each 2 mentions); Privacy policy, Media services, GreenPrint, Off-Campus Access, Library instruction for courses, Engineering (under resources by subject) (each mentioned once).

*Unfamiliar terms/functions: DCAL (7 mentions); DartDoc (3 mentions); BorrowDirect (2 mentions); Media services, Departments, Course reserves, Article Citation Finder, library instruction for courses (each mentioned once).

*Categories/functions would like to find on webpage: Map of the libraries (3 mentions); Google Scholar, Interlibrary loan (each 2 mentions); Login, Search engine, Feedback link, Contacts, Sources citations, Stacks guides (each mentioned once).

5. Banner search box

*Title searching (4), author searching (2), keyword searching, subject searching, Search360 (2), casual reading, audio-visual

6. Search360

*Four students knew about it, three accessing it through “Find It” and one through the banner quick search.

*Three students didn’t know about it, but would use it now that they know about it.

7. Expand Your Search

*Six of the seven students limited their search by date and audio recordings

*Most students guessed they could expand their search to include items not owned by Dartmouth by searching WorldCat, but none of them knew what WorldCat was.

*Four students knew to go to Search360 to expand search to include journal articles.

*Four of the students knew to go to RefWorks or “Cite this item” to import bibliographic information

Comments and Recommendations:

1. The top library web page does not have too many categories (most are useful) and students are clear about scrolling down to view the entire page. Keep as is, for the most part.

2. The top page is readable to those tested in regards to page size, font, color, contrast, clarity and resolution. Keep as is, for the most part. Possibly further test students with visual impairment.

3. Many, but not all students will find Subject Guides through “Resources by subject”, but others may need to have them pointed out by librarians and faculty. The feature could use some more marketing, because students who knew about the subject guides were enthusiastic.

4. Students were enthusiastic about Search360, and many had learned about the feature through First-Year seminars. The concept of federated searching is worthy of retention and refining.

5. Acronyms and coined terminology are not easy for students to understand, such as DCAL, DartDoc, and BorrowDirect. Consider further explaining acronyms and pointing towards interlibrary loans and document delivery in a more general way.

6. Put “Maps of the libraries” on the top page under “About the libraries”.

7. Students use the banner search box for a variety of purposes, and like having it as an option.

8. Students who know about Search360 like it, and use it both through “Find It” and the banner search box. Continue to educate users about the feature.

9. Students seem to understand the possibility of limiting catalog searches, and like the possibility of expanding their searches. Consider more general terminology such as “Find copies elsewhere”, “Find journal articles”, “Find on the web” and “Import citations”, since students don’t necessarily know what WorldCat, RefWorks, etc. are.

Appendix: Usability Test Results (Week of October 20, 2008)

1. Looking at the main page of the library site, do you use the quick links?

What are quick links? – 5

Yes, I would. I’ve looked up books on the catalog

I usually use the search bar, would use stacks guide, arts and humanities

Can you see the entire page on your computer screen?

Yes if I scroll down -- 7

2. Are the formats and styles used on this web page easy to read?

Page size Font size

Yes All 7 thought the page size and font were fine

No

Does the overall color of the website offer enough contrast, clarity or resolution?

Top banner Center of webpage Side banner Bottom banner

Yes All 7 said yes to all with some comments about going from green to grey

No

3. Let’s set up a scenario. You are taking your first class in sociology and you've been assigned a paper. You need to get started on research and don't know what library resources there are for doing research in this area. Where would you go on the web site to know what library resources there are for doing research in sociology? [If they go to the catalog or Search360, let them. Then, ask them to go to the library web site and look at the page and see if there is anywhere they could go to find library resources on doing research in sociology.]

Resources by subject – 4

Looks over page, I would probably go to search keyword in the catalog, notices resources by subject and scrolls through

I would start in the catalog for books, and then search 360 for articles

Looking under “Find it” for library catalog and eJournals

Other comments- Had library instruction on Search 360 in class

Love the reference librarians I’ve use them already

I usually do not go to resources by subject but I will now

Need more refined locations by subject

4. You are now looking at the top page of the library website [each person being tested gets a printed page].

*Please circle three categories or functions that you use frequently or are most likely to use. Search 360 -- 3

Hours -- 3

Locating/Borrowing Materials -- 2

Social Sciences (under resources by subject) -- 2

RWIT -- 2

eJournals -- 3

Library Catalog -- 3

Paddock Music Library (under the Libraries)

Get Research Help

Library Instruction for courses

*Please cross out three categories or functions that you are least likely to use.

Friends of the Dartmouth college library - 2

Digital Library for Alumni – 4

Privacy Policy

Media Services

GreenPrint Release Stations

Off-campus Access

About our Collections -- 2

Visitor Information -- 2

Library Instruction for courses

DCAL -- 2

Blackboard – 2

Engineering (under resources by subject)

*Please put an "x" beside any categories or functions or terms that are totally unfamiliar to you.

DCAL -- 7

Media Services

Borrow Direct – 2

DartDoc -- 3

Department

Course Reserve

Article citation Finder

Library Instruction for courses

*Are there any categories or functions you would expect to find on the library webpage that you do not see here?

Google Scholar

Map of the libraries – 3

Interlibrary loan

Login

Search engine

Feedback link

Contacts

Articles we don’t have and how I get them

It’s fairly thorough, I would search Google scholar

Is there someplace on the library website that shows how to cite sources?

Clearer stack guides – oversized collection

5. How would you use the Banner search box?

Title searching – the Open House was useful and enabled me to find RWIT

I’d type in keywords if I were looking for a book; or author or title – thought that author title were directions of options you could enter in to the search box

If I was looking for a book, or an author – type them in and see what comes up

Title or article or book, I don’t find this as useful as Search 360

I search for regular books for casual reading not research

Search 360, audio/visual, and catalog – It’s easier to follow the links

I wouldn’t use it for general searches, probably by subject or a specific topic or library specific inquires. Is it similar to Google?

Other comments

The most popular majors should have links at the top of the page

6. Do you know about Search 360?

YES--Would you show me how you use/search it?

YES, “Find it” search 360 also uses the limit functions

YES, to find articles. Picked examples with no info other than title

YES, I love it. Did a quick search and for more specific info would go to subjects. Had a BI by librarian in class.

YES, “Find It” search 360. Selected a few catalogs to see the article. It would be nice if it defaulted to all articles. Where do these results come from?

NO--Tester would explain what Search 360 is. Would you now use Search 360? Could you demonstrate how you would search/use it?

NO, explained, yes, I would use it now. Searched GOVT and single transferable vote. Very slow to load and find search

NO, explained

NO, explained. I definitely think I would use it now.

7. Tester: Go to the new main catalog page (still in development):

Question: Suppose you are looking for an audio recording of Erik Satie’s “Trois gymnopedies”. Search the catalog for audio recordings only of this piece, issued 1980 or later.

Searched keyword, year, and audio/music

Entered trios gym, change entire collection to audio recordings, looked at when it was issued (after 1980)

Went to Title tab; chose audio recordings from dropdown, typed trios gymonopedie into search box, clicked on “Limit Sort” (although said the 1984 is what he’s want.) Doesn’t know where Paddock is

Limit by audio, 1980 published, search title

Searching for audio, search title and 1980

Audio recordings refine search after 1980 searched by author

Searched by keyword author and title

How might you expand your search to include:

-other recordings of this piece not owned by Dartmouth?

Not sure, WorldCat? Not clear what it is

Went back to search page: go to off campus access? I don’t know? Tester explained “expand your search” and WorldCat

Looked at “Expand your search”—no idea what WorldCat is so he click there. Tester explained what WorldCat was; entered search trios and look at libraries near you

Expand search-WorldCat—didn’t know what it was

Additional searches --Using the right hand box found CRL

Had no idea

“Expand your Search” didn’t know what WorldCat is but selected correct item

-journal articles about this piece?

Search 360 -- 3

Change dropdown menu to journal articles/newspaper. Tester prompted for Search 360

Don’t know--expand search for journals?

Went to search eJournals and newspapers

Looking for Search 360 but found Academic Search Premier

-importing the bibliographic information of one of the search results to add to your own bibliography?

RefWorks

Found “Cite this Item” immediately. If I were using MLA style I’d copy and paste

Looked around page, saw “Cite this item”; here we go…I’d just copy and paste. Tester explained RefWorks.

Finally located “cite this item”

Not sure I know how to do that—Right hand pull down menu back to the catalog

Had no idea

Would use NoodleBib or Citation Machine – Tester explained Refworks

Other Comments-

Box is small. You can’t see “expand your search” when scrolling for results. There should probably by another box on the bottom.

Professors should mention RefWork on their syllabus

Useful if Search 360 was from the Banner

I brag all the time about the library to my friends going to different colleges

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download