Author: Bethany Messersmith
Information Literacy Librarian/College Liaison
Southwest Baptist University Libraries
bmessersmith @ sbuniv dot edu
Website usability studies are by no means brand new. In many respects they are a close relative to the focus group session because they utilize prompts to assess opinions, as well as the usability of a product. According to Dr. Jakob Nielsen, a leading web usability consultant, “Usability allows us to make everyday life more satisfying by empowering people to control their destiny and their technology rather than be subjugated by computers” (qtd. in Chow, Bridges, and Commander 254). While website usability testing is a practice that was adopted a little over a decade ago by the library sector, librarians have always invested time in assessing user wants and needs (Battleson, Booth, and Weintrop 190). Today public and academic librarians are devoting more attention to seeking user feedback on the ease of website tasks, as “the library as ‘place,’ traditionally defined in a physical building, has expanded into a virtual environment” (Bakoyema and Groves). In light of this, developing partnerships with web design experts is critical in the academic library setting.
There are some discrepancies in the literature regarding the number of libraries who currently utilize website usability best practices. In the Spring 2014 issue of Reference & User Services Quarterly, Chow, Bridges, and Commander reported that of the “1,266 participants [academic and public libraries in the United States] who completed a portion of the Library Website Survey (LWS) with a completion rate of 79.6 percent or 1,016 respondents…more than 70 percent of the libraries who responded…have not conducted usability testing” (263). These statistics only reflect “14.1 percent of the nation’s public and academic libraries,” so it is difficult to generalize the results (Chow, Bridges, and Commander 263). According to a survey of the Association of Research Libraries (ARL) that was conducted during 2008, of the 84 member institutions who responded, “71 (85%) have conducted usability testing” (Chen, Germain, and Yang 68). Of the literature that currently exists on this topic, however, there is no mention of libraries facilitating partnerships with computer science students to impact website renovations, a practice that Southwest Baptist University Libraries has engaged in since the spring 2014 semester.
In 2011, the University Libraries hired its first Digital Services Librarian whose primary responsibility was to manage the library’s website and all digital subscriptions. As a result, website usability testing became a regular component of the website design process. The Digital Services Librarian resigned from his position with the University Libraries in 2013, however, and the position remained unfilled until May 2015. Assuming a position elsewhere, the former Digital Services Librarian agreed to adjunct remotely for Southwest Baptist University Libraries, redesigning the website every summer. As a result of his remote contact with the campus, the Dean of University Libraries and the Information Literacy Librarian assumed the role of usability testing.
Website usability testing is more of a priority than ever before at Southwest Baptist University Libraries, as the website is becoming the first point-of-contact not only for distance learners, but for students attending each of the four campuses. As of the Fall 2014 semester, there were 1,550 undergraduates enrolled at Southwest Baptist University’s Bolivar campus. The 2013 Student Library Survey asked students from all four of Southwest Baptist University’s campuses to indicate the frequency of face-to-face versus virtual website visits they make to the library per semester. As Figure 1 illustrates, type of visit varies by campus. At the main campus, however, the frequency of virtual visits occurring at least once or twice a month is greater than the face-to-face visits that occur for that category.
Figure 1. Face-to-face versus virtual library visits by campus.
Although an analysis of website sessions – unique visits to the website, regardless of the pages viewed by a single user, contains some data gaps due to website redesigns not being recorded by Google Analytics for a few months, traffic to the website is pretty consistent. Please see Table 1 below for the number of sessions from September 2014-May 2015.
Table 1. Website session views by month.
As a result of librarian workloads, forming partnerships with faculty in other departments became critical in successfully recruiting students for website usability testing. Initially, the Information Literacy Librarian attempted to rally student participants via advertising on the library’s website, campus monitors, chapel announcements, the Facebook page, etc. Incentives for participating were also promoted, including a coupon for a free drink at the café on-campus, as well as free food directly following the session, but responses lagged. Due to the impending summer 2014 website redesign, the library faculty did not want to make modifications to the site without hearing from its primary constituents.
There is almost no discussion in the library literature on the challenges of recruiting students for website usability feedback sessions (Bakoyema & Groves; “Usability Reports”; Becker & Yannotta). Some libraries noted holding focus group sessions, while others recruited students in a public space, asking them to take a few minutes that day to offer insight on the ease of website tasks (Blakiston 37).
It was necessary to adopt a new approach in generating student participation, so a partnership with a faculty member outside of the library was formed. The Information Literacy Librarian contacted a professor in the Department of Computer and Information Sciences, seeking to recruit students pursuing Computer Information Science and Web Systems and Design Majors. The professor personally encouraged students to participate. As a result, eleven students signed-up for the spring 2014 session. The library faculty were pleased with this number, which was more than adequate according to the Nielsen Norman Group’s theory that five participants will provide the feedback needed (Blakiston 13).
Heat map tracking was set-up to record clicks, but due to server issues during the study, this was quickly dispensed with and testers focused on what students verbally conveyed regarding task completion and overall website satisfaction/dissatisfaction. The session was recorded and transcribed by the Information Literacy Librarian, who established the study as qualitative at the outset. According to Steve Krug’s book entitled, Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems, he writes – “In a quantitative test, you’re interested in proving something,” whereas in a qualitative test “the purpose isn’t to prove anything; it’s to get insights that enable you to improve what you’re building” (13-14).
The Dean of the University Libraries began the website usability session with some general questions about the website overall. These questions were concerned with first impressions of the website. The remaining questions asked students to complete tasks, which may be found in Table 2 below.
|1. When you look at our website as it is today, critique it for us. What do you like and what do you dislike? What do you think works well and what do you think doesn’t work well? How would you improve it?|
|2. Where would you go to get immediate help from a librarian?|
|3. Where would you go to find a listing of all our databases?|
|4. Where would you go to find articles needed for research?|
|5. Where would you go to search for a book you need for research?|
|6. Where would you find your personal librarian’s contact information?|
|7. Where would you find a research guide for a class?|
|8. Where would you go to find tutorials on using the library and its resources? (Ex: how to request books and articles from other libraries, using specific library databases, etc.)|
|9. Where would you go to find the libraries’ hours of operation?|
|10. Where would you go to locate a course reserve for a class?|
|11. Where would you go to request a book from another library?|
|12. Where would you go to request an article from another library?|
Table 2. Website usability task list.
While the above tasks were by no means atypical for a website usability session, the feedback reflected a knowledge of website design best practices, representative of the sample of computer science students who participated.
The University Libraries gleaned invaluable feedback as a result of the spring 2014 website usability session. Research Review Board (RRB) approval was not acquired prior to the session, as it was conducted for internal assessment purposes. The qualitative data below is a result of the spring 2015 usability session, which took place with an entire computer science class of twenty this time. After the usability session, the professor distributed a link to an RRB-approved survey developed by the Information Literacy Librarian, asking students to answer five questions related to the website. See Table 3 below for an overview of student comments.
|What do you like about the University Libraries’ current website? Please list.||· “I like how it’s trying to mimic the SBU website.”
· “I like the prominent search bar.”
· “I like the options to get in contact with somebody, they are simple, easy to use, and easy to find.”
· “I like the color scheme.”
· “The drop downs are quick.”
· “It [the website] has a helpful ribbon and there is duality of access through the ribbon at the top and links at the bottom. The databases are fairly easy to use and search for published articles, journals, etc.”
· “It looks very organized. I don’t think I would have any problem finding what I’m looking for. The ask a librarian feature is nice.”
· “The search bar is right in the middle of the page and the center of attention when you load the page.”
· “Easy Access to the database articles”
· “The simplicity of the layout/interface and most of the information and links being useful to the average student. I also like the inclusion of the library hours on the main mobile page.”
|What do you dislike about the University Libraries’ current website? Please list.||· “It doesn’t quite match up with SBU’s website. There’s definite stylistic differences.”
· “I don’t like how the footer is so big. There’s a lot to look at, and it overwhelms me. My preference is for smaller footers and main functionality somewhere else.”
· “The box that you search in…seems to be up to [sic] far…”
· “There is a random check box when I try to SearchEverything and Articles.”
· “SearchEverything should be two words and it seems like it is one word.”
· “I am currently using an older browser for google chrome and the course reserves is on a separate line than the rest of the search options…”
· “On the home page you have ask a Librarian 4 times, this might be over done but I do see why you put them in the places that they are. Also, the Ask A Librarian on the right hand side of the home page above the hyperlinks doesn’t do anything, I think it is supposed to go to a different page.”
· “I would make all of the hyperlinks and ways to get in contact with the library staff the same color. It’s kind of off putting when there are random color breaks between black and blue.”
· “I think that the adds [sic] are too big.”
|What is the most difficult information to find? Please list.||· “I would need to spend a lot more time on the home page to get more comfortable and familiar with it in order to find things easily. I try to spend as little time as possible on the home page and jump immediately to the page where I want to be. Sometimes I’ll even just do a search on SBU’s website for what I’m wanting on the library page.”
· “I had no trouble finding anything.”
· “I really only use the website for databases so I have had no issues”
|What are some recommendations you would make for improving the website? Please list.||· “Advertisements/Announcements not in such prime location; that’s not what I’m looking for when I got to your website. I want the other features of the library; Functionality in main area.”
· “The links at the bottom could have less white space to give more space to the center/middle of the page.”
· “Eliminate all the content in the footer above the bible verse.”
· “Replace the ads with an ask librarian section, and make that the only ask librarian section on the page. When you leave the page, then make the ask librarian section a fixed tab on the side of the page.”
· “Fix placeholder information, move advertisements, drop down menus are represented with all options on the bottom, so that space could be utilized. Keep the search functionality and the tabs at the top, they are very helpful”
· “Add server resources. Change the mobile site to add functionality to the menu button and get rid of the information in the purple space at the bottom by adding it to the menu button’s functionality.”
|Would you engage in a library website usability session again? Why or why not?||· “This is a much better website than when I first started SBU, Good Job on the improvements. A few tweaks here and there and it will definitely be an awesome website.”
· “Yes, So that I could help out in any way to improve the interaction design of the library website.”
· “Yes, to find what hours the library is open and to find resources, usually from the databases. I sometimes will ask a librarian for help as well if I’m struggling to find the information I’m looking for.”
Table 3. Website usability comments- spring 2015.
As a result of technical difficulties with the website during the second usability test, the University Libraries was unable to complete the website usability task list with students this spring. Despite this seemingly unfortunate setback, the library faculty received feedback through a second requirement that the professor added to the assignment. Each student in the computer science class recruited a student from their department to engage in a website usability test before the end of the semester. This provided at least twenty unique feedback opportunities. All class enrollees then wrote reports summarizing findings. The professor submitted an overview of the results to the library faculty.
It is critical to recognize website changes that are feasible but do not necessarily lead to a complete website overhaul. Some of the suggestions students made during the last study were quite easy to implement. For example, students indicated that the “Ask a Librarian” feature was highlighted too many times on the home page and should be eliminated in some cases. Other students expressed concern on the server’s tendency to crash when a group of students tried to access it simultaneously. Students reported that advertising on the home page was a distraction, and content in the footer field was excessive. The library faculty took this feedback into consideration and the Digital Services Librarian made changes accordingly. “Ask a Librarian” was removed from several sections of the home page, the University Libraries finally transitioned from WordPress to LibGuides to manage its website content, the advertising was minimized, so that social media buttons, the college verse, and library contact information were the only items left in the footer.
Figure 2. Website with New Footer.
Figure 3. Website with Old Footer.
Other feedback will take some time to implement and will require future studies. For example, one student reported that: “It [the website] doesn’t quite match up with SBU’s website. There’s definite stylistic differences.” The University Libraries receives comments from students periodically that the University’s website is hard to navigate. During the Fall 2015 semester, the University will unveil a website redesign that it has been working towards for the past several months. Since the University Libraries engages in significant website modifications during the summer months when students are not enrolled full-time, changing the look and feel of the University Libraries site to reflect the University’s new website will not occur until then. These changes will require conversations with key stakeholders to insure that the usability of the University Libraries’ website is not impeded by these changes.
It might also be useful to study the participants’ experience of usability studies. While RRB approval did not cover this aspect in the present study, Feedback from computer science students’ perspectives on participating in one-on-one website usability studies would be of interest going forward.
A streamlined website design is critical to finding information in the twenty-first century. As a result, creativity is important to providing constituents with information in a compact and accessible format. An untapped constituent group that academic libraries should consider in the website redesign process is students pursuing web design and computer science programs at their institutions. These partnerships are twofold – providing students with practical job experience and library personnel with the expert advice needed to maximize their website’s potential.
Bakoyema, Fagdeba, and Christy Groves. “Engaging the Users Through Usability Testing.” Tennessee Library Association 59.2 (2009): n. pag. Web. 19 Feb. 2015.
Battleson, Brenda, Austin Booth, and Jane Weintrop. “Usability Testing of an Academic Library Web Site: A Case Study.” Journal of Academic Librarianship 27.3 (2001): 188-198. Professional Development Collection. Web. 19 Feb. 2015.
Becker, Danielle A., and Lauren Yannotta. “Modeling a Library Website Redesign Process: Developing a User-Centered Website Through Usability Testing.” Information Technology & Libraries 32.1 (2013): 6-22. Education Source. Web. 19 Feb. 2015.
Blakiston, Rebecca. Usability Testing: A Practical Guide for Librarians. Lanham: Rowman & Littlefield, 2015. Print.
Chen, Yu-Hui, Carol Anne Germain, and Huahai Yang. “Web Usability Policies/Standards/Guidelines Do Not Influence Practices at ARL Academic Libraries.” Evidence Based Library and Information Practice 4.4 (2009): 67-70. Web. 19 Feb. 2015.
Chow, Anthony S., Michelle Bridges, and Patricia Commander. “The Website Design and Usability of US Academic and Public Libraries.” Reference & User Services Quarterly 53.3 (2014): 253-265. Education Source. Web. 19 Feb. 2015.
Krug, Steve. Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. Berkeley: New Riders, 2010. Print.
“Usability Reports.” University Libraries. University of Washington, n.d. Web. 19 Feb. 2015.