Focus on Assessment - Mar 14, 2011

In 2009, Lisa German asked the Next Gen Team to evaluate whether a Table of Contents
(TOC) Service would enhance our users’ discovery experience. The TOC service provides
access to the table of contents for monographs in the CAT, which are then discoverable via
keyword searches. A subgroup of Next Gen, led by Stephen Woods, examined the options for
receiving TOCs for CAT records. Working together with Rebecca Mugridge and John Attig, they
recommended that the University Libraries subscribe to the BackStage TOC enhancement
service for 18 months to determine whether or not the service was useful. An assessment would
be made to continue/discontinue the service at that time. Because our subscription expires in
June, it is now time to assess use and to make a decision regarding continuation or
cancellation. The University Libraries does retain permanent access to the TOC obtained during
the subscription period. In order to ascertain use and utility of the service, we designed a
survey that gathered data and comments on the use and perceived value of the TOC service.
Ginger Kowalchuk sent out the survey to all full-time employees in the University Libraries in
early December 2010. There were a total of 76 respondents, including 47 who identified
themselves as being from University Park and 26 from the campuses. Below is a summary of
the findings.

Are you familiar with the TOC service? (76 total respondents)
Yes, very = 29
Moderately = 34
No, not at all = 13

How often do you use/examine the TOC in a CAT Record? (50 total respondents)
Very frequently = 15
Often (every week) =12
Occasionally (A few times per month) =9
Infrequently (Once per month or less) =7
Never = 3

If you have used TOC, do you find it useful, why or why not? (39 total respondents)
Yes=29
No=7 (Note: Several of the “no” responders report a preference for Google Books or Amazon)
Undecided=3

Is the TOC worth the cost? Why or Why Not? (37 respondents. Survey instrument stated that
cost for year one
was $12,000)
Yes=22
No=9
Undecided=6

Should the University Libraries continue to provide the TOC? Why or why not?
Yes=22 (with various comments on its usefulness)
No=9 (should use the money for other collections; use Google Books or Amazon instead)
Undecided=8

There were also questions asking for general comments regarding how patrons use this service
and for additional evaluative comments regarding TOC.

Next Steps
While the survey findings generally show support for the continuation of this service, in these
tight budgetary times the acquisition of resources such as TOC must be discussed in the
context of our overall strategic goals for collections and related services. Therefore, instead of
relying strictly on survey findings to drive future decisions, we also need input as to how this
product fits into overall collections priorities. There will be a public forum on March 29, 9:30-
11:00 a.m., in Mann Assembly Room (with speaker John Attig) to discuss this service and to
gather more information. Ultimately, the information from this survey and forum will be given to
the Libraries administration, who will make the renewal decision. The evaluation of this service
through a simple survey demonstrates how the University Libraries can quickly and easily
gather data to inform decisions. While no survey instrument is perfect, the information from this
survey and from the upcoming forum will assist the Libraries in making decisions in how to best
allocate and use our resources.