Focus on Assessment - Mar 21, 2011

By Nonny Schlotzhauer

On March 14, I joined 29 other librarians gathered at the InterContinental Hotel in Toronto for
the 7th ARL Service Quality Evaluation Academy. Attendees hailed from a diverse array of
universities and research institutes from across North America and were joined by one librarian
from South Africa. The academy was an intensive five-day program that focused on both
qualitative and quantitative methods for collecting and analyzing library service quality data. The
participants shared assessment activities at their institutions, and we all came away from the
week with a better understanding of increasingly important role of assessment in academic
libraries. The academies goals are to enhance the pool of librarians with advanced assessment
skills by creating an infrastructure for libraries to design and develop outcomes-based library
assessment programs. It is hoped that the participants will return to their home institutions to
build the capacity for assessment through advocating its use and to establish programs and
projects to ensure that services meet, and preferably exceed, user expectations. After brief
introductions by everyone, day 1 began with an overview of basic statistics by Bruce Thompson,
distinguished professor of educational psychology and professor of library science at Texas
A&M University. He then proceeded to talk about quantitative methods, different data collection
techniques and how we might apply these to our work as librarians. Professor Thompson
continued on day 2 with more foundations of statistics, and we then proceeded to engage in
hands-on activities using SPSS to look at LibQUAL data. On day 3, we were joined by Colleen
Cook, newly appointed dean of Libraries at McGill University in Montreal. She spoke of the
importance of focus group research to gather needed information in order to address problem or
improve service. Dean Cook presented a hypothetical situation, one that in theory all libraries
face – how to plan for the future - and we discussed possible tools to apply to solving the
problem. We split into four groups to brainstorm and come up with the best approaches to
address the issue. Days 4 and 5 were devoted to time spent on learning how to use of ATLAS.ti,
a powerful tool for the qualitative analysis of content that can be used for data gathered from
interviews, focus groups or surveys. This two-day workshop was led by Nick Woolf, an
instructional designer who is as an independent educator, trainer, and consultant in qualitative
research methods. Atlas.ti, new to all but a few of us at the academy, appeared complicated at
first but as I gradually got a feel for it I realized it has a great deal of flexibility and functionality. It
could prove to be a valuable tool to help us tackle, in a systematic way, much of the
unstructured data that we have gathered over the years at the Libraries. Assessment has been
said to be equal parts art and science. After a week in Toronto with ARL colleagues, I realize
that holds true for the work we do in trying to improve our service, our collections, and our
teaching. I am looking forward to sharing the skills and knowledge I learned at the academy with
everyone at the Libraries.