Focus on Assessment - Apr 4, 2011

By Debora Cheney

It’s really hard to design products by focus groups. A lot of times, people don’t know what they
want until you show it to them. – Steve Jobs

The UX (User Experience) movement has taken off and Steve Jobs is right in the thick of it. The
debate may be yet another one of those “which comes first” dilemmas—the chicken or the egg?
Where should we incorporate the user in the feedback loop? Some would argue, as Jobs does,
that “design by focus group” isn’t all that productive. Others argue users/consumers are already
giving us feedback by what they use (or don’t use) and buy—we just need to be tuned in to what
they are telling us. Jobs would probably argue if we do that, we will be providing our users with
ideas that they can respond to—with their behavior (visiting our website) or with their credit
cards. Clearly, libraries need to listen to our users but just how can we do that most effectively
and efficiently? As we worked to finalize the furniture decisions for the Knowledge Commons we
decided that we should receive some feedback on furnishing ideas being considered in our
collaborative spaces.

How did it work:Two different kidney shaped tables and seven chairs were delivered to the
West Pattee and installed at the approximate location they will be installed in a few short
months—our “test bed” setup. We installed a library computer on the surface of one table so
they could compare with and without a computer. The chairs and tables were numbered for
easy identification and I created a 13-question survey to solicit information on which piece(s) of
furniture they preferred, but also what features they felt were most important.

Feedback from two different groups
We scheduled a “formal” focus group with the Undergraduate (UG) Advisory Committee—a
group of students who receive free pizza and soda every now and then to serve as a sounding
board for the Libraries. We provided them with an overview of the Knowledge Commons (KC)
progress and then a walking tour of the Leisure Reading Room and the two new group study
rooms and then moved to West Pattee to our “test bed” setup. What I heard as students looked
at furniture was immediate reaction to the furniture, but more importantly to specific features
they considered absolutely necessary. I was able to ask follow up questions and encouraged
them to record not only their answers to the questions on the survey but also their free-form
responses and reactions to the furniture. In order to round out the perspective of UG Advisory
Committee, we created an opportunity for other students and staff to provide feedback as well. I
placed signs near the “test bed” set-up to solicit feedback on the survey and dropped by
regularly to observe how the furniture was being used. I asked students who were using the
tables and chairs their opinions and asked them to complete surveys. We asked all of you to
send your student workers up to evaluate the furniture. We also invited people from ITS/ DLT,
library staff and librarians to visit the furniture and complete a survey. We received feedback
from 34 students, five staff/librarians and one ITS staff. The student’s immediate reaction to the
look and feel of the furniture was positive (question one on the survey), and we also learned
some specifics:
• Access to power at the table surface for laptops, phones, etc. is very important—we
responded by modifying the table design to include outlets at the table surface and carefully
considered the design of the outlets so they would accommodate a wide range of plug designs.
This feedback was useful in other furnishings decisions and we made sure to meet this
need.
• Computers were still important on the kidney shaped tables—while there is a growing
movement toward laptops students indicated they like not to have to borrow or carry their
laptops all the time.
• Group size and table size: we learned students felt these tables would really work for a
maximum of three students, not 3-5 students and interestingly, there were lots of comments
referring to the existing large rectangular tables in the room. Many students wanted a large table
surface both for individual and for larger groups. They preferred the CPU on the table surface
rather than installed underneath to provide more room for groups clustered around the kidney
shaped table.
• The “look” of the furniture is important—comments on “wood” color; shape of table—they are
design conscious, but also practical. They asked about cost, but wanted the room to be a neat
place in look and feel.
• Privacy is important but varies for individuals—there wasn’t consensus on the height of the
privacy screen, so, we didn’t change it—this told us that some users want privacy; others like to
see and be seen.
• Chairs: comfort/cushioned; casters; arms (moveable)—Chairs with these features absolutely
won out in the votes for specific chairs and for features desired. Comfortable chairs were always
mentioned and emphasized.

What we learned about asking our students for feedback
• Don’t ask a focus group to design the room or furniture but give them specific options (a “testbed”
set up) they can respond to; they will tell you what is missing and what works.
• Ask both structured questions and open ended responses—students provided feedback about
the large rectangular tables we didn’t ask about—use an informal focus group approach to
follow up immediately with how students respond; balance results with more free-form
approaches for soliciting information.
• A fairly small number of students provided a great deal of information with a lot of agreement.
Had we asked more students it is likely the responses would have clustered around these same
response rates. We also received information we hadn’t anticipated or asked about in their
comments and in follow up questions.

One more time
We used this same approach with the furniture and technology setup to be used in the group
study rooms in the KC. Three different vendors set up furniture and technology side-by-side and
we allowed each vendor to demo their product and furniture to administrators, librarians, and
students, We used a survey designed for these furniture/technology setups to solicit feedback
but also observed questions and interactions with the vendor reps. We also brought in the UG
Advisory Committee to test the setups one evening. It was exciting to watch them evaluate the
different options; to wrestle through their differences of opinion; and to ultimately be able to see
where their preferences were for the furniture/technology setups. In the end we used this
feedback to make final decisions about what furniture and technology was selected for the
group study rooms. We’ll let all students be the judge when the KC opens whether we made the
right choices based on their colleague’s feedback. But I can say now, I agree, at least with part
of what Jobs says, to paraphrase—students may not know what they want until you show it to
them—but when you show them ideas, they responded in ways that allowed us to make clear
choices in the final stages of selecting furniture and technology that would facilitate collaborative
work in the KC.