In response to the assignment to select a 'user centered' article to review as a means for selecting a tour topic, I decided to canvas current literature on Digital Library evaluation. I came across a recent article by Hong Xie titled "Evaluation of digital libraries: Criteria and problems from users' perspectives." I used this article as the basis for this discussion & through reading it formed a potential tour topic question.
Hong argues that previous evaluations of digital libraries lacked user focus and that, in this rapidly developing area, an overall lack of focus on evaluation criteria exists.
Hong points to the varying definitions of digital libraries, emphasizing two perspectives (resource/service access based and resource/service management based). She also points to the ARL definition which emphasizes links between users/objects, universal access, and diverse new formats (434). Hong continues by discussing the limitations of previous user centered studies of digital libraries, stating that previous studies "were mainly usability studies" (437). She lists a number of evaluation criteria common to both digital and traditional libraries by Saracevic (collection, information, use, standards) and expands possible evaluation criteria for digital libraries to include service evaluation, impact on communities, and evolving evaluation criteria (436). Hong asserts that the study reported in this article emphasizes user perspectives by allowing users to both define the evaluation criteria for digital libraries and then apply those criteria to the evaluation of a digital library.
The study involves an extended survey (2 years) during which 2 studies were conducted. In each case, users were asked to form evaluation criteria for a digital library, then to select a DL and evaluate it using those criteria. Her findings pointed to an emphasis on usability features (discovery, output, community) and a lack of emphasis on library based (preservation, stability, etc). Interestingly, the evaluation criteria that bubbled up from the survey put service evaluation third (behind usability & collections). Another interesting finding was an emphasis on the need to have the DL assist users with Authority judgments (449). Hong points to limitations of the study, mentioning that the participants were all LIS students who had significant backgrounds of awareness of digital libraries. She goes on to recommend evaluation of digital libraries from within the social and organizational context of the user.
In thinking about this article, I was disappointed to see little discussion of the state/type of user. Although Hong recommends further user-centric research, this article winds up focusing more on perceived importance of system functionality, not on what context drives the use of these systems. In addition, the use of LIS students most likely has a significant impact on the types of recommendations generated. Further, suggesting previously defined criteria (the ARL study) to participants prior to their definition of evaluation criteria influences their feedback.
Despite these limitations, the list of criteria presented on page 440 (grouped into major categories: Usability, Collection quality, Service quality, System performance efficiency, and User opinion solicitation) provides a high level framework within which more specific questions could be asked.
Driven in part by this article I considered a potential tour topic and came up with a few questions. What does 'user centered' mean in a digital library? Are the definitions/interests of users and perspectives different in a digital environment than in traditional environments? How could you engage in user based studies of digital libraries (how do you determine context in a digital environment)? How are information seeking theories such as SenseMaking, Anomalous States of Knowledge, Berrypicking, or Question-negotiation addressed in digital library designs?
I am not sure how/if these questions work together. Perhaps questions 2 and 3 are refinements of question 1 while 'information seeking theories'(4) is altogether separate. Given the need to limit scope of the tour topic, perhaps a more focused statement of the topic is:
In information science literature, the 'user' is evaluated using a number of different tools (surveys, interviews, ethnographies). Often these studies seek to define the context of the user in relation to a specific environment/task. The varying definitions of digital libraries (collection vs. service, resource access vs. resource organization and storage) are underwritten by differing definitions and assumed perspectives of a typical 'user.' How can the real perspectives and context of the digital library user be studied? Are there contexts/traits/actions that define a user differently in a digital environment than traditional environments?
Is this a realistic topic? Does it meet the investigative/research requirements of the assignment?