Thoughts on Time/Timing. . .

The readings from this week(10/31) focus on the impact of time/timing on information use. Solomon provided two articles that discussed some issues related to how users change over time (particularly the evolving behavior of the children searching the OPAC) and what impact time has on the information seeking process. The third article by Taylor lays out a framework for defining information use by professionals and does include time/change as a component of that definition. I was left wondering how successful Taylor was in pursuing this research over the course of the last decade and whether or not anyone found his definitions relevant. In retrospect, I wondered how some of the seemingly artificial distinctions (focus on 'active' information seeking, structured definition of goals) would impact the relevance of the research. Taylor seemed to be building on Dervin/Nilan's work though and mentioned some important concepts which have been reflected in more recent research, notably the idea of context and the concept that 'problems change over time'.

In relation to the time/timing articles Solomon asks a few questions. 'Why gather data over time, what does it give us?', 'How is time incorporated into methods, frameworks, theories?', and 'where would a time based perspective lead us?'. Clearly longitudinal studies appear to give us some valuable data about how users 'change' over time. Solomon was able to observe the learning of the elementary school children' use of the OPAC by introducing a time element to the study where a single point study might have missed important interactions/learning actions. Taylor mentions an important point though in discussing how problems change over time. Given the difficulty that a researcher would have in defining a single point of a user/problem, I would imagine that preserving/updating that definition over a longitudinal study would prove difficult.

A number of the cognitive/affective theories include some idea of change in state of the user (iterative seeking actions, the result of 'realization') but do these count as 'time' based? For instance, does Dervin's gap succeed in describing a discrete moment as well as a long term wrangling with a large issue? Is the issue on the other side of the gap static in Dervin's model? What about Bate's berrypicking model? While it is easy to infer a time component there (berries tend to be picked individually or in small groups after all), does the model ever revisit the user state, problem definition, or think about changing solutions?

In relation to the tangible concepts of metadata/technology that have been discussed at LITA, how have the solutions that are being discussed (new types of OPACs, internet site archiving, web 2.0 technologies) addressing the impact of time on the user/question/solution? An interesting discussion in a session about how to use Worldcat as your local catalog (implications for staffing, user seeking, etc) revisited the idea that libraries have had a flawed user model to base their ILS on for some time. There was a significant amount of frustration in the room regarding possible solutions, conflicting ideas of who/what the user is/wants, and the best/most efficient/most permanent way of addressing these needs. What was interesting was a concept which was universal here but not openly discussed: the impact/conflict that the different timelines of the web and library world are having on the ILS. In the commercial/open web world, technologies & indexes come and go in six months. In the library/bureaucratic world, technologies / indexes are multi-year/decade investments. The assumption in the room was that our users were changing at the web rate of time, while libraries were (by comparison) standing still. As a result, the group discussion wound around the idea that we had to do something quick (taking little time to define best solution was, and recognizing the idea that in web 2.0 solutions move on a much faster scale). Is there an issue here with the idea of non-normal time progression? When we discuss time in these senses, the discussion does not center on measured time but rather perceived state changes/events which only occur as time moves.

Let's think Internet archive for a moment & the impact of time. The IA looks to separate content from a timeline (and marks it) for posterity. In this way, they help define state/sense in a way that perhaps other information sources cannot. In relation to the mechanisms of social software in use now, the Internet Archive, is providing discrete snapshots of user states (however well defined) that are not susceptible to the changing vagaries of the web. Does this introduce a new research methodology?

Two articles & some further tour topic thoughts

The previous entry discusses some readings on digital libraries & some potential tour topic questions. ' The varying definitions of digital libraries (collection vs service, resource access vs resource organization and storage) are underwritten by differing definitions and assumed perspectives of a typical 'user.' How can the real perspectives and context of the digital library user be studied? Are there contexts/traits/actions that define a user differently in a digital environment than traditional environments?"

In doing some quick research in IEEE, ACM DL, and Cambridge Sci Abs on digital libraries & social/user components I came across a number of articles that addressed user perspectives. An interesting book from 2003 (yet to be retrieved) may provide a good definition of the topic. So clearly this topic is not new and the questions asked are still a bit too broad. I selected several articles to read and wound up choosing two which seemed promising. The first article, by Catherine Marshall and Sara Bly "Sharing encountered information: digital libraries get a social life" turned out to be a study in how/when/for what purpose people share encountered information. The study focused in part on digital libraries but really centered on the contexts of information sharing. They concluded with several interesting observations regarding the impact that digital library design has on sharing behaviors.

The second article I selected by Nancy Kaplan, Yoram Chisik, and Debra Levy "Reading in the wild: Sociable literacy in practice" described a research project which sought to create an online reading club of middle school aged children using a system called Alph. The online system included both a digital copy of the book to be read and a suite of annotation tools that were designed to give the participants 'community building' interactivity. The study used 5 participants who agreed to use the electronic experience rather than reading the book in print. Given the small number of participants, the study was not able to generalize some findings due to the fact that the behaviors noted could be attributed to one or two individuals. In particular, 2 individuals reported having less than adequate access to a computer and another proceeded through the book so fast that there were no other individuals reading those sections when they were.

The study did find that the participants stayed focused on a traditional linear approach to reading, that they made extensive use of the annotation tools, that the annotations were on-topic with the book being read, and that overall, reactions to the social components were positive. The study also found though that there was little 'community building' in that there were few responses to annotations, that users did not spend much time searching for other's comments, but concluded that this may be due to the limited survey population.

I found this article to be an interesting study if only because it attempts to use a technology to enhance traditional library reading programs. The article concludes by asking whether "Re-envisioning reading and re-inventing the book may bring a generational divide into sharper focus"(104). This idea refers to an assertion that the nature of reading & interaction is changing in the online environments and that the change represents a significant shift.

Neither article helped me refine my tour questions much so I am going to proceed with some further investigation & readings.

Reaction to article (10/17) and a potential tour topic

In response to the assignment to select a 'user centered' article to review as a means for selecting a tour topic, I decided to canvas current literature on Digital Library evaluation. I came across a recent article by Hong Xie titled "Evaluation of digital libraries: Criteria and problems from users' perspectives." I used this article as the basis for this discussion & through reading it formed a potential tour topic question.

Hong argues that previous evaluations of digital libraries lacked user focus and that, in this rapidly developing area, an overall lack of focus on evaluation criteria exists.

Hong points to the varying definitions of digital libraries, emphasizing two perspectives (resource/service access based and resource/service management based). She also points to the ARL definition which emphasizes links between users/objects, universal access, and diverse new formats (434). Hong continues by discussing the limitations of previous user centered studies of digital libraries, stating that previous studies "were mainly usability studies" (437). She lists a number of evaluation criteria common to both digital and traditional libraries by Saracevic (collection, information, use, standards) and expands possible evaluation criteria for digital libraries to include service evaluation, impact on communities, and evolving evaluation criteria (436). Hong asserts that the study reported in this article emphasizes user perspectives by allowing users to both define the evaluation criteria for digital libraries and then apply those criteria to the evaluation of a digital library.

The study involves an extended survey (2 years) during which 2 studies were conducted. In each case, users were asked to form evaluation criteria for a digital library, then to select a DL and evaluate it using those criteria. Her findings pointed to an emphasis on usability features (discovery, output, community) and a lack of emphasis on library based (preservation, stability, etc). Interestingly, the evaluation criteria that bubbled up from the survey put service evaluation third (behind usability & collections). Another interesting finding was an emphasis on the need to have the DL assist users with Authority judgments (449). Hong points to limitations of the study, mentioning that the participants were all LIS students who had significant backgrounds of awareness of digital libraries. She goes on to recommend evaluation of digital libraries from within the social and organizational context of the user.

In thinking about this article, I was disappointed to see little discussion of the state/type of user. Although Hong recommends further user-centric research, this article winds up focusing more on perceived importance of system functionality, not on what context drives the use of these systems. In addition, the use of LIS students most likely has a significant impact on the types of recommendations generated. Further, suggesting previously defined criteria (the ARL study) to participants prior to their definition of evaluation criteria influences their feedback.

Despite these limitations, the list of criteria presented on page 440 (grouped into major categories: Usability, Collection quality, Service quality, System performance efficiency, and User opinion solicitation) provides a high level framework within which more specific questions could be asked.

Driven in part by this article I considered a potential tour topic and came up with a few questions. What does 'user centered' mean in a digital library? Are the definitions/interests of users and perspectives different in a digital environment than in traditional environments? How could you engage in user based studies of digital libraries (how do you determine context in a digital environment)? How are information seeking theories such as SenseMaking, Anomalous States of Knowledge, Berrypicking, or Question-negotiation addressed in digital library designs?

I am not sure how/if these questions work together. Perhaps questions 2 and 3 are refinements of question 1 while 'information seeking theories'(4) is altogether separate. Given the need to limit scope of the tour topic, perhaps a more focused statement of the topic is:

In information science literature, the 'user' is evaluated using a number of different tools (surveys, interviews, ethnographies). Often these studies seek to define the context of the user in relation to a specific environment/task. The varying definitions of digital libraries (collection vs. service, resource access vs. resource organization and storage) are underwritten by differing definitions and assumed perspectives of a typical 'user.' How can the real perspectives and context of the digital library user be studied? Are there contexts/traits/actions that define a user differently in a digital environment than traditional environments?

Is this a realistic topic? Does it meet the investigative/research requirements of the assignment?

Observations on Sense Making impact on system design

In reading Solomon's third act on the WPP project, I was struck with the gathering of information seeking/sense making behaviors into distinct groups. Solomon differentiates 7 'styles' of behavior (holist, manager, analyst, organizer, processor, presenter/communicator, and transmitter) based on three analyzed components of observation (cognition, affect, conation). It seems that Solomon stops short of creating a comprehensive list of styles but rather groups evident behaviors/actions into styles that fit the organization that he studied. For each style, he addresses Sense Making orientation (bridging components into large picture, resolving conflict, plan vs. action, creation of argument or point of view, organizing components into categories, and selecting messages to communicate).

He refers back to these Sense Making behaviors in his conclusion in a call to bridge the divide between people and information systems, recognizing the variability, uncertainty and complexity of information use/seeking in people. He recommends systems that are 'subtle, intrinsic, and natural.'(1137) I can recognize the importance of looking at these grouped styles/sense making behaviors but perhaps fail to see concrete outcomes of such analysis (processor point of view!). The recommendations expressed here do match up with the stated goals of the user-centered library2.0 trend and perhaps go a step further down the path of recognizing not only that an organization/system should be designed with the user at the center but also that the user is a flexible/changing/non-specific thing. There are numerous examples of web 2.0 technologies / interfaces that address this variability in user approaches to inforamtion seeking, whether or use models similar to Solomon's Sense Making styles to create their concrete examples of data browsing, user feedback, user-created categories/information is unknown.

The ability of users to interact on the social web in these different roles perhaps could be a developing trend. This dynamic is found already in sites like Wikipedia where users get to take on 'expert' or 'organizer' or 'editor' roles. In fact, each of the styles described by Solomon could be observed there. Do those styles work in more 'authority' based sites like store fronts or research databases? How would you encourage someone with a 'manager' or 'holistic' style to make a contribution in these situations? Can the styles reported by Solomon be generalized to other organizations/interactions or are they specific to the WPP by nature of the interaction of the users in that environment?

Given my tendency towards the organizer perspective, I would argue that well marked data (metadata/xml) enables systems to be flexible and user centered without a high amount of design. It seems in contrast that applying a rigid or highly designed interface works in opposition to this goal by extensively defining a single type of user interaction/sense making. Likewise, attempting to use complicated systems & information retrieval algorithms to determine document relevance works in opposition to 'user-centered', multi-purpose designs. Granted, the expectation that a user could make use of more raw forms of data without significant background in information processing is unreasonable, but perhaps there is a middle ground there with interactive constructs based on metadata/interoperability that encourages new combinations of data & use.

Theory vs practice & a tour topic idea. . .

This past week I spent wed/thu/fri riding from Burlington to Kinston (part of cycle north Carolina). While the primary goal of the trip was entertainment/exercise, I also setup a secondary goal of catching up on some optional reading and filling in what I saw as some gaps with regards to my understanding of research methods. I focused on several chapters from a book on social science research methods & made it through some overview material on quantitative/qualitative research. In conjunction with this I have been reading the chapters from the handbook on evidence based librarianship & considering 'tour topics.' Having hit week 6 already it is beginning to look as if the semester will be quickly complete & I wanted to be sure that I was considering these topics early enough in the semester. .

I found the chapters on EBL very interesting in the quantifiable question approach (is it true/relevant, is it reproducible, is it applicable) & was wondering if/how this approach to what Bruce & Bice refer to as 'Critical Appraisal' could be implemented in an Information Literacy class. Their differentiation between extensive (authority, reliability) and intensive (validity, methodology) is an interesting way of grouping the evaluative questions (authorship, authority, relationship to other literature, perspective, bias) often used in info lit classes and might provide a more useful structure for thinking about evaluation in general.

I was also intrigued by the discussion of how librarians should/could integrate research review into their work & planning processes. This touches on a continuing push & pull (from my perspective) regarding the use of research/theory in LIS communities, the emphasis/lack of continuing education/certification in LIS, and the use of structured (quantitative) evaluative measures vs. more unstructured (qualitative). Perhaps each of these issues are unrelated but it seems that the opposites of each issue describe polar LIS communities (practice based, skill-centered continuing education, casual observation evaluation (direct usage statistics)) vs (theory fueled, externally conceived continuing education, more structured observation (user ‘research’, LibQual)).

In relation to this idea i have been wondering how I might be able to use the information from information literacy class syllabi in conjunction with student reviews to draw parallels between practical/theoretical content in information literacy courses and student reactions to them. The content of the syllabi could provide data regarding the relationship between theory/research & practical instruction offered in the class (in the form of class topics/readings) while student reviews could provide the user reaction to these combinations. My dataset could be the syllabi & reviews from the last 6 semesters of information literacy courses. I could quantify the subject matters addressed in each class & attempt to label each subject section as either a practical or theoretical centered topic (find a book (practical), what is information literacy (theoretical)). It might be difficult to specifically tie student comments to courses/topics but two of the questions on the evaluation form asks students to comment specifically on what they found most useful/least useful in the class. These two questions could provide some data regarding their opinion about the usefulness of specifically mentioned topics or the orientation of the course in general. I suspect that it will be possible to distinguish student reactions to course content but it could also be that attempting to segment topics into practical/theoretical introduces too many subjective decisions.

Is this a specific enough/too specific question? Is it answerable? How would the results be useful?