Hannah Marshall

January 2015

Preliminary Findings: A Comparative Study Of User- And

Cataloger-Assigned Subject Terms

1. What is the level of correspondence between users and catalogers in the subject analysis of images?

Based on evidence that description is a strong indicator of search behavior, this will lend insight into our users’ search behaviors and the accessibility of subject metadata in our image collections

2. What is the level of correspondence in the type of subject analysis being performed by each group?

When users’ and catalogers’ terms differ, do the terms that they assign reflect similar or differing types of interpretation and analysis?

The subject analysis of images has traditionally been viewed as a graduated scale that identifies different levels of interpretation and analysis. We are interested to see the degree to which each group uses them.

3. Does coaching users on performing subject analysis change their search behaviors?

Findings can be leveraged into outreach efforts and in-house cataloging best practice

RESEARCH QUESTION(S)

December 2013 - In i t ia l proposal Literature review Methodology Schedule Value of research

June 16- June 26 t h 2014 - Inst i tute for Research Design in L ibrar ianship ( IRDL)

July – October 2014 - Study Design Selected images and exported metadata Designed surveys (Qualtrics) Drafted recruitment email and informed consent form Identified the study population, recruitment method, data analysis techniques Pre-testing survey Drafting and submitting Request for Exemption from IRB Review

October 5, 2014 - IRB Exempt ion granted October 24 t h 2014 - Data col lect ion begins December 8 t h 2014 – Data col lect ion ends (Spr ing 2015 – Data col lect ion begins again)

RESEARCH PROCESSTIMELINE

Data collection instrument: Qualtrics survey 10 images 9 subject terms per image Control Group

“Please review the image and then enter terms that you think describe the image in the spaces provided.”

Variable Group “What is the image of?” “What is the image about?” “What is the image a good

example of?’

RESEARCH PROCESSMETHODOLOGY

Survey active October 24th – December 8th

80 responses – 20% response rate Distributed to all undergraduates enrolled in art

history or classics courses Approximately 400 students

Randomly divided into a control and variable group 39 responses in control group

Completion rate = 31.7% 41 responses in variable group

Completion rate = 35.9%

RESPONSES

Comparison of l iteral values Research Question #1: What is the level of correspondence

between users and catalogers in the subject analysis of artworks? Participants’ responses vs. existing subject terms Control group vs. variable group

Research Question #3: Does coaching users on performing subject analysis change their search behaviors?

Comparison of coded values Research Question # 2: What is the level of correspondence in the

type of subject analysis being performed by each group? Participants’ responses vs. existing subject terms Control group vs. variable group

Research Question #3: Does coaching users on performing subject analysis change their search behaviors?

ANALYSIS

Comparing existing metadata to part icipants’ responses 31% correspondence (for control and

variable groups) for each of the existing terms assigned to

an image, at least one participant assigned the exact same term

Equal rate of correspondence between control and variable groups

Comparison of control and variable groups (no great differences) Participants in the variable group

supplied more terms per image Average # of terms provided by variable

group = 104 Average # of terms provided by control

group = 88

PRELIMINARY FINDINGSLITERAL VALUES

Strata Existing Metadata

Participants (all)

Participants(control)

Participants(variable)

Primary 72% 20% 22% 18%

Secondary 9% 11% 10% 12%

Tertiary 13% 3% 4% 3%

PRELIMINARY FINDINGSCODED VALUES

Strata:Primary – perception of the work’s pure form

“What is the work/image of?”Secondary – incorporates cultural and iconographic knowledge

“What is the work/image about?”Tertiary – looks at art as the product of a historical environment

“What is the work/image a good example of?”

*Note – roughly 66% of responses included terms that related to non-subject metadata like culture, period, materials and techniques, and work type. These were not coded

Finish coding responses and complete the comparisonsExamine differences in the literal and coded responses for

each image to detect differences between different types of images Figurative vs. abstract Western vs. non-western Cultural objects and artifacts vs. expressive works

Analyze the non-subject related terms that were provided

Next Steps: Continued analysis Spring 2015 data collection

FURTHER ANALYSIS TO BE DONE

THANK YOU!