assessing our assessment: failures and successes at ut-austin aj johnson, michele ostrow, meghan...
TRANSCRIPT
![Page 1: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/1.jpg)
Assessing our Assessment: Failures and Successes
at UT-Austin
AJ Johnson, Michele Ostrow, Meghan Sitar
Library Instruction ServicesUniversity of Texas Libraries
![Page 2: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/2.jpg)
FAILURE 1Didn’t ask for help at the beginning
http://www.flickr.com/photos/inju/246717376/
![Page 3: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/3.jpg)
FAILURE 2Didn’t recognize the difference between faculty and librarian
culture
http://fc04.deviantart.com/fs16/f/2007/160/7/f/WilliamSnape2_by_WilliamSnape.jpg
http://www.hermionegrangerswand.com/
![Page 4: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/4.jpg)
FAILURE 3Didn’t anticipate low response
rates
http://www.houseofzing.com/book/images/ignore.jpg
ww
w.h
ou
seofzin
g.co
m/b
oo
k/imag
es/ign
ore.jp
g
![Page 5: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/5.jpg)
FAILURE 4Didn’t consider how faculty view
librarians
htt
p://
ww
w.f
lickr
.com
/pho
tos/
bibl
iona
/238
9790
54/
htt
p://
ww
w.f
lickr
.com
/pho
tos/
kais
hin/
1660
2380
46/
![Page 6: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/6.jpg)
FAILURE 5 Didn’t hire an army of instruction
librarians and statisticians
"United States Army, The." Online Photograph. Encyclopædia Britannica Online. 22 May 2008 <http://www.britannica.com/eb/art-82331>.
![Page 7: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/7.jpg)
FAILURE 6 Didn’t pre-test
![Page 8: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/8.jpg)
FAILURE 7Still didn’t pre-test
![Page 9: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/9.jpg)
FAILURE 7Still didn’t pre-test
![Page 10: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/10.jpg)
FAILURE 8 Designed our post-test around
faculty characteristics
![Page 11: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/11.jpg)
FAILURE 8 Designed our post-test around
faculty characteristics
![Page 12: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/12.jpg)
FAILURE 9Didn’t match pre-tests to post-
tests
![Page 13: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/13.jpg)
FAILURE 9Didn’t match pre-tests to post-
tests
![Page 14: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/14.jpg)
FAILURE 10 Project design didn’t reflect flexibility of our instruction
programs
![Page 15: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/15.jpg)
FAILURE 10 Project design didn’t reflect flexibility of our instruction
programs
![Page 16: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/16.jpg)
FAILURE 11 Didn’t define a control group
“No controle (in control)” by renatotarga on Flickr, http://www.flickr.com/photos/rtarga/223776150/
![Page 17: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/17.jpg)
FAILURE 12 Thought writing questions would
be the easy part
![Page 18: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/18.jpg)
FAILURE 13Didn’t base questions on learning
outcomes
![Page 19: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/19.jpg)
FAILURE 13Didn’t base questions on learning
outcomesOLD: Question - You found an article in a database but it isn’t full-text. How do you find the full text of the article somewhere else? (Circle all that apply)1.Search the Library Catalog. Use the article title as your search terms.2.Search the Library Catalog. Use the title of the journal, magazine or newspaper as your search terms.3.Follow the “FIND IT UT” link
![Page 20: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/20.jpg)
FAILURE 13Didn’t base questions on learning
outcomesNEW: Learning Outcome - Students will be able to determine
where scholarly material will be found. Question - You are doing research on border security along the
U.S. Mexico border. Choose the BEST place to find a research study by a political science professor.Library article databasewww.google.comPublished encyclopediaDon’t know
![Page 21: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/21.jpg)
FAILURE 14 Forced to use multiple choice tool to measure critical thinking skills
![Page 22: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/22.jpg)
FAILURE 14 Forced to use multiple choice tool to measure critical thinking skills
Old:When evaluating web sites to see if you should use them in your research, which factors should you consider? (Choose all that apply)1.Site was on the first page of results in Google2.Author expertise3.Page design4.Bias5.Don’t know
![Page 23: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/23.jpg)
FAILURE 14 Forced to use multiple choice tool to measure critical thinking skills
Newer:You find a web page about media mergers and know nothing about the author. What would be your best clue as to whether the page was a reliable source to use for your paper?1.date the page was last updated2.the URL (http://www...)3.the quality of the page’s graphic design4.the advertisements on the page5.don’t know
![Page 24: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/24.jpg)
FAILURE 14 Forced to use multiple choice tool to measure critical thinking skills
Newest:You are doing research on the effect of illegal immigration on border violence. You find a website from an organization called The Illegal Immigration Research Coalition, which includes statistics on incidents of border violence. List at least three criteria you would use to evaluate this site for bias and credibility.
![Page 25: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/25.jpg)
FAILURE 15Created questions with more than
one right answer1. Google searches will find information from library databases
True False Don’t know 2. Choose the BEST place to find a recent report from a media
watchdog organizationlibrary article databasewww.google.compublished encyclopediadon't know
![Page 26: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/26.jpg)
FAILURE 16Created trick questions
You are doing Internet research about border security. Choose the domain of a website most likely to contain a U.S. presidential candidate's stance on immigration issues
.org
.edu
.gov
.comdon't know
![Page 27: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/27.jpg)
FAILURE 17Didn’t define success
before we started
http://www.easywebcreator.com/applications/WebBlogManager/inc_webblogmanager.asp?CategoryID=227&ItemID=1061&action=refer
![Page 28: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/28.jpg)
FAILURE 18Didn’t identify all the variables
http://www.flickr.com/photos/jdawg/172131301/
![Page 29: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/29.jpg)
FAILURE 19Can’t really use the data
![Page 30: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/30.jpg)
FAILURE 19Can’t really use the data
![Page 31: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/31.jpg)
FAILURE 19 Can’t really use the data
![Page 32: Assessing our Assessment: Failures and Successes at UT-Austin AJ Johnson, Michele Ostrow, Meghan Sitar Library Instruction Services University of Texas](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649c745503460f94926cc9/html5/thumbnails/32.jpg)
Presentation and Questions Created by Participants available
at:
http://www.lib.utexas.edu/services/instruction/ostrow.html