1

2012-2013 Assessment Report Department/Unit: Office of University Writing

Office of University Writing

Expected Outcome 1: Students and faculty will be aware of and use the services offered by the Office of University Writing

Students and faculty will be aware of and use the services offered by the Office of University Writing in support of writing and writing instruction. Assessment Method 1: Usage data of Miller Writing Center from WCOnline Scheduler Assessment Method Description The appointment scheduler system we use (WCOnline) allows us to collect and analyze data about the students who use the Miller Writing Center. We compared this data to previous years following, in particular, overall use by unique users compared to overall number of appointments (repeat users), use by location, and use by colleges as a percentage of their enrollment. Findings Slightly more appointments were made at the MWC in 2012-13 (4,069) than in 2011-12 (4,038) but fewer unique users made those appointments (2,048 in 2012-13 compared to 2,081 in 2011-12), an indication of increased repeat usage.

The RBD Library's Learning Commons remains the most well-used location and our adjustments to the scheduling of tutors created a drop in overall usage at the Library. However, we are still close to full capacity at this site (80% overall usage is considered full use because the data counts early weeks of the semester where usage is typically low, and we lose some time because we allow students to schedule in 30, 45, or 60 minute blocks but beginning on the quarter hour). Forestry and the Architecture Library, which are funded by those Colleges, saw an increase in usage even though the overall numbers are still lower than other locations.

2

A closer examination of the Architecture location (which is funded by the School of Architecture), indicated that despite the relatively low use, 84% of the students coming from Architecture courses were using the location in Architecture (LADC). As the graph below shows, the total number of appointments coming from courses in the Department of Architecture (Architecture, Interior Architecture, and Landscape Architecture) was 63 with 53 of those made at the LADC location. However, the overall traffic to that location was primarily students from outside the College of Architecture (67% of the appointments at this location were made by students who are listing their major as outside the college).

Usage by students with different majors remains quite variable, but with more students from the College of Liberal Arts making use of the Writing Center. However, a comparison by the percentage of the colleges' enrollment shows a more even distribution across most colleges with a higher percentage of Nursing students using the MWC. Comparing the 2011-12 academic year with the 2012-13 academic year shows that 5 of the 13 colleges had an increase in the percentage of their total enrollment making use of the MWC, with the School of Pharmacy showing the largest increase as a percentage of enrollment (from 0% to 3%). Education, Engineering, and undeclared students showed the largest increases in use by raw numbers. Business,

3

Forestry, Human Sciences, and COSAM showed declining use in both raw numbers and as a percentage of their total enrollment.

How did you use findings for improvement? 1. We shared site specific data with all location partners. In the case of the Architecture site, comparison of Architecture vs non-Architecture students made it difficult for Architecture to justify continuing to assign graduate student assistants to staff the location at the LADC. However, since the data suggests that most of the students from Architecture using the MWC were making their appointments at this location, the Office of University Writing agreed to staff this location, advertise it more heavily to increase traffic, but reduce the number of tutoring hours until usage increases. 2. To increase traffic at all satellite locations we will launch an advertising campaign to make it clearer to students that these sites are open to everyone. 3. We suspect that the increase in usage by Pharmacy students is related to

4

our being able to hire tutors from that College. We will continue to recruit broadly and seek to balance the tutoring staff to reflect all disciplines. Additional Comments Assessment Method 2: Goggle Analytics of website Assessment Method Description We use Goggle Analytics to collect data about the use of our website. We compared the use of the site from August 1, 2012 to July 31, 2013 with data from the prior year. We were especially interested to see if the ePortfolio Project materials altered the use of the site. Findings The pattern of use of the website is very similar across both years, echoing peak periods for the Miller Writing Center and the call for grant proposals for the ePortfolio Project (April 2013), but with usage generally greater in 2012-13 than in 2011-12. We lost data in the early part of 2011 when the site migrated to a different server (thus the flat line in August-October of that year). The one exception to the pattern of increased use in 2012-13 over 2011-12 is the rise in use of the site when we released the 24/7 videos in January of 2012. It appears that the increase use can be attributed almost entirely to the ePortfolio Project and the pages that were created to support faculty and students involved in that project.

Overall, use of the website saw a significant growth in every category. Total number of visits increased by 47.49%, unique users increased by 99.12%, page views increased by 51.82%. Duration of visits increased from an average of 1 minute 55 seconds to 2 minutes 7 seconds (a 10.52% increase).

5

Only the bounce rate declined from 63.58% leaving the page they entered without interacting with it to 62.25% doing so, and indication that users are spending more time reading the information on the page.

A closer look at the most frequently accessed pages shows that the Miller Writing Center remains the most frequently accessed page, but the ePortfolio pages are next, and visitors stay on those pages longer than they do on the Miller Writing Center page. The data suggests that students continue to come to the MWC page to schedule appointments, but faculty and students access the ePortfolio pages to get information of varying kinds including looking at the gallery of examples.

6

How did you use findings for improvement?

1. We hired graduate students as summer fellows to create additional content for the website.

2. We are announcing to the AU community when new content goes onto the website.

3. We will investigate putting an RSS feed on the site so that faculty and students can subscribe to specific features of the site and receive automatic updates.

4. We recognize that not all visitors to the site are Auburn students and faculty. We will explore how best to filter the data available to us to better identify usage patterns of Auburn users.

Additional Comments

Expected Outcome 2: Students and faculty will be satisfied with our services

We expect students and faculty to be satisfied with the quality of our services and programs. Assessment Method 1: WriteFest Survey Assessment Method Description We sent qualtrics surveys to all participants after each session. Of the 76 total participants in the fall term, we had 40 responses to the survey. Of 42 participants in the spring term, 19 responded to the survey. We also sent a survey to all 20 participants after the WriteFest Thesis and Dissertation Bootcamp held in June 2013 and received comments from all participants. The surveys remained open for two weeks. Comments were also solicited from the faculty who presented at WriteFest during the spring of 2013. Findings Graduate students surveyed after the WriteFest sessions during fall and spring term were clear that they are less interested, and perhaps less motivated by, free food and beverages than they were by the opportunity to write, talk about writing, and get help with a range of topics and questions related to writing. Data was more mixed about whether working on writing and getting writing done was more important than the information about writing conveyed at the discussion tables. On the other hand, the WriteFest Bootcamp survey from the summer indicated that having time to write and having a clear structure of goal setting were the most important factors. All

7

20 participants rated their overall experience good to excellent (12 "excellent", 7 "very good", 1 "good"). Qualitative responses included comments like:

“[BootCamp] got rid of distraction and allowed focused writing” “I enjoyed having the structured writing time. It really allowed me to

better organize myself” “the atmosphere and the schedule were very conducive towards

productive writing”

8

9

10

11

12

13

How did you use findings for improvement? We do not need to make any changes to WriteFest Bootcamp because graduate students showed up, participated fully, achieved the goal of spending time on their writing, and indicated satisfaction with the program. However, despite the expressed satisfaction with the fall and spring WriteFest program, students did not consistently attend. Therefore,

1. We will adapt the structure that was so well received at WriteFest Bootcamp to the fall and spring WriteFest sessions by focusing more on goal setting.

2. We will offer two WriteFest Dissertation and Thesis Bootcamps -- one in spring 2014 and one in the summer.

14

Additional Comments Assessment Method 2: Evaluations of practicum workshop Assessment Method Description We worked with TAs in 2 departments in 2 different colleges with approximately 45 total TAs over multiple encounters. We have been given the comments from TAs collected by one of those departments. There are 21 responses, but not all of the TAs attended the sessions we facilitated on designing group activities and peer review groups (referenced as "workshopping" by some responders). We read these responses to highlight any mention of the sessions we conducted. Findings A total of nine graduate students commented specifically on the programs we delivered as part of their preparation to teach practicum.

"Designing Effective Writing Tasks (Margaret Marshall) was absolutely useful. Please do more of this! Her explanations are incredibly useful and insightful. I'd like to design a whole unit as a class! That would be so useful to us who are nervous about being the teacher of record next year."

"I enjoyed Travis Adams' session on peer review sessions as I believe it provided a lot of context in regards to a well-known yet somewhat ambiguous classroom practice. His delivery was enjoyable, and I felt that requiring us to look back on our own experiences with peer review as students was an interesting way to introduce a topic that the majority of us seemed biased against. I also liked all of the handouts, as they enable me to leave with a take away that I can refer back to at a later date."

"Margaret Marshall's presentation was also helpful." "Topics that I found particularly helpful and immediately applicable to

my life were group work, class discussions, and structuring writing assignments."

"The sessions of Practicum that featured Margaret Marshall and Travis Adams were particularly useful in the way they provided concrete pragmatics in structuring group work and peer response. They need to happen earlier in the term."

"I found the handouts and practical advice about assignment design, leading discussion, and effective peer response very useful."

"I found Practicum to be incredibly helpful. We covered topics that were relevant to my experiences as a GTA, particularly structuring effective group discussions. I think it would have been helpful to have those earlier in the semester. I also really enjoyed Dr. Adams' discussion on effective peer workshopping. I know that is something that I can definitely transfer over."

"I really enjoyed paper workshopping and responding to student

15

writing, as the field (as a new teacher) is completely new to me." "Additionally, there were several guest speakers that came (which I

really enjoyed and found useful), but those speakers mentioned that we didn't need to worry about taking notes because they would email us the PowerPoints or the outline of their ideas, but that never actually happened."

How did you use findings for improvement?

1. We will continue to work with departments as requested to supplement the TA training they provide.

2. We urged the department to move the peer review session in particular to an earlier date in the term and were successful in getting that session into October instead of in late November.

3. We will no longer rely on the faculty members in the department distributing our PowerPoints, but will instead prepare a handout of key slides that students can use to take notes during the presentation.

Additional Comments Assessment Method 3: Spontaneous testimonials Assessment Method Description We collect any emails that faculty, students, or colleagues send us about our services and/or programs. Findings Comments sent to us spontaneously are uniformly positive. Workshops and presentations in particular are consistently praised as helpful no matter who is delivering the content. In the work with targeted departments we see both a breadth of faculty writing in appreciation of the services and longer, more detailed interactions with the faculty most deeply engaged.

June 11, 2012: (other professor) and I could not have been prouder to be associated with Auburn—and truthfully, with Margaret Marshall! The Auburn panels I heard were superb and it's clear that in just a few years you've created a program that supports innovative, energetic, and intellectually engaged teaching. I hope our department can become more fully involved, beginning with the xx course.

June 11, 2012: I really enjoyed the (IWAC) conference, Margaret, and am really glad you convinced us to participate. It was well worth it and I feel much more enthusiastic about thinking through our program for the fall!

July 30, 2012: Hurray for you and the nice write-up the Writing Initiative got in the Auburn Report—so good to see your hard work getting the recognition it deserves!!

16

August 21, 2012: Just a note to say - I saw the announcement about WriteFest in Auburn Daily. Lovely idea! I'm so encouraged about the kind of writing culture we are creating here.

September 6, 2012: I've already overheard my students mention the Writing Center and that it was very useful!

September 28, 2012: I just wanted to thank you for taking the time to explain the resources of the Miller Writing Center to my students in xxx last week, and to show them how to use Peer Response to improve their business memos last Monday. I just got through grading and recording Homework #2 and am really pleased with the results: the class average was higher than in the past four semesters, and the coefficient of variation was 20-50 percentage points lower. Plus my grading speed was a good deal faster, not having to stop and correct as many spelling, grammar, and typo errors as in the past. So, all in all, a very worthwhile addition to my class routine! I appreciate the availability and willingness of the Miller Writing Center in helping improve my students' academic success, and thank you especially for all your efforts.

November 9, 2012: Thank you for your excellent presentations on e-portfolio. They were well organized, concise and very motivational. Our staff and (students) are excited about working with this in the near future.

June 6, 2013: I attended WriteFest seminar on Effective Poster Presentations in the Fall 2012 semester. I just wanted to THANK YOU SO MUCH for all your guidance and advice in revising my poster, how to have confidence in my research when presenting, how to draw people in as they walk by and how not to wait until someone ask about my poster and the proper way to answer questions asked my the judges. I won first place in the Graduate Student category which is the "American Society for Microbiology President's Award". All of your advice helped me tremendously and I would recommend the WriteFest to all students from all academic backgrounds or classifications. Thank you in advance for all that you have done and may do in the future.

How did you use the findings for improvement?

1. Working in a sustained way with target departments (or faculty) does seem to elicit deeper engagement but we find collecting unsolicited comments of limited use in making subsequent decisions. We have begun to investigate how others measure the depth of faculty

17

engagement in order to create a survey or other instrument that could be administered more broadly. We have found institutional and organizational change theorists as well as K-12 professional development programs helpful.

2. The work we did with one of the departments last year was so positive that we have continued to consult with that department, though without the intensive workshops for the faculty. We will repeat the workshops for the graduate students in that program.

3. Our work with the ePortfolio cohort faculty has been informed by our sustained work with writing in these departments. We will refine our objectives for faculty to emphasize depth of engagement.

4. We created three new on-campus opportunities for faculty to share their work with writing and learn from others -- Conversations in Celebration of Teaching, WriteBites Lunch, and WriteBites Online.

Additional Comments

Expected Outcome 3: Students who work for us as tutors, interns, or ePortfolio Ambassadors will demonstrate an ability to think critically about their work.

We employ both undergraduate and graduate students in a variety of roles. In each case, however, we design training, assign tasks, and promote practices that encourage students to be critically reflective about their practices by synthesizing experiences, challenging assumptions, seeing problems from multiple points of view, and creating new knowledge that can be shared with others by integrating theory and practice. Assessment Method 1: MWC consultant self-reflections Assessment Method Description The Miller Writing Center assessed the student staff’s ability to think critically about their work through an end-of-semester reflection writing exercise. Each writing consultant and front desk worker was required to submit an essay that asked them to describe and reflect on their experiences, their personal development over the semester, their progress toward the goals they set, their engagement with the professional development activities, and their response to the student surveys from WC Online. (See attached forms) Following Kember et al (2008) (also attached), these responses were

18

organized in four categories of reflective thinking: 1=Habitual action: no reflective thinking or thoughtful engagement, merely repetition of the prompt and empty cliché. 2=Understanding: a limited review of events and concepts without connections made or depth of analysis; while there is evidence that knowledge has been acquired, there is no clear critical engagement. 3=Reflection: events and concepts are not just reviewed, but connections are made between diverse concepts and experiences, and meaning is developed through active, applied thought. 4=Critical Reflection: events, concepts, theories are synthesized using significant high-level critical thinking, leading to significant transformation of the writer’s understanding of her/himself and the world. The rubric (attached) identified these levels of reflection across four categories, based on the prompts given.

Discussion of Tutoring Experiences Goal Setting and Personal Development Engagement with peers, professional development

opportunities, and/or student surveys Self-Assessment

All categories did not have to be addressed in every reflection. Following Kember et al (2008), each writing sample was read as a whole with the moment of highest reflection recorded. Kember et al. (2008) on Reflection can be provided upon request.

Miller Writing Center Semester Reflection

Please complete the following reflection by or before Friday, December 7th. Write as much or as little as you feel necessary. Send your reflection as an email attachment in Word or .pdf format to the writing center email account: writctr@auburn.edu

Name:

Semester: Fall

Year: 2012

First, please write a reflection on your work with the Miller Writing Center this semester. Include some discussion of what went well, what you may have struggled with, and what you learned:

Then, (if applicable) read over the surveys that have been filled out by writers you worked with and write a brief reflection on these surveys. What do they suggest about your

19

consulting this semester? What do you learn from them? What seems important about them? If you did not receive any surveys, write briefly

Number of surveys _____

Finally, discuss what goals you have for your next semester with the MWC, what aspects of your work you would like to improve upon, and what you might do to achieve those goals or make improvements:

MWC Reflective Writing Rubric 2012-13

Following Kember et al (2008), the rubric was organized in four categories of reflective thinking:

1=Habitual action: no reflective thinking or thoughtful engagement, merely repetition of the prompt and empty cliché.

2=Understanding: a limited review of events and concepts without connections made or depth of analysis; while there is evidence that knowledge has been acquired, there is no clear critical engagement.

3=Reflection: events and concepts are not just reviewed, but connections are made between diverse concepts and experiences, and meaning is developed through active, applied thought.

4=Critical Reflection: events, concepts, theories are synthesized using significant high-level critical thinking, leading to significant transformation of the writer’s understanding of her/himself and the world.

Each of these categories was divided into four sub-categories, based on the prompts.

Discussion of Tutoring Experiences 

Goal Setting and Personal Development 

Engagement with peers, professional development opportunities, and/or student surveys 

Self‐Assessment 

All elements did not have to be addressed in every reflection. Following Kember et al (2008), each writing sample was read as a whole with the moment of highest reflection assessed.

20

Semester Reflection Rubric, 2012-13

A “4” Reflection

A “3” Reflection A “2” Reflection A “1” Reflection

Discussion of Tutoring Experiences

Experiences are synthesized and lead to thoughtful conclusions, with new learning about tutoring clearly articulated.

Detailed descriptions that are connected together to establish patterns of understanding about the individual’s tutoring

Reasonably detailed descriptions of experiences that show more depth of engagement with events

Vague descriptions of experiences that show little or no understanding and reflection

Goal Setting and Development

Deep understanding of goals are synthesized from experience and development activities

Goals and progress thoughtfully analyzed and connected to experience, but not deeply connected to broader theoretical ideas and/or future progress

Goals are described with reasonable clarity, with a gesture toward change over time.

Goals are indefinitely and mechanically defined in response to prompt

Engagement with peers, development, and/or student surveys

Developmental activities and tutoring experiences are synthesized into new, clearly articulated understanding of tutoring

Connections and recognize patterns across tutoring experiences and developmental activities

Descriptions show reasonable understanding of developmental activities, but not depth of thinking

Developmental activities are described vaguely and mechanically.

Self-Assessment

Patterns of strengths and weaknesses are synthesized into a new, well-articulated sense of self-awareness of the writer’s approach to her/his work as a tutor

Thoughtful recognition of patterns of strengths and weaknesses, with thought given towards future development as a tutor.

Articulation of strengths and weaknesses without significant depth

General and clichéd in approach and tone.

21

Complete the following reflection by or before Friday, May 3rd and send as an email attachment to writctr@auburn.edu

INFORMATION

Name: Click here to enter text. Semester: Choose an item.

Review the goals you made for yourself at the beginning of the semester (see attached goals document). List those here: Click here to enter text.

Number of staff meetings missed: Click here to enter text. Completed missed staff meeting reflections: Click here to enter text.

Number of surveys: Click here to enter text.

Check all writing center related events/processes you participated in this semester:

☐Workshop(s) ☐Hiring Committee ☐Staff Meeting Notes ☐Library Tailga

☐Class Visit(s) ☐Conference Proposal Writing/Submission

☐Distance Tutoring ☐Library OASI

☐WriteFest ☐Conference Attendance ☐MWC Orientation Planning/Facilitator

☐Writing cente

☐Lead Consultant ☐Conference Presentation Given

☐WC Theory/Practice Course

☐Other (describ

Other: Click here to enter text.

REFLECTION

Based on the information from above, write a reflection on your work with the Miller Writing Center this semester. Your reflection should 1) describe, 2) analyze, and 3) make connections between your work, goals, learning, and experiences. For your reflection, choose a focus of particular interest or relevance for you and consider responding to some of the prompts included for that focus. Write as much or as little as you feel necessary, and feel free to write about anything else you feel is relevant to your work, learning, or experiences with the MWC this semester

22

Focus Prompts

Goals

Describe the progress you made toward the goals you set. Did you meet your goals? Why or why not? How did your goals, or your work toward them, connect to other aspects of your work in the MWC? Discuss your goals for next semester with the MWC. What aspects of your work would you like to improve upon, and what might you do (or need from us) to achieve those goals or make improvements?

Staff Meetings

What is one concept, suggestion, strategy, etc. you took from a staff meeting and applied this semester? How did your attendance at weekly staff meetings help or hinder your development as a consultant or front desk worker, your learning, and/or your sense of belonging to the writing center community? Identify one contribution you made to a MWC staff meeting and discuss how that contribution helped you, others on the staff, or the MWC as a whole.

Surveys

What do the surveys filled out by writers you worked with suggest about your consulting this semester? What do you learn from them? What seems important about them? If you did not receive any surveys, write briefly about what you might do to get students to complete surveys in the future.

Experiences

For each item you checked, please describe what you learned or gained from that experience and how that experience connected to other aspects of your work with the MWC. If you did not participate in any activities other than tutoring or working at the front desk, please elaborate on why not, and/or what your future interest might be.

Reflection: (write here or on another page/document) Click here to enter text.

Findings Score Fall 2012 Spring 2013 4: Critical Reflection 3 10 3: Reflection 6 13 2: Understanding 14 4 1: Habitual Action 3 0 Average 2.35 3.22 Total Reflections 26 27 Total Staff 41 43 Missing Reflections 15 16

The fall average of 2.35, barely above “Understanding,” indicates that MWC staff had initial difficulty demonstrating significant critical thinking about their work across any of the categories they were prompted to consider. But the dramatic rise in the spring to 3.22, well in the range of “Reflection,” suggests that ongoing development work was successful in helping students think and

23

reflect more actively about their work.This rise may have been affected by the shift in the prompt which asked for a more focused response (see attached forms). Also, a significant number of staff members did not complete the exercise and only 18 of the 27 spring respondents also responded in the fall. How did you use findings for improvement?

1. We believe the increase in critical reflection that our analysis shows is the result of the professionalizing experiences we offer students through the regular staff meetings. We will continue those activities and continue to encourage students to propose conference proposals that give them an audience for their reflections.

2. We will continue to ask students to prepare written reflections at the end of each term using a standardized prompt.

3. We will take steps to increase participation by linking the reflective essay to a new peer evaluation program and to our rehiring process.

4. In the future we will use the rubric to identify levels of reflection in each of the categories we prompt students to address. This will give us a more detailed picture of what activities generate reflective thinking and whether the staff as a whole avoids reflecting on particular categories.

Additional Comments Assessment Method 2: Professional activity and awards Assessment Method Description We encourage all students working in the Office of University Writing to join the larger conversations about writing, tutoring, etc. by submitting proposals to appropriate regional and national conferences, and submitting written scholarship to appropriate journals. Since professional conferences evaluate the proposals suitability and rigor during the selection process, acceptance is a good measure of the students' depth of thinking. Likewise, journals accept scholarship only after considerable review. Finally, on campus awards are competitive and given only to candidates who demonstrate sophisticated thinking and unusual engagement in the intellectual work. We tallied the conference acceptances, travel grants received, publications accepted, and awards won by any of our students. Findings Three out of four graduate assistants working for the Office of University Writing presented at conferences during 2012-13, including one assistant who presented at four different conferences, one who presented at two different conferences and one who presented at one conference. In addition, one graduate student tutor presented at a state conference, and five undergraduate tutors presented at two different regional conferences. Three undergraduates received competitive travel grants to attend two different conferences, and two graduate students received competitive travel support

24

for presenting at the Conference on Higher Education Pedagogy. These same two graduate assistants received competitive awards here at Auburn based, in part, on their activities with the OUW. In total, Auburn was represented by student presenters at two international conferences (Association for Authentic, Experiential, Evidence-Based Learning and the International Writing Centers Association), four national conferences (Conference on College Composition and Communication, National Conference on Peer Tutors in Writing, Conference on Higher Education Pedagogy, and Quinnipiac University Writing Across the Curriculum), and three regional conferences (AAEEBL Southeast Conference, Southeastern Writing Center Association Alabama Spring Conference, and Southeastern Writing Center Association). In addition, one graduate assistant submitted a scholarly essay to the Writing Lab Newsletter, was asked to revise, and successfully resubmitted the essay which is forthcoming. How did you use findings for improvement?

1. Each of the presenters shared their own presentations with either the OUW staff or the Miller Writing Center staff as part of the on-going professionalizing experiences we offer our student employees.

2. Those who attended conferences shared what they had learned from other presentations with the staff as well. Such connections to the larger academic conversations related to our work deepen the understanding and critical engagement of our tutors, interns, Ambassadors, and graduate assistants.

3. We encouraged students to create proposals for the National Conference of Peer Tutoring in Writing to be held in November 2013 and sixteen students had proposals accepted and fourteen will be attending.

4. We will continue to encourage student employees to submit proposals for appropriate conferences.

5. We will also encourage student employees to submit written scholarship to appropriate journals.

Additional Comments