Graduate Program Assessment Report

University of Central Florida

Mission

Communication M.A. Program is dedicated to serving its students, faculty, the Central Florida community and the professions associated with the field of communication. The mission of the program is to offer high-quality, academically challenging graduate education in Mass and Interpersonal Communication; to mentor students in the conduct of research and creative activities; to provide the program’s students with the educational development that will enhance the intellectual, cultural, environmental, and economic development of the metropolitan region; to develop students' academic and professional competencies; to establish UCF as a local and global leader in the communication field; and to support the mission and vision of the University of Central Florida as a whole.

University of Central Florida

Assessment Process

The assessment process is designed to measure student competencies using direct and indirect assessments of student learning of academic, research, and professional skills.

Direct measures of student competencies include evaluations of specific sections of students’ theses, comprehensive exam responses, and writing samples.

 Indirect measures of students’ preparedness for doctoral work and perceptions of program relevance to professional goals are measured using an online survey of recently graduated students.

University of Central Florida

Outcomes (reporting period = 2012-2013 academic year)

Outcome: 1 At least 90% of students will demonstrate satisfactory or above satisfactory knowledge of the literature in their field.

Measurement: Pass rate for comprehensive exams; evaluation of thesis by committee

Results: Comps (N = 26)

88.6% Above Satisfactory or Satisfactory Thesis (N = 10)

Satisfactory/Above Satisfactory = 100% Total

Satisfactory/Above Satisfactory = 92.3%

University of Central Florida

Outcome 2: At least 90% of students will demonstrate satisfactory or above satisfactory knowledge of quantitative research methods.

Measurement: review of comprehensive exam responses and methods sections in theses.

Results Comprehensive Exams (N = 25)

Satisfactory/Above Satisfactory = 60% Thesis (N = 4)

Satisfactory/Above Satisfactory = 100% Total

Satisfactory/Above Satisfactory = 74% Last year = 77%

University of Central Florida

Outcome 3: At least 90% of students will demonstrate satisfactory or above satisfactory knowledge of qualitative research methods.

Measurement: review of comprehensive exam responses and methods sections in theses.

Results Comprehensive Exams (N = 25)

Satisfactory/Above Satisfactory = 77% Thesis (N = 6)

Satisfactory/Above Satisfactory = 100% Total

Satisfactory/Above Satisfactory = 86% Last year = 81%

University of Central Florida

Outcome: 4 At least 90% of graduates entering doctoral programs will feel prepared to succeed at the doctoral level.

Measure 4.1- 4.3: graduates who entered doctoral programs (in last three years) will be asked how well UCF MA prepared them for doctoral work overall, and in the areas of theory and methodology.

Results:

Overall 4/4 rated NSC MA satisfactory or above Theory 4/4 rated NSC MA satisfactory or above Methodology 3/4 rated NSC MA satisfactory or above

University of Central Florida

Measure 4.4: At least 80% of our graduates who apply to doctoral programs will have published one or more peer-reviewed articles and/or presented one or more conference papers as a student in our program.

Results 4/4 published or presented a manuscript

Relatedly: 55% of all respondents reported publishing or presenting research with a faculty member or under the direction of a faculty member.

University of Central Florida

Outcome 5: At least 80% of graduates will report favorably on the impact of the Program’s relevance to professional career advancement.

Measure: 5.1 Response to survey item, “Completion of the Communication M.A. program has had (will have) a favorable impact on my professional advancement.”

Results: (N=29) Strongly agree or agree = 90% (n=26) Neither agree nor disagree = 10% (n = 3) Strongly disagree or disagree = 0% (n=0)

University of Central Florida

Measure 5.2: Response to survey item, “The overall Communication M.A. program content is relevant to (can be applied to) one or more professional settings.”

Results:  (N = 38)

Strongly Agree or Agree = 97% (n=28) Neither Agree or Disagree = 3% (n=1) Strongly disagree or disagree = 0% (n = 0).

University of Central Florida

Outcome 6

At least 90% of graduates will demonstrate satisfactory critical/analytical thinking skills.

Measure 6.1: Writing Samples (N= 19) No Data Yet

Rubric: Critical Thinking

3 = Above Satisfactory: Student synthesizes studies/areas of research in a logical way; provides well-reasoned critique of research; uses logical rationale in developing ideas and/or supporting claims made in the paper.

2 = Satisfactory: Student satisfies all but one of the characteristics in the “above satisfactory” category

1 = Below satisfactory: Student fails to satisfy two or more characteristics in the “above satisfactory” category

University of Central Florida

Outcome 7: At least 90% of graduates will demonstrate satisfactory written communication skills.

Measure 7.1Students taking comprehensive exams will be required to submit a writing sample consisting of any paper written for a course in the Communication M.A. program. A panel of faculty will assess the student’s writing using the rubric on the next slide. At least 90% of students will score Above satisfactory or Satisfactory.

University of Central Florida

Rubric: Writing Skill

3 = Above Satisfactory: Writing shows high competence in the areas of precision, organization (including effective use of transitions), use of grammar, and language usage

2 = Satisfactory: Writing is below the competence level in one or more areas listed in the Above Satisfactory category, but is at least adequate in three of the four areas

1 = Unsatisfactory: Writing is below adequate and needs improvement in two or more areas listed in the Above Satisfactory category

University of Central Florida

Measure 7.1 Results No Data Yet

University of Central Florida

Measure 7.2 Students completing internships will be assessed on their workplace

communication skills by their supervisors. Upon completion of the internship, supervisors will be asked to complete an online questionnaire consisting of the Communicator Competence Questionnaire. As a group, internship students will reach an average of 5.5 (Agreement and above range) on a 7 point scale on the CCQ for a satisfactory assessment.

University of Central Florida

Example Items: My intern is a good listener: 100% Agree or Strongly Agree My intern writes effectively: 70% Agree or Strongly Agree (30% no opportunity to observe) My intern’s email messages reflect the level of professionalism expected by my

organization: 100% Agree or Strongly agree

Results (N = 11) Overall mean = 6.33 out of possible 7 10 of 11 averaged 5.5 or above out of possible 7 on

CCQ

University of Central Florida

Perceived Weaknesses (from survey) Program needs more structure for beginning students Too many/not enough PhD development courses Not enough real world applications Courses were not challenging enough – some students

did little work but still passed Could not get elective courses in Mass Communication Too much focus on theory No one told me how to get a job with this degree

University of Central Florida

Perceived Program Strengths

Possibility of doing original research and faculty interest in, and encouragement of, developing ideas into research projects

Feel like faculty are approachable Learning theories that were tied directly to real world

problems Professional applications of class concepts Challenging classes

University of Central Florida

Plan for changes to assessment procedures/plan: Change to annual rather than academic year reporting Refine measurement of communication skills measure for

interns to look at specific items rather than an overall mean. Four potential items:

Listening skill Public speaking skill Conflict skill Writing skill

University of Central Florida

Changes resulting from previous assessment data Review sessions for students taking quantitative comps Fall 2013 launch of required course: Introduction to Graduate

Study in Communication. Addition of Theories of Public Relations course Students must pass ALL core comprehensive exams (starting

with this incoming class)

University of Central Florida

Planned and potential changes based on assessment data Require at least one methods course in each of two first

semesters Require theory in the first year (potential) At least partially standardize methods courses by finding

common outcomes and building comp exam questions around outcomes instead of classes