11

Spanning the Divide BetweenSpanning the Divide BetweenInstitutional Research andInstitutional Research and

Classroom AssessmentClassroom AssessmentSan Diego Community College DistrictSan Diego Community College District

San Diego Mesa CollegeSan Diego Mesa CollegeBill Grimes, M.A., Bill Grimes, M.A., Director of District Institutional ResearchDirector of District Institutional Research and Planningand Planning

Yvonne Bergland, Ph.D., Yvonne Bergland, Ph.D., Dean of Instructional Services, Resource Dean of Instructional Services, Resource Development, and ResearchDevelopment, and Research

Gail Conrad, M.A., Gail Conrad, M.A., Mesa Interim VPSS, Former SLOAC CoordinatorMesa Interim VPSS, Former SLOAC CoordinatorSusan Mun, M.A., Susan Mun, M.A., Research Analyst (CBR)Research Analyst (CBR)

Strengthening Student Success, San Jose, CAStrengthening Student Success, San Jose, CAOctober 3 – October 6, 2007October 3 – October 6, 2007

Presentation Learning Outcomes

• Explain the key elements of a Campus-Based research model

• Identify the steps that colleges need to take for implementing this model

• Evaluate the advantages and challenges of this model

33

Recommendation from WASC Recommendation from WASC (Accreditation Process)(Accreditation Process)

““The college should develop and The college should develop and implement a plan to meet current and implement a plan to meet current and future needs for institutional research that future needs for institutional research that is accurate, timely and actionable…” is accurate, timely and actionable…”

44

Recommendation from WASCRecommendation from WASC(continued)(continued)

“… “… Toward this end, the college should Toward this end, the college should carefully consider how instructional carefully consider how instructional research is positioned in the college so research is positioned in the college so that it may support the entire institution that it may support the entire institution from a fair, unbiased and informed stance, from a fair, unbiased and informed stance, thereby strengthening various planning thereby strengthening various planning and institutional improvement efforts. and institutional improvement efforts. (I.B).”(I.B).”

55

Recommendation from WASCRecommendation from WASC(continued)(continued)

““In order to build upon their efforts to In order to build upon their efforts to strengthen institutional effectiveness and strengthen institutional effectiveness and to foster a “culture of evidence” throughout to foster a “culture of evidence” throughout the district, the district office and the the district, the district office and the colleges should cooperate in the colleges should cooperate in the development of an enhanced research development of an enhanced research function with both strong district and function with both strong district and strong college components. (I.B.3, I.B.6, strong college components. (I.B.3, I.B.6, IV.B.2.b.)”IV.B.2.b.)”

66

Preparation ChecklistPreparation Checklist Central contact person(s) and locationCentral contact person(s) and location Mechanism to inform the college of research activities Mechanism to inform the college of research activities Mechanism for identifying research needed for making Mechanism for identifying research needed for making

effective data-driven decisions.effective data-driven decisions. A year-long research agenda that: 1) integrates A year-long research agenda that: 1) integrates

campus-wide information, research activity and needs, campus-wide information, research activity and needs, 2) takes advantage of existing district research (avoids 2) takes advantage of existing district research (avoids duplication of effort), 3) identifies and addresses duplication of effort), 3) identifies and addresses information gaps as expressed by the campus information gaps as expressed by the campus community.community.

A College Research Committee to support and guide A College Research Committee to support and guide the functions listed above.the functions listed above.

(See CD document : “Preparation Checklist”)(See CD document : “Preparation Checklist”)

77

Setting the StageSetting the Stage

San Diego Community College District (SDCCD)San Diego Community College District (SDCCD)• City, Mesa, and Miramar College and Continuing EducationCity, Mesa, and Miramar College and Continuing Education• Second largest community college district in CaliforniaSecond largest community college district in California• Total enrollment exceeds 140,000 studentsTotal enrollment exceeds 140,000 students

Mesa CollegeMesa College• Largest of the three colleges in the SDCCDLargest of the three colleges in the SDCCD• Functioning Research CommitteeFunctioning Research Committee• Well-established Program Review process and Student Learning Well-established Program Review process and Student Learning

Outcomes Assessment Cycle (SLOAC)Outcomes Assessment Cycle (SLOAC)• Integration of Academic and Student Services Program ReviewsIntegration of Academic and Student Services Program Reviews

88

District / CBR ModelDistrict / CBR Model

IRP Director

Chief Researcher (vacant)

District Analyst

Office Assistant

City College CBR

Mesa College CBR

Miramar College CBR

District Analyst

District Analyst

Continuing Education (50%)

Assistant Chancellor, Student Services

Director, Institutional Research and Planning

District-wide Research

Committee

PresidentPresident’s Cabinet

Vice President, Student Services

Vice President, Admin. Services

Vice President, Instruction

Dean, Instructional Services, Research and Resource Development

Campus-Based Researcher

Mesa’s Research

CommitteeProgramReview

SLOAC

SAN DIEGO MESA COLLEGEOFFICE OF INSTRUCTIONAL SERVICES, RESOURCE DEVELOPMENT AND

RESEARCHRESEARCH FUNCTION – LINES OF RESPONSIBILITY

DISTRICT COLLEGE

Office of Instructional ServicesResource Development and Research

Resource Planning andDevelopment

Special Projects

Faculty EvaluationProcess

EnrollmentManagement

Schedule Development

Curriculum Review Process

Staff Development

President VPI Deans Staff Faculty Chairs District College Committees

Functions

Areas Results are Forwarded

1111

District/Campus CollaborationDistrict/Campus Collaboration

Mesa CollegeDistrict IRP

Campus-based Researcher

Supports data-driven decisions that increase

student access, retention, and success

Basic Skills

Program Review

SLO Support

Accreditation support

Surveys

District-wide studies

Chancellor’s Cabinet reports

State reports

District IR web site

Basic Skills

Program Review

SLO Support

Accreditation support

CCSSE/CCFSSE

Research Agenda

Faculty/Staff requests

Educational Master Plan

Marketing & promotion

1212

Campus-Based ResearchCampus-Based Research

College-wide Strategic Planning

Research

RequestsSurveys

1313

Examples ofExamples ofRequests for Research ReportsRequests for Research Reports

• Student Athletes StudyStudent Athletes Study• Student success (ESOL, Math, Fast Track HS, PERG, Student success (ESOL, Math, Fast Track HS, PERG,

Learning Communities, short-term courses)Learning Communities, short-term courses)• Marketing, outreach & recruitment (GIS maps, BLAS, Marketing, outreach & recruitment (GIS maps, BLAS,

CHIC, transfer, Bridges program, students on CHIC, transfer, Bridges program, students on probation/DQ, applicants who did not enroll)probation/DQ, applicants who did not enroll)

• Graduate follow-up surveys (Chemistry, PTA) Graduate follow-up surveys (Chemistry, PTA) • TransferTransfer

(see CD handout “CBR Project Log”)(see CD handout “CBR Project Log”)

1414

Requests for Research to Support Requests for Research to Support Assessment of Student LearningAssessment of Student Learning

Support direct assessment of SLOs Support direct assessment of SLOs • Spanish 101 Multiple Independent Scoring of EssaysSpanish 101 Multiple Independent Scoring of EssaysIndirect Assessment of SLOsIndirect Assessment of SLOs• CCSSE/CCFSSE/Basic Skills OversampleCCSSE/CCFSSE/Basic Skills Oversample• Silver ProcessSilver ProcessExamination of Learning IndicatorsExamination of Learning Indicators• Instruction: Successful course completion rates, Instruction: Successful course completion rates,

retention, persistence, etc.retention, persistence, etc.• Student Services Programs: Retention, persistence, etc.Student Services Programs: Retention, persistence, etc.

1515

Educating the CampusEducating the CampusFlex Day PresentationsFlex Day Presentations

District Research - Ways It Can Assist District Research - Ways It Can Assist Developing Effective Research QuestionsDeveloping Effective Research Questions

Using Rubrics in Assessment (Part I)Using Rubrics in Assessment (Part I)Using Rubrics in Assessment (Part II)Using Rubrics in Assessment (Part II)

Pivot Tables: An Interactive Tool to Support DecisionsPivot Tables: An Interactive Tool to Support DecisionsFTES Basics (Part I)FTES Basics (Part I)

CBR and the Research AgendaCBR and the Research AgendaHow to Develop a SurveyHow to Develop a Survey

Follow up to CBR and Mesa College Research Agenda: Follow up to CBR and Mesa College Research Agenda: RetrospectiveRetrospective

Student Services SLOs & College PlanningStudent Services SLOs & College Planning

((See CD document See CD document “Flex presentations”)“Flex presentations”)

1616

Educating the CampusEducating the CampusResearch Planning Agenda, GIRPA, & CCSSEResearch Planning Agenda, GIRPA, & CCSSE

President’s CabinetPresident’s CabinetAcademic Senate Academic Senate Classified SenateClassified Senate

Associated StudentsAssociated StudentsPresident’s Executive Staff President’s Executive Staff Academic Affairs CommitteeAcademic Affairs Committee

School meetings (7)School meetings (7)Chairs Committee Chairs Committee

Dean’s CouncilDean’s CouncilResearch Committee Research Committee

Student Services AdministratorsStudent Services AdministratorsStudent Services CouncilStudent Services Council

See CD document “Research Planning Agenda”See CD document “Research Planning Agenda”

1717

District / Campus CommunicationDistrict / Campus Communication

• Bi-monthly joint meetingsBi-monthly joint meetings• Weekly Research & Planning meetings including Weekly Research & Planning meetings including

all District analysts and CBR(s)all District analysts and CBR(s)• District-Wide Research Committee meetingsDistrict-Wide Research Committee meetings• District representation at campus committee District representation at campus committee

meetingsmeetings• District / campus liaisonsDistrict / campus liaisons• Project Tracker / Research LogProject Tracker / Research Log

1818

BenefitsBenefits

• Collaboration between campus and district Collaboration between campus and district research office has become substantially research office has become substantially stronger and richer. stronger and richer.

• Sophistication of research initiatives has Sophistication of research initiatives has increased. increased.

• Research has been a pathway to integrationResearch has been a pathway to integration• Interactions between student services programs Interactions between student services programs

and academic success are becoming more and academic success are becoming more clearly in focus.clearly in focus.

1919

ChallengesChallenges

• Growing painsGrowing pains• Need to strengthen connection to SLO Need to strengthen connection to SLO

development, measurement, and linkages development, measurement, and linkages to traditional institutional researchto traditional institutional research

• Challenge of new or redefined policies Challenge of new or redefined policies based on new requests, context and based on new requests, context and campus frameworkcampus framework

2020

““Serving Two Masters (or more?)”Serving Two Masters (or more?)”

Director of District IRP: Supervises policy-related Director of District IRP: Supervises policy-related issues, project plans, and weekly meetings issues, project plans, and weekly meetings amongst the analysts in the SDCCD research amongst the analysts in the SDCCD research communitycommunity

Dean of Instructional Services, Resource Dean of Instructional Services, Resource Development, and Research: Supervises day-Development, and Research: Supervises day-to-day operation, fulfillment of campus research to-day operation, fulfillment of campus research requests, and committee work (with exception requests, and committee work (with exception of District-Wide Research Committee)of District-Wide Research Committee)

2121

Measures of ProductivityMeasures of Productivity

• Research activity and strategic involvement in Research activity and strategic involvement in SLO development has skyrocketedSLO development has skyrocketed

• Sophistication of the requests and the Sophistication of the requests and the stakeholders has been increasingstakeholders has been increasing

• Rubric design for rating sophistication of Rubric design for rating sophistication of research requestsresearch requests

• Measures of increased faculty campus Measures of increased faculty campus participationparticipation

• Measures of district-wide benefitsMeasures of district-wide benefits

2222

SurprisesSurprises• Research-based momentum has sustained itself Research-based momentum has sustained itself

through changes in leadership, circumstances, through changes in leadership, circumstances, and policiesand policies

• Similarities between District and Campus Similarities between District and Campus Research OfficesResearch Offices• Exponential increases in workloadExponential increases in workload• UrgencyUrgency• PrioritizationPrioritization• Sharing / exchangeSharing / exchange• Requests for researchRequests for research

2323

Guidelines for ImplementingGuidelines for Implementingthe Research Planning Agendathe Research Planning Agenda

FERPA FERPA GIRPA GIRPAData sensitivity: Access, security, use, disseminationData sensitivity: Access, security, use, dissemination

• Level I:Level I: public information which is highly aggregated, or broadly public information which is highly aggregated, or broadly categorized, such as enrollment figures, transfer rates, or any categorized, such as enrollment figures, transfer rates, or any other institution-wide dataother institution-wide data

• Level II:Level II: General Requests for Research Reports, survey data, General Requests for Research Reports, survey data, and data that are disaggregated, or broken out by categories, to and data that are disaggregated, or broken out by categories, to some extent, such as success rates or student progress at the some extent, such as success rates or student progress at the program levelprogram level

• Level III:Level III: Special Requests for Research Reports and sensitive Special Requests for Research Reports and sensitive information that is highly disaggregated, such as student contact information that is highly disaggregated, such as student contact information, data at the Course Reference Number (CRN) level, information, data at the Course Reference Number (CRN) level, student records student records

(See CD documents “GIRPA” and “GIRPA FAQs”)(See CD documents “GIRPA” and “GIRPA FAQs”)

2424

Questions???Questions???

District Web Site: www.sdccd.net/research/Mesa Web Site: www.sdmesa.edu

Phone: (619) 388-6942 (District) (619) 388-2509 (Mesa)

bgrimes@sdccd.eduyberglan@sdccd.edugconrad@sdccd.edusmun@sdccd.edu

Email:

2525

Operational LogisticsOperational Logistics• The Research Office continues to support District-wide projects The Research Office continues to support District-wide projects

where it is easier to complete two or more colleges’ work.where it is easier to complete two or more colleges’ work.• Projects are shared between the CBR and Research Office to Projects are shared between the CBR and Research Office to

maximize efficiency of time, data collection, analysis, or reporting. maximize efficiency of time, data collection, analysis, or reporting. • CBR is the lead on a project self-contained to the campus and its CBR is the lead on a project self-contained to the campus and its

unique research agenda. (Campus Liaison would support or unique research agenda. (Campus Liaison would support or provide temporary backup)provide temporary backup)

• Essential role of documentation to: 1) enhance learning, 2) allow Essential role of documentation to: 1) enhance learning, 2) allow for constructive review, 3) provide historical perspective for future for constructive review, 3) provide historical perspective for future researchers, and 4) maximize efficiency. Reinforces principle of researchers, and 4) maximize efficiency. Reinforces principle of sharing and teaching among research community. sharing and teaching among research community.

• Secure and shared network space located at each campus for Secure and shared network space located at each campus for downloading and uploading files in a safe and secure manner.downloading and uploading files in a safe and secure manner.

• Continue process of maintaining individual daily “To Do” list Continue process of maintaining individual daily “To Do” list

2626

Research at MesaResearch at Mesa

SSupport the Mesa College mission upport the Mesa College mission

CCreate a culture of evidencereate a culture of evidence

AAssist faculty, staff, & administration with ssist faculty, staff, & administration with researchresearch

RReinforce data-driven decision-makingeinforce data-driven decision-making

EEducate the campus communityducate the campus community

??

2727

Research at MesaResearch at Mesa

CCreate a culture of evidencereate a culture of evidence

AAssist faculty, staff, & administration with ssist faculty, staff, & administration with researchresearch

RReinforce data-driven decision-makingeinforce data-driven decision-making

EEducate the campus communityducate the campus community

SSupport the Mesa College mission upport the Mesa College mission

2828

Research questionResearch question

Research report requestedResearch report requested

Reason and rationale for requestReason and rationale for request

Date research report is neededDate research report is needed

Parameters of research reportParameters of research report

Signatures from requestor, Department Chair, Signatures from requestor, Department Chair, and Dean or Managerand Dean or Manager

Request for Research Report FormRequest for Research Report Form

See CD document “Request for Research Report” formSee CD document “Request for Research Report” form

2929

Web LinksWeb Links

Request for Research Report Form:Request for Research Report Form:http://www.sdmesa.edu/instruction/research/pdf/http://www.sdmesa.edu/instruction/research/pdf/

RESEARCHREQUESTFORM.pdfRESEARCHREQUESTFORM.pdf

Research Planning Agenda:Research Planning Agenda:http://www.sdmesa.edu/instruction/research/pdf/http://www.sdmesa.edu/instruction/research/pdf/

ResearchPlanningAgendaApproved2-27-07.pdfResearchPlanningAgendaApproved2-27-07.pdf

GIRPA:GIRPA:http://www.sdmesa.edu/instruction/research/pdf/DataSensitivityGIRPA.pdfhttp://www.sdmesa.edu/instruction/research/pdf/DataSensitivityGIRPA.pdf

GIRPA FAQs:GIRPA FAQs:http://www.sdmesa.edu/instruction/research/pdf/http://www.sdmesa.edu/instruction/research/pdf/

DataSensitivityGIRPAFAQs.pdfDataSensitivityGIRPAFAQs.pdf