assessing institutional effectiveness through the electronic portfolio

Download Assessing Institutional Effectiveness Through the Electronic Portfolio

Post on 10-Jan-2016




0 download

Embed Size (px)


Assessing Institutional Effectiveness Through the Electronic Portfolio. Lynda Barner West, Ed. D. Associate Vice President – Technology and Information Services Andrea Beranek, M.A. Assistant to the Vice President for Academic Affairs, Program Support. Institutional Background. - PowerPoint PPT Presentation


  • Assessing Institutional Effectiveness Through the Electronic Portfolio

    Lynda Barner West, Ed. D.Associate Vice President Technology and Information ServicesAndrea Beranek, M.A.Assistant to the Vice President for Academic Affairs, Program Support

  • Institutional BackgroundRoman Catholic Sisters of MercyUrban CampusOakland Area of Pittsburgh Pennsylvania81% undergraduates commute90,000 college students in regionPrivate Masters Colleges and Universities I2000 Students80% Undergraduate95% Women at the Undergraduate Level22% minority55% First Generation College

  • Why Assessment Issues Are at the Forefront Now?New Core Curriculum ImplementationFall 2003 implementation of new undergraduate Core Curriculum Middle States StandardsAssessment PlanShift from strongly recommended to requiredStudent learning outcomes

  • Charge from VPAASystematic assessment of student learning is essential to monitoring quality and providing information that leads to improvementDevelop a workable plan that addresses individual student and cohort learning outcomes at institutional, program, and individual course levels

  • Beyond Data Collection

  • Sub CommitteeFacultyDirector of Core Curriculum ImplementationChair of Curriculum CommitteeChairs of Skills Integration Sub-committeesWriting, research, technology, quantitative reasoning, speaking Academic Affairs StaffAVP Technology and Information ServicesAst to VP Academic Affairs

  • The Assessment Puzzle Individual GrowthCore EffectivenessMajor SequenceInstitutional Effectiveness

  • Four LevelsIndividual StudentBaselineProgress Through the CurriculumPerformance at ExitCore CurriculumSkillsConceptsProgram and MajorSkillsConceptsSequencesInstitutional LevelOverall Effectiveness

  • Levels Across TimeIndividualCoreProgramInstitutionEntry Baseline End First Year End Second Year End Third Year End Fourth Year

  • Carlow's Achievement Tracking ePortfolioCareer LiteracyCritical ReasoningQuantitative ReasoningResearch Service LearningSpeakingTechnologyWriting


  • Conceptual Design ConsiderationsUse and UsabilityWhoIndividual StudentFaculty CommitteesInstitutional ReviewersHow IndividuallyLocalRemote

  • Access and SecurityLevelsIndividual Portfolio ContributorViewer of GroupsFacultyAdvisorAssessorSystems Administrator

  • Views of DataContributor Individual ViewAssessorSelection of Statistical Sample or CohortAdministratorDatabase Layer

  • Portfolio User Roles And Views StudentIndividual and DetailedAdvisorGroup, Mid Level, and Across Many TopicsFacultyIndividual and Associated With a CourseAssessorGroup, Aggregate, AnonymousAdministratorThe Behind the Scenes Technical Operations

  • Evaluating Existing Models and ProductsNotebookPhysical NotebookA digital notebook (e.g. CD)Web EnvelopeStudent has a web site and required elementsWeb TemplateFramework with categories of items and slots

  • Meeting Our NeedsInstitutional NeedsQuery the ContentsSelect SamplesReport on ContentsFor AdvisorsFor Assessors

  • Evaluation ConsiderationsEase of use regardless of roleEase of maintenance (low support requirements in IT)View as individualView as advisorView as curriculum assessorStandard ReportsAd hoc queriesCostPersistence of data

  • Technical overviewWindows server 2003Underlying Database is Sequel serverCold Fusion is used to talk to the databaseInformation is imported from the backend administrative database Accounts are maintained through cold fusion to create the person record rather than creating them as windows users on the server - account is not a windows account Each user has a unique login and directory spaceEach user has a person record in the database

  • Front EndWritten in Cold Fusion codeCFML (cold fusions markup language)HTML (standard hypertext)Password encrypted by Cold Fusion in the databaseUse Cold Fusion to create filing areas (directory maintenance) and other administrative tasks (account creation and storage area)Not using windows authentication

  • TemplatesInstitutionalSet by assessorHighly flexiblePersonalSet by individualHighly flexible

  • TemplateDescription of Expected ArtifactDesired Student Learning OutcomesEligible Courses

  • Brief Tour

    Patent Pending

  • ReportsWritten in Cold FusionQueries in sequel and pasting them into Cold Fusion

  • SecurityAccessible via the internetTurned off directory browsingOwner can make items public (sets flag)Queries pick up the template but only those items that are marked public are availablePasswords encrypted (via Cold Fusion routines)

  • Levels of accessAdministrative 1 create accounts and directories, manage batch processing Administrative 2 add and inactivate template for the person, assign templates to students, reset passwordHelp Desk Password maintenanceInstitutional Assessor reporting across all portfoliosAdvisor / faculty see assigned studentsDesignation located in administrative database and importedIndividual sees own materials and advisor comments

  • Data Analysis IssuesHow to use the data to create informationIndividual progressImpact of Core CurriculumProgram effectiveness in the MajorHow to use the data to informCurriculumInstitutional practice

  • In ProcessRubric DevelopmentArtifact DescriptorsEvaluative MethodologyFocus deeply and from multiple views on select student learning outcomesNarrow and deep vs wide and shallow

  • Other Critical IssuesFaculty buy inFaculty trainingStudent buy in and training

  • Additional ResourcesAAHE Electronic Portfolio Project, T.A. and K.P. Cross. Classroom Assessment Techniques: A Handbook for College Teachers. San Francisco: John Wiley, 1993.Cambridge, Barbara L. Ed. Electronic Portfolios: Emerging Practices for Students, Faculty, and Institutions. Edwards, N., C. Heider, and R. Port. Common Rubrics Approach. Washington, DC: AACTE, 2001.Huba, Mary E. and Jann E. Freed. Learner Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning. Boston: Allyn and Bacon, 2000.

  • Resources (cont)Palomba, C. A. and T. W. Banta. Assessment Essentials: Planning, Implementing and Improving Assessment in Higher Education. San Francisco: Jossey-Bass, 1999.Siegal, Michael J. The College Student Experiences Questionnaire: Assessing Quality of Effort and Student Gains. First Year Assessment, May 10, 2003.- - - - -. Primer on the Assessment of the First College Year, Brevard, NC: Policy Center of the First Year of College, 2003.Student Learning Assessment: Options and Resources. Philadelphia: Middle States Commission on Higher Education, 2003.Suskie, L. Assessment to Promote Deep Learning: Insights from AAHEs 2000 and 1999 Assessment Conferences. Washington, DC: American Association for Higher Education, 2001.

  • Resources (cont)Swing, Randy Ed. Proving and Improving: Strategies for Assessing the First College Year (Monograph No. 33). Columbia, SC: University of South Carolina, National Resource Center for the First-Year Experience and Students in Transition, 2001.Walvoord, B. E. and V. Johnson-Anderson. Effective Grading: A Tool for Learning and Assessment. San Francisco: Jossey-Bass, 1999.Yogan, Lissa. Using Qualitative Methods to Develop Faculty as Stakeholders in Assessment: A Case Study. First Year Assessment Listserv, September 7, 2001.

    List serv on Assessment Policy Center on the First Year of College FYA-LIST@LISTSERV.SC.EDU

    Web templateProsThinking throughProcess buy inAccess controlTraining issues


View more >