outcomes assessment results - minot state u

Minot State University College of Business, IACBE Annual Report of Learning Assessments, 2011-2012

1

Outcomes Assessment Results For Academic Year: 2011-12

Section I: Student Learning Assessment

Student Learning Assessment for Master of Science in Management

Intended Student Learning Outcomes for MS program in Management :

1. Make managerial decisions that add value to an organization within an environment of risk and uncertainty.

2. Demonstrate strategic and operational leadership skills.

3. Communicate in a manner that enhances the overall effectiveness of an organization.

4. Strategically apply technology in a managerial context.

5. Demonstrate professionalism in both behavior and attitude.

6. Strategically apply entrepreneurship concepts in a managerial context.

7. Strategically apply cohort-particular international/global business concepts in a managerial context.

8. Apply knowledge management tools and techniques in a management environment.

Assessment Tools for Intended Student Learning Outcomes— Direct Measures of Student Learning:

Performance Targets/Criteria for Direct Measures:

1. Rubrics for written business Plan in BADM 598 (Capstone

Experience) Every student learning outcome will have a mean that is at least at the

“Proficient” level (i.e., 3.00 or higher).

2. Rubrics for oral business plan in BADM 598 (Capstone Experience) Every student learning outcome will have a mean that is at least at the


Assessment Tools for Intended Student Learning Outcomes— Indirect Measures of Student Learning:

Performance Targets/Criteria for Indirect Measures:

1. Focus group discussion of graduating studens Consensus that all MSM learning outcomes of the MSM were met.


2

2. Survey of graduating students

Every student learning outcome will have a mean that is at least at the


Summary of Results from Implementing Direct Measures of Student Learning: Performance Target Was…

Met Not Met

1. One student learning outcome reported a mean that was at least at the “proficient” level (i.e., 3.00 or higher). This goal was not met.

X

2. No student learning outcome reported a mean that was at least at the “proficient” level (i.e., 3.00 or higher). This goal was not met.

X

Summary of Results from Implementing Indirect Measures of Student Learning: Performance Target Was…

Met Not Met

1. There was a consensus that the goals of the MSM program were met. This goal was met. X

2. Fifteen of the twenty (75.0%) of the means of the twenty student learning outcomes were at least at the “proficient” level (3.00 or higher). Five means (25.0%) were lower than 3.00. Four of these had means of 2.93 (very close to 3.00); one had a mean of 2.73 Three of these low means were on outcomes that dealt with financial information; this is understandable, given that the only finance course most students have taken is the one in this program. This goal was not met, although the results were very close.

X

Proposed Courses of Action for Improvement in Areas for which Performance Targets Were Not Met:

1. Examine the appropriateness of Student Learning Outcome 6.1

2. In the future, oral presentations of the capstone projects should be evaluated real time by multiple assessors.

3. Revise the capstone project assessment instruments so they are more useful for arrange of capstone projects.


3



Student Learning Assessment for Master of Science in Information Systems

Intended Student Learning Outcomes for MS program in Information Systems :

1. Demonstrate technology skills at a sophisticated level.

2. Develop information systems infrastructures.

3. Communicate an information systems strategy.

4. Provide information systems consulting services.



1. (Rubric for written project in BIT 582, Professional Consulting

(capstone experience project) 95% of MSIS graduate students will perform sophisticated or proficient

on the written project in BIT 582.

2. Rubric for project oral presentation in BIT 582, Professional

Consulting (capstone experience project) 95% of MSIS learners will perform sophisticated or proficient on the oral

project in BIT 582.



1. Graduate Student Perception of Learning Survey 90% of graduate students will perceive a high level of satisfaction from

their learning experience (90% of responses = Sophisticated or Proficient)

2. Employer Survey 80% of employers will be satisfied with MSU/MSIS graduate/employee

(80% of responses = Sophisticated or Proficient)


Met Not Met


4

1. The average score for each respondent was 3.21 out of 4.00; the aveage score for each objective was 3.1 out of 4.0 X

2. The average total score for each respondent was 3.38 out of 4.0; the average total score for each objective was 3.1 out of 4.0

X


Met Not Met

1. Mean =3.70. All 12 respondents had a mean of at least 3.18. Nolearning outcome was scored below 3.0. X

2. This assessment was not conducted this year; the next administration is scheduled for AY 2014-15. NA NA


1. Review the assessment documents to ensure that wording is understandable to all external raters.

2. Continue to work with students throughout the program to ensure all rate at “comprehensive” or “sophisticated” rate.

3. Redesign assessment tools to better fit the variety of types of projects and still capture the required assessment data.

4. Students will be allowed to choose the objectives that match their capstone projects. This will provide more accurate assessment data by eliminating the blank responses or N/A responses.


5



Student Learning Assessment for BS in Accounting

Intended Student Learning Outcomes for BS program in Accounting :

1. Demonstrate knowledge of and apply the principles and procedures of financial accounting.

2. Demonstrate knowledge of and apply the principles and procedures of managerial accounting.

3. Demonstrate knowledge of and apply auditing and attestation theories, standards, and procedures.

4. Demonstrate and apply knowledge of federal taxation, ethics, professional and legal responsibilities, and business law.

5. Integrate skills related to technology, research, problem solving, and communications.

6. Demonstrate knowledge of and apply principles and procedures to the following area, if selected: fraud examination, governmental and not-for-

profit accounting, accounting information systems, managerial accounting, law or auditing.



1. Written Core PBL (CBE) Examination Overall examination average of at least 65%.

2. Program Written Examination (no longer conducted) Graduating accounting majors will obtain an overall examination average

of at least 65%.

3. Course-embedded Assessments (new assessment) Students will earn an overall average of 75%.



1. Graduate Exit Survey

At least a 4.0 (on a 6-point scale) overall average from graduating

students when rating how well the accounting program prepared students

to meet program learning outcomes.


6

2. Alumni Survey (every three years)

At least a 4.0 (on a 6-point scale) overall average from alumni when

rating how well the accounting program prepared alumni to meet program

learning outcomes AND 80% of alumni would recommend MSU to

students who are interested in an accounting career.


Met Not Met

1. Accounting students earned 73.8% overall on the PBL Examination; College of Business students earned 71.8%. X

2. This direct measure is no longer conducted. NA NA

3. The student average on the embedded question assessment was 81.7%, exceeing the goal. X


Met Not Met

1. The overall weighted average was 5.00. The lowest weighted average was 4.44. X

2. Not completed in 2010-11; will be completed in AY 2012-13. NA NA


1. As it is new, the Accounting faculty will work toward improving the embedded assessment measures, despite the strong performance of students on the first administration. Also, the faculty will conduct the alumni survey in AY 2012-13.


7



Student Learning Assessment for BS in Energy Econonics and Finance

Intended Student Learning Outcomes for BS program in Energy Economics and Finance :

1. Meet the financial analysis needs of the energy industry in North Dakota and the Great Plains.

2. Prepare future business managers to meet energy challenges.

3. Provide methods for examining alternative energy sources for their economic and financial suitability in meeting energy users’ needs.

4. Contribute to our nation’s energy independence and security by educating a workforce dedicated to energy economic and financial analysis.



1. Written project and presentation in FIN 459 95% of students will achieve 80% proficiency in analytical and simulation

skills, as measured by rubric

2. Final exam in FIN 459 (to graduating seniors) 80% of students will achieve score of 70% or higher



1. Survey of graduating seniors: Student perception of their ability to

meet program learning outcomes Average of 4.0 on 6-point scale

2. Survey of alumni 2 and 6 years after graduation At least 80% of alumni would recommend program to others.


Met Not Met

1. Students in the EEF program were measured on their project in FIN 459, Advanced Project Evaluation and Risk Analysis, the capstone course for the EEF program. This goal was deemed to have been met.

X


8

2. The exam has been created but has not been given to students, due to the low number of graduates in the first three years. The test was not administered in Spring 2012, due to illness of the instructor and instructor substitution. However, in place, the EEF students took the PBL Examination. They earned higher percentages than other College of Business students on the following components: Accounting (82.9% vs. 78.0%); Finance (79.2% vs. 70.9%); Economic Environment (77.8% vs. 66.5%); ), Social Environment (79.2% vs. 65.7%), and Marketing (81.3% vs. 71.1%).

NA NA


Met Not Met

1. The overall weighted average was 5.09. The goal was met. X

2. This measure is scheduled to be administered in two years (six years after the first graduating class). As the program was started in 2009, the six-year mark has not been reached. Due to the low number (4) of graduates in the program because of its newness , the director is in close contact with program graduates and their employers.

NA NA


1. This program has grown steadily and is now in its fourth year, achieving 39 declared majors. Most of these students are currently in their freshman and sophomore years. Students take most of the courses in this major in their junior and senior years. The director will continue to monitor students’ progress through the direct and indirect measurses as well as informal contact with graduates and employers.


9



Student Learning Assessment for BS in Finance

Intended Student Learning Outcomes for BS program in Finance :

1. Demonstrate knowledge of and apply the principles, tools and techniques for effective and efficient financial management.

2. Demonstrate knowledge of and apply investment strategy and analysis from basic investing to the theory of portfolio construction and risk

management.

3. Demonstrate an understanding of the management, performance, and regulatory aspects of financial instructions and markets.

4. Demonstrate the theories, knowledge, and financial tools necessary to start, grow, and harvest a successful business venture.



1. Written Core PBL (CBE) Examination Overall examination average of at least 60%.

2. Program Written Examination 80% of graduating finance majors will receive a score of 60% or higher.



1. Graduate Exit Survey

At least a 4.0 (on a 6-point scale) overall average from graduating

students when rating how well the finance program prepared students to

meet program learning outcomes.

2. Alumni Survey At least 80% of alumni would recommend MSU’s finance program to

others.


Met Not Met

1. Finance students who took the exam earned 71.6% overall. X


10

2. The three students who took the exam earned an overall average of 71.3%. X


Met Not Met

1. The overall weighted average was 5.71 (n=9); no outcomes had a weighted average of less than 4.0. All students responding to the survey would recommend the program.

X

2. This survey was not completed during the academic year. NA NA


1. Finance program goals have been met. The alumni survey will be administered in AY 2012-13.


11



Student Learning Assessment for BS in International Business

Intended Student Learning Outcomes for BS program in International Business :

1. Apply knowledge of international business at the strategic, tactical, and operational levels in a changing global environment.

2. Apply cross-cultural business skills and expertise.

3. Apply a variety of business and management skills in evaluating domestic and international business and non-business opportunities.

4. Demonstrate effective decision-making, organizing, and leadership skills.



1. Comprehensive Business Exam (focus on COB Core) All International Business majors will score at least 70% on

Comprehensive Business Exam.

2. Rubrics used for evaluating written/oral presentations of capstone

projects

All International Business majors will score at least 80% in written/oral

presentations in BADM 462 (International Business Strategy).



1. Student Satisfaction Survey 90% of International Business majors indicate high satisfaction with

learning.

2. Focus group discussions General consensus that student learning outcomes were met.


Met Not Met

1. The overall average score of International Business majors on the CBE was 59.43%. Five of twenty students (25.0%) scored at least 70% on the CBE. This goal was not met.

X


12

2. Five written projects and sixteen oral presentations were evaluated; none of these scored at least 80%. This goal was not met.

X


Met Not Met

1. 54.2% of International Business majors reported means on the International Business questions at the “Very Good” or “Excellent” levels (33.3% reported “Excellent” and 20.8% reported “Very Good;” 25.0% reported “Fair” and 020.8% “Poor.” While a greater percentage than last year reported at the “Very Good” level, the goal was not met.

X

2. Approximately 25% of international business graduates received employment related to their major field or continued with their education. This goal was not met.

X


1. Student Exit Survey: examine trend analysis for past 5-6 years; Core Committee should examine questions 1-19 and 27a-27f; propose revisions to IB questions to ensure that they align with new program goals and learning outcomes; revise structure of survey.

2. CBE: apparent mismatch between our program and CBE questions. IB curriculum and assessment committees will examine alignment between IB program/SLOs and CBE exam. Consider using CBE as pre- and post-test to show added value.

3. BADM 462 Capstone Project: consider greater emphasis on economics.

4. Department will discuss “engaging experiences” with menu of options. Students may benefit from significant “out of classroom” experiences in lieu of BADM 462, with some caveats, as approved by department chair.

5. Consider additional resources and/or program coordinator for IB program.

6. Refer results to IB Curriculum and Assessment Committee as part of its program review.


13



Student Learning Assessment for BS in Management

Intended Student Learning Outcomes for BS program in Management :

1. Apply management principles at strategic, tactical, and operational levels.

2. Apply management skills from an entrepreneurial perspective.

3. Demonstrate effective decision-making skills in the basic areas of international business, strategy, and operations.

4. Analyze personal leadership style.

5. Demonstrate human relations skills in diverse organizational settings.



1. Written exam developed by CBE (focus on COB Core) All Management majors will score at least 70% on Comprehensive

Business Exam.

2. Rubrics used for evaluating written/oral presentations in BADM 489

(Entrepreneurship and New Venture Creation)

All Management majors will score at least 80% in written/oral

presentations in BADM 489 (Entrepreneurship and New Venture

Creation).



1. Survey of graduating students 90% of Management majors indicate high satisfaction with learning.

2. Focus group of graduating students 80% of Management majors indicate satisfaction at the “Very Good” or

“Excellent” levels.


Met Not Met


14

1. The overall average score of Management majors on the CBE was 64.87%. Nine of twenty-five students (36.0%) scored at least 70% on the CBE.

X

2. Due to the pilot study currently underway, this assessment goal was not measured directly. Due to completion of revisions to the BS curriculum and BS-Outcomes Assessment Plan, this assessment goal has been changed for subsequent years.

NA NA


Met Not Met

1. 66.7% of management majors reported means on the management questions at the “Very Good” or “Excellent” levels (40.5% reported “Excellent” and 26.2% reported “Very Good;” 16.7% reported “Fair” and 6.7% “Poor.” While the results are somewhat satisfying – and representated a slight improvement over the previous year – the goal was not met.

X

2. Approximately 60% of BS-Management graduates received employment related to their major field or continued with their education. Approximatley 90% were employed in related and/or unrelated positions. This goal was not met; however, the Department is very pleased with the overall placement results.

X


1. Student Exit Survey: BADM Department will examine trend analysis over five-year period. Core Committee should analyze selected questions; Management questions should be revised for alignment with program goals and student learning outcomes.

2. CBE: appears to be mismatch between program and CBE measurement. Largest differences in scores are between management and accounting/finance. Could compare our results with all schools instead of just peers; consider using CBE as pre- and post-test. Core Committee could analyze CBE results.

3. BADM 489 capstone simulation: Fall 2011 results are dramatically lower than previous years, most likely due in part to effects of the flood. Continue to use CapSim; examine construct and criterion-related validity. Examine CompXM for match with our outcomes.

4. Placement Report: change goal to “unemployed not greater than the national unemployment rate.”

5. Management students weakest in quantitative/data analysis. Strengthen these components in program.

6. Department will consider “engaging experiences” for students, including out-of-classroom experiences.


15



Student Learning Assessment for BS in Management Information Systems

Intended Student Learning Outcomes for BS program in Management Information Systems :

1. Define the theoretical background to develop, document, troubleshoot and implement Management Information System programs in a variety of

environments.

2. Develop and implement information technology solutions that enhance organizational performance.

3. Gain breadth and depth in the technical aspects of the discipline.

4. Apply Management Information Systems principles and values in a production environment.

5. Plan and prepare for careers or graduate education where a strong Management Information Systems foundation is required.



1. Rubric for written final project (Capstone Integrated Project) 90% of MIS students will perform at Sophisticated or Proficient level on

the written final project in BIT 470.

2. Rubric for oral presentation of final project (Capstone Integrated

Project)

90% of MIS students will perform at Sophisticated or Proficient level on

the oral presentation of final project in BIT 470.



1. Student Perception of Learning Survey 80% of graduating MIS majors will perceive a high level of learning

(sophisticated or proficient)

2. Employer Survey 80% of employers are satisfied with MSU/MIS graduate/employee

(sophisticated or proficient)


Met Not Met


16

1. There were 10 students divided into 2 group projects that were assessed. Goals 1, 6, 15, and 18 were met at the sophisticated level (100%). Goals 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 16, and 17 were met with 50% at the sophisticated level and 50% at the comprehensive level.

X

2. The students in the online section presented their final projects online through the use of Adobe Connect while those in the campus class presented to a group of College of Business faculty. Goals 1, , 8-15, and 18 were met at the sophisticated (100%). Goals 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 16, and 17 were met with 50% at the sophisticated and 50% at the comprehensive.

X


Met Not Met

1. The survey instrument was administered to the students with only 5 students returning the survey. All MIS program goals were met at 80% “sophisticated” or “proficient” levels.

X

2. This goal was not measured this year; it will be next measured in 2014. NA NA


1. Reword objectives to reflect actions.

2. Develop embedded assessment techniues to better capture accurate assessment of each individual rather than groups, for the three courses in the CoB Core.


17



Student Learning Assessment for BS in Marketing

Intended Student Learning Outcomes for BS program in Marketing:

1. Apply marketing analysis and planning and control concepts at the strategic, tactical, and operational levels in an ever changing dynamic

marketing environment.

2. Apply promotional entrepreneurial, communication, and cross-cultural marketing skills and competencies.

3. Apply consumer behavior knowledge in marketing products, services, and ideas.

4. Apply marketing concepts in consumer, industrial, and government markets.

5. Apply marketing skills in evaluating domestic and international marketing environments.

6. Demonstrate effective decision-making, planning, organizing, leading, and controlling knowledge, skills, and abilities.

7. Apply research tools and skills for exploring marketing opportunities and for solving market problems.



1. Comprehensive Business Exam (focus on COB Core) All Marketing majors will score at least 70% on Comprehensive Business

Exam.

2. Rubrics used for evaluating written/oral presentations in BADM 488

(Marketing Strategy)

All Marketing majors will score at least 80% in written/oral presentations

in BADM 488 (Marketing Strategy).



1. Survey of graduating students 90% of Marketing majors indicate high satisfaction with learning.

2. Focus group of graduating students 80% of marketing majors indicate satisfaction at the “Very Good” or

“Excellent” levels.


18


Met Not Met

1. The overall score of Marketing majors on the CBE was 62.86%. Six of sixteen (37.5%) Marketing majors scored at least 70% on the CBE. This goal was not met.

X

2. Five of fifteen (33.3%) capstone projects scored at least 80% on the written projects. An additional three (20.0%) projects scored between 75 and 79%. This goal was not met.

X


Met Not Met

1. 57.7% of all Marketing majors reported means on the marketing questions at the “Very Good” or “Excellent” levels (23.1% reported “Excellent” and 34.6% reported “Very Good;” 26.9% reported “Fair,” and 15.4% reported “Poor.”) This goal was not met.

X

2. Approximately 77% of marketing graduates received employment related to their mjor field or continued with their education. This goal was met.

X


1. Student Exit Survey: Examine trend analysis for five years; revise marketing questions; have CoB Core Committee examine selected questions.

2. CBE: mismatch between program and CBE questions; faculty will examing alignment of CBE with program SLOs; seek broader comparator group than just peers; consider pre- and post-tests to show value added.

3. Quantitative areas are weakest; strengthen program to emphasize budgeting and finance, and quantitative/data analysis areas.

4. Consider “engaging experiences” for students, with menu of options including out-of-class experiences to strengthen applications and relevance.


19



Student Learning Assessment for BAS in Applied Management

Intended Student Learning Outcomes for BAS program in Applied Management:

1. Communicate clearly, effectively, and professionally.

2. Use technology effectively.

3. Comprehend and/or exhibit professional behavior.

4. Demonstrate and apply management skills at the tactical and operational levels.

5. Demonstrate effective decision-making skills in the basic areas of business and operations.

6. Demonstrate human relations skills in diverse organizational settings.

7. Analyze personal leadership styles.

8. Integrate major program knowledge and skills in a capstone experience.



1. Comprehensive Business Exam (focus on COB Core) All Applied Management majors will score at least 70% on the

Comprehensive Business Exam.

2. Rubrics used for evaluating student report on internship experience All Applied Management majors will score at least 80% on their

internship experience.



1. Student Satisfaction Survey 90% of Applied Management majors will indicate high satisfaction with

learning.

2. Focus Group Discussions 80% of Applied Management majors will indicate satisfaction at the

“Very Good” or “Excellent” levels.


20


Met Not Met

1. The overall average score of Management majors on the CBE was 64.87%. Nine of twenty-five students (36.0%) scored at least 70% on the CBE. This goal was not met.

X

2. During the past year, there were no internships completed by BAS-Applied Management students. NA NA


Met Not Met

1. 66.7% of management majors reported means on the management questions at the “Very Good” or “Excellent” levels (40.5% reported “Excellent” and 26.2% reported “Very Good.” 16.7% reported “Fair” and 16.7% “Poor.” While the results are somewhat satisfying – and represented a slight improvement over the previous year - the goal was not met.

X

2. Approximately 60% of BS-Management gradutes received employment related to their major field or continued with their education. Approximately 90% were employed in related and/or unrelated positions. The goal was not met; however, the Department is very pleased with the overall placement results.

X


1. Student Exit Survey: examine trend analysis for past five years.; Core Committee should analyze selected questions; revise management questions for alignment with program goals.

2. CBE: examine differences between management and accounting/finance; mismatch between CBE measures and program, should compare our results with other schools beyond peer institutions; consider using CBE as pre- and post-test for comparison.

3. Placement Report: goal should be changed to: unemployed not greater than the national unemployment rate.

4. Management students are weakest in quantitative/data analysis area. Strengthn these components of the program.


21



Student Learning Assessment for BAS in Business Information Technology

Intended Student Learning Outcomes for BAS program in Business Information Technology :

1. Demonstrate the technical skills to develop, document, troubleshoot, and implement information technology programs in a variety of

environments.

2. Develop and implement information technology solutions that enhance organizational performance.

3. Gain breadth and depth of knowledge in the technical aspects of the discipline.

4. Apply technical Business Information Technology principles and values in a production environment.



1. Rubric for written proposal in BIT 497; Internship Coordinator

Evaluation

90% of BAS ABIT students will perform at Sophisticated or Proficient

level on their written proposal.

2. BIT 497; Internship Supervisor Survey 80% of internship supervisors will be satisfied with MSU/BAS ABIT

intern (80% of responses = Sophisticated or Proficient).



1. Student Perception of Learning Survey 80% of graduating BAS ABIT majors will perceive a high level of

learning (80% of responses = Sophisticated or Proficient).

2. Employer Survey 80% of employers will be satisfied with BAS ABIT graduate/employee

(80% of responses = Sophisticated or Proficient).


Met Not Met


22

1. No students completd an internship during this assessment period, therefore, no written proposals were submitted.

NA NA

2. No students completed internships, therefore, no data were gathered to assess the satisfaction of the supervisors. N NA


Met Not Met

1. Student surveys were sent to students completing the BAS-ABIT program, but no responses were returned. NA NA

2. This goal was not assessed during this assessment period. The next assessment period will be reported in AY12-13 assessment report.

NA NA


1. Modify assessment documents to reflect the new curriculum.

2. Implement method to ensure that student satisfaction survey is completed and returned.

3. Develop assessment method to assess capstone project when internship is not completed.

outcomes assessment results - minot state u

Documents