category 7 - southern arkansas university

Southern Arkansas University – Systems Portfolio – June 1, 2013

Category Seven: Measuring Effectiveness Page 93

AQIP Category Seven: MEASURING EFFECTIVENESS examines how your organization collects,

analyzes, distributes, and uses data, information, and knowledge to manage itself and to drive

performance improvement.

Processes related to Measuring Effectiveness vary in their level of maturity. Processes related to

selecting, managing, and distributing data related to programs, planning, and improvement (7P1 and 7P2)

are aligned. These processes are stable, well-managed by the University’s vice presidents, and

continuously improved in collaboration with affected units. Stakeholders understand the processes, have

input in the selection and analysis of data, and share results. Results related to these processes track

progress on the University’s key strategic and operational goals. Processes for selecting, managing, and

distributing data at the department level (7P3) are systematic, with data collected through repeatable

processes that are being evaluated for improvement. Preparation of the University’s first systems

portfolio, however, revealed an obvious area for improvement—a number of process teams discussed

ways to improve access to institutional information and recommended improvements related to collection

of data for the annual faculty performance reviews and related to centralizing online access to more

institutional data.

Processes related to analyzing and sharing data at the institutional level (7P4) are aligned. These

processes are stable, well-managed by the University’s vice presidents, and continuously improved in

collaboration with affected units. Institutional analysis is expanding to incorporate measures required by

the state’s new performance funding system. The Vice Presidents Council (VPC) provides key leadership

in analyzing and sharing information with stakeholders.

Processes related to determining the needs for comparative data (7P5) are systematic. These processes

provide valid, comparable, longitudinal data for institutional and unit planning. Processes for ensuring

that department and unit goals align with organizational goals (7P6) are also systematic. The VPC, deans,

and directors work together to ensure that goals, objectives, and action steps address University goals.

Processes for ensuring timeliness, accuracy and security of information system (7P7) are systematic.

Procedures for internal control of information are well-managed, under the oversight of the vice president

for finance and the director of information technology services.

PROCESSES

7P1. How do you select, manage, and distribute data and performance information to support your

instructional and non-instructional programs and services?

Southern Arkansas University uses multiple methods to select, manage, and distribute data and

performance information to support instructional and non-instructional programs and services. The

University selects data to be collected based on the informational needs of the data users. These data

needs may be defined by strategic goals, state and federal requirements, specialized accreditation

requirements, and general education and program assessment requirements. In addition, departments may

identify data needed to effectively manage work processes and measure departmental operating goals.

As previously discussed in Category 1, the University has clearly defined learning goals and a systematic

process for assessing student learning. The Office of Institutional Effectives (OIE) compiles assessment

data and prepares an annual report, which is maintained and shared as a LiveText project. As previously

discussed in Category 1 and Category 6, the University has a systematic process for assessing support

programs and services that is under the oversight of program directors and the Vice Presidents Council.

Table 7-1 provides a list of data used to support instructional and non-instructional programs and services.



Table 7-1: Data Supporting Instructional and Non-instructional Programs and Services

Data Selected Managed By Distribution Method

Admissions data Dean of Enrollment Services

EMAS platform

Daily enrollment and weekly application

reports shared within unit and with VPC

Student information: ACT scores,

transcripts, current schedule

Institutional Research

Registrar

POISE database

Online access for faculty and staff for

individual students, specific reports

available upon request

Student schedules Information Technology Services

Campus Connect platform

Online access for students, faculty and

staff

Instructional resources, student

grades, online instruction, course

embedded assessments

Information Technology Services

Blackboard LMS

Online access for students, faculty, and

staff

Course evaluations by students Office of Human Resources

Blackboard LMS

Results of course evaluations distributed

to faculty, chairs, deans

Student learning outcomes –

program level

Assessment Review Council and

OIE

LiveText web-application

Shared online access

Student learning outcomes –

general education

General Education Committee

and OIE


Shared online access

Student success rates in transitional

classes

Director of Developmental

Studies

Shared within division, with VPC;

summary results posted on website

Student success rates for

supplemental instruction program

Director of Student Support

Services

Shared within division, with VPC; report

posted on website

Tracking data for early alerts Director of Early Intervention

Services

Shared within division, with VPC

ACT scores for incoming students Dean of Enrollment Services

Campus Connect platform

Advising folders and online access for

advisors

Library usage data Director of Library Shared within division, with VPC

Program reviews Academic Department Chairs Shared with department and VPAA;

submitted to ADHE

Specialized accreditation reports Academic Deans Shared within program and with VPAA

and president; submitted to accrediting

agency

Continuous Quality Improvement

Initiatives

AQIP Council and AQIP

Executive Council

Updates shared through council meetings,

faculty and staff meetings; annual reports

on HLC website

NCAA student athlete reporting:

enrollment, retention, grades,

graduation rates

Athletic Director Shared with VPC, submitted to NCAA

Institutional Fact Book: student

demographics, enrollment, SSCH,

ACT scores, retention, degrees

awarded, faculty composition

Institutional Research Shared with VPC, deans, department

chairs, and directors, Institutional Fact

Book posted on website

7P2. How do you select, manage, and distribute data and performance information to support your

planning and improvement efforts?

Addressing Core Component 5D

SAU uses multiple methods to select, manage, and distribute data and performance information to support

planning and improvement efforts. The University selects data based on the information needs of the

institution that are defined by strategic goals, state and federal requirements, and specialized accreditation

requirements, with a commitment to collecting data that helps the University improve performance.

http://web.saumag.edu/developmental/

http://web.saumag.edu/support/si/

http://web.saumag.edu/its/ir-information/




Planning for continuous quality improvement takes place at multiple levels. At the college and

department level, deans and department chairs base planning on enrollment trends, student assessment

data, course and faculty evaluation data, requirements for specialized accreditation, and feedback from

student satisfaction surveys. At the institution level, strategic planning incorporates key performance

indicators, such as enrollment, retention, and completion rates; local student satisfaction surveys; and

recommendations emanating from continuous quality improvement initiatives projects. The University

also relies on comparative student satisfaction data provided by the NSSE and SSI to identify areas for

improvement. Table 7-2 identifies data that is used to support planning and improvement. (5D)

Table 7-2: Data that Supports Planning and Improvement

Data Selected Managed By Distribution Method

Strategic plan Strategic Planning Council, VPC,

president, and Board of Trustees,

Submitted to ADHE, shared with deans and

directors, goals posted on University

website

Blue and Gold Vision President and SAU Foundation Distributes regular reports on progress and

giving. Updates posted on Foundation

website.

SAU Budget Book VP Finance Budget book is distributed yearly to deans,

vice presidents, president and board of

trustees

Budget reports VP Finance

POISE software

Online access to current budget activity and

balances available for department chairs,

deans, directors, faculty, administrators

Admissions and enrollment

data

Dean of Enrollment Services and

Registrar

Office of Admissions sends daily

enrollment and weekly application reports

to VPAA. Registrar sends reports to ADHE.

Retention and progression data VPSA, Institutional Research

(IR)

VPSA distributes yearly and semester

reports to deans, VPC, and directors. IR

sends reports to IPEDS, ADHE.

Graduation rates Registrar, VPAA Registrar distributes semester reports to

deans, VPAA.

Student diversity Office of Multicultural Affairs Distributes yearly minority report to VPC

and submits to ADHE.

Safety and security data University Police Department Cleary reports distributed to VPC and

available on the University website.

Dining surveys Aramark Results provided to VPSA and VPC

Occupancy rates in residential

living

VPSA Results shared with dean of students, VPC

Housing exit surveys VPSA Results shared with dean of students, VPC

Cost containment initiatives VP Facilities Results shared with VPC and available on

the University website

Student learning outcomes OIE


Shared online access and summary reports

provided for VPAA

Course evaluations by students Office of Human Resources

Blackboard LMS

Results distributed to faculty, chairs, deans,

VPAA

Faculty composition Office of Human Resources Results distributed to VPC

Faculty and staff retention Office of Human Resources Results distributed to VPC

Average faculty salaries Office of Human Resources Results distributed to VPC

Faculty peer evaluations Office of Human Resources

Blackboard LMS

Results distributed to faculty, chairs, deans,

VPAA

Staff evaluations Office of Human Resources Results distributed to staff member,

supervisor

Faculty annual summary of

professional activity

Office of Human Resources Results distributed within departments to

facilitate peer review, filed in HR

http://web.saumag.edu/aqip/strategic-plan/

http://www.saufoundation.org/vision/

http://web.saumag.edu/police/clery/

http://web.saumag.edu/administration/cost-containment/



Promotion and tenure

documents

Office of Human Resources and

VPAA

Recommendations forwarded from VPAA

to president and Board of Trustees

Advising surveys Dean of Enrollment Services Results are distributed to VPSA and

advising director

BAM Orientation surveys Dean of Enrollment Services Results are distributed to VPSA,

admissions, and advising director

Continuous Quality

Improvement Initiatives

AQIP Council and AQIP

Executive Council

Updates shared through council meetings,

faculty and staff meetings

NCAA student athlete

reporting: enrollment,

retention, grades, graduation

Athletic Director Shared with VPC, submitted to NCAA

SAU Foundation Report Director of Foundation Distributed to president and financial

reporting posted on website

Institutional Fact Book: student

demographics, enrollment,

SSCH, ACT scores, retention,

degrees awarded, faculty data

Institutional Research Shared with VPC, deans, department chairs,

and directors, Institutional Fact Book

posted on website

IPEDs: enrollment, student

financial aid, degrees awarded,

retention, graduation rates,

institutional resources

Institutional Research Submitted to U.S. Department of

Education; online access for VPC, extracted

data provided as requested

Annual state reporting,

including performance funding

data

Institutional Research Submitted to ADHE, shared with VPC;

reports distributed by ADHE to all

institutions

National Survey of Student

Engagement

VPSA Reports shared with VPC, distributed to all

departments

Noel Levitz Student

Satisfaction Inventory

VPSA Reports shared with VPC, distributed to all

departments

In 2009, the State of Arkansas adopted a performance funding system (PFS) for institutions of higher

education. This model is based on outcomes, such as retention, degree completion, and job placement,

rather than on inputs, such as enrollment. By adopting this model, the state is rewarding effective and

efficient management, which is essential as institutions face increasing demands on higher education and

reductions in state funding. The state will phase in the PFS beginning in the next fiscal year, with 95% of

funding based on established needs and 5% based on performance. Original legislation called for

allocations to shift by 5% annual increments until funding reached 75% for need and 25% for

performance. Recent legislation, however, froze future performance funding at 10% until all institutions

reach needs funding at the 75% level. As presented in 7R3, SAU achieved the performance targets for the

first measurement year and received the allocated performance funding. Table 7-3 briefly summarizes the

mandatory and optional measures included in the Arkansas PFM. (5D)

7P3. How do you determine the needs of your departments and units related to the collection,

storage, and accessibility of data and performance information?

Table 7-3: Arkansas Performance Funding Measures

Mandatory

Measures

4 point max

Optional Measures

6 point max

Compensatory

Measure

1 point max

Bachelor Degrees Minority Credentials Non-Traditional Credentials Students Receiving Pell

All Credentials Remedial Credentials Transfer Student Credentials

STEM Credentials High Demand Course Completions

Progression of Cohort External Grants and

Awards Expenditures

Regional Critical

Need/Economic Development

http://www.saufoundation.org/about/financials/

http://www.saufoundation.org/about/financials/


https://www.dropbox.com/s/ia7ibiscex2j4vt/University%20Performance%20Report%20Card_2013-4-22_FINAL-1.pdf



Each department or unit plays a role in determining their own needs for data and performance

information. All faculty and staff have access to the University’s primary data collection and sharing

systems—POISE and Campus Connect—which provide access to current student data and budget

information. All staff members in the Office of Enrollment Services has access to EMAS, the

University’s data collection and sharing system for enrollment management. All faculty advisors have

access to the degree audit program. All stakeholders have access to the University’s Fact Book, which is

accessible on the University website. The Fact Book contains historical data related to enrollment,

retention, degrees awarded, student demographics, and faculty composition, as well as comparative data

for other state institutions. Any department may request data from the Institutional Research, which

maintains IPEDS data and state reporting data, or from the registrar, who maintains the POISE database

related to current students.

Recommendations for improvements in data collection, storage, and accessibility usually emerge from

dialog between department chairs, deans, committees, quality improvement teams, and administrators.

Table 7-4 summarizes recommendations and solutions to recent data needs.

Table 7-4: Methods for Determining Department-level Information Needs

Recommended by Need Solution

Faculty and department

chairs

Need for online resources to support advising Degree audit program

Assessment Review

Council

Need for integrated online sharing of assessment

reports

LiveText portfolios for

assessment

Action Project team Need for integrated online sharing of general

education assessments

LiveText portfolios for general

education assessment

AQIP coordinating team Need for online data sharing for systems portfolio SugarSync cloud sharing

College of Education

NCATE team

Need for online data sharing for preparing NCATE

report

Drop Box cloud sharing

Office of Institutional

Effectiveness

Need for online data sharing for preparing state

program review

LiveText portfolio for program

review

Department chairs Need for online tools to collect and compile annual

faculty performance reviews

Assigned to Action Project

team for solution

Department chairs Need for better system for student advising Assigned to Action Project

team for solution

Collection and accessibility of advising information has been the focus of recent dialog between

department chairs and the administration. The University’s degree audit program assists advisors in

reviewing degree requirements and courses taken to track a student’s progress toward graduation. Using

degree audit, advisors can better identify students satisfying associate degree requirements—one of the

performance indicators included in the state’s performance funding model. Last year degree audit

identified 75 additional students who had completed associate degree requirements. However, advisors

have encountered problems using the degree audit program, particularly in evaluating transfer students.

Department chairs communicated these concerns, as well as other issues with the advising process, to the

administration. To provide immediate relief for a clearly-identified problem, the University adjusted

planned staffing to add a half-time transcript analysis evaluator. In addition, the University launched a

new action project in fall 2012 to identify and implement strategies for improving degree audit and the

advising process.

As discussed in 1I1, the University has significantly improved the processes for collecting, storing, and

accessing data related to student learning outcomes. To implement these processes campus-wide, the

University adopted the LiveText web-based application as SAU’s platform for collecting general

education and program assessment data.




Preparation of SAU’s first systems portfolio revealed areas for improvement in the collection and

accessibility of data. Several teams discussed ways to improve the process for collecting and compiling

data for faculty performance reviews, which resulted in an action project to identify online solutions for

collecting and compiling this data. Many teams discussed the need for centralizing online access to more

institutional data. For example, the team collecting data about faculty research (Table 2-4) reported that

scholarship data was difficult to aggregate. SAU recognizes this as an opportunity for improvement.

7P4. How, at the institutional level, do you analyze data and information regarding overall

performance? How are these analyses shared throughout the institution?

Addressing Core Component 5D

At the organizational level, SAU’s senior administrators participate in state meetings and conferences

with legislators and the ADHE to discuss significant educational issues related to the new performance

funding system and related to the University’s needs-based funding. SAU was a member of the four-year

planning committee that made recommendations related to developing the performance funding process.

The University collects standardized data and submits reports for the Integrated Postsecondary

Educational Data System (IPEDS). The University provides annual reports to the ADHE, including

mandatory state reporting, performance funding data, and the University’s annual budget. Senior

administrators review IPEDS and ADHE summary reports to identify areas of strength and opportunities

for improvement. For example, the most recent reports indicated that SAU’s faculty composition is more

than 100 percent of that for comparable schools, but classified and professional staff composition ranges

from 40-60 percent. Faced with budget reductions for 2012-2013, the University chose to implement a

reduction in force related to non-tenured faculty rather than staff. (5D)

Since enactment of performance funding in 2009, SAU has worked aggressively to monitor and improve

measures identified in Table 7-3. For example, SAU recognized an immediate need to improve retention

and remediation. In response, a continuous quality improvement team was appointed to design and

implement improvements; this resulted in redesign of the transitional development curriculum and

creation of the Office of Early Intervention Services. These improvements are further discussed in 1R2.

As another example, in May 2013 the VPC appointed a team to review all 810 students who did not make

satisfactory progress. The team analyzed academic records, developed a plan for each student to get back

on track through summer school, and assisted students in finding summer jobs to pay for the classes. (5D)

At the institutional level, senior administrators analyze data and share information on overall performance

through formal and informal processes. The University distributes the Fact Book, which contains current,

longitudinal, and comparative key performance indicators, to senior administrators, deans, directors,

department chairs, and the library. The Fact Book is also posted on the SAU website and can be accessed

by any stakeholder. The president presents regular reports about University performance to the Board of

Trustees, Faculty Senate, Staff Senate, SAU Foundation Board of Governors, and at faculty and staff

meetings. Informally, senior administrators meet regularly to analyze data and information about

performance. Table 7-5 lists significant institutional data that is analyzed and distributed. (5D)

Table 7-5: Institutional Data Analysis and Distribution

Analysis by Performance Data Reporting and Distribution

President Foundation fundraising SAU Foundation Board of Governors

Updates on capital projects Board of Trustees, legislature, ADHE

Operational data for all divisions Board of Trustees with recommendations

Performance funding report Board of Trustees, VPC, deans, directors,

faculty and staff



VP Academic

Affairs

Annual college assessments Deans, chairs, VPC

Summary of pending legislation, new acts Legislative liaisons, president, VPC

VP Administration Summaries of course evaluations by students Faculty members, department chairs and

deans

Summaries of peer evaluations Faculty members, department chairs and

deans

Summaries of staff evaluations Staff member, appropriate VP

VP Student Affairs Enrollment, retention, and graduation data VPC, deans and directors

Housing applications and occupancy VPC, dean of students

Annual departmental assessments Department heads, VPC

Support program assessments (BAM, SI) Groups administering program, VPC

NSSE, SSI VPC, deans, directors, department chairs

VP Finance Comparison of budget to actual expenditures President, VPC, Deans, Directors

ADHE financial reporting ADHE, legislature, VPC, public

information

Grant reporting ADHE, USDED, ADE, appropriate grant

project coordinator, and senior

administrator

University comprehensive financial

statements

VPC, Board of Trustees, general public,

ADHE, DFA, division of Legislative Audit

Title IV Student Financial Aid data USDED

VP Facilities Quarterly and annual job performance ratings Supervisors

In-house construction project updates Senior staff and supervisors in physical

plant

Energy usage Senior staff in physical plant, VPC

Safety compliance reports Senior staff at physical plant; VPC

Since transitioning to a focus on quality improvement, the University has increasingly shared information

about continuous quality improvement initiatives (CII). The AQIP Council, which includes over 25

faculty and staff members, meets each month to share progress reports on action projects and other CIIs.

The three-member AQIP coordinating team meets each month with the AQIP Executive Council to share

these updates. The coordinating team also presents regular reports at faculty and staff meetings.

7P5. How do you determine the needs and priorities for comparative data and information? What

are your criteria and methods for selecting sources of comparative data and information within and

outside the higher education community?

The University determines comparative data needs and priorities by reviewing key processes that are

essential to attaining SAU’s mission, vision, and strategic plan. Comparative data is primarily derived

from reporting provided by the state (ADHE) and from Federal reporting (IPEDS). These comparative

and validated sources are readily available when needed, but these sources are often one year behind the

current calendar year. Within the full-composition of state institutions, SAU primarily compares our data

to peer institutions within the state—institutions with similar levels of enrollment and similar programs.

SAU also participates in ACT and CSRDE studies that gather retention and graduation data on cohorts.

That data is then compared to participating institutions of our selection.

Comparative data is required for the ten-year reviews of academic programs coordinated by ADHE. State

reporting and IPEDS provide comparative data related to enrollment, retention, graduation, etc.

Additional sources of comparative data are needed to evaluate student and employee satisfaction. To meet

this need, the University uses collects data on a three-year cycle through the National Survey of Student

Engagement (NSSE), the Noel Levitz Student Satisfaction Inventory (SSI), and the Chronicle of Higher



Education survey of ―Greatest Colleges to Work For.‖ Results of these surveys are linked to their

respective normalized data and provide comparison with peer groups and national averages.

The Arkansas performance funding score card also provides comparative data. The first score card was

recently released by ADHE and was presented at the April 26, 2013, meeting of the Arkansas Higher

Education Coordinating Board.

7P6. How do you ensure department and unit analysis of data and information aligns with your

institutional goals for instructional and non-instructional programs and services? How is this

analysis shared?

As described in 8P3, the SAU Strategic Planning Council communicates the University’s strategic goals

to the colleges, departments, and divisions. In response, each unit develops action plans to support the

achievement of the goals and objectives. Each unit also selects measurable outcomes associated with the

action plans. The University is completing the first-year of a new three-year strategic planning process,

and units are currently preparing progress reports. Table 7-6 provides methods in place to ensure that data

analysis and information aligns with organization goals and describes how unit results are shared.

Table 7-6: Alignment of Unit and Organizational Goals

Administrator Action Assurance Dissemination

Instructional Goals

VPAA Department chair submits strategic

plans to deans, VPC

VPC review Analysis shared with dean

Budgets linked to strategic plan,

submitted by chairs through deans

VPAA reviews and

forward to VPC

Budget and analysis shared

with deans, chairs

Fact Book used as basis for enrollment

and graduation data by academic unit

Fact Book maintained by

IR and reviewed by VPC

Analysis shared with dean

Projected enrollment data included in

proposals for new courses and

programs

Reviewed by Academic

Affairs Committee and

VPC for budget

implications and need

Minutes of AAC

published, new programs

reviewed by Board of

Trustees

Personnel requisitions to substantiate

need and budgeting for new positions

Reviewed by VPAA and

HR

Approved positions

advertised through HR

Department chair reviews faculty’s

development plan and prepares

performance evaluation

Reviewed by dean and

VPAA

Individual meetings for

evaluation confidentiality

Requests for library resources Revised by dean, librarian,

library committee

Acquisitions available on

library website

President Reviews and prioritizes proposed

quality improvement initiatives

Reviews by AQIP

Executive Council

New CII presented at


Non-Instructional Goals

VP Finance Financial goals and objectives must

provide fiscal support to University

and data necessary for compliance

VPC reviews Analysis distributed to

Board of Trustees, ADHE,

auditors

VP Student

Affairs

Directors submit strategic plans to

VPSA

VPC reviews Analysis shared with

directors

Budgets linked to strategic plan,

submitted by directors

VPSA reviews and

forward to VPC

Budget and analysis shared

with directors

VP

Administration

Reviews state and federal updates

related to HR and updates handbooks.

VPC reviews Handbooks on website,

provided to new

employees

Reviews strategic planning process

with SPC and Administrative Council

VPC reviews Analysis provided at

faculty/staff meetings, and

to units and departments



7P7. How do you ensure the timeliness, accuracy, reliability, and security of your information

system (s) and related processes?

Each department at SAU has a responsibility to collect and enter data accurately and in a timely manner.

Student data initiates in the Admissions Office, where staff members enter student admission data into

EMAS, the enrollment management system. After admission, student data is transferred by the Office of

the Registrar into the POISE system. The Registrar also enters new courses and programs, which have

been approved by Academic Affairs Committee and by ADHE, into the POISE system. Course

information is maintained and schedule information updated by the Office of Academic Affairs with data

integrated from the POISE system into Campus Connect, the platform for registration management.

Faculty verify rosters and enter and verify grades through Campus Connect, from which grades are posted

onto academic transcripts in the POISE system.

Reporting of mandatory state and federal data is centralized in the Institutional Research. Access to the

data is restricted, primarily for security purposes. Specialized training is necessary to query, extract, and

format valid, meaningful data from the POISE system. The ITS staff, including the director of

institutional research (IR), are well trained to effectively and accurately use the system. Data compiled for

state and federal reporting are summarized in the Institutional Fact Book, which is posted on the website.

When requested, IR extracts and provides data need by the offices, departments, and colleges across

campus. Integration of some data, particularly data which related to graduate studies, currently lies

outside the current parameters for student data collection for tracking retention and enrollments. The new

upgrade for the EMAS software will allow greater integration of POISE and EMAS and incorporate the

graduate department into the system.

Information Technology Services supports and manages the software and hardware necessary for POISE,

Campus Connect, and Blackboard, the University’s learning management system. Many controls are in

place for security, prevention of erroneous data entry (data validation), and error report checks after data

entry. Campus data, including budgets and student records, are centralized in the POISE database system.

The POISE system is based on Open VMS, one of the most secure operating systems in the world.

OpenVMS and POISE are very stable less expensive than similar systems, requiring less staff to manage.

SAU’s financial information system, under the oversight of the vice president for finance, incorporates

internal control procedures appropriate for higher education. These internal controls are reviewed

annually by the Arkansas Division of Legislative Audit to determine the audit procedures that will be

applied in order to express an opinion on the financial statements. Internal controls extend to control over

the administrative systems and student information system, which are centrally managed by director of

information technology services. Both systems have generator backup systems that automatically start in

the event of loss of power. This configuration provides high availability for our business critical systems.

ITS also monitors system performance and provides downtime notification. SAU has 24/7 access,

uninterruptable power, backups (regular and off-site disaster recovery backup) of the administrative

system. The security for SAU’s system requires user authentication, application access, off-site disaster

recovery backup, intrusion detection/monitoring, SSH encryption, and web site encryption.

VP Facilities Crosswalks organizational goals with

Blue and Gold Vision campus

development plan

VPC reviews Construction progress

updates for faculty, staff,

and Board of Trustees

President Reviews and prioritizes proposed

quality improvement initiatives

Reviews by AQIP

Executive Council

New CII presented at



Category Seven: Measuring Effectiveness 2

RESULTS

7R1. What measures of the performance and effectiveness of your system for information and

knowledge management do you collect and analyze regularly?

Table 7-7 lists the key performance indicators for measuring effectiveness.

Table 7-7: Key Performance Indicators for Measuring Effectiveness

Indicators Data Presented in

Student entry indicators Admissions updates

ACT scores for beginning students Figures 7-1 and 7-2

Demographics of beginning students Fact Book (p. 2+)

Process indicators Daily enrollment updates

Enrollment data for university Figures 7-3–7-6

Enrollment data by major and by college Fact Book (p. 41+)

International enrollment Fact Book (p. 2+)

Diversity enrollment Fact Book (p. 2+)

Enrollment data by feeder schools Fact Book (p. 55+)

Retention rates Figure 7-7

Graduation data Figures 7-8—7-11

Housing occupancy Figure 7-12

Housing exit surveys See Figures 6-2

Dining surveys See Table 6-8

Student learning outcomes See 1R2 and 1R3

NSSE Table 7-12

SSI Figure 7-14

University resource indicators SSCH by department and college Fact Book (p. 81+)

Faculty composition See 4R2

Faculty/staff diversity See 4R2

Faculty scholarship and creativity See 2R2

Faculty salary comparisons Table 7-11

Employee satisfaction surveys Figure 7-13

Financial management Utility usage and cost Table 7-8

Cost containment initiatives

External accountability Mandatory performance funding measures for graduation

rates and cohort progression Table 7-9

Optional performance funding measures, including

graduation rates for minority students, remediation students,

transfer students, students in critical needs programs, and

completion data based on SSCH

Table 7-10

IPEDS

7R2. What is the evidence that your system for Measuring Effectiveness meets your organization’s

needs in accomplishing its mission and goals?

SAU’s system for measuring effectiveness responds to environmental changes, changes in stakeholder

needs, and

continuous quality improvement initiatives. The University is committed to communicating with all

constituencies to ensure that the University community is aware of progress in meeting various

objectives. The data presented below represent key measures of effectiveness.

Student Entry Indicators. The Office of Admissions provides weekly updates to the VPSA related to

the number of students applying for admission and completing admission to SAU. Student admission data

https://www.dropbox.com/s/ml79d6fqrxerifr/Fact%20Book%202011.pdf








0

200

400

600

800

20

00

20

02

20

04

20

06

20

08

20

10

2500

2600

2700

2800

2900

3000

20

00

20

02

20

04

20

06

20

08

20

10

19.5

20

20.5

21

21.5

22

2000 2002 2004 2006 2008 2010

Figure 7-1: Average ACT Scores

0

20

40

60

80

100

ACT>27

ACT<16

Figure 7-2: Enrollment by ACT

Figure 7-3: Total Enrollment Figure 7-4: Beginning Freshmen

Figure 7-5: Undergraduates Figure 7-6: Graduate Students

50.00%

55.00%

60.00%

65.00%

70.00%

Figure 7-7: Retention Rates

provides a profile of entering students, including demographics, high

school transcripts, and ACT scores. As shown in Figure 7-1, the

average ACT scores for beginning freshmen continues to trend upward.

As shown in Figure 7-2, data also indicates an upward trend in entering

students with ACT scores about 27, coupled with a downward trend in

entering students with ACT scores below 16. The University compiles

and analyzes demographic data of entering students to monitor student

diversity initiatives. Data on diversity is provided in the SAU Fact Book

(p. 2+)

Process Indicators. The University collects and analyzes data to measure

the effectiveness of its instructional and no-instructional

programs, including enrollment and retention data, graduation

data, and measure of satisfaction with key services.

Enrollment data indicates the effectiveness of SAU’s

recruiting efforts, instructional processes, and support services.

At the beginning of each semester, the Office of Enrollment

Services provides the deans and VPC with daily updates on

enrollment until the official census date. Institutional research

compiles current and longitudinal data on enrollment. As shown in

Figure 7-3, total enrollment has

increased since 2003—with total

enrollment of nearly 3,400 in 2010

and 2011. Enrollment of beginning

freshmen, shown in Figure 7-4, has

increased after a modest decline in

2007. Graduate enrollment, shown in

Figure 7-5, has increased

significantly. The University’s

strategic plan includes an objective

to increase the number of graduate

programs and graduate enrollment.

Data indicates that the University is

accomplishing this objective.

Additional analysis of enrollment

data, including analysis by college

and major, based on diversity, and by

feeder schools, is provided in the

SAU Fact Book.

Retention data indicates effectiveness of the University’s retention efforts,

instructional programs, and support services. The University compiles

current and longitudinal data on first-year retention and cohort progression.

As shown in Figures 7-7, the University’s retention rates have not

consistently improved. The University identified retention as an area for

improvement in 2010 and appointed an Action Project team to develop and

implement new retention strategies. Data for the next two years will

0

100

200

300

400

500

20

00

20

02

20

04

20

06

20

08

20

10




0

200

400

600

800

20

00

-01

20

02

-03

20

04

-05

20

06

-07

20

08

-09

20

10

-11

0

50

100

150

200

20

00

-01

20

02

-03

20

04

-05

20

06

-07

20

08

-09

20

10

-11

Figure 7-8: Total Degrees

Figure 7-9: Associate Degrees Figure 7-10: Bachelor’s Degrees Figure 7-11: Master’s Degrees

Figure 7-12: Residence Hall Occupancy

indicate the degree of effectiveness of these new initiatives.

Graduation data indicates success in accomplishing the University’s

mission and effectiveness of the University’s retention efforts,

instructional processes, and support services. The University compiles

current and longitudinal data on graduates from associate, bachelors,

and masters programs. As shown in Figures 7-8 through 7-11, total

degrees awarded by SAU continue to increase in all categories. As

previously noted, advisors are now using the degree audit system to

better identify students satisfying associate degree requirements,

which resulted in identifying 75 additional students who had

completed associate degree requirements in 2011-12.

Residential occupancy and satisfaction

are indicators of effectiveness related to

campus life. As shown in Figure 7-12,

trend analysis indicates that SAU is

meeting the housing needs of students.

With occupancy nearing full capacity, the

University continues to add new

residential living facilities. As previously

reported in 6R2 (Figure 6-2), residence

hall students complete exit surveys at the

end of each semester. Results consistently

show that over 85 percent of residents

were satisfied with their room.

Dining services are an essential service provided for students, faculty, and staff. The University collects

and analyzes two measures of satisfaction with campus dining services—surveys by the dining service

provider and results of the SSI questions related to dining. Results of the dining survey can be referenced

in 6R2 (Table 6-8) and consistently indicate lower scores related to variety of food. Results of the most

recent SSI corroborates this weakness with results that are statistically lower than the national averages

(4.04 versus 4.57; significant at the .001 level).

The University identified dining services as an area for improvement in 2011 and requested input from

the Student Government Association about options for improving the variety of food. An SGA poll

indicated student’s preference for replacing Quizno’s with Subway and for adding Chick-fil-A and Grill

Works. By late 2012, the three new restaurants opened in the food court. Results of future surveys will

indicate if these changes improved satisfaction.

0

50

100

150

200

20

00

-01

20

02

-03

20

04

-05

20

06

-07

20

08

-09

20

10

-11

0100200300400500

20

00

-01

20

02

-03

20

04

-05

20

06

-07

20

08

-09

20

10

-11



Student learning outcomes are a key measure of the effectiveness of the University’s academic

programs and support services. Student learning outcomes are discussed at length in 1R2 and 1R3.

University Resource Indicators. SAU collects and analyzes data to measure the effective use of faculty

and staff resources, including analysis of student semester credit hour production by department and by

college. This analysis is available in the Fact Book (p. 81+). The University monitors faculty productivity

in terms of scholarly contributions and creativity; these measures are discussed in 2R2. The University

monitors faculty composition by rank and diversity; these measures are discussed at length in 4R2.

Financial Management. Cost containment measures are key indicators of effective management of

financial resources. Energy consumption and costs are tracked by the physical plant as a targeted area for

critical cost containment. As shown in Table 7-8, the University’s energy usage and per unit energy costs

have also increased significantly over the last five years. Increased usage is attributed to three factors:

new construction that

increased campus square

footage by 14%, enrollment

increased 7.9%, and record-

high summer temperatures.

Faced with increasing utility

costs, the University

invested $1.4 million in the new district heating and cooling system, which is providing estimated annual

savings of $340,000 and resulted in SAU’s energy costs dropping to $1.20 per square foot, which is

among the most efficient in the state. Selecting energy efficient equipment has resulted in over $100,000

in rebates from power and gas companies. Other cost containment initiatives are shared on the University

website.

External Accountability. Arkansas’ performance funding measures include four mandatory measures

related to graduation and cohort progression. In terms of graduation data, the PF formula established

2009-10 as the base year and set goals of increasing graduation rates by 4.73% each year for bachelor’s

degrees, total degrees, and

degrees in science, technology,

engineering, and mathematics

(STEM). Cohort progression is

a measure of two-year retention, in which the goal is to increase the current two-year progression rate

over the average rate for the three preceding years. Table 7-9 summarizes SAU’s performance results

released in April 2013. As shown, SAU exceeded targets for three of the four measures.

Each institution may choose to be evaluated based on up to six optional Arkansas performance funding

measures. At this time, SAU has elected to be evaluated on five optional categories plus the compensatory

category that factors in a low-income student population. Each optional measure is based on the goal of

increasing the institution’s current two-year results over the average rate for the three preceding years and

one point is awarded for meeting each goal. Table 7-10 summarizes the points awarded on SAU’s

performance score card released in April 2013. As shown, SAU earned full points in the five optional

measures with additional points based on the number of students who receive Pell grants.

Based on SAU’s results, the University received the full allocation of performance funding provided.

Table 7-8: Analysis of Energy Usage and Costs 2008-2012

Electricity Natural Gas Water Wastewater

Increase in use 4.6% 15% 22% 13.5%

Change in unit cost .4%

Increase

15%

Decrease

102%

Increase

96%

Increase

Percentage of total utility 60% 21% 11% 8%

Table 7-9: Performance Funding Scorecard for Degrees and Progression

Bachelor’s Degrees Total Degrees STEM Degrees Progression

97% 101% 127% 106%

Table 7-10: Performance Funding Scorecard for Optional and Compensatory Measures

Minority

Degrees

Remedial

Degrees

Transfer

Student Degrees

Regional Critical

Needs Degrees

Course

Completion

Compensatory Points

for Pell

1 point 1 point 1 point 1 point 1 point .56


http://web.saumag.edu/administration/cost-containment/



0% 20% 40% 60% 80%

Survey Average

Fairness

Communication

Senior Leadership

Pride

Policies, Resources & Efficiency

Compensation, Benefits &…

Teaching Environment

Figure 7-13: Employee Satisfaction Survey

7R3. How do your results for the performance of your processes for Measuring Effectiveness

compare with the results of other higher education organizations and, if appropriate, of

organizations outside of higher education?

SAU processes for measuring effectiveness are evaluated using comparative data to identify market

trends, areas of success, and opportunities for improvement. Comparative data related to enrollment,

graduation rates and employees are provided by IPEDS and ADHE reporting and are summarized in the

SAU Fact Book. Comparative data related to student satisfaction is provided through regular

administration of the National Survey of Student Engagement (NSSE) and the Noel Levitz Student

Satisfaction Inventory (SSI). Comparative data related to employee satisfaction is provided through

participation in the Chronicle of Higher Education’s survey of the Greatest Colleges to Work For and

analysis of average faculty salaries provided by AAUP.

At the institutional level, the administration

carefully monitors faculty salaries in comparison to

state and national averages. As shown in Table 7-

11, AAUP data indicates that faculty salaries at

SAU compare favorably with state averages but are

below the national averages.

As shown in Figure 7-13, results of

the Chronicle survey identified an

opportunity for improvement relation

to communication between faculty,

staff, and administrators. Reflecting

on these results, the University has

taken steps to improve

communication between constituents.

For example, the AQIP coordinating

team strategically selected faculty and

staff members to jointly co-chair each

team working on the systems

portfolio and continuous quality

improvement initiatives.

Results provided by the NSSE generally

indicate that student satisfaction at

SAU is similar to that at peer

institutions. The NSSE results in

Table 7-12, indicate statistically

significant differences related to

the level of academic challenge,

which is below peer averages, and

student-faculty interaction, which

is above peer averages.

Summary results of the 2012 SSI

are provided in Figure 7-14 and

indicate a relatively high level of

student satisfaction with little

difference when compared to peer

Table 7-11: Average Faculty Salaries

Based on 2012 AAUP Data

SAU Arkansas National

Professor $74,700 $67,125 $88,021

Associate Professor $57,300 $56,745 $70,312

Assistant Professor $49,600 $49,721 $59,318

Color

Coding:

Red Flag

0%-

44%

Yellow Flag

45%- 54%

Fair to Mediocre

55%- 64%

Good

65%- 74%

Very Good to

Excellent

75%- 100%

Table 7-12: 2012 NSSE Results

SAU Southeast

Public

Sig Carnegie

Class

Sig NSSE

2012

Sig

Level of Academic Challenge

First-year 51.3 53.1 54.8 * 54.5 *

Senior 54.6 57.1 * 58.6 *** 58.4 ***

Active and Collaborative Learning

First-year 43.8 43.5 45.8 44.2

Senior 54.1 51.8 53.7 52.1

Student Faculty Interaction

First-year 37.0 36.4 37.5 35.9

Senior 50.3 44.1 *** 44.5 ** 42.9 ***

Enriching Educational Experiences

First-year 27.0 28.1 28.0 28.4

Senior 39.4 39.7 39.8 40.4

Supportive Campus Environment

First-year 62.5 64.0 64.8 63.5

Senior 64.2 60.8 * 62.2 60.5 * *p<.05 **p<.01 ***p<.001



1 2 3 4 5 6

Academic Advising

Campus Life

Concern for the Individual

Recruitment & Financial Aid

Responsiveness to Diverse…

Service Excellence

How Satisfied Are Our Students Compared to

other Institutions?

National Four-Year Publics

Figure 7-14: Student Satisfaction Inventory

results. Results of detailed questions on

the SSI, however, identify opportunities

for improvement, such as the results

previously discussed related to dining

services.

7I1. What recent improvements have you made in this category? How systematic and

comprehensive are your processes and performance results for Measuring Effectiveness?

The University’s processes and performance results for measuring effectiveness are comprehensive,

providing information to those engaged in planning and managing the institution. Performance results are

available and shared to support data-based decision making.

The University has made many improvements related to this category. For example, effective collection

and sharing of information is linked directly to the University’s reliable web-based information systems.

Over the past few years, our Campus Connect web-based information system has become more and more

utilized by the faculty and staff of SAU. The Campus Connect portal is a web extension of POISE which

allows a few of the commonly used academic applications from POISE to be accessed anywhere. Course

enrollments, student advising, degree audits, grade submissions, and student demographic data are all

accessible through the Campus Connect web portal. This has greatly streamlined data access for students

and faculty. Campus Connect also allows faculty to complete routine tasks online, such as certifying

rosters, submitting early alert referrals, and printing student transcripts.

7I2. How do your culture and infrastructure help you to select specific processes to improve and to

set targets for improved performance results in Measuring Effectiveness?

SAU uses many processes to improve its performance. Since transitioning to a focus on continuous

quality improvement in 2010, the University community has worked together to improve processes

related to retention, remediation, assessment of student learning outcomes, and employee development.

Teams continue to work on issues related to improving academic advising and the faculty evaluation

process.

category 7 - southern arkansas university

Documents