Every A&SSS unit should have an active mission statement, which is "a broad statement of the direction and values of the [academic and student support services department/office/center]. For each unit the mission statement should reflect how the unit contributes to the education, development, and experiences of students at the institution. The mission statement also should describe the services provided by the unit." (p. 14 in UCF Administrative Assessment Handbook).

A well-defined mission statement includes the following components:

1. Purpose of the unit - the reason(s) why you perform your major activities or operations. Your office's focus may be helping students graduate on time, assisting faculty with research proposals, or supporting transfer students at FSU.
2. Unit's Stakeholders - groups of individuals who participate in your programming and/or are benefiting from the provided services. For example, graduate students, female students in STEM disciplines, assistant professors seeking tenure, or university staff employees.
3.  Primary functions - your unit's most important functions, operations, services and/or offerings that directly or indirectly support student success. For instance, offering course design seminars to instructional faculty, coaching students from under-resources backgrounds, or providing data-based decision support to university leadership.
4. Connection to University's mission - alignment between your unit's mission statement and the mission statement of FSU. Your unit's mission may be focused on leadership education and community engagement, which supports part of FSU's mission to "instill strength, skill and character". Another example is a university unit striving to "support the faculty in the important and difficult work of crafting transformative learning experiences", which directly aligns with FSU's dedication to excellence in teaching.

Below is an example of Office of Institutional Performance and Assessment mission statement, with the four components of a well-defined mission statement underlined.

We are dedicated to supporting the University community (2) in our continuous improvement efforts (1). Our goal is to facilitate systematic, comprehensive, and data-based planning and evaluation processes (3) focused on enhancing institutional quality and effectiveness (4).

POs of A&SSS units should focus our efforts on improving direct support to faculty and students, enhancing indirect support for student learning, or serving a specific co-curricular mission that supports the college experience. Most Outcomes for A&SSS units are chosen because of their assumed or proven positive impact on faculty and student success. Chosen POs should be a result of a unit’s analysis of its program’s strengths and weaknesses and should reflect its commitment to improving the most important services and outcomes. Dependent on the purpose and primary functions of the A&SSS unit, POs selected by an individual department/office/center typically fall into three categories:

1. Outcomes focused on the efficiency, breadth and/or quality of unit’s support services. For example, FSU Office of Financial Aid may have a PO to optimize processing time in order to disburse financial aid at least two weeks before classes begin. Office of Faculty Development and Advancement may have a PO focused on strengthening junior faculty mentoring by providing additional programming and incentives.
2. Outcomes focused on developing certain knowledge, skills, behaviors, values and attitudes among students, faculty and/or staff. FSU Academic Center for Excellence may have a PO aimed at improving student study skills and habits. FSU Health Services may have a PO aimed at increasing financial literacy among students and employees. FSU Office for Human Subjects Protection may choose a PO that targets adherence to ethical principles and laws.
3. Outcomes focused on research, creative endeavors, and service to public and community. FSU Innovation Hub may have a PO that captures their unit’s efforts in promoting innovative scholarship and projects among undergraduate students. FSU Fraternity and Sorority Life may have a PO focused on fundraising and stewardship that benefit local families.

In addition, POs are typically conceptualized as:

• Desired final products and results (e.g., increased graduation or retention rates, greater number of event attendees, improved reported well-being among students, more diverse departmental faculty);
• Activities and strategies aimed at achieving a goal (e.g., enhancing the quality of academic advising as a way to support timely graduation, increasing the number of students completing Financial Wellness course module as a way to decrease graduates' student loan default rates).

Depending on the level of control an A&SSS unit has over a specific outcome, either the activities/strategies or their final results may be chosen. POs, like many university-wide metrics, may require planning for an outcome that is a product of multiple strategies. For example, increasing student graduation rates is a PO that requires successfully implemented efforts aimed at retaining students at FSU, their timely completion of gateway courses, ensuring they have financial means to attend collect, etc. In addition, enhancements put into effect in a given academic year may yield noticeable results at a later point. For instance, it may require several years for new graduation-focused strategies to lead to sizable changes because it may take a new cohort of students 2-4 years to fully benefit from the expanded range of programmatic improvements.

It is unusual to have a PO that is pursued for only one year; typical ‘lifespan’ of a PO is 3-6 years. A longer implementation period allows for more thoughtful planning, consistent multi-year assessment, and data-based, sustained enhancement efforts. Reasons for ‘retiring’ a PO may include: the Outcome that the unit wanted to attain has been achieved, the Outcome is no longer a priority or is no longer under the purview of the unit, the existing Outcome has been modified and replaced by its much narrower or much broader version, etc. A&SSS units may contact the Office of IPA to consult on sunsetting existing POs and/or selecting new POs.

Do not select POs that resemble a 'to-do' list or a plan to accomplish a task or series of tasks, especially if they can only be completed if new funding is requested and received (e.g., hiring an employee, renovating office suite, buying new computers).

Below and in subsequent sections, all IE assessment report components (as they are requested in the IE portal) are illustrated using an example PO from the FSU Advising First program.

• Provide a succinct name for the PO:
  PO Name: PO - Academic advising services
• Identify the improvement to be made at the administrative unit level:
  Program Outcome: The quality of academic advising sessions provided by Advising First advisors to students will be high.

Importantly, all POs should be clearly connected to the institutional goals as they are outlined in the FSU Strategic Plan. In addition, the Goals and Initiatives of the FSU Strategic Plan and A&SSS units' POs should be in alignment with budgetary decisions and resource allocation. Finally, all planning, assessment, and implementation activities should relate to, and advance, the University's mission. The system of relationships between these elements is illustrated in the figure below (adapted from Hoefer, 2019).

The University's mission is stated as follows:

Florida State University preserves, expands, and disseminates knowledge in the sciences, technology, arts, humanities, and professions, while embracing a philosophy of learning strongly rooted in the traditions of the liberal arts. The university is dedicated to excellence in teaching, research, creative endeavors, and service. The university strives to instill the strength, skill, and character essential for lifelong learning, personal responsibility, and sustained achievement within a community that fosters free inquiry and embraces diversity.

The 6 Goals and 17 Initiatives of the 2017-2022 FSU Strategic Plan Implementation are:

In order to sustain "a systematic and documented process of assessing institutional performance with respect to mission" (p. 56 in SACSCOC Resource Manual), A&SSS units are asked to align POs that they identified and (plan to) pursue with one or more Strategic Plan Initiatives. This process is known as 'institutional back mapping'; it allows for a visual representation of the link between the goals of individual university units and strategic priorities of the institution (pp. 62-66 in Nichols & Nichols, 2005). Below are several examples of different POs' alignment with the Initiatives of the FSU Strategic Plan.

 ​

Assessment methodology for a PO should be focused on accurately measuring the extent to which an A&SSS unit achieved the established Outcome. The unit may choose to assess a unit-level PO through measuring the efficiency of its day-to-day operations, quality of provided services, number of individuals who received support, etc. Institutional POs are typically the university-level final ‘products’ that unit-level activities support, such as faculty and student gains/benefits resulting from receiving the unit’s services, including those of educational nature.

It is important to evaluate POs with appropriate assessment instruments, within the context of a unit’s functions, and in a methodologically consistent fashion to allow for year-over-year assessment. When designing an assessment methodology, it is useful to adhere to the S.M.A.R.T. guidelines – program-level outcomes should be Specific, Measurable, Achievable, Relevant, and Time-bound. When feasible, IPA and Office of Institutional Research can provide additional data and analytic support to A&SSS units in need of custom reports and datasets.

• Describe how the assessment of the PO will be conducted:
  Assessment Process: PO - Academic advising services The quality of provided advising services will be measured via a post-advising session survey, the link to which will be sent to students through email. Student responses will be aggregated per academic year define as Fall and Spring terms. The survey was created internally several years ago and is being used to measure various aspects of advising meetings. It contains several items/questions. The question that will be used to measure this PO asks students to rate on a 5-point Likert scale the following statement: "How satisfied were you with your advisor's knowledge about liberal studies, major, and university wide requirements?"

The preferred approach to ascertaining whether a PO is achieved is setting multi-year quantitative goals/benchmarks using data that can be counted, measured, and expressed using numbers. Qualitative (descriptive, verbal, non-numeric) information may also be used to measure POs, but this assessment approach is not recommended due to the potential difficulty to ascertain whether the PO has been achieved or not.

An A&SSS unit can choose to set a goal for a PO, a benchmark or both. For the purposes of IE assessment, a goal denotes a desired numeric change between two values. For example, a 5 percent increase can mean growing the number of students who attended an event from 20 students to 21 students. Alternatively, a 5 percentage point increase can mean growing the headcount from 70% to 75%. (There is more information available about the difference between a percent and a percentage point.)

A benchmark denotes a minimum or a maximum numeric threshold that the unit will strive to meet. For example, having at least 90% of students exhibit desired behavior after they had participated in the unit’s programming. Or not exceeding two weeks between a data request date and the request being fulfilled.

When units decide how high they should set the PO goal/benchmark, they should study relevant industry standards, performance of similar units at peer institutions, and/or review the unit’s own past levels of achieving the PO. The goal/benchmark should be set at a level that is ambitious, yet achievable with some effort. Over the years, goal(s)/ benchmark(s) for the same PO may be changed: if you choose to increase/ decrease the goal/benchmark, record those changes in the IE Portal in the 'Goal/Benchmark' field and specify the timeframe (e.g., "Beginning with the 2021-2022 fiscal year, the benchmark used to measure success of this PO will be increased from 300 training attendees per year to at least 350 attendees.")

Importantly, “[w]ithin institutional effectiveness, departments (academic and nonacademic) are free to stretch themselves to the limit and to attempt innovative approaches to provide services without fear of failure. Within institutional effectiveness, departments are not held accountable for failure or success, only for having in place a process for stating outcomes, measuring accomplishments, and using the results to improve programming.” (Nichols, 1995).

Within the IE Portal, units can include any documentation they deem relevant to the Assessment Process and/or the Goal/Benchmark (e.g., copies of survey questions, consultant's report, data tables and graphs, unit's annual reports).

• Specify a measurable assessment standard that defines success:
  Goal/benchmark: According to the most recent available data, 88% of students who were surveyed and responded indicated that they were satisfied or very satisfied with advisor's knowledge. Over the next five years, we want to increase this number to at least 93%. 
  • Baseline: 2016-17 year = 88% of students
  • Year 1 Plan: 2017-18 = at least 89%.
  • Year 2 Plan: 2018-19 = at least 90%.
  • Year 3 Plan: 2019-20 = at least 91%.
  • Year 4 Plan: 2020-21 = at least 92%.
  • Year 5 Plan: 2021-22 = at least 93%.

During the academic/fiscal year, A&SSS units deliver support services to students, faculty, staff, and community members as planned at the beginning of the year. At the end of each assessment cycle, units aggregate necessary information/data and report the levels at which POs were achieved. A proper results statement for each PO provides a plethora of methodological details and is largely quantitative (i.e., includes counts and percentages out of total where appropriate). Results can be compared to last year’(s’) results to provide additional context. In some cases, qualitative results statements can be appropriate.

• Present information regarding the levels at which the PO was achieved:
  Results Statement: The quality of provided advising services was measured via a survey. In 2018-19 academic year (Fall 2018 and Spring 2019), 1,014 students responded to the survey. Out of them, 724 self-reported that they were ‘very satisfied’ and 184 self-reported that they were ‘satisfied’ with their academic advisor’s about liberal studies, major, and university wide requirements. In summary, 89.5% of students who were surveyed and responded indicated that they were satisfied. Compared to the student satisfaction level of 88.1% in 2017-18 academic year, a greater proportion of students reported satisfaction this year. The goal/benchmark for the 2018-19 year to increase the satisfaction level to 90.0% was not achieved, but there is a positive trend indicating improvement.

The culmination of the annual IE Assessment process is the analysis of why the Outcome was achieved at the level that it was. Using facts and their expert opinion, each unit should try to determine the reason(s) why the PO was attained at this particular level. Most reasons will include specific factors, decisions, actions, people, and events that negatively and/or positively influenced the results. Units need to make sure that they 'close the loop' on prior year'(s') improvement action(s) by explicitly stating whether those changes were indeed implemented as planned and whether they had the intended positive effect on the PO.

A strong analysis of results should: 1) be focused on the take-aways from internal discussions or investigations regarding the data, 2) form the link between the data and the improvement action(s), and 3) highlight areas of success in addition to areas needing enhancements. In addition, if the unit concludes that the way the PO is assessed needs to be changed, an evaluation of the assessment methodology should be provided.

In the IE portal, the file bank associated with each PO allows users to upload any relevant supporting documentation, such as data tables and visuals/graphics, minutes/notes from the unit’s meeting(s) where results were analyzed and discussed. These kinds of records provide documented evidence of assessment and improvement efforts and are required by SACSCOC, so each unit should include these files when available.

• Examine the reason(s) for the attained results:
  Analysis of Results: We believe that the increase in student satisfaction levels was due to the enhancements to the training resources provided to the academic advisors in the Advising First program. Specifically, before the start of the 2018-19 academic year, every continuing advisor participated in the refresher training and every new advisor participated in an enhanced training that included a new module explaining how best to address frequently occurring issues. We hypothesize that the increase in student satisfaction level was not as high as we set out to achieve because of a high turn-over rate. About 1/5 of Advising First academic advisors voluntarily separated from their positions in the 2017-18 academic year, which required hiring 12 new employees to fill the vacancies. The newly hired advisors were not as experienced and therefore their student ratings were not as high. The voluntarily separated advisors cited inadequate pay as one of their main reasons for leaving.

The most intensive component of the IE assessment process is devising and implementing actions to enhance a unit’s services and operations. Formulating sound improvement actions requires participation, engagement and meaningful contribution from as many members of the A&SSS unit as possible. Whether POs have been met or not, it is the responsibility of the department/office leadership and assessment coordinators to determine a plan of action for the next year.

When an Outcome does not meet the desired goal/benchmark, the unit should figure out why and then develop a way to improve its functions and services. These plans should be well-thought-out and describe specific changes to be implemented, ranging from small-scale enhancements to significant changes in a unit’s operations. Improvement actions may also include new or modified assessment approaches, changes to job responsibilities, purchasing third-party products, adopting new practices, attitudes and behaviors, etc.

In cases when a PO is being consistently achieved at a high level, it is recommended to either increase the desired goal/benchmark, focus on a different aspect of the same PO, or derive a new PO that would address other important aspects of the unit's work. If these changes are not feasible, the units should consider how they expect to maintain a high level of performance.

Most importantly, “[t]he institution should be using the data to inform changes based on evaluation of its findings. Plans to make improvements do not qualify as seeking improvement, but efforts to improve a program that may not have been entirely successful certainly do.” (page 68 in SACSCOC Resource Manual).

• Describe specific actions to improve or sustain performance:
  Improvement Plan: In order to continue improving student satisfaction levels with their academic advising services, the Assistant Director of Advising First program will design and oversee the training for new advisors and the Director will request increases in salary of the advisors whose compensation is below the median in their job class. For additional details, please see the two attached supporting documents:
  • The draft of the training protocol designed specifically for first-year advisors. It contains description of the new one-on-one peer mentoring program and outlines specific conversation topics to be covered during weekly meetings with supervisor;
  • Salary analysis of all currently employed advisors and recommended increases. The 17 advisors whose compensation is below the median will be prioritized in salary increase considerations.