Policies and Guidelines for Assessment of Student Learning
Assessment Handbook - May 2013
Developed by
the Academic assessment Council
Assessment Handbook (PDF)
| Table of contents |
|---|
Cal Poly Assessment Goals |
Introduction |
Cal Poly's Assessment Strategy: The Three Legged Stool
|
Annual Program Assessment |
University/GE Assessment |
Program Review/Provost's Initiatives |
Glossary |
Appendices |
Cal Poly's Assessment Goals
University Goals (1)
- The university has created a plan to assess the University Learning Outcomes (ULOs) with special emphasis on the five Western Association of Schools and Colleges (WASC) core competencies (2), identify standards of performance for all students in the institution, and implement a coordinated assessment of student learning over a single ten-year cycle of review. As a result, the university will be able to provide evidence of student learning, take steps in areas where improvement is needed and demonstrate where learning has improved. Assessment is intended to be an ongoing effort for continuous improvement of student learning.
Program Goals (1)
- Programs will create a plan to assess all Program Learning Outcomes, identify standards of performance for students in the program, and implement a comprehensive assessment of student learning over a single cycle of program review. As a result, the program will be able to provide evidence of student learning, take steps in areas where improvement is needed and demonstrate where learning has improved. Program assessment is intended to be an ongoing effort for continuous improvement of programs and student learning.
1) The University and Program Goals were adopted by the Academic Assessment Council, Spring 2013 and subsequently approved by the Provost.
2) WASC core competencies: written communication, oral communication, quantitative reasoning, critical thinking and information literacy
Introduction
What Assessment is and Why We Do It
Assessment is evaluating and improving student learning through four steps:
- establishing learning goals/objectives/outcomes
- providing learning opportunities
- assessing student learning
- using the results of assessment to improve student learning and the assessment process itself (3)
Student learning is assessed to:
- ensure that students are learning what we want them to learn.
- provide an opportunity to make programmatic improvements based on assessments.
- document student learning for interested stakeholders (the university as a whole, the CSU system, the state legislature, parents, students, outside accrediting agencies, and the general public).
- use valid and reliable evidence to demonstrate student learning and to improve it.
(3) Adapted by Linda Suskie, Assessing Student Learning: A Common Sense Guide. New York; Jossey-Bass, 2009, p.4. Print.
back to top
General Principles of Assessment
Cal Poly’s assessment process is intended to embody the following principles:
- Design assessment efforts to be useful for multiple purposes and audiences including course improvement, program improvement, program review, external accreditation and WASC accreditation.
- Gather useful assessment evidence efficiently; this includes encouraging externally accredited programs to use their existing assessment efforts to fulfill university assessment expectations to the degree possible.
- Support programs with resources to help with assessment, including rubrics, descriptions of best practices, model timetables, model reports, model evidence collection and storage practices, and so on.
- Empower faculty and programs with maximum feasible control over how and when they assess their Program Learning Objectives (PLOs).
- Centralize and standardize evidence collection and storage, and make assessment evidence easily available to programs.
- Reaffirm that assessment evidence is not to be used to evaluate individual faculty members or in the RPT process.
These processes were agreed on by the Academic Assessment Council in Spring 2013.
back to top
The Purpose of This Document
-
This document explains the assessment procedure that the Academic Assessment Council (4) has proposed and the Provost has approved. It explains the model of assessment that Cal Poly currently plans to follow for the next 10 years (until the next time the university comes up for reaccreditation). This document is intended to be a guide to policies and procedures for programs and deans, as well as a reference source for anyone involved in assessment on campus.
(4) Academic Senate resolution AS-735-11 empowered the Academic Assessment Council to act as the university’s main policymaking body for assessment of student learning.
Learning Objectives: The Goal
- Learning objectives exist on many different levels, and identify what we want students to know or be able to do by the time they complete a learning experience (graduate, complete a course, complete participation in an extracurricular activity, etc.).
- Listed below are brief descriptions of the following learning outcomes/objectives:
WASC Core Competencies, University Learning Objectives, Diversity Learning Objectives, Sustainability Learning Objectives, General Education Focused Learning Objectives and Program Learning Objectives.
Western Association of Schools and Colleges (WASC) Core Competencies
WASC (Cal Poly's regional accrediting body) expects all schools that it accredits to both achieve and assess these competencies.
Cal Poly's University/General Education assessment spans over a five-year cycle, covering one of these WASC core competencies each year. This schedule complements the workshops offered by WASC and the CSU Chancellor’s office.
- written communication
- oral communication
- quantitative reasoning
- critical thinking
- information literacy
back to top
Cal Poly's University Learning Objectives (ULOs)
When students graduate from Cal Poly, they should be able to:
- Think critically and creatively
- Communicate effectively
- Demonstrate expertise in a scholarly discipline and understand that discipline in relation to the larger world of the arts, sciences, and technology
- Work productively as individuals and in groups
- Use their knowledge and skills to make a positive contribution to society
- Make reasoned decisions based on an understanding of ethics, a respect for diversity, and an awareness of issues related to sustainability
- Engage in lifelong learning
ULO 6 states that all Cal Poly graduates should be able to make reasoned decisions based on a respect and awareness of diversity an an awareness of issues related to sustainability.
Because of the complexity of these objectives, the Academic Senate adopted the Diversity Learning Objectives (DLOs) in 2008 and the Sustainability Learning Objectives (SLOs) in 2009, both as addenda to the ULOs.
back to top
Diversity Learning Objectives (DLOs)
All students who complete an undergraduate or graduate program at Cal Poly should be able to make reasoned decisions based on a respect and appreciation for diversity.
Students should be able to:
- Demonstrate an understanding of relationships between diversity, inequality, and social, economic, and political power both in the United States and globally
- Demonstrate knowledge of contributions made by individuals from diverse and/or underrepresented groups to our local, national, and global communities
- Consider perspectives of diverse groups when making decisions
- Function as members of society and as professionals with people who have ideas, beliefs, attitudes, and behaviors that are different from their own
Sustainability Learning Objectives (SLOs)
Cal Poly defines sustainability as the ability of the natural and social systems to survive and thrive together to meet current and future needs. Cal Poly students should be able to consider sustainability when making reasoned decisions.
Students should be able to:
- Define and apply sustainability principles within their academic programs
- Explain how natural, economic, and social systems interact to foster or prevent sustainability
- Analyze and explain local, national, and global sustainability using a multidisciplinary approach
- Consider sustainability principles while developing personal and professional values
General Education Program Learning Outcomes
GE PLO #1 ~ Construct and critique arguments from a logical perspective.
GE PLO #2 ~ Use appropriate rhetorical strategies to connect with diverse audiences through oral, written, and visual modes of communication.
GE PLO #3 ~ Address real world problems by demonstrating broad disciplinary knowledge, skills, and values in arts, humanities, sciences, and technology.
GE PLO #4 ~ Understand the value of a general education in relation to a major course of study.
GE PLO #5 ~ Collaborate with people of different backgrounds, values, and experience.
GE PLO # 6 ~ Evaluate global and local issues and their impact on society.
GE PLO # 7 ~ Use intention and reflection to develop and improve one’s own learning.
Program Learning Outcomes
Every program at Cal Poly has program learning outcomes (PLOs) which translate the ULOs into terms relevant to each specific discipline and define expectations for all graduates of the program. For example, "demonstrating expertise in a scholarly field" means something different in the Theater Department than in does in the Chemistry Department. It is appropriate that each program specify what abilities and knowledge students should have to achieve the expected degree of expertise for that discipline. PLOs should be explicitly mapped to the ULOs.
Cal Poly's Assessment Strategy: The Three-Legged Stool
- Over the next ten years, Cal Poly will assess the ULOs in three different but complementary ways: Annual Program Assessment, University/General Education Instituational Assessment and Program Review/Provost's Initiatives.
Annual Program Assessment
- During each program review cycle, each program at Cal Poly should assess all of its program learning objectives. (Program review cycles vary in length among programs; most are 6 years.) In each year, each program should engage in some aspect of the assessment process (gathering evidence, analyzing evidence, reflecting on the evidence, implementing changes, assessing those changes, assessing the assessment process). Programs decide which PLOs to assess when, and they also decide how to assess them. The university will provide advice and guidance (rubrics, information about best practices, workshops and training opportunities, templates, and so on). Programs that are already engaged in assessment efforts to maintain external accreditation are encouraged to use those efforts to fulfill their Cal Poly assessment obligations as well, to the degree possible. The main purpose of program-level assessment is to ensure student learning at the program level, and to enable program improvement as necessary.
University/General Education Assessment
- Cal Poly is conducting an institutional level University/GE assessment that parallels program-level efforts and is intended to assess learning outcomes across the university. The tentative ten-year plan is designed to align the WASC core competencies, the University Learning Objectives and the GE Program Learning Outcomes. Programs may want to consider aligning their assessment efforts with this schedule to take advantage of the various resources. Listed below are two documents that illustrate the assessment flow visually with graphs.
Program Review/Provost's Initiatives
- Program Review / Provost’s Initiatives – As programs undergo periodic program review, they will generally be asked to undertake an additional assessment effort focused on an area of special interest to Academic Programs. For example, programs might be asked to assess the effectiveness of their Senior Project as a capstone experience. Academic Programs will provide programs with information about this additional assessment effort at least one year in advance, to allow for planning. In addition, Academic Programs will also coordinate university-wide assessment-related efforts such as Cal Poly’s participation in the National Survey of Student Engagement (NSSE); these university-wide efforts will generally require little from individual programs.
Annual Program Assessment
Program Learning Objectives (PLOs): Where It All Starts
- Program Learning Objectives (PLOs) are what the faculty of a program want the students in their program to know and to be able to do by the time the students graduate. For each program, the people who are best equipped to identify those desirable outcomes are the program's faculty. Cal Poly has University Learning Objectives, but they’re intentionally stated in general terms to allow each program to apply them to its specific disciplinary needs.
- Every program has expectations about what its students will know and will be able to do by the time they graduate, however, sometimes those expectations are unspoken and vague. It is the faculty in the program that know what knowledge and skills someone with a bachelor’s degree in Chemistry or English or Food Science and Nutrition should have.
- Further, all faculty are engaged in judging whether the students have obtained the specific knowledge and skills, by grading their work in major classes and by evaluating and grading their senior projects. Some of the judging of student work is quantitative (for example, multiple-choice tests), and some of it is qualitative (for example, grading an essay or assessing a recital). But all of it is potentially measurable or observable—that is, all such evaluation of student work could be done in ways that allow different students’ work to be compared using common criteria and for the findings to be reported in a systematic way. The goal of writing PLOs is to spell out faculty expectations clearly and explicitly.
back to top
Measurable/Observable Learning Outcomes and Standards/Benchmarks
- All programs have established Program Learning Outcomes (PLOs). The next step is to make sure that these outcomes are defined or operationalized in a measurable or observable way. A concise set of PLOs, summarizing expectations for all students in an academic program, is likely to be written at a level of generality that is difficult to assess. A program will also need specific and measurable (or observable) learning outcomes.
- Depending on the program, discipline, and the role of external accrediting agencies (if any), such outcomes may be defined at the course level (in which case they would overlap with course learning outcomes), or they may operate at a level between the course objectives and the program objectives. They might be called student learning outcomes, annotated program learning objectives, measurable attributes, student performance criteria, etc. The name doesn’t matter. What does matter is that the outcomes specify a student skill in relationship to an artifact or performance that can demonstrate the development of that skill. For example, if one of a program’s PLOs is “Demonstrate mastery of expository writing,” the associated learning outcome might be “Write at least three 20-25 page papers at an advanced level of proficiency.”
- Once a program has identified the measurable or observable learning outcomes for each PLO, the next step is to identify standards or benchmarks for evaluating whether the students in the program are achieving those outcomes at an appropriate level. There are many ways to do this.
- Here is an example of one way to do it. To measure or observe the expository-writing PLO and observable learning outcomes (given as examples above), a program might decide to collect a sample of 20-25-page papers from upper-level major courses and evaluate them using a rubric such as the University Writing Rubric
- The University Writing Rubric looks at five characteristics of papers (Purpose, Synthesis, Support, Style, Mechanics), and scores each one from 0-4. The program might decide that their internal standard or benchmark of student success is that 70% of the sampled papers should get a 3 or higher on each of the characteristics. If the program then finds that 70% of the sampled papers earn 3s or better on Purpose and Synthesis, but only 50% do so on Support, and only 40% on Style and Mechanics, that provides some useful data about where the program’s students are doing well and where they could use additional instruction.
Assessment Plan and Curriculum Map
- After a program has PLOs, measurable/observable learning outcomes, and benchmarks/standards in place, the next step is creating a multi-year assessment plan. The basic idea is that during every program review cycle (approximately 6 years), each program will assess all of its PLOs. When the next program review takes place, each program should have sufficient information about whether its students are achieving its PLOs. (And when the next university re-accreditation process takes place, the university as a whole should be in the same position.)
- To create an assessment plan, the program will need a curriculum map (available in the appendix) which is a grid that shows all of a program’s courses, including support courses and GE areas, as well as any special components that the program includes (exit exam, exit interview, periodic alumni survey, etc.). The curriculum map should indicate in which courses or components students should have achieved or be able to demonstrate an Introductory, Developmental or Mastery level regarding each PLO and ULO. Each program should decide when and where in the curriculum to assess each PLO. (See below for information about reporting these plans.)
Assessment Reporting
- Programs will be asked to report on their assessment activities both annually and during periodic program review. To make that process easier, Academic Programs sends out two documents each year
- called the Assessment Plan Progress Report (available in the appendix). The templates are designed to capture information about PLOs, measurable/observable learning outcomes, annual assessment efforts, the program’s timetable for assessment, and efforts to improve the program based on assessment results.
- The templates are also designed to be cumulative—programs simply update them each year. By the next program review, most of the information the program will need should already be in the most recent annual report. All programs were asked to submit an Assessment Plan Progress Report during the 2012-2013 academic year, so this basic process should be familiar.
back to top
University/General Education Assessment
- The General Education program has an independent assessment process that parallels program-level efforts and is intended to assess learning outcomes across the university. The tentative five-year assessment plan is designed to align the WASC core competencies, the University Learning Objectives and the GE Focused Learning Outcomes. Programs may want to consider aligning their assessment efforts with this schedule to take advantage of the various resources. Listed below are two documents that illustrate the assessment flow visually with graphs.
- University/GE Assessment Cycle (PDF)
What University/GE Assessment Will Ask of Programs
- In general, University/GE assessment will require little from the programs and most activities will be voluntary. For example, faculty may be asked if they would like to join a WASC learning committee. Faculty teaching GE might be asked to pilot a rubric or embed an assignment in one of their classes and then report the results.
- For information about how to obtain evidence or participate in the University/GE Assessment, contact the GE Program office by phone at x62228 or by email (mtool@calpoly.edu).
What University/GE Assessment Will Provide Programs
- Evidence about how well their students are achieving the various ULOs
- Opportunities for programs to participate in and benefit from the various working groups that are planning and carrying out University/GE assessment.
back to top
Program Review/Provost's Initiatives
Program Review
- All academic programs at Cal Poly undergo periodic program review. The frequency of review varies widely among programs, though most are on a 6-year cycle. Program review is already a well-established process at Cal Poly, and extensive information is available on the Academic Programs website.
Assessment comes up in two ways in the program review process:
1) Reporting past efforts
- Starting in 2012-2013, all programs have been asked to report their assessment efforts annually, using the Assessment Plan Progress Report templates provided by Academic Programs.
| YEAR | Annual Assessment PLAN |
review of annual assessment Plan |
|---|---|---|
| 15-16 | ||
| 16-17 |
The templates are designed to be cumulative?, so that the program adds each year’s information to the previous year’s report. The templates are also designed to capture nearly all of the assessment information programs will be asked to submit during program review. Thus, by the time program review comes around, each program’s cumulative report of past efforts should contain most of the assessment information that will be required.
- Programs should also expect to be asked to provide a brief report closing the loop on their experience with assessment since the last program review: reporting any changes they have made or plan to make to improve student learning, and any changes they have made or plan to make to improve their assessment process.
2) Additional Assessment Efforts
- All programs (including those that are externally accredited) should expect to engage in some additional assessment effort as part of program review. For example, Academic Programs may ask programs to assess the effectiveness of the senior project as a capstone experience, by using a standard rubric to assess a sample of senior projects.
- Programs will be informed about this additional assessment effort at least one year in advance, to give them time to prepare (and perhaps to piggyback other assessment efforts onto this one).
Provost's Initiatives
- The Provost’s office oversees a variety of additional efforts that may produce evidence relevant to assessment, such as Cal Poly’s participation in the National Survey of Student Engagement (NSSE) and the Collegiate Learning Assessment (CLA). Evidence from those efforts is made available to programs through Academic Programs.
back to top