Senior Project Assessment: Guidelines

 

Assessing the Senior Project as a Capstone

• Two assessment efforts on the effectiveness of the senior project were made during the academic year 2010-2011. The results of both were reported to WASC as part of Cal Poly’s self-study. For further information see Senior Project Assessment FAQs.
• All academic programs were asked to assess their senior projects as capstone experiences using a WASC capstone rubric (PDF). This can be done without examining the evidence of student work.

---

Guidelines for Using the WASC Capstone Rubric

1. USING THE RUBRIC -  The WASC rubric is authoritative and should be used as is.

2. METHODOLOGY -  There are several ways to complete this task, but any method selected should use the rubric as the basis of assessment. The faculty should be engaged, as should students. For example, a department might use a faculty meeting or the fall retreat to examine the senior project as it relates to the rubric, with student input obtained through a survey or focus group.
3. COMMUNICATING THE RESULT -  Reports were due in Academic Programs and Planning  on March 21, 2011 
   
   Deliverables of benefit to the program included:
   1. Comments on the rubric including recommendations for improvement.
   2. The result of the rubric-based assessment as interpreted by the program.
   3. A description of the method(s) used to arrive at the result.
   4. Recommendations for improvement and/or conclusions about the senior project as a capstone.
   5. The evidence to support these recommendations/conclusions.

---

Assessing the Senior Project as an Artifact of Student Learning

Program Review 2010-12

Programs that went through  review in 2010-12 were asked to take on the more ambitious task of assessing their senior projects for writing and critical-thinking skills using both the

• University Expository Writing Rubric (PDF)
• Critical Thinking VALUE Rubric (Word doc).

Programs whose senior projects did not address these outcomes at the mastery level (e.g., because their senior projects do not regularly include a written component) worked with their associate deans to identify alternate outcomes based on the University Learning Objectives (ULOs).

Programs that would like to make the senior project a focus of their annual assessment efforts are encouraged to use the writing and critical-thinking rubrics referenced above.

Using the Rubrics

1. The writing rubric has been developed at Cal Poly as an interdisciplinary tool for use in ULO-based assessment; it should be used as is.
2. Programs may work with their associate deans to revise the critical thinking rubric in ways that make sense to individual disciplines or groups of disciplines, but the criteria and four-point structure must remain the same.

Methodology

1. Sampling the projects. Programs should work with their associate deans to produce a random sample of collected projects; Institutional Planning & Analysis (IP&A) can provide guidance and assistance.
2. Examining the projects. Programs should use a method similar to the Writing Proficiency Exam (WPE). The “reading” should be a blind one, with names masked and projects known only by EMPL ID. Readers should calibrate their expectations by reading a few projects together in advance of the actual scoring. Pairs of readers should examine each project with the intention of arriving at a reliable result. The program should designate one or more third readers who can provide a consistent judgment when the initial reading of a project varies by more than one point on any criterion.
3. Communicating results. Preliminary reports will be due in APP on May 16, 2011. Final versions will be part of the program’s self-study report. Deliverables of benefit to the program should include the following:
   1. Rubrics as used.
   2. Comments on the rubrics including recommendations for improvement.
   3. An account of the method of creating the population of projects collected and the sample of projects read, including numbers of both.
   4. Results of rubric-based assessments as interpreted by the program. EMPL-identified results should be submitted to IP&A, which will provide a spreadsheet for this purpose. IP&A will also produce aggregated results for use by the program in its reports. All results should be presented in the form of a distribution across each rubric.
   5. Recommendations for the improvement of senior project policies/procedures at all levels. These include university policies on the Senior Project and Completion of Senior Project, as well as the Academic Senate Resolution on Archiving Senior Projects (PDF).
4. Sample projects. Programs should submit four examples of student work demonstrating minimal, average, good, and superior attainment in the senior project, accompanied by holistic statements defining attainment at each level. Programs with more than one type of project (research projects, creative projects, etc.) should present a range of samples from one project type only.

---

Findings and best practices

Based on the results of the assessments described in sections I and II (capstone, writing, and critical thinking), faculty representatives in each college were asked to develop a summary of findings and a statement of best practices. These were due on June 10, 2011 (end of Spring Quarter). Issues included:

1. The nature of the senior project as a capstone defined in relationship to the ULOs.
2. The relationship of the project to an assessable artifact.
3. The cost of instruction.
4. The integration of student learning in GE, the major, and the co-curriculum.
5. The tension between student advising and autonomy as well as other factors affecting completion.
6. The use of external benchmarks.
7. Service learning and external engagement.
8. Archiving and sharing projects.
9. Other compelling issues.

---

WASC Timeline

1. Fall 2010: assessing senior project begins.
2. March 21, 2011: report on assessing senior project as capstone (sec. I) due in APP.
3. May 16, 2011: preliminary report on assessing senior project for writing and critical-thinking (sec. II) due in APP.
4. Spring 2011: results of assessing senior project incorporated into EER report.
5. June 10, 2011: college-level report on findings and best practices (sec. III) due in APP.
6. Summer 2011: further results of assessing senior project incorporated into EER report.
7. Fall 2011: EER report circulated and revised before being sent to WASC.
8. February 2012: WASC team visits campus.

---

Using the Results

Departments and colleges used the results of this assessment for the purpose of program improvement. In addition, the results should lead to the improvement of senior project policies/procedures at all levels.  They will inform the campus plan for assessment as it develops during 2010-2011, and they served as an important source of evidence for the WASC self-study as it moved into the final EER phase. Every department and college has a part to play in this shared effort.