Guidelines for Self-Studies - Graduate Programs

“First and foremost, the purpose of the academic program review (APR) process at Cal Poly is to acknowledge the strengths of our academic programs and, at the same time, to seek ways of further enhancing their quality.”   In general terms, this is accomplished by a self-study process which provides for a review of the educational effectiveness of the program as it relates to:

• The mission of the program, college and institution.
• The curriculum through which the program's mission and goals are pursued.
• The extent to which the program is achieving its objectives for student learning.
• The quality and diversity of the faculty and staff and their contributions to achieving the program's mission and goals.
• The quality of the infrastructure supporting the program (e.g., library and other educational resources; physical facilities, etc.).

Note: Student learning is as important in graduate programs as it is in undergraduate programs. Moreover, graduate programs have the responsibility to ensure that learning occurs at an advanced level and is not simply an extension of knowledge beyond the undergraduate curriculum. The Cal Poly catalog states:

"In graduate courses, students cope with more complex ideas, problems, techniques and materials than in undergraduate courses. Graduate study requires searching and exhaustive analysis, identification and investigation of theories and principles; application of theory to new ideas, problems and materials; extensive use of bibliographic and other resource materials, with emphasis on primary sources for data; and demonstration of competence in scholarly presentation of the results of independent study."

Programs are strongly encouraged to involve their faculty in all stages of the APR process, including design of the self-study. The Program Head/Chair, the Dean of the College, the Dean of Research and Graduate Programs, and the Vice Provost all sign off on the self-study proposal and the timeline for its preparation.

The self-study report, which documents the review, is shared with the Dean of the College, the Dean of Research and Graduate Programs, the Vice Provost, and the reviewers. It provides a context for the subsequent site visit and the action planning phase of the review process. Therefore, the self-study must demonstrate an evidence-based and consultative approach to the review of the educational effectiveness of the program.

Self-Study Content

All programs are expected to use in their analysis and include as a part of their self-study the following (in no particular order):

General information about the program

• An updated progress report on the action plan from the previous APR cycle (where applicable).
• An executive summary of data on students, faculty, staff, facilities, etc. Some data will be supplied by the Office of Institutional Planning & Analysis at the request of Academic Programs Office. Requirements for assessment data are described below:
  • A record of the titles of completed student theses/projects since the last self study.
  • A list of faculty who participate in the delivery of the program and a bulleted summary of their research and/or professional development interests
  • A description of the attention the program has paid to factors affecting graduation rates and other elements of student success (e.g., realistic curricular flowcharts, with courses scheduled to make timely graduation possible; procedures to maximize the likelihood of successful completion of thesis/project, etc.).

  Please use the summary to draw conclusions from the data/evidence presented. Avoid presenting raw data. If necessary, this can be added to an appendix, provided on a CD or by some other electronic access.

Data from the assessment of student learning outcomes

Use the most recent submission from the department of the WASC program assessment inventory (PDF) as the outline for this section of the self-study.

The most important part of this section will be the description you provide about how faculty are using the evidence to improve learning experiences for students.

The self-study report should include:

• The program level learning outcomes for students (preferably expressed as measurable outcomes using Observable Verbs for Cognitive Domain Instructional Objectives (PDF).

  Note: Conduct a self assessment of the program learning outcomes by using the WASC Rubric for Assessing the Quality of Academic Program Learning Outcomes (PDF).

• An indication of where the program outcomes are publicized to students or will be publicized (see WASC Rubric for Assessing the Integration of Student Learning Assessment into Program Reviews (PDF)).
• The current plan for assessment of these outcomes. The plan should include the direct and indirect measures that are used for assessing the program learning outcomes.
• Indicate who has reviewed the plan in the past six years (program faculty, faculty in the college, faculty at the university, academic administrators in the college, academic administrators in central administration, external reviewer and/or consultants).
• An executive summary of assessment results since the last program review. (Append the raw data only if deemed necessary for the external reviewers.)
• Indicate which examples of student work (preferably work that was used as an assessment artifact) will be available for reviewers and whether reviewers will be meeting with students. Please do not include the samples of student work in the self-study but make them available for the reviewers either on blackboard site (to which the reviewers are provided access) or by some other means (see WASC Rubric for Assessing the Integration of Student Learning Assessment into Program Reviews (PDF)).
• A description of how the faculty uses the evidence (e.g., the decision making processes that leads to data analysis, data interpretations and recommendations for improvements at the curricular or programmatic level). This is the most critical aspect to focus on in the assessment section of the self-study report (see WASC Rubric for Assessing the Integration of Student Learning Assessment into Program Reviews (PDF)).
• Where possible, provide evidence of the impact of the curricular or programmatic change.

Note: For more information on assessing student learning outcomes at the graduate level, see the Addendum on Assessment of Graduate Programs. Assistance with assessment efforts is available from the college assessment council or other individuals identified by the Dean’s office. The Associate Vice Provost for Academic Programs, in consultation with the Academic Assessment Council, can provide consultation and additional information on program level assessment (see the WASC Rubrics for The Evolution of Program Level Assessment (PDF), Assessing the Quality of Academic Program Learning Outcomes (PDF), Assessing the Use of Portfolios for Assessing Program Outcomes (PDF), and Assessing the Use of Capstone Experiences for Assessing Program Outcomes (PDF)).

Alignment of university learning objectives with program curricula

The goal of this section of the self-study is for the program to describe, using a map or matrix, where in the curriculum students are introduced to (I), develop (D) or master (M) university learning objectives (where applicable include college learning objectives). A sample map will be provided.

Alignment of university diversity learning objectives with curricula/program experiences

University diversity learning objectives were approved in the ’07-’08 academic year. We are asking all departments up for review in 2008-2012 to include evidence of how the program provides or intends to provide opportunities for students to meet these objectives through the curricula and/or program experiences.

One outcome for the GE Assessment plan for 2007-2009 will be to propose measurable outcomes for various aspects of skills, knowledge and abilities associated with the university diversity learning objectives. As information becomes available, we will provide it to departments undergoing program review. There may be some applicability of this work to graduate programs.

Evidence of an appropriate culminating experience

As graduate programs have evolved to meet both marketplace and student demands, many have become more interdisciplinary and encourage team approaches and cooperative learning. This has led to innovative ways to provide culminating experiences for students. Whether employing these new methods or using more traditional ones, programs must still demonstrate how they comply with Title V and Cal Poly catalog requirements for the culminating experience for each student. (If no changes to the culminating experience have been made since the last review, simply indicate this in the self-study report.) Describe assessment data and/or other forms of evidence to demonstrate that the culminating experience, as practiced in this degree program is having the intended outcomes (see the Addendum on Assessment and specific guidance on culminating experiences).

Specific issue of concern

When there is an issue of particular interest or concern to the program in regard to its educational effectiveness, and/or an issue shaped by the context of what is currently important to the college, the Dean of Research and Graduate Programs or university, please include this in the self-study. This issue is proposed by the program faculty to the Dean of the College, Dean of Research and Graduate Programs and Vice Provost for their approval. Some examples of issues that might be raised in a review are governance, curriculum, faculty, resources and access and assessment (see “Features of Quality Graduate Programs” below).

Features of Quality Graduate Programs

In its 2000 report entitled “Graduate Education at Cal Poly,” Cal Poly's Task Force on Graduate Education provided a set of features that define quality graduate programs. These features can provide an outline for the self-study. They are:

Governance

• The department or program has a graduate coordinator.
• The graduate coordinator has adjustment of teaching load in order to effectively administer the program.
• The department or program has a committee to deal exclusively with graduate issues.
• Graduate students are involved in program evaluation processes.
• The program coordinator works cooperatively through a campus-wide graduate committee.

Curriculum

• A core curriculum exists that emphasizes integration of knowledge and preparation for specialization and the intellectual rigor of this curriculum is clearly beyond that of the baccalaureate level.
• At least four regular graduate courses are available per year in addition to supervised and independent study.
• No more than 20% of unit requirements for graduate level work involve independent study (excluding thesis). In disciplines which are research-intensive, 30% is allowable.
• Graduate level (i.e., 500-level) courses should make up 70% of a program.
• Availability of graduate level coursework is increased by "pooling" or cross-registration between related departments.

Faculty

• Teaching faculty has Ph.D. or other appropriate terminal degree and relevant professional experience.
• Faculty has a strong commitment to professional development in the scholarship of discovery, the scholarship of integration and/or the scholarship of application.
• Faculty interacts with students in a professional and civil manner in accordance with University policies.
• Faculty members create supervisory relations with students that stimulate and encourage in the classroom, lab, or studio.
• Advisors develop a clear understanding with their students about their specific research responsibilities, including the time lines for completion of research and the thesis or project.
• Faculty members create an ethos of collegiality so that learning takes place within a community of scholars.
• Faculty encourages graduate students to participate in professional meetings or perform or display their work in public settings.
• Faculty acknowledges student contributions to research presented at conferences, in professional publications, or in applications for copyrights and patents.

Resources

• Space, facilities, equipment, and support staff are committed to carry out meaningful research and/or creative activities.
• The department or program is able to provide opportunities for research, scholarship, and creative activities.
• Each graduate student has a major professor and faculty committee.
• All students have the opportunity to be involved as teaching assistants where appropriate to the discipline.
• Some form of financial support (TA's, RA's, scholarship, fellowship, aid) is available for all students.
• Offices or study areas are provided within the department to provide interaction with faculty.

Access and Assessment

• Discriminatory barriers to present and potential students are absent.
• Graduate student population has high diversity (e.g., ethnic, geographic, etc.).
• Certificate programs are used to respond to the need for occupationally related graduate coursework.
• Program is reviewed and evaluated regularly to ensure that the graduate program is given specific attention separate from the other offerings of the department.
• Students make timely progress toward degree completion.

Self-Study Report Format Guidelines

Although no single presentation format is prescribed for the self-study report, the report should respond to the self-study content items #1-5 (and #6 as applicable) listed above, and follow the general guidelines listed below. For the benefit of reviewers, please keep in mind that the self-study report should contain relevant summations of any data and read like an executive summary.

1. A cover page reflecting the name of the program being reviewed, the name of the college, the name of the department chair, and the date of the self-study.
2. A table of contents.
3. A “Standard Program Information” page immediately following the table of contents. This page will provide a general overview of the program for the reviewers and should include the information listed below:
   • Name of program, department, and college
   • Program mission statement
   • Department location
   • Department web link
   • Complete list of programs offered in the department including concentrations/specializations and minors.
   • Number of faculty, instructors, lecturers, and staff
   • Number of students enrolled in program
   • Total operating budget (including expenses)
   • List of specialized facilities
   • List of urgent space needs
4. The body of the report should be narrative and may include relevant tables, charts, or graphs that summarize data findings.
5. Web links can be included in the body or appendix of the document. Avoid printing web pages to include them in the self-study.
6. Raw data, print outs, faculty vitae, lists of publications, and similar documentation should be burned onto a CD or added to a Blackboard site for the reviewers.
7. The length of the report should be at least 25 pages but no more than 50. Double sided printing is encouraged.
8. List page numbers in the bottom right corner of the document.
9. Use a spiral/comb binding or a staple in the top left corner to bind the report in lieu of a three ring binder.
10. Accredited programs will provide two copies of the final self-study report; one for the Academic Programs Office and one for the internal reviewer. Non-accredited programs will provide one copy for the Academic Programs Office, one for the internal reviewer and one for each of the external reviewers. In addition, both accredited and non-accredited programs will provide an electronic copy of the final self-study report to the Academic Programs Office.