Quantitative Reasoning Report (2017)
Quantitative Reasoning: AY 2015-16 Scoring Session Results
The assessment of Quantitative Reasoning began with work completed by the QR Learning Community in 2015-16. The community, comprised of faculty members representing each college on campus, developed a working definition of QR as “…the ability to make (or critique) a persuasive argument about a real-world or discipline-specific problem based on numerical evidence”. Based on this definition, the community developed a rubric with four traits: Problem Identification, Quantitative Analysis, Visual Presentation and Oral/Written Communication.
For this assessment, over 240 student assignments were collected from four lower-division GE courses PSY 202, ECON 222, MATH 112, STAT 130) with approximately 60 artifacts from each course. On June 27 and June 28, 2016, twenty faculty members scored the assignments after norming was completed using the four-point Quantitative Reasoning Rubric. Levels of performance on the QR rubric range from 1 for “Limited Proficiency” to 4 for “High Proficiency.” Each assignment was scored twice by the readers and discrepancy scores greater than 1 point were scored by a third reader. Results of the scoring sessions are below.
Averages and standard deviations of scores for each trait by subject:
A third reading was used whenever there was a two-point discrepancy in scores on any trait between the two readers of an artifact. If the third reading coincided with either of the first two reader’s scores, the third reading would replace the non-matching score. If the third reading was in between the first two readings, a coin would be flipped to decide whether the score should be rounded up or rounded down. If the coin said to round up, the third score would replace the lower of the first two and vice versa for rounding down. 21 artifacts (approximately 9%) required a third reading. Of these 21 artifacts, 12 contained a discrepancy in one trait, 7 contained a discrepancy in two traits, and 2 contained a discrepancy in three traits. A breakdown of the 32 total discrepancies by trait follows
