Senate Resolution No. 0304-05R
University at Albany RESOLUTION
on General Education Assessment
Introduced by SUNY Faculty Senator Richard Collier, 4/19/04
Whereas the University at Albany has already taken a campus position against any
system-wide assessment (last year's Senate Resolution No. 0203-02R, attached) and finds
nothing in the two attached documents from the “Campus-based Assessment Committee”
that persuades us to change this position,
Whereas our campus-based assessment, developed in accordance with SUNY's guidelines
and category definitions and fully approved by GEAR, infuses the Written
Communication and Critical Thinking requirements throughout our curriculum and
embeds their assessment in the full structure of our courses,
Whereas we are completing the second year of our three-year cycle of General Education
assessment, with embedded assessment of Written Communication and Critical Thinking
that does not involve or lend itself to one or more examinations,
Whereas the latest version of System-wide Assessment speaks of "instruments" (which
we deem to be examinations) and requires our results to be correlated statistically with
the results from nationally-normed external testing, and
Whereas we do not find either compelling or intellectually valid the argument that this
latest version should be supported because portions of it may be less objectionable than
prior (or future) versions,
BE IT RESOLVED THAT: the faculty of the University at Albany will continue to follow
the General Education Assessment Plan of its own, and not support, embrace, or willingly
participate in a SUNY-mandated standardized General Education Assessment Plan such
as that recently proposed by the “Campus-based Assessment Committee.”
RATIONALE:
The documents “Strengthened Campus-based Assessment” and “Proposed [GEAR]
Revisions (Spring 2004)” were produced by the ad hoc “Campus-based Assessment
Committee” that was charged with modifying System-wide assessment proposals
previously rejected by the University Faculty Senate, the Faculty Council of Community
Colleges, and the Student Assembly. The committee included representatives of the
Senate, the Faculty Council, the Student Assembly, SUNY System, and one Trustee (the
student member). The committee completed its work during its two meetings (March,
2004).
The University Faculty Senate has requested each SUNY campus to review and consider
endorsing the two attached documents and report its decision back to the University
Faculty Senate, which will decide whether or not to endorse the proposed documents at
its next Plenary Session, at Syracuse Health Sciences, April 22nd-24th.
The Undergraduate Academic Council carefully studied the proposal at its March 25th
meeting and determined that its requirement of a nationally-normed “instrument” to
assess Writing and Critical Thinking or a “locally developed instrument…demonstrated
to correlate statistically (i.e., have concurrent validity) with nationally-normed measures”
directly conflicts with our on-going, GEAR-approved assessment of these two important
General Education categories.
It was strongly emphasized we do not have an “exam” and will not develop a local
instrument for critical thinking or writing. We do not want an instrument as a test of
something not necessarily well tested by such a device. Our faculty members have
participated extensively in designing the assessment process and have agreed that at the
beginning of the semester the instructor will inform the students how the various
components will count into their final grade. The instructor is also required to explain to
them the connection between the class objectives and SUNYGER category objectives. At
semester end, the instructor reports to Albany's General Education Assessment
Committee students' achievement on those portions of course requirements relevant to the
general education category or categories being assessed.
The UAC notes that our learning objectives/outcomes are based on the SUNY definition
for the “Critical Thinking” category: “Students will identify, analyze, and evaluate
arguments as they occur in their own or others’ work; and develop well-reasoned
arguments.” In contrast, the proposal documents seem to imply the existence of generally
agreed upon national “norms” for critical thinking.
While the conflict with our assessment of writing and critical thinking is sufficient
grounds to reject the latest proposal, the UAC also questioned the usefulness of a System-
wide effort to assess the base level mathematics ability common to the many hundreds of
disparate SUNY courses that have been approved to satisfy SUNYGER Mathematics.
The campus-based assessment of each calculus, statistics, etc., course is both feasible and
may improve the course, in contrast to a statewide testing of such minimal high school
level mathematics as may be common to all those courses.
The UAC concluded this proposal for implementing System-wide assessment, although
arguably less objectionable than previous versions, remains unacceptable and urges the
University Senate to reject the proposal.
Senate Resolution No. 0203-02R
University at Albany RESOLUTION
WHEREAS an email sent from Associate Provost Donald Steven on the SUNYGED
Listserv on March 21, 2003 indicated that the Provost's office "would like to begin
discussing the Task Force recommendations on university-wide assessment…"; and
WHEREAS the purpose of general education assessment has been understood to be an
evaluation of a campus' programs, not an assessment of individual students; and
WHEREAS it is our belief that assessment needs to be a campus-based activity for the
following reasons:
Such a standardized system-wide assessment would interfere explicitly with the
fundamental academic freedom of instructors in discussing their subject, an
important element of higher education in a democracy.
A standardized system-wide assessment would appear to presuppose a system-
wide curriculum, which would tend to undermine the rigor and comprehensive
nature of each campus' General Education Program.
The University at Albany created a General Education Assessment Plan mandated
by SUNY System Administration which was approved by the University at
Albany's University Senate and submitted to SUNY GEAR, and which is
currently in its first year of a three-year cycle of implementation.
The University at Albany adopted the SUNY Learning Outcomes and Guidelines
for the Approval of State University General Education Requirement Courses and
in addition developed for our campus a more in-depth set of learning objectives.
The logistics of such system-wide standardized assessment would appear to be
insurmountable, given that general education is specifically defined as a four-year
program, with students following numerous individual routes in choosing from
the intellectually rich diversity that is appropriate to a university education; and
given that general education requirements can be met through coursework taken
at other institutions, including high schools and study abroad programs, making
an examination given at one campus an evaluation not only of that campus but of
many other teaching and learning environments.
Uniform system-wide standardized assessment at any point prior to the
completion of the four-year general education experience would not reflect the
accomplishments of the graduate.
Any funds which might be spent developing such a vehicle would be better spent
on enhancing individual campus-based undergraduate assessment.
BE IT RESOLVED THAT: the faculty of the University at Albany will continue to follow
the General Education Assessment Plan of its own, and not support, embrace, or willingly
participate in any SUNY-mandated standardized General Education Assessment Plan.
Strengthened Campus-based Assessment
Summary
Background
It is not the intent of this summary to retrace the University’s many efforts over the years
to develop a high quality assessment program. As we all know, colleagues have been
engaged for a long time in serious and conscientious efforts to develop a State University
Assessment Initiative that represents both “best assessment practices” and our own “best
thinking.” We can be very proud of what has been accomplished to date.
In its final report, the Provost’s Advisory Task Force on the Assessment of Student
Learning Outcomes underscored its belief that assessment serves two complementary
functions in higher education today: ‘Assessment as improvement’ and ‘Assessment as
accountability,’ and that both of these functions have an appropriate place in the SUNY
Assessment Initiative and can strengthen the University’s institutions and the System as a
whole.
In recent years, we have struggled to try to find a way to address both of these goals in a
way that faculty and stakeholders agree meets their concerns and expectations. After
many attempts, we believe that we have discovered an approach that is based on good
assessment practice, that addresses our faculty’s many legitimate concerns, and that
meets System goals as expressed by the Chancellor.
The Elements of a Proposal
We believe that proposal for a strengthened campus-based assessment process could be
structured as follows:
1. Existing campus-based assessment plans need only be revised and incorporated in the
proven and ongoing GEAR process. There would be no need for a second, System layer
of assessment.
2. Only the learning outcomes in three foundational areas: Mathematics, Basic
Communication (Written), and Critical Thinking (Reasoning), should be assessed
incorporating external measures and the assessment of these “building blocks” of general
education could be done in various ways that would provide real flexibility for campuses.
3. An instrument such as the National Survey of Student Engagement (NSSE), the
Community College Survey of Student Engagement (CCSSE), or a revised, extended
Student Opinion Survey (or similar measures) could be used to understand the campus
academic climate and how that might relate to the assessment results. This analysis of
assessment results could be incorporated into campus procedures.
4. System Administration should cover the cost of all externally referenced measures and
surveys for a representative sample of students.
5. Value-added assessment should be optional.
6. The proposed policy should contain language which will safeguard assessment data
from misuse.
Proposed Revisions to the GEAR Guidelines
To incorporate these elements into the GEAR process, very few changes are actually
needed:
Re “externally-referenced measures”, to GEAR Criterion #3, add the following:
For the learning outcomes in Mathematics, Basic Communication (Written), and
Critical Thinking (Reasoning), are externally referenced measures of the campus’s
choice — either nationally- or SUNY-normed — included?
For campuses opting to attempt to determine the growth in learning achieved by
SUNY undergraduates in some or all of general education (“value-added”), is
there an adequate description of when measures will be administered and how
problems commonly related to pre- and post-testing (e.g., student motivation,
attrition) will be controlled?
Every effort has been made to describe “externally referenced measures” in a way that
provides for meaningful external referencing while providing maximum flexibility for
campuses. Not only will this address the longstanding concern of faculty that no proposal
should promote a “one size fits all” approach, but it also renders any attempt to engage in
inter-campus comparisons essentially meaningless. Campuses will continue to use their
current approved assessment plans for all of the SUNY-GER learning outcomes. For the
learning outcomes in the three “building blocks,” there are a number of ways in which
campuses can meet the requirement for “externally-referenced” measures: for example,
through the use of nationally-normed tests such as the Academic Profile, the California
Critical Thinking Skills Test or the Quant-Q, or by using SUNY-normed measures in lieu
of nationally-normed measures, using one of two approaches:
i. A locally developed instrument that measures the learning outcomes in one or more of
these three areas and that is demonstrated to correlate statistically (i.e., have concurrent
validity) with nationally-normed measures.
ii. A locally developed instrument that measures the learning outcomes in one or more of
these three areas that is reviewed and approved by GEAR. As part of this process, GEAR
will rely on discipline-based panels of distinguished SUNY faculty that will develop
standards and rubrics campuses may use to assess student performance. Campuses
choosing to use their own standards and rubrics must demonstrate to GEAR that their
standards and rubrics are essentially equivalent to those developed by the discipline-
based panel.
Campuses opting for this approach would be expected to periodically provide GEAR
with samples of student work for each standard of student performance, to be reviewed to
ensure ongoing validity and reliability of the measure. Note that it is also possible to
combine these approaches: for example, a campus might choose to use the CCTST for
Critical Thinking and SUNY-normed measures for Mathematics and Basic
Communication (Written).
Re the “campus academic environment”, add a new GEAR Criterion (as #6) to:
Ask that “Mechanisms for assessing the campus academic environment are
described.”
Re student involvement, to address the need for greater student involvement:
Revise GEAR Criterion #7 — related to governance — to require that the plans
show evidence of student involvement in the development of revisions to the
assessment plan.
Safeguards re the Utilization and Reporting of Assessment Results
To ensure that assessment results are used appropriately, it is essential that this process be
guided by a set of principles that will safeguard assessment data, along the following
lines:
Assessment results will never be used to punish, publicly compare, or embarrass
students, faculty, courses, programs, departments, or institutions either
individually or collectively.
Assessment results will never be used to make public comparisons among groups
of students based on gender, race, ethnicity, or other demographic factors. A basic
value of the State University is that all students can learn and the University’s
programs are intended to provide educational opportunities to students as
individuals, not by virtue of their membership within a particular demographic
category.\
Given the vast diversity that exists among SUNY campuses — reflecting their
unique missions and constituent groups — the public dissemination of assessment
data for accountability purposes will take place only through aggregate reporting
for SUNY as a whole and by sector (Doctoral Degree-granting Institutions,
Comprehensive Colleges, Colleges of Technology and Community Colleges).
Campus-specific assessment data will be used for confidential in-house
discussions as part of the ongoing improvement process, whether between faculty
and administrators on a particular campus or between campus representatives and
System Administration officials.
Faculty support for, and participation in this initiative is predicated on the
responsible and collegial adherence to these principles.
Costs
The costs of the purchase and scoring of nationally-normed measures — as well as for
the NSSE or CCSSE — should be paid for by System Administration for State-
operated/funded institutions and community colleges, based on a sample size of up to
20% of the undergraduate student body on a three-year cycle.
Next Steps
System Administration has indicated that its goal is to move assessment forward through
consensus and the Chancellor has said that if a faculty proposal met his expectations and
has the support of governance, he would be willing to consider revisions to the June 17
Board resolution.
Proposed Revisions (Spring 2004)
General Education Assessment Review (GEAR) Group
Review Process Guidelines
I. Introduction
The General Education Assessment Review (GEAR) Group has been established upon
the recommendation of the Provost’s Advisory Task Force on the Assessment of Student
Learning Outcomes and has been formed jointly by leadership from the University
Faculty Senate, the Community College Faculty Council, System Administration and the
Student Assembly.
Comprised primarily of faculty from throughout the University, GEAR also includes
students, campus chief academic officers, and campus professional staff (particularly
from Institutional Research). GEAR is co-chaired by Dr. Patricia Francis, Professor of
Psychology and Executive Assistant to the President at the College at Cortland and Dr.
Donald Steven, Executive Vice Provost for Academic Affairs. GEAR’s Web page, which
includes a summary of its activities as well as many useful resource and reference
materials, may be accessed at http://cortland.edu/oir/gear/.
II. Goals
The GEAR Group’s goal is to work with campuses as they develop and implement their
campus-based plans for assessing student learning outcomes in General Education,
following the guidelines contained in the Task Force report as well as subsequent
discussions involving faculty and campus and System leadership. GEAR intends to
function as a resource and a colleague, making itself available to campuses to the extent
that they would welcome and in ways that they feel would be helpful, engaging them in a
dialogue as they develop and carry out their assessment plans. In its “process review” of
campus General Education assessment plans, GEAR will focus exclusively on the
campus’s assessment processes and procedures, not the assessment outcomes themselves.
III. Process
Each campus is responsible for determining the particular structure and content of its
campus-based General Education assessment plan, following its own existing governance
processes.
The task of developing and implementing a campus-based assessment plan for General
Education should fall primarily to the faculty members who teach in the program, with
the assistance of professional staff and students when appropriate. (Indeed, it may well be
the case that on some campuses a full-time staff and/or faculty assessment person may be
in a leadership role.) Campus-based assessment plans should be submitted to, and
approved by, the campus’s Faculty Senate or Faculty Council prior to being submitted to
the GEAR Group for formal review.
GEAR’s Expectations of Campus General Education Assessment Plans
In its initial review of campus assessment plans, the GEAR Group will use eight nine
criteria in evaluating a plan’s comprehensiveness and rigor. In addition to reflecting
widely recognized best assessment practices in higher education, these criteria are
consistent with the general guidelines included in the Task Force Report and subsequent
discussions, the expectations for assessment of the Middle States Commission on Higher
Education, and regulations proposed by the New York State Education Department as
part of its Quality Assurance Initiative in Higher Education.
In its initial review, the GEAR Group will seek to ascertain for each campus plan that:
1. The objectives for student learning in General Education relate directly to the
student learning outcomes defined in the Implementation Guidelines of the
Provost’s Advisory Task Force on General Education. The GEAR Group is likely to
agree that this criterion is met if all outcomes from the Implementation Guidelines are
reflected in the campus’ statement of General Education learning objectives for its
program. (It is important to note that campuses may also include additional learning
objectives that are specific to their own program.)
2. Programmatic activities intended to accomplish the campus’ objectives for
student learning in General Education are described. The GEAR Group is likely to
agree that this criterion is met by the campus providing GEAR with its guidelines or
procedures for designating courses as General Education courses.
3. The measures developed to assess student learning are designed to provide
credible evidence of the extent to which students have achieved the learning
outcomes or skills stated in the objectives. The GEAR Group is likely to agree that this
criterion is met if, for each learning objective, appropriate assessment measures have
been established for determining the degree to which students have mastered the
objective. In judging the appropriateness of a specific measure, the GEAR Group will
rely on answers to the following questions:
Will it directly measure student learning (i.e., as differentiated from the perception
that learning has taken place)?
Will it measure the objective it is intended to measure (i.e., will it have reasonable
face validity)?
Will the plan provide assurances that the measure is reliable, particularly with
respect to the ability of two independent scorers to rate it similarly (i.e., will it
have inter-observer reliability)? While this issue is less important for objective
measures (e.g., multiple choice exams), it is critical for qualitative approaches
(e.g., portfolios), which do not yield “one correct answer.”
For the learning outcomes in Mathematics, Basic Communication (Written), and
Critical Thinking (Reasoning), are externally referenced measures of the campus’s
choice—either nationally- or SUNY-normed —included?
Will the data that are reported be representative? It may not be feasible for
campuses to assess all students on a particular measure, nor is it necessary. The
campus assessment plan should therefore make it clear how representative
sampling of students will be assured when collecting assessment data.
For campuses opting to attempt to determine the growth in learning achieved by
SUNY undergraduates in some or all of general education (“value-added”), is
there an adequate description of when measures will be administered and how
problems commonly related to pre- and post-testing (e.g., student motivation,
attrition) will be controlled?
4. The plan proposes standards to which student performance relative to the
learning outcomes in the objectives can be compared. The GEAR Group is likely to
agree that this criterion is met if campus assessment plans include, for each learning
objective, the standard defining what level of student performance the faculty considers
as “exceeding,” “meeting,” “approaching,” and “not meeting” standards.
5. The anticipated results of the assessment are able to affirm the degree to which
the learning objectives have been achieved and thus make it possible to identify
areas that need to be addressed in order to improve learning. The GEAR Group is
likely to agree that this criterion is met if it is clear from the assessment plan that
mechanisms exist for sharing assessment results with appropriate faculty and staff and for
making programmatic improvements based on the assessment results (if necessary).
6. Mechanisms for assessing the campus academic environment are described. The
GEAR Group is likely to agree that this criterion is met if it is clear that the assessment
plan provides for the periodic administration of a survey that yields indicators reflecting
the campus academic environment (e.g. the National Survey of Student Engagement, the
Community College Survey of Student Engagement or a revised, extended SUNY
Student Opinion Survey or similar instrument.) and a report on what has been learned
from the campus’s consideration of the possible relationship between academic
assessment results and these environmental influences.
7. The assessment plan has been reviewed and approved through the appropriate
curriculum and faculty governance structures and shows evidence of student
involvement in the development of revisions to the assessment plan. The GEAR
Group is likely to agree that this criterion is met if the assessment plan includes a section
describing the process through which the plan was developed and approved on the
campus prior to being shared with the GEAR Group, as well as the efforts made to
include students in the process of revising the initial plan.
8. The plan adheres to the timetable established by the GEAR Group and agreed to
by the University Provost. The GEAR Group is likely to agree that this criterion is met
if it is clear that the assessment of all of the General Education learning objectives in the
Knowledge and Skills Areas and Competencies takes place within a three-year cycle.
(The campus plan should include the schedule for the assessment cycle.)
9. The assessment process includes provisions for evaluating the assessment process
itself and disseminating assessment results to the appropriate campus community.
The GEAR Group is likely to agree that this criterion is met if processes are described in
the assessment plan for evaluating the assessment process once complete, making
changes in the process if necessary, and sharing assessment results with the appropriate
campus community.
Initial Review
GEAR will receive and critique campus assessment plans and approve those that meet its
expectations for effective assessment; campuses will be advised in writing of revisions
that would likely lead to approval, as appropriate. GEAR will place a strong emphasis on
the extent to which campuses demonstrate they will use assessment results to improve
their General Education programs.
Ongoing Review
After the initial review process, the GEAR Group will review campus General Education
assessment plans on a biennial, staggered basis, applying the same criteria as above, with
greater emphasis on how campuses are using assessment data to improve their General
Education programs.
IV. Reporting
GEAR will establish a clear protocol and a standardized reporting format—consistent
with the recommendations of the Provost’s Advisory Task Force on the Assessment of
Student Learning Outcomes—for campuses to use to report assessment results in General
Education to System Administration for the purpose of accountability. This annual report,
to be submitted by the Chief Academic Officer at each campus directly to the Office of
the Provost, will include specific information on its students’ progress in mastering the
learning outcomes outlined in the General Education Implementation Guidelines. System
Administration will use these data—in accord with the Utilization and Reporting of
Assessment Results principles in the Task Force report—in the preparation of summary
reports to external stakeholders for accountability purposes.
V. Summary
The GEAR Group will continue the long tradition of involving existing faculty
governance and curriculum review structures on individual State University campuses in
the process of assessment. This involvement of SUNY faculty was central in the early
1990’s when the State University was playing a leadership role nationally in the
assessment movement, and it has certainly characterized the deliberations of the Task
Force on the Assessment of Student Learning Outcomes that has provided the raison-
d’être for GEAR.
Approved by GEAR: October 16, 2001, updated: December 5, 2003
Proposed revisions, Spring 2004
Appendix
Nationally-normed Measures
There are many nationally-normed measures available that are designed to assess
learning outcomes in Mathematics, Basic Communications (Written), and Critical
Thinking (Reasoning). The table below shows a number of examples of some which
campuses may find useful:
Discipline
AP
CAAP CCTST
CRA
QUANT-Q
Mathematics
x
x
x
Basic Communications
(Written)
x
x
Critical Thinking
(Reasoning)
x
x
x
x
x
AP Academic Profile, Educational Testing Service (www.ets.org/hea/acpro/)
CAAP Collegiate Assessment of Academic Proficiency, ACT (www.act.org/caap/)
CCTST The California Critical Thinking Skills Test, INSIGHT Assessment
(www.insightassessment.com/test-cra.html)
CRA California Reasoning Appraisal, INSIGHT Assessment
(www.insightassessment.com/test-cra.html)
QUANT-Q Quant-Q, INSIGHT Assessment (www.insightassessment.com/test-cra.html)
Costs
The costs of the purchase and scoring of nationally-normed measures—as well as for the
National Survey of Student Engagement or the Community College Survey of Student
Engagement—will be paid for by System Administration for State-operated/funded
institutions and community colleges, based on a sample size of up to 20% of the
undergraduate student body on a three-year cycle.
SUNY-normed Measures
Campuses wishing to include SUNY-normed measures in lieu of nationally-normed
measures in Mathematics, Basic Communication (Written), and Critical Thinking
(Reasoning) may use one of two approaches:
iii. A locally developed instrument that measures the learning outcomes in one or more of
these three areas and that is demonstrated to correlate statistically (i.e., have concurrent
validity) with nationally-normed measures, including those listed above.
iv. A locally developed instrument that measures the learning outcomes in one or more of
these three areas that is reviewed and approved by the GEAR Group. As part of this
process, GEAR will rely on discipline-based panels of distinguished SUNY faculty that
will develop standards and rubrics campuses may use to assess student performance.
Campuses choosing to use their own standards and rubrics must demonstrate to GEAR
that their standards and rubrics are essentially equivalent to those developed by the
discipline-based panel.
Campuses opting for this second approach would also be expected to periodically provide
GEAR with samples of student work for each standard of student performance, to be
reviewed by GEAR to ensure ongoing validity and reliability of the measure. GEAR will
provide campuses with feedback and, possibly, recommendations regarding any scoring
adjustments that may be required.
Mixing and Matching
It is also possible to combine these approaches: for example, a campus may choose to use
the CCTST for Critical Thinking and SUNY-normed measures for Mathematics and
Basic Communication (Written).
