COUNCIL ON ACADEMIC ASSESSMENT (CAA)
MINUTES, SEPTEMBER 26, 2005
LC-31J; 1:00 – 2:30 PM
Bruce Szelest, Maria Brown, Joel Berkowitz, Marjorie Pryse, Sue Faerman, Bill Lanford,
Sean Walmsley, Malcolm Sherman, Carol Jewell, Kristina Bendikas, Olimpia Pelosi,
Barbara Wilkinson
Sherman began the meeting with a welcome and an explanation of the purpose of the
council, and its operating policies and procedures. External authorities require that all
departmental degree programs, both graduate and undergraduate, be assessed on a seven
year cycle. The council receives and reviews departmental self-study documents and
external reviewer reports. The council plans to expand the scope of these reviews to
include minors and service courses. The documents submitted by departments and the
reports of external reviewers are kept in Wilkinson’s office and are available to faculty
and administrators who request them. Off campus requests for these documents will be
processed with a slight time delay. Last year the Council was divided into two
committees (whose titles are only roughly descriptive), a qualitative committee and a
quantitative committee, each of which receives half of the mandated departmental
assessments. We will likely continue this arrangement. Because this council has existed
for only one year, things are less settled than with other councils.
To start the academic year, introductions were made and meeting times discussed. It was
decided that the council would meet on Mondays from 1:30-3:00, on days when no
Senate meeting was scheduled. The council will likely meet monthly.
The council next heard the quantitative group report on the Chemistry self-study
document and external reviewer report. Sherman asked whether this report was sent to
the Dean. Lanford responded that it had not, that it is a draft. Lanford commented that
the council needs to be careful about including external reviewer suggestions in the
report, so it doesn’t appear as if the council is taking a stand on the substance of
recommendations as opposed to whether the self-study covered the required ground.
Sherman noted that the weaknesses were understated, but that the comments would serve
to alert the dean to look at specific portions of the self-study document deserving of
particular attention. Pryse asked what was the point of reviewing the external reviewer
report if we don’t include any comments? Faerman explained two potential roles the
council might adopt: one that would focus on the process of assessment, and the other on
the outcome. The former may be addressed by asking whether the Chemistry Department
is working on programs during the seven years between reviews. The latter may be
addressed by asking whether there are discrepancies between the report and what we
know. Brown asked whether the council focus is to be on the process? Lanford replied
that the role of the council is not to evaluate the program, but to point to significant parts
of the assessment and to makes sure the assessment was done properly. The Chemistry
Department’s assessment of student learning is based on student’s scores on an external
exam sponsored by the American Chemical Society. The Chemistry self-study document
doesn’t indicate how few students become senior chemistry majors. Few students,
therefore, take the exam. Lanford explained that the exam is nationally normed, that
there are several ACS exams, and that it represents a far more rigorous measure of
student learning than is used or available in other departments. Retention is identified as
a problem in the weaknesses section of the CAA report, and in the Chemistry
Department’s self-study document. Sherman returned to an earlier question about how
the external evaluators were chosen. Szelest explained that five to six names were
provided by the department and that Pryse and Faerman make decisions based mainly on
securing external reviewers whose experience covers the programs offered. Faerman
suggested adding a second paragraph to the CAA report, stating that the department
needs to focus attention on the learning environment in order to address retention.
Sherman suggested that this is conveyed under section D of the report. Lanford added
that another option would be a follow-up by the assessment office to find out why there
were so few majors. Faerman drew a parallel to the General Education Program’s
learning outcomes and assessment of engagement, using either the campus Student
Experience Survey or the NSSE. She suggested that the Chemistry Department should
look into this. Concern was expressed that the tone of the report is so positive as to
overshadow items C and D under the weaknesses section. It was suggested that the
quantitative committee take out the complementary language. Lanford responded that in
terms of thoroughness and frankness, the self-study document is exceptionally well done.
Sherman added that the report first complements the department to build them up, then
slips in the weaknesses. Berkowitz explained that when the other group reviewed self-
study documents and external reviewer reports, they focused on how well the department
assessed the programs. Lanford mentioned that the lack of retention issue was addressed
in the external reviewer report, and suggested hiring a Science Education staff person to
address the problem. It is not our role to support this suggestion in our report. Sherman
suggested adding a recommendation for the department to look at student engagement
and retention. Brown asked about the format of the CAA reports. Lanford asked
whether the Chemistry report was similar to the others. It was suggested that the Senate
be added to the list of recipients of our reports. Faerman suggested that each report
include wording to indicate that the focus will be on the assessment process, and that we
rarely take stands on substantive issues. Brown said that other reports focused on areas
that were missing from the self-study document.
Sherman moved to the addendum document. Paper copies were then distributed to
council members. Sherman explained that this was a joint project of several of last year’s
council members, with an appendix supplied by Collier. While many hands were
involved, the effect was mainly to make the document shorter and clearer. The purpose
of the document is to require departments to assess the effectiveness of instruction of
minors and service courses. At a previous council meeting, it was agreed that the section
referring to “LC courses” be removed. Szelest explained that this realization that minors
and service courses needed to be included for review came out of reviewing self-studies
such as Chemistry and languages, the bulk of whose teaching is directed toward non-
majors. This document will go to the Senate as a report and will be woven into the
assessment template.
Sherman asked, when will 2005-2006 program review self-study documents be ready to
be read? Wilkinson responded that the first is likely to be ready in a month or two, and
that the rate of submissions will increase as the academic year progresses.
Walmsley noted that self-study documents assess the majors when undergraduate
programs include majors and minors. But in the case of a joint minor, in which
department’s self-study are they to be assessed? No general answer was given. Faerman
commented that Sociology, for example, has both a major and a minor program.
Presently, little attention is paid to the minor program and students. Journalism was a
minor in the English Department, but operated quite independently. Lanford suggested
that every student has a major and it may be possible to get at everything by focusing
only on majors. It might be good to assess minors, but we haven’t mastered majors yet.
Berkowitz added that in Judaic Studies many more students are minors than are majors.
Pryse suggested adding “asks reviewed academic units” to the second paragraph, before
the bolded text. Sherman asked whether the council approved the addendum with the
proposed changes. The council would then send it as a report to the Senate. Szelest
proposed a modification to the second paragraph of the Phase-In section. The
assessment of the minor would become effective beginning with the 2006-2007 program
reviews. Lanford suggested that we submit it as a report to the Senate, and say we are
unsure we have the authority to adopt it. Sherman suggested that we replace “proposes”
with “plans” in the text. The original document was likely written by Joshua Smith, who
was the former Director of Assessment. Faerman asked whether the Provost’s
Assessment Advisory Committee was the place to make this decision. Sherman argued
that the CAA has the authority to determine both the scope of the assessment and the
procedural rules. Sherman asked for a vote on the addendum document. Eleven
approved. One dissented.
Sherman moved to the numbers of faculty on the committee. Due to two faculty
members being unable to participate on the committee, we currently have the minimum
number of faculty members. We may wish to seek a quantitative social scientist to join
the council. Would anyone like to recommend someone?
The next meeting was scheduled for Monday, November 7th from 1:30 – 3:00.
Respectfully submitted by Barbara Wilkinson
