By Phillip Wankat and Frank Oreovicz
STUDENTS EVALUATIONS ARE HERE TO
STAY, SO WE SHOULD GET AS MUCH OUT OF THEM AS POSSIBLE.
Many engineering professors are opposed to
student evaluations because they are subjective and can unfairly
cast a bad light on a teacher's abilities. There are
a number of scientific studies about their reliability. Two
that are easily accessible are Reflective Faculty Evaluation
(Jossey-Bass, San Francisco, 1993) by John Centra, and chapter
16 in our book Teaching Engineering (www.purdue.edu/ChE/News_and_Events/publications/Teaching_Engineering).
The factors that students consider when filling out their
evaluations include how much they learned, the professor's
enthusiasm, their rapport with him or her, course content,
and grades. Students cannot evaluate the appropriateness of
content, but they have the advantage of seeing all aspects
of a course including lectures, homework, handouts, tests,
office hours, and teaching assistants. And partly because
they do see everything, summative (end of semester student
ratings) are considered reliable. Typical correlation coefficients
for reliability are r = 0.69 (10 students), 0.81 (20 students),
and 0.89 (40 students). Contrary to anecdotal evidence, alumni
ratings of overall performance are highly correlated with
student ratings (r values from 0.75 to 0.83).
The validity of student ratings (they do correlate with student
learning) has been extensively studied. The highest correlation
coefficient is between the instructor's global skill
as a teacher and student learning (r = 0.50). Thus, students
tend to learn more from teachers they rate highly. [To calibrate
these numbers, note that the correlation between high school
and college grades is approximately r = 0.45. The correlation
between professors' student ratings and measures of
research productivity is r = 0.12.]
Formative (during the semester) evaluations are obviously
more useful for course improvement than the summative—the
course is still in session and there is time for improvement.
With summative evaluation, specific questions about teaching
methods are useful for improving subsequent courses, but one
needs to work with a coach to experience significant benefits.
Formative evaluations can be as informal as asking the class
to write what you can do to help them learn.
A major problem with student evaluations is the number of
extraneous variables that affect ratings. The most important
of these are the students' initial motivation and expectations
for the course. Students who expect to like the course usually
do like it (r: 0.45). Class size is important, and classes
with fewer than 15 students receive significantly higher ratings
than larger classes. Since students tend to learn more in
smaller classes, the increased rating of the course has some
validity. Engineering, mathematics, and science students tend
to rank professors slightly lower than students in other disciplines.
Elective, lecture, and senior courses tend to garner higher
ratings than required, laboratory, and first-year courses,
Because extraneous factors affect ratings significantly,
comparisons of raw ratings for administrative purposes such
as assigning teaching awards or merit raises are not justified.
The good news: Most of the extraneous factors can be statistically
adjusted for. The bad news—these adjustments are seldom
To improve the student evaluation process, teachers should
insist that statistical corrections be made before using evaluations
for administrative purposes. And since the overriding purpose
of teaching evaluation is to improve teaching, other methods
should be strongly encouraged, such as personal reflection,
attending teaching workshops, consultation with experts, video-taping
lectures, and preparing faculty portfolios.
Phillip Wankat is head of interdisciplinary engineering
and the Clifton L. Lovell Distinguished Professor of Chemical
Engineering at Purdue University. Frank Oreovicz is an education
communications specialist at Purdue's chemical engineering
school. They can be reached by e-mail at firstname.lastname@example.org.