By Phillip Wankat and Frank Oreovicz

STUDENTS EVALUATIONS ARE HERE TO STAY, SO WE SHOULD GET AS MUCH OUT OF THEM AS POSSIBLE.

Many engineering professors are opposed to student evaluations because they are subjective and can unfairly cast a bad light on a teacher's abilities. There are a number of scientific studies about their reliability. Two that are easily accessible are Reflective Faculty Evaluation (Jossey-Bass, San Francisco, 1993) by John Centra, and chapter 16 in our book Teaching Engineering (www.purdue.edu/ChE/News_and_Events/publications/Teaching_Engineering).

The factors that students consider when filling out their evaluations include how much they learned, the professor's enthusiasm, their rapport with him or her, course content, and grades. Students cannot evaluate the appropriateness of content, but they have the advantage of seeing all aspects of a course including lectures, homework, handouts, tests, office hours, and teaching assistants. And partly because they do see everything, summative (end of semester student ratings) are considered reliable. Typical correlation coefficients for reliability are r = 0.69 (10 students), 0.81 (20 students), and 0.89 (40 students). Contrary to anecdotal evidence, alumni ratings of overall performance are highly correlated with student ratings (r values from 0.75 to 0.83).

The validity of student ratings (they do correlate with student learning) has been extensively studied. The highest correlation coefficient is between the instructor's global skill as a teacher and student learning (r = 0.50). Thus, students tend to learn more from teachers they rate highly. [To calibrate these numbers, note that the correlation between high school and college grades is approximately r = 0.45. The correlation between professors' student ratings and measures of research productivity is r = 0.12.]

Formative (during the semester) evaluations are obviously more useful for course improvement than the summative—the course is still in session and there is time for improvement. With summative evaluation, specific questions about teaching methods are useful for improving subsequent courses, but one needs to work with a coach to experience significant benefits. Formative evaluations can be as informal as asking the class to write what you can do to help them learn.

A major problem with student evaluations is the number of extraneous variables that affect ratings. The most important of these are the students' initial motivation and expectations for the course. Students who expect to like the course usually do like it (r: 0.45). Class size is important, and classes with fewer than 15 students receive significantly higher ratings than larger classes. Since students tend to learn more in smaller classes, the increased rating of the course has some validity. Engineering, mathematics, and science students tend to rank professors slightly lower than students in other disciplines. Elective, lecture, and senior courses tend to garner higher ratings than required, laboratory, and first-year courses, respectively.

Because extraneous factors affect ratings significantly, comparisons of raw ratings for administrative purposes such as assigning teaching awards or merit raises are not justified. The good news: Most of the extraneous factors can be statistically adjusted for. The bad news—these adjustments are seldom done.

To improve the student evaluation process, teachers should insist that statistical corrections be made before using evaluations for administrative purposes. And since the overriding purpose of teaching evaluation is to improve teaching, other methods should be strongly encouraged, such as personal reflection, attending teaching workshops, consultation with experts, video-taping lectures, and preparing faculty portfolios.

Phillip Wankat is head of interdisciplinary engineering and the Clifton L. Lovell Distinguished Professor of Chemical Engineering at Purdue University. Frank Oreovicz is an education communications specialist at Purdue's chemical engineering school. They can be reached by e-mail at purdue@asee.org.