By Alvin P. Sanoff

DESPITE ITS LIBERAL ARTS FOCUS, ALVERNO COLLEGE MAY BE ABLE TO TELL ENGINEERING SCHOOLS A BIT ABOUT ASSESSMENT.

On the surface it would seem that a small Catholic women's college in Wisconsin that specializes in the liberal arts would have little in common with the nation's engineering schools. But dig beneath the surface and you find that Milwaukee's Alverno College and engineering institutions share a concern with assessing student learning and ability-based education.

This focus is still somewhat rare in higher education. Most colleges require only that students attain acceptable grades in a set of required courses to receive their degree. They expect students to demonstrate, through tests and papers, that they have acquired specific bits of knowledge and mastery of certain concepts. But students are not expected to demonstrate mastery of specific abilities, such as showing they can work in teams, and to prove that they have learned to apply their knowledge.

Zoreh Emami, associate dean for academic affairs at Alverno, says "there is a culture in higher education that holds the view that ‘we don't care what students are doing so long as we deliver the knowledge.' " There is a tendency, she adds, "to view thinking as a passive act and to look down on" the idea of actually having students do things, which is often dismissed as "vocational education." But Alverno is unapologetic about the fact that doing is a central part of the way it educates students.

It is the emphasis that Alverno places on assessment and outcomes—assuring that students have the knowledge and ability to meet specified performance goals—that prompted the ExxonMobil Foundation to give the school a $125,000 grant two years ago. The foundation believed that the wisdom Alverno had accumulated in more than three decades of building its education program around assessments and outcomes could benefit engineering schools striving—in some cases struggling—to meet the revised standards adopted by the Accreditation Board for Engineering and Technology. Those standards include requirements that schools demonstrate that their graduates are capable, not just in content areas but in areas considered vital for workplace success, such as functioning on multidisciplinary teams, communicating successfully, and understanding professional and ethical responsibility.

While some of the ABET outcomes are comparable to those at Alverno, others are not. At Alverno, students must demonstrate mastery in eight areas: communication, analysis, problem-solving processes, social interaction, developing a global perspective, making value judgments and independent decisions, effective citizenship, and aesthetic engagement, i.e. involvement with the arts

Within each of these eight areas, Alverno has defined six levels of achievement. Students must reach the sixth level in their major and the fourth level in other parts of the curriculum. The Alverno approach is difficult to grasp in the abstract. It becomes clearer when cast in terms of a specific discipline. In chemistry, for example, majors are expected to: communicate effectively using the language, concepts, and models of chemistry; use the methodology of chemistry to define and solve problems independently and collaboratively; use a wide variety of laboratory techniques with accuracy, precision, and safety; find, select, and use appropriate scientific information to support their work; use values and scientific information to make responsible decisions about the use of chemical materials and knowledge; use different strategies and models of chemistry to analyze and synthesize chemical data; critique data, strategies, and models of chemistry; and apply learning in an off-campus, professional setting.

Ann van Heerden, associate professor of chemistry at Alverno, has structured her junior-level biochemistry course around several of the abilities that students are expected to master. For example, to help students develop the ability "to use values and scientific information to make responsible decisions about the use of chemical materials and knowledge," she requires students to take part in a panel discussion on a controversial biochemistry topic and then to write a self-assessment in which they provide evidence from the discussion to show that they have met the assessment criteria. She, in turn, provides students with feedback on their self assessment.

Van Heerden says that to achieve desired outcomes a course must be well thought out and carefully structured. Trying to graft assessment components onto an existing course is not effective, Alverno faculty members say. The idea is to design the course based on "the kind of thinking you want students to do rather than starting with the material you want the students to cover," says Tim Riordan, associate dean for academic affairs at Alverno. He acknowledges that "getting faculty members to articulate clearly the things they want students to demonstrate is a challenge."

Judith Miller, director of the Center for Educational Development, Technology, and Assessment at Worcester Polytechnic Institute (WPI) in Massachusetts, says that Alverno's approach represents "the gold standard" in assessment. The Milwaukee school's reputation was instrumental in the ExxonMobil Foundation's decision to approve the Alverno grant.

The foundation did not expect its modest grant to be transformative. Truman Bell, program officer for education and diversity at the ExxonMobil Foundation, says the grant funded "a seed program." The foundation viewed the grant as an opportunity to expose engineering administrators and faculty to Alverno's approach.

Alverno used a variety of methods, including a series of workshops, to explain its approach to engineering educators. One workshop, held at the 2004 ASEE Annual Conference, drew some 95 faculty members and administrators. Riordan says engineering administrators and faculty want to learn about Alverno's methods, but have questions about how what Alverno does can be translated into engineering education.

On-site Visit

Some engineering school faculty and administrators who have participated in Alverno's workshops feel that the school has much to offer them. Tom Brumm, assistant professor of agricultural and biosystems engineering at Iowa State University, says that even prior to Alverno's receipt of the grant, he had visited the Milwaukee school several times to "observe and learn from what they have done in assessment. The basic principles of assessment that they have demonstrated are very important to what we are doing" to meet ABET requirements and improve the education of students.

In Brumm's program, students must meet 14 competencies. To prove their mastery they create an electronic portfolio, something that students at Alverno also do. The Iowa State portfolio might include examples of a student's work, such as a capstone design project, as well as a video of the student's speeches. By focusing on a competency-based approach, Brumm says, students come to see their education "as more of an integrated whole rather than as a checklist of courses they have to take."

For the faculty, he says, this shift requires a major re-conceptualization of how a course is constructed and taught. "We are changing the way we do business," Brumm says. The response from employers to the competency-based approach has been overwhelmingly positive. "They are amazed we are focusing on competencies because that is exactly what they are doing" with their employees, Brumm says.

WPI's Judith Miller says that her school has drawn on the methods Alverno uses to assess its students on teamwork and problem solving. WPI students are put on a team, given a problem to solve, and have a few hours to work on it. When time is up, teams present their results. Each group's work process and presentation is videotaped and sent to external judges who pick a winner. The winning team's members receive a cash prize.

Miller says that while some facets of Alverno's approach can be adapted by engineering schools, others might be more difficult to emulate. In general, the Alverno approach is very hands-on and labor intensive for faculty, and that might make it difficult to institute at a large school with huge classes. "When I describe what Alverno does to colleagues, they say it sounds really cool, but we couldn't do it here," Miller says.

Moreover, she says, at Alverno much of the faculty research and scholarship revolves around assessment, while that is decidedly not the case at schools of engineering. So there is no incentive for engineering faculty members to pursue innovations in assessment.

Alverno administrators say they don't expect engineering schools to follow in their footsteps. They feel that institutions should utilize those facets of the Alverno system that work for them, while devising their own version of assessment. "We don't expect schools to replicate what we do," Riordan says.

But whatever approach they take, Riordan says, engineering schools need to bear in mind that assessment is most successful when done on an individualized basis. He says that a lot of engineering faculty and administrators seem to think about assessment as a way to see where their programs need to be improved, and thus collect aggregated data. But he argues that "if you are not assessing at the individual student level," then the value of assessments is considerably diminished. For one thing, that means some students may be graduating without having the requisite abilities.

There is no question that the public increasingly wants colleges to prove that they have made a difference for students—that they have added to what students know and can do. As a result of public demand, the desire of employers, and growing pressure from government, various accrediting bodies are following ABET's lead and incorporating outcomes into their requirements. But it is slow going, since it introduces an element that is alien to the way the academy has traditionally done business.

Alverno was clearly decades ahead of other institutions in understanding the importance of assessing student abilities. Although engineering schools are relative newcomers to this approach, they are still ahead of most others in higher education. Iowa State's Brumm says that "because of ABET, engineering is further along than other fields."

Alvin Sanoff is a freelance writer based in Bethesda, Md.