PRISM Magazine Online - April 2000
Web Extras

1.  Remarks by the President at Science and Technology Event
          President Clinton
2. 
Engineering Education for the 21st Century
          James J. Duderstadt
3. 
Caution:  Outcomes Assessments in Use
          George A. Hazelrigg


REMARKS BY THE PRESIDENT AT SCIENCE AND TECHNOLOGY EVENT

California Institute of Technology
Pasadena, California

          THE PRESIDENT:  Thank you so much.  Dr. Moore, President Baltimore; to the faculty and students at Cal Tech, and to people involved in NASA's JPL out here.  I want to thank Representatives Dreier, Baca and Millender-McDonald for coming with me today, and for the work they do in your behalf back in Washington.  I want to thank three members of our Science and Technology team for being here -- my Science Advisor Neal Lane; Dr. Rita Colwell, the NSF Director; and my good friend, the Secretary of Energy, Bill Richardson, who has done a great job with our national labs to keep them being innovators in fields from computational science to environmental technology.
          One person who would have liked to have been here today and I can tell you thinks that he would be a better representative of our administration on this topic is the Vice President.  When we took office together, the fact that I was challenged scientifically and technologically was standing joke.  (Laughter.)  And he wants all of you to know that he's campaigning all over the country with a Palm 7 on his hip.  (Laughter.)
          He wants you to know that he loves science and technology so much, he's not even angry that Cal Tech beat out Harvard for top spot in the U.S. News rankings this year.  (Laughter.)  I think it has something to do with the relative electoral votes of California and Massachusetts.  (Laughter.)
          But before I came out here I told Dr. Moore and Dr. Baltimore that it was a real thrill for me to meet Dr. Moore, that even I knew what Moore's Law was; and that before the Vice President became otherwise occupied, we used to have weekly lunches and I'd talked to him about politics and he'd give me lectures about climate change.(Laughter.)
          But we once got into this hilarious conversation about the practical applications of Moore's Law, like it explains why every cable network can double the number of talk shows every year that no one wants to listen to.  (Laughter.)  And so it's a real thrill for me to be here.  (Laughter.)
          Actually, I come with some trepidation.  An eight-year-old child met me at the airport, and she and her brother came with their father, who is a friend of mine, and she brought me a letter from her third grade class.  And the letter had all these questions:  What was your favorite book when you were in the third grade?  What did you collect then?  What do you collect now?  And one of the questions was, are you ever nervous when you're speaking before large audiences.
          And the answer -- and I was writing all these answers so we could type up a letter -- I said, not usually.  But I mean, I'm sort of nervous here today.  (Laughter.)  And I told somebody I was nervous, one of the wags back at the White House with a sense of humor, and he said, well, you know the Einstein millennial story, don't you -- trying to help me get unnervous.  (Laughter.)  I said, so I said, no -- you always learn to be patient in the face of other people's jokes.  It's one of the great social skills that an American can develop.  (Laughter.)
          So I said, no.  And he said, well, God decides to give America a millennial gift, and the gift is to send Einstein back to Earth for a few days to talk to ordinary folks, because he was the greatest brain of the last millennium.  And they have the first meeting in a nice little hall like this.  And it's absolutely packed, and these three big, burly guys push their way to the front, shoving everyone else to the side.  So Einstein politely takes them first and he says to the first guy, well, what's your IQ, young man?  And he said 240.  He said wonderful, let's talk about how I thought up the theory of relativity.  And they have a terrific conversation.
          The second guy, he says, what's your IQ?  He said, 140.  He said, let's talk about globalization and its impact on climate change.  And they had a terrific conversation.  And the third guy kind of hung his head, and he said, what's your IQ?  And he said, 40.  And Einstein said, oh, don't worry.  You can always go into politics.  (Laughter.)
          I want you to know, though, in preparation for this day I've been spending a lot of time trying to get in touch with my inner nerd.  (Laughter and applause.)  And my wife helped me, because she's been having these Millennium Lectures at the White House to discuss big things.  And the other night, she had Vince Cerf, who was one of the founders of the Internet, and Eric Lander, who's helped to develop many of the tools of modern genome research.  And that really got me thinking, and I want to say some more serious things about that in a moment.  And then my staff challenged me to actually order Christmas gifts over the Internet.  And I did that.  And while doing that, I learned that with just a click of a mouse, I could actually order -- and I did this, I'm embarrassed to say -- I ordered Arkansas smoked ham and sausage delivered to my door.  (Laughter.)  So I think the 21st century has more for me than I had originally thought.  (Laughter.)
          As all of you know, Albert Einstein spent a lot of time here at Cal Tech in the 1930s.  And three weeks ago, Time Magazine crowned him the Person of the Century.  The fact that he won this honor over people like Franklin Roosevelt and Mohandas Gandhi is not only an incredible testament to the quantum leaps in knowledge that he achieved for all humanity, but also for the 20th century's earth-shaking advances in science and technology.
          Just as an aside, I'd like to say because we're here at Cal Tech, Einstein's contributions remind us of how greatly American science and technology and, therefore, American society have benefitted and continue to benefit from the extraordinary gifts of scientists and engineers who are born in other countries, and we should continue to welcome them to our shores.  (Applause.)
          But the reason so many of you live, work and study here is that there are so many more questions yet to be answered:  How does the brain actually produce the phenomenon of consciousness?  How do we translate insights from neuroscience into more productive learning environments for all our children?  Why do we age -- the question I ponder more and more these days.  (Laughter.)  I looked at a picture of myself when I was inaugurated the first time the other day, and it scared me to death.  (Laughter.)  And so I wonder, is this preprogrammed, or wear and tear?  Are we alone in the universe?  What causes gamma ray bursts?  What makes up the missing mass of the universe?  What's in those black holes, anyway?  And maybe the biggest question of all:  How in the wide world can you add $3 billion in market capitalization simply by adding .com to the end of a name?  (Laughter.)
          You will find the answers to the serious questions I posed and to many others.  It was this brilliant Cal Tech community that first located genes on chromosomes and unlocked the secrets of chemical bonds and quarks.  You were the propulsive force behind jet flight and built America's first satellites.  You made it possible for us to manufacture microchips of ever-increasing complexity and gave us our first guided
tour on the surface of Mars.  With your new gravitational wave observatory, you will open an entirely new window on the mysteries of the universe, observing the propagating ripples which Einstein predicted 84 years ago.
          Today, I came here to thank you for all you're doing to advance the march of human knowledge and to announce what we intend to do to accelerate that march by greatly increasing our national investments in science and technology.
          The budget I will submit to Congress in just a few days will include a $2.8 billion increase in our 21st century research fund.  This will support a $1 billion increase in biomedical research for the National Institutes of Health; $675 million, which is double the previous largest dollar increase for the National Science Foundation in its entire 50-year history; and major funding increases in areas from information technology to space exploration to the development of cleaner sources of energy.
          This budget makes research at our nation's universities a top priority, with an increase in funding of more than $1 billion.  University-based research provides the kind of fundamental insights that are most important in any new technology or treatment.  It helps to produce the next generation of scientists, engineers, entrepreneurs.  And we intend to give university-based research a major lift.
          The budget supports increases not only in biomedical research, but also in all scientific and engineering fields.  As you know, advances in one field are often dependent on breakthroughs in other disciplines.  For example, advances in computer science are helping us to develop drugs more rapidly, and to move from sequencing the human genome to better understanding the functions of individual genes.
          My budget supports a major new national nanotechnology initiative worth $500 million.  Cal Tech is no stranger to the idea of nanotechnology, the ability to manipulate matter at the atomic and molecular level.  Over 40 years ago, Cal Tech's own Richard Symonds asked, what would happen if we could arrange the atoms one by one the way we want them?  Well, you can see one example of this in this sign behind me, that Dr. Lane furnished for Cal Tech to hang as the backdrop for this speech.  It's the Western hemisphere in gold atoms.  But I think you will find more enduring uses for nanotechnology.
          Just imagine, materials with 10 times the strength of steel and only a fraction of the weight; shrinking all the information at the Library of Congress into a device the size of a sugar cube; detecting cancerous tumors that are only a few cells in size.  Some of these research goals will take 20 or more years to achieve.  But that is why -- precisely why -- as Dr. Baltimore said, there is such a critical role for the federal government.
          As I announced yesterday, this budget also includes an historic initiative to make higher education more affordable.  I am well aware of the fact that I would not have become President of the United States without loans and grants and jobs that helped me get through college and law school; and that more and more, given the cost of higher education, a higher and higher percentage of our students need more of all those things.  This has been a virtual obsession for me ever since I became President.  I was determined to leave office saying we had opened
the doors of college to all Americans.
          We have come a long way, by changing the student loan program to make it less expensive and to give young people more options forpaying off their loans, including as a percentage of their income when they leave school.  We've increased the number of work-study grants from $700,000 to $1 million.  We've dramatically increased the Pell Grant program, and the HOPE Scholarship tax credit and the Lifetime Learning tax credits we adopted in 1997 last year alone had almost 5 million beneficiaries in institutions of higher education in the United States.  (Applause.)
          Yesterday, I proposed that, for the first time, we make college tuition tax deductible, and that we do it in a way that would benefit even more people on more modest incomes so that they could get the same 28-percent benefit, even if they're in the 15-percent tax category.  I think this is very important.  (Applause.)
          The budget contains another increase in Pell grants, special initiatives to help minority students get into science and engineering and graduate.  (Applause.)  Special efforts -- that is basically a test program for several thousand students now -- to try to do something about the extraordinarily high dropout rate from college.
          Now, over two-thirds of the high school graduates are actually going to go into college this year.  That's an increase of over 10 percent in the last seven years.  That's quite a large increase in a short time.  But the dropout rate has increased correspondingly.  We want to know why.  Is it for financial reasons?  Is it because people weren't prepared?  Could they all be just idiosyncratic personal reasons?  And we intend to do everything we can with a very large test group to see what we can do to turn this situation around.
          And, finally, we're going to double the size of our Gear-Up program to 1.4 million young people.  That's the program where people in universities and college all across America mentor middle-school kids
who are at risk to try to help them develop the skills and the belief that they can go to college, and simultaneously to tell them and their parents exactly what they can expect in the way of aid under current law
if they do go, so they will know.  Many people still don't know that the barriers to their going on to college have been removed.  So I hope you will also support this part of our budget, because the young people of our country and their families need it.
          In addition to announcing our new research budget and our efforts to make colleges more affordable, I'd like to try to achieve one other mission here today.  First, I want to take a step back to acknowledge that we have not done a good enough job in helping all Americans to understand why we need very, very large investment in science and technology.
          Far too many of our citizens think science is something done by men and women who are in white lab coats behind closed doors that somehow leads to satellite TV and Dolly the sheep.  And it's all a mystery.  It is our responsibility to open the world of science to more of our fellow citizens; to help them understand the great questions science is seeking to answer and to help them see how those answers will actually affect their lives and their children's lives in profoundly important and positive ways.
          First, we have to make sure Americans understand the contributions science and technology are making right now to the present level of economic growth, something Dr. Baltimore referred to.  For example, because of our early investments in the Internet, America now leads the world in information technology, an industry that now accounts for a third of our economic growth, although only 8 percent of our work force; that generates jobs that pay 80 percent more than the private sector average.
          If you look at that -- what does that mean to ordinary people, and what does it mean to the nature of the economy we're living in?  I have never told the American people that we had repealed the ordinary laws of supply and demand, or the business cycle.  But we have stretched them quite a lot.
          In February, next month, we will have the longest economic expansion in the history of the United States -- outstripping even those that required full mobilization for war.  Now, part of that is because we have pursued, I believe, sound policies -- to get rid of the deficit; to start running surpluses, the first back-to-back surpluses in 42
years; to keep our markets open, with 270 trade agreements; to argue, as I have, that not only exports are benefited by open markets, we also benefit from the imports, because they're a powerful brake on inflation and allow us to continue to grow.
          But the real reason this thing keeps going on and on and on is that -- all we did in the government was to set the conditions, and provide the tools, for the American people to succeed.  The real reason is the exponential growth in information technology, and how it is rifling through every other sector of our economy and reinforcing the material science revolution, which proceeded it by a few years, but which continues to the present day.
          When I became President, there were only 50 sites on the Worldwide Web -- 50.  When I became President -- that seemed like a long time ago to the students, but the rest of you will know -- (laughter) --it's just like yesterday.  There are now over 50 million.  Think of it. In seven years, from 50 to over 50 million.  It is changing everything about the way we work and live and relate to each other.
          I was in Northern California a few weeks ago with a lot of really fascinating young people who work with E-bay -- a lot of you have probably bought things, maybe you've even sold things on E-bay.  But for example, one of the things I learned is that in addition to the employees of E-bay, there are now 20,000 people whose primary source of income is buying and selling on E-bay.  They do it for a living.  And several of them -- not insubstantial number of them -- were on welfare before they found a way to bring their entrepreneurial skills to bare by trading on E-bay.  It has changed everything.
          So we have to say to people, if you like the fact that we have the lowest unemployment and welfare rolls in 30 years, the lowest minority unemployment rates ever recorded, the lowest female unemployment rate in 40 years, the lowest poverty rates in 20 years, the lowest single household poverty rate in 46 years, you have to understand that all that, at least in large part, is because of the ability of the discoveries of science and technology to rifle through our ordinary lives.  And it is very, very important that all of us do a better job of that.
          I have proposed in this budget a 36 percent increase in information technology research alone, so that researchers will be able to tackle a wide array of other challenges.  How do we find, precisely, the piece of information we're looking for in an ever-larger ocean of raw data.  How do we design computers that are usable by everyone including people with disabilities.
          One of the most fascinating relationships I've developed -- we were talking on the plane ride out here about one of the great things about being President is nearly anybody will come to talk to you -- once, anyway.  (Laughter.)  And we were talking about all the people I had been privileged to meet in the last seven years.  You know, I have developed quite a good personal friendship with Steven Hawking, who, as all of you know, has lived longer with Lou Gehrig's disease, as far as we know, than any person who's ever lived -- partly, I am convinced, because of not only the size of his brain, but the size of his heart.  But it is fascinating to see what technology has permitted this man to do.
          Just a few years ago, he could have had the biggest brain in the world, and no one could have known it, because it could not have gotten out.  There is no speaking capacity, almost no movement left.  He can just move his thumb, and hold in his hand this remarkable little tracer that goes through a whole dictionary of words that he has, that he runs through with rapid speed.  He picks the word he wants, puts the sentences together, and then an automated voice tells you what he just said.
          How can we make it even easier for him?  How can we make it even easier for other people?  This will be a huge issue.  Make no mistake about it, the liberation of Americans with disabilities is also in no small measure the product of the revolution in science and technology.
          There are also other uses.  I read the other day that manufacturers are soon going to introduce a refrigerator that can scan the bar codes of empty packages and expired goods -- (laughter) -- and order new groceries for you over the Internet.  (Laughter.)  Now, everybody who's ever poured out a carton of bad milk will love this.  (Laughter.)  You don't have to smell your bad milk anymore.  It won't be long before the computer will refuse to order what's bad for you -- (laughter) -- and only pick items off Dean Ornish's diet.  And then we'll all be in great shape.  (Laughter.)
          The second thing I think we have to do is let Americans know how investments in science and technology, broadly stated, will allow us to lead longer, healthier lives.  Everybody knows now that you can put money into cancer research -- and thank God we've discovered two of the genes that are high predictors of breast cancer, for example, in the last couple of years -- but we need for more Americans to understand why we need a broad research agenda in science and technology, for the health of Americans.  (Applause.)
          In the 20th century, American life expectancy went from 47 years to almost 77 years, thanks to penicillin and vaccines for many childhood diseases.  We were talking the other day about the impact -- I'm old enough to remember the first polio vaccine.  And I remember how our mothers herded us in line and made us stand there waiting for our shot.  And it was like they were all holding their breath, praying and hoping that we would get our shot before we got polio.  It's something that young people today can hardly imagine, but it hung like a cloud
over the families of my parents' generation.  Now, we have this incredible life expectancy -- today, the average American who lives to be 65 has a life expectancy of 83 -- already.  And we are clearly on the cusp of greater advances.
          Later this year, researchers expect to finish the first complete sequencing of the genome -- all 3 billion letters and 80,000 genes that make up our DNA code.  Since so many diseases have a genetic component, the completion of this project will clearly lead to a revolution in our ability to detect, treat and prevent many diseases.
For example, patients with some forms of leukemia and breast cancer soon may receive sophisticated new drugs that elegantly actually target the precise cancer cells with little or no risk to healthy cells.  That will change everything.
          Our new trove of genomic data may even allow us to identify and cure most genetic diseases before a child is even born.  Most people just take it as a given now that within the next few years, when young mothers bring their babies home from the hospital, they will bring along a genetic map of their children's makeup, what the problems are, what the challenges are, what the strengths are.  It will be scary to some extent, but it also plainly will allow us to raise our children in a way that will enhance the length and quality of their lives.
          But it's important to recognize that we never could have had the revolution in the genome project without the revolution in computer science as well, that they intersected.  Research at the intersection between biomedical research and engineering will also lead to amazing breakthroughs.  Already, scientists are working on -- we've seen it on television now -- an artificial retina to treat certain kinds of blindness, and methods of directly stimulating the spinal cord to allow people who are paralyzed to work.  Now, you think of that.
          Last year, for the first time, to give you an idea of the impact of technology on traditional medical research, last year, for the first time, medical researchers transplanted nerves from the limbs to the spine of a laboratory animal that had its spine severed and achieved movement in the lower limbs for the first time.  That had never happened before.
          Now, because of advances in the intersection between science and engineering, we may not have to keep working on that.  We may actually be able to program a chip that will stimulate the exact movements that were prevented by the severing or the injury of a spine.  And all the people that we have seen hobbled by these terrible injuries might be able to get up and walk.  Because there was medical research, yes, but there was also research on the engineering, nonbiological components of this endeavor.  We have to do a better job of explaining that to the American people.
          Third, advances in science and technology are helping us to preserve our environment in ways that preserve more sustainable and widespread economic growth.  And that is very important.
          Let me just give you an example.  Not far from here in Southern California, a couple years ago the Department of Energy, working with the National Homebuilders and HUD, helped to construct a moderate- and low-income housing community, with glass in the windows that keeps out four or five times as much heat or cold, and lets in even more light.  And that, coupled with the latest insulation technology and the latest lighting in the house, enabled the houses to be marketed to people of modest incomes, with the promise that their electric bills would average 40 percent below what they would in a home of that size built in the traditional manner.  I can tell you that after two years, the power bills are averaging 65 percent less.  And we can't build enough houses for the people that want them.
          The Detroit Auto Show this year is showcasing cars that, I'm proud to say, were developed as part of our partnership for new generation vehicles that the Vice President headed up, and we started way back in '93.  We brought in the auto workers and the auto companies and we said, look, instead of having a big fight about this, why don't we work together and figure out how to use technology to dramatically increase mileage.  And a lot of you are probably familiar -- they're using fuel-injection engines, which cuts a lot of the greenhouse gas emissions; some using developed mixed-fuel cars that start on electricity, switch to fuel after you reach a certain stage, and then go back to electricity when you slow down back to that speed, because 70 percent of the greenhouse gas emissions are used in starting and stopping cars.
          And there are all kinds of other things being developed.  But this year the Detroit Auto Show has cars making 70, 80 miles a gallon, that are four-seater cars, that will be on the market in a couple of years.  You can buy Japanese cars this year on the market that get about 70 miles to the gallon, but they're small two-seaters.  Last year I went and saw cars that are 500 to 1,000 pounds lighter than traditional cars, and score at least as well on all the damage tests -- again because of the revolution in material science, with composite materials being used in the cars.
          And the big thing that's coming up in this area is, before you know it, I believe we will crack the chemical barriers to truly efficient production of biomass fuels.  One of the reasons you see this whole debate -- in the presidential campaign, if you're following it, you know the big argument is, is it a waste of money to push ethanol or not, if it takes seven gallons of gasoline to make eight gallons of ethanol.  But they're on the verge of a chemical breakthrough that is analogous to what was done when crude oil could be transferred efficiently into gasoline.  And when that happens, you'll be able to make eight gallons of biomass -- not just from corn, but from weeds, from rice hulls, from anything -- for about one gallon of fuel.  That will be the equivalent therefore, in environmental terms, of cars that get hundreds of miles a gallon.  And the world, the environmental world, will be changed forever.  And that's -- one-third of our greenhouse gas emissions are in transportation.
          Now, I just want to kind of go off the script a little to hammer this home, because big ideas in science matter.  And once you make a big breakthrough, then thousands and thousands of things follow that have immense practical significance.  But you must also know and believe that being in the grip of a big idea that is wrong can be absolutely disastrous.
          So today, in Washington and in much of the world, there is a debate that goes something like this:  The overwhelming evidence of science is that the climate is warming at an unsustainable rate due to human activity.  And then there's this old idea, which says, well, that's really too bad, but a country can't grow rich or stay rich and sustain a middle-class lifestyle, unless every year it puts more greenhouse gases into the atmosphere than it did the year before.  And you certainly can't drastically cut them, and maintain your level of wealth.
          Our administration spent hundreds of thousands of dollars last year complying with requests to appear before a House subcommittee that believes that our passion about climate change is some sort of subversive plot to wreck the American economy.  (Laughter.)  Either that or -- you know, I've been reading too many kooky books or something.  (Laughter.)  They think it's just crazy.  Why?  Because they can't face the fact that we would do anything to hurt the American economy, and they really believed it would.  So I would argue to you that here is a place where we're in the grip of an idea that is wrong.
          Our efforts to get India and China and other big countries that will soon surpass us in greenhouse gas emissions to cooperate with us, not in regulation, but in new technologies, to help them grow rich differently, always keep running up against the barrier of suspicious officials who believe somehow this is kind of an American plot to keep them poor.  Why?  Because they're in the grip of an idea that isn't right anymore.  It is simply not true that to grow rich, you have to put more greenhouse gases in the atmosphere.
          So again, I say we have to do a better job of explaining the contribution that science and technology can make to saving the planet and allowing us to still have prosperous lives -- and, I would argue, to allow us to have more prosperous lives and better lives that would otherwise be the case, certainly within 40 to 50 years, if we don't act and act now.  This is profoundly important.  (Applause.)
          Finally, I think we have to do a better job of having an open debate about the responsibilities that all these advances and discoveries will clearly impose:  The same genetic revolution that can offer new hope for millions of Americans could also be used to deny people health insurance; cloning human beings; information technology which helps to educate children and provide telemedicine to rural communities could also be used to create disturbingly detailed profiles of every move our citizens make on line.
          The federal government, I think, has a role to play in meeting these challenges as well.  That's why we've put forward strict rules and penalties to limit the use and release of medical records; why we've worked with Congress to ban the cloning of human beings, while preserving our ability to use the morally and medically acceptable applications of cloning technology, which I believe are profoundly important; why we're working with the Internet industry to ensure that consumers -- consumers -- have control over how their personal information is used.
          It's up to all of us to figure out how to use the new powers that science and technology give us in a responsible way.  Just because we can do something doesn't mean we should.  It is incumbent, therefore, upon both scientists and public servants to involve the public in a great debate to ensure that science serves humanity -- always -- and never the other way around.
          On this campus nearly 70 years ago, Albert Einstein said, "Never forget this, in the midst of your diagrams and equations:  concern for man himself and his fate must always form the chief interest of all technical endeavors."  Today, at the dawn of this new millennium, we see for all of you, particularly the young people in this audience, an era of unparalleled promise and possibility.  Our relentless quest to understand what we do not yet know, which has defined Americans from our beginnings, will have more advances in the 21st century than at any other time in history.  We must be wise as we advance.
          I told you earlier that the First Lady sponsored a Millennium Evening with Vince Cerf and Professor Lander.  One of the most interesting things he said about his genomic research confirmed not other scientific research, but the teachings of almost every religion in the world.  He said that, genetically, we are 99.9 percent the same.  And, he said, furthermore, that the genetic differences among individuals within a given racial or ethnic group are greater than the differences between groups as a whole -- suggesting that we are not only our brothers' and sisters' keepers, but in fundamental genetic ways, we are our brothers and sisters.
          And I leave you with this thought.  I think the supreme irony of our time is that I can come here as President and have the high honor of discussing these unfathomable advances wrought by the human intellect, that have occurred and the even greater ones yet to occur, in a world where the biggest social problem is the oldest demon of human society -- we are still afraid of people who aren't like us.  And fear leads to distrust, and distrust leads to dehumanization, and dehumanization leads to violence.
          And it is really quite interesting that the end of the Cold War has marked an upsurge in ethnic and racial and tribal and religious hatred and conflict around the world; and that even in our own country we see countless examples of hate crimes from people who believe that others are different and, therefore, to be distrusted and feared and dehumanized.
          You have the power to put science and technology at work advancing the human condition as never before.  Always remember to keep your values at the core of what you do.  And tell every one of your fellow citizens, and indeed people with whom you come in contact all across the world, that every single scientific advance confirms over and over again the most important facts of life -- our common humanity.

Thank you very much.  (Applause.)


Engineering Education for the 21st Century
          James J. Duderstadt
May 9, 1998

     A centennial celebration is always something very special.  It is even more noteworthy when it honors an academic department that has provided leadership throughout its history.  When the Department of Chemical and Metallurgical Engineering was founded at the University of Michigan in 1898, it was the second such program in the nation (MIT being the first in 1888).  Ironically, two years earlier Michigan's College of Literature, Science, and Arts had abolished its chemistry major, so that for the next two decades the engineering school provided all chemistry instruction for the University.
     To learn more about what engineering education was like a century ago, I went back into the University archives and pulled out a sample curriculum.  I was surprised to find it remarkably similar to today's program.  In 1898 we required students to take 130 credit hours of courses in mathematics, physics, and chemistry with a concentration in applied courses in chemical engineering and metallurgy.  If one swaps yesterday's requirement for surveying and mechanical drawing for today's courses on computers, the two curricula are almost identical.  Of course, the actual content of these courses has changed considerably—or so one would hope.
     With one major exception, the actual structure of the engineering curriculum has remained roughly the same over the past century.  But that exception is an important one.  The 1898 curriculum placed far more stress on the importance of a liberal education, with more courses in humanities, arts, and social sciences.  In fact, one might even suggest that we have regressed over the past century, overloading our current curriculum with highly specific technical courses at the expense of broader educational opportunities for our students.
     Of course, engineering practice today is dramatically different than it was a century ago.  Indeed, it is quite different from that of just a few years ago, when most faculty were educated.  And this is the theme for my remarks today.  This raises an important question:  Is engineering education today adequately preparing our students for a world of practice and citizenship that is quite different from the one that we have known?  More broadly, is the university as we know it today capable of serving the rapidly changing needs of contemporary society?
     Since stepping down as the president of the University of Michigan two years ago, I have had the luxury of focusing more of my attention on the future of higher education.  Through a variety of activities at the state, national, and international level, my own vision of the future of the university has been increasingly dominated by the theme of change and transformation.  In fact, these days I am frequently invited by university presidents to visit their campuses and scare the hell out of their faculty. 
     You probably recall the well-worn expression that the best way to get a mule to move is to first whack it between the eyes with a 2x4 to get its attention.  Well, in the mind of many college presidents, I have become the 2x4 in their efforts to get the attention of their institutions and prepare them for change. 
     Since much of the context for issues swirling about engineering education today is provided by the broader challenges of change facing higher education, I will begin by remarks by summarizing briefly my spin on just where the university may be headed.  In a nutshell, I believe that the forces of change in higher education are far stronger than most realize.  Furthermore, I believe that engineering education will not be exempt from these changes, but may be swept along at the crest of the wave of university change.
     Put another way, I believe there is little likelihood that the engineering curriculum will continue to preserve its century-old structure in the century—indeed, in the decade—ahead.  Indeed, I might suggest that this is likely to be the last centennial celebration for our Department of Chemical Engineering, since such highly specialized academic departments are unlikely to survive much longer.  In fact, there are some who believe that even the university itself is unlikely to survive intact, at least as we know it today, over the next century.

The Best of Times and the Worst of Times

Remember the opening lines from Charles Dickens' The Tale of Two Cities?

    It was the best of times, it was the worst of times,
    It was the age of wisdom, it was the age of foolishness,
    It was the epoch of belief, it was the epoch of incredulity,
    It was the season of Light, it was the season of Darkness,
    It was the spring of hope, it was the winter of despair.

These do indeed seem like both the best of times and the worst of times for higher education in America.  On the one hand, in an age of knowledge in which educated people and their ideas have become the wealth of nations, the university has never been more important, and the value of a college education never higher.  The educational opportunities offered by the university, the knowledge it creates, and the services it provides are key to almost every priority of contemporary society, from economic competitiveness to national security to protecting the environment to enriching our culture.  There is a growing recognition that few public investments have higher economic payoff than those made in higher education.  In 1997 the federal government made the largest commitment to higher education since the GI Bill through $40 billion of tax incentives to college students and their parents as part of the budget-balancing agreement.  In 1998 Washington took further action by proposing the largest increase in the funding of academic research in decades.  And both the administration and Congress promise balanced budgets and generous support for years to come.
     Yet, there is great unease on our campuses.  The media continues to view the academy with a frustrating mix of skepticism, ignorance, and occasional hostility that erodes public trust and confidence.  Although an unusually prosperous economy has provided both state and federal governments with the resources to halt the erosion in public support of higher education, the danger of intervention in the name of accountability remains high.  Throughout society we see a backlash against earlier social commitments such as affirmative action, long a key mechanism both for diversifying our campuses and providing educational opportunity to those suffering discrimination in broader society.  And the faculty feels the stresses from all quarters:  There is fear that research funding will decline again when the economy cools and entitlement programs grow, a sense of loss of scholarly community with increasing specialization; and a conflict between the demands of grantsmanship, a reward structure emphasizing research, and a love and sense of responsibility for teaching. 
     To continue paraphrasing Dickens, while we may be entering an age of wisdom—or at least knowledge—it is also an age of foolishness.  Last year, the noted futurist Peter Drucker shook up the academy when, during an interview in Forbes, he speculated:  "Thirty years from now the big university campuses will be relics.  Universities won't survive.  It's as large a change as when we first got the printed book."  One can imagine the network of interactions that ricocheted across university campuses in the months following Drucker's conjecture.  It was fascinating to track the conversations among the University of Michigan deans on electronic mail.  Some, of course, responded by blasting Drucker, always a dangerous thing to do.  Others believed it to be moot.  A few even surmised that perhaps a former president of the University of Michigan might agree with Drucker.  (He doesn't, incidentally.)
     So what are we facing?  A season of light or a season of darkness?  A spring of hope or a winter of despair?  More to the point, and again in a Dickensian spirit, is higher education facing yet another period of evolution?  Or will the dramatic nature and compressed time scales characterizing the changes of our time trigger a process more akin to revolution? 
     To be sure, most colleges and universities are responding to the challenges and opportunities presented by a changing world.  They are evolving to serve a new age.  But most are evolving within the traditional paradigm, according to the time-honored processes of considered reflection and consensus that have long characterized the academy.  Is such glacial change responsive enough to allow the university to control its own destiny?   Or will a tidal wave of societal forces sweep over the academy, both transforming the university in unforeseen and unacceptable ways while creating new institutional forms to challenge both our experience and our concept of the university?
     To illustrate these issues, consider two sharply contrasting futures for higher education in America.  The first is a rather dark future in which strong market forces trigger a major restructuring of the higher education enterprise.  Although traditional colleges and universities play a role in this future, they are both threatened and reshaped by aggressive for-profit entities and commercial forces that drive the system toward the mediocrity that has characterized other mass media markets such as television and journalism.
     A contrasting and far brighter future is provided by a vision of a culture of learning in which universal or ubiquitous educational opportunities are provided to meet the broad and growing learning needs of our society.  Using a mix of old and new forms, learners are offered a rich array of high quality, affordable learning opportunities.  Our traditional institutional forms, including both the liberal arts college and the research university, continue to play key roles, albeit with some necessary evolution and adaptation.
     Although market forces are far more powerful that most realize, we also believe that it is possible to determine which of these or other paths will be taken by higher education in America.  Key in this effort is our ability as a society to view higher education as a public good that merits support through public tax dollars.  In this way, we may be able to protect the public purpose of the higher education enterprise and sustain its quality, important traditions, and essential values.
     If we are to do this, we must also recognize the profound nature of the rapidly changing world faced by higher education.  The status quo is no longer an option.  We must accept that change is inevitable and use it as a strategic opportunity to control our destiny, retaining the most important of our values and our traditions.

The Forces Driving Change

     There are powerful forces driving an increasing societal demand for higher education services.  In today's world, knowledge has become the coin of the realm, determining the wealth of nations.  It has also become the key to one's personal standard of living, the quality of one's life.   We are in a transition period where intellectual capital—brainpower—is replacing financial and physical capital as the key to our strength, prosperity, and well being.  In a very real sense, we are entering a new age, an Age of Knowledge, in which the key strategic resource necessary for prosperity has become knowledge itself, that is, educated people and their ideas.  Our society is becoming ever more knowledge-intensive.
     As knowledge and educated people become key strategic priorities, our society has become more dependent upon those social institutions that create these critical resources, our colleges and universities.  Yet there is growing concern about whether our existing institutions have the capacity to serve these changing and growing social needs—indeed, even whether they will be able to survive in the face of the extraordinary changes occurring in our world.
     The forces of change of most direct concern to higher education can be grouped into four areas:  i) financial imperatives, ii) changing social needs, iii) technology drivers, and iv) market forces.

Financial Imperatives

     Since the late 1970s, higher education in America has been caught in a financial vise.  On the one hand, the magnitude of the services demanded of our colleges and universities has increased considerably.  Enrollments have grown steadily; the growing educational needs of adult learners have compensated for the temporary dip in the number of high school graduates associated with the post-war baby boom/bust cycle.  University research, graduate education, and professional education have all grown in response to societal demand.  Professional services provided by colleges and universities also continue to grow in areas such as health care, technology transfer, and extension—all in response to growing needs.
     The costs of providing education, research, and service per unit of activity have grown at an even faster rate, since these university activities are dependent upon a highly skilled, professional workforce (faculty and staff); they require expensive new facilities and equipment; and they are driven by an ever-expanding knowledge base.  Higher education has yet to take the bold steps to constrain cost increases that have been required in other sectors of our society such as business and industry.  This is in part because of the way our colleges and universities are organized, managed, and governed.  But, even if our universities should acquire both the capacity and the determination to restructure costs more radically, it is debatable whether those industrial sector actions designed to contain cost and enhance productivity could have the same impact in education.  The current paradigm of higher education is simply too people- and knowledge-intensive.
     As the demand for educational services has grown and the operating costs to provide these services have risen, public support for higher education has flattened and then declined over the past two decades.  The growth in state support of public higher education peaked in the 1980s and now has fallen in many states in the face of limited tax resources and the competition of other priorities such as entitlement programs and corrections.  While the federal government has sustained its support of research, growth has been modest in recent years and is likely to decline as discretionary domestic spending comes under increasing pressure from federal budget-balancing efforts.  There has been significant downsizing in federal financial aid programs over the past two decades, with a corresponding shift from grants to loans as the predominant form of aid.  While the new federal budget agreement is good news to middle-class parents, it is unlikely to bring new resources to higher education.
     To meet growing societal demand for higher education at a time when costs are increasing and public support is declining, most institutions have been forced to sharply increase tuition and fees—substantially faster than the Consumer Price Index.  While this has provided short-term relief, it has also triggered a strong public concern about the costs and availability of a college education, along with accelerating forces to constrain or reduce tuition levels at both public and private universities.  As a result, most colleges and universities are now looking for ways to control costs and increase productivity, but most are also finding that their current organization and governance makes this very difficult.
     The higher education enterprise in America must change dramatically if it is to restore a balance between the costs and availability of educational services needed by our society and the resources available to support these services. 
     The current paradigms for conducting, distributing, and financing higher education may not be able to adapt to the demands and realities of our times.

Societal Needs

     The needs of our society for the services provided by our colleges and universities will continue to grow.  Significant expansion will be necessary just to respond to the needs of a growing population which will create a 30 percent growth in the number of college-age students over the next decade.  But these traditional students are only part of the picture; we must recognize the impact of the changing nature of the educational services sought by our society.
     Eighteen to twenty-two-year-old high school graduates from affluent backgrounds no longer dominate today's undergraduate student body.  It is comprised also of increasing numbers of adults from diverse socio-economic backgrounds, already in the workplace, perhaps with families, seeking the education and skills necessary for their careers.  When it is recognized that this demand for higher education may be significantly larger than that for traditional undergraduate education, it seems clear that either existing institutions will have to change significantly or new types of institutions will have to be formed.  The transition from student to learner, from faculty-centered to learner-centered institutions, from teaching to the design and management of learning experiences, and from student to a lifelong member of a learning community—all suggest great changes are ahead for our institutions.
     The students entering college today require a different form of education in which interactive and collaborative learning will increasingly replace the passive lecture and classroom experience.  The student has become a more demanding consumer of educational services, although frequently this is directed at obtaining the skills needed for more immediate career goals.
     We are beginning to see a shift in demand from the current style of "just-in-case" education in which we expect students to complete degree programs at the undergraduate or professional level long before they actually need the knowledge, to "just-in-time" education in which education is sought when a person needs it through non-degree programs, to "just-for-you" education in which educational programs are carefully tailored to meet the specific lifelong learning requirements of particular students.  So too the shift from synchronous, classroom-based instruction to asynchronous computer network-based learning to the provision of ubiquitous/pervasive learning opportunities throughout our society will demand major change.
     The needs for other higher education services also are also changing dramatically.  The relationship between the federal government and the research university is shifting from a partnership in which the government is primarily a patron of discovery-oriented research to a process of procurement of research aimed at addressing specific national priorities.  The academic medical center has come under great financial pressure as it has been forced to deal with a highly competitive health-care marketplace and the entry of new paradigms such as managed care.  While the public appetite for the entertainment provided by intercollegiate athletics continues to grow, our colleges also feel increasing pressures to align these activities better with academic priorities and national imperatives (such as the Title IX requirements for gender equity).
     Even as the nature of traditional activities in education, research, and service change, society is seeking new services from higher education, e.g., revitalizing K-12 education, securing economic competitiveness, providing models for multicultural societies, rebuilding our cities and national infrastructure.  All of this is occurring at a time when public criticism of higher education is high, and trust and confidence in the university is relatively low.
     The inability of our existing institutions to meet the growing need for higher education is magnified many times throughout the world.  Just consider for a moment that over half of the world's population is under twenty years of age, most seeking education as the key to their future quality of life.  To meet this staggering demand, a major new university would need to be created every week.  Yet in most of the world, higher education is mired in a crisis of access, cost, and flexibility.  Unless we can address and solve this crisis, billions of people in coming generations will be denied the education so necessary to compete in—indeed, to survive in—an age of knowledge.
     Sir John Daniels, Chancellor of the Open University of the United Kingdom, observes that although the United States has the world's strongest university system this seems ill-suited to guiding us out of this global education crisis.  Our colleges and universities continue to be focused on high-cost, residential education and to the outmoded idea that quality in education is linked to exclusivity of access and extravagance of resources.  In fact, the American concept of the campus university would deny higher education to nearly all of the billions of young people who will require it in the decades ahead. 
     Again there are many signs that the current paradigms are no longer adequate for meeting growing and changing societal needs.

Technology Drivers 

     As knowledge-driven organizations, it is not surprising that colleges and universities should be greatly affected by the rapid advances in information technology—computers, telecommunications, and networks.  In the past several decades, computers have evolved into powerful information systems with high-speed connectivity to other systems throughout the world.  Public and private networks permit voice, image, and data to be made instantaneously available across the world to wide audiences at low costs.  The creation of virtual environments where human senses are exposed to artificially created sights, sounds, and feelings liberate us from restrictions set by the physical forces of the world in which we live.  Close, empathic, multi-party relationships mediated by visual and aural digital communications systems lead to the formation of closely bonded, widely dispersed communities of people interested in sharing new experiences and intellectual pursuits created within the human mind via sensory stimuli.  Rapidly evolving technologies are dramatically changing the way we collect, manipulate, and transmit information.
     This technology has already had dramatic impact on our colleges and universities.  Our administrative processes are heavily dependent upon information technology—as the current concern with the approaching date reset of Year 2000 has made all too apparent.  Research and scholarship depend heavily upon information technology, e.g., the use of computers to simulate physical phenomena, networks to link investigators in virtual laboratories or "collaboratories," or digital libraries to provide scholars with access to knowledge resources.  Yet, there is an increasing sense that new technology will have an its most profound impact on the educational activities of the university and how we deliver our services.
     Most significant here is the way in which emerging information technology has removed the constraints of space and time.  We can now use powerful computers and networks to deliver educational services to anyone anyplace anytime, confined no longer to the campus or the academic schedule.  Technology is creating an open-learning environment in which the student has evolved into an active learner and consumer of educational services, stimulating the growth of powerful market forces that could dramatically reshape the higher education enterprise.
     Again, we must face the possibility that the current paradigm of the university may not be capable of responding to the opportunities or the challenges of the new knowledge media or the needs of the digital generation.

Market Forces

     We generally think of public higher education as public enterprise, shaped by public policy and actions to serve a civic purpose.  Yet market forces also act on our public colleges and universities.  Society seeks services such as education and research.  Academic institutions must compete for students, faculty, and resources.  To be sure, the market is a strange one, heavily subsidized and shaped by public investment so that prices are always far less than true costs.  If prices such as tuition are largely fictitious, even more so is much of the value of education services, based on myths and vague perceptions such as the importance of a college degree as a ticket to success or the prestige associated with certain institutions.
     In the past, most colleges and universities served local or regional populations.  While there was competition among institutions for students, faculty, and resources—at least in the United States—the extent to which institutions controlled the awarding of degrees, that is, credentialing, led to a tightly controlled competitive market.  Universities enjoyed a monopoly over advanced education because of geographical location and their monopoly on credentialing through the awarding of degrees.  Today all of these market constraints are being challenged, as information technology eliminates the barriers of space and time and as new competitive forces enter the marketplace to challenge credentialing.
     As a result, higher education is rapidly evolving from a loosely federated system of colleges and universities serving traditional students from local communities to, in effect, a global knowledge and learning industry.  With the emergence of new competitive forces and the weakening influence of traditional regulations, the higher education enterprise is evolving like other "deregulated" industries, e.g., health care, or communications or energy.  In contrast to these other industries which have been restructured as government regulation has disappeared, the global knowledge industry is being unleashed by emerging information technology which releases education from the constraints of space and time, even as its credentialing monopoly begins to break apart.  And, as our society becomes ever more dependent upon new knowledge and educated people—upon knowledge workers—this global knowledge business must be viewed clearly as one of the most active growth industries of our times.
     Many in the academy would undoubtedly view with derision or alarm the depiction of the higher education enterprise as an "industry" or "business," operating in a highly competitive, increasingly deregulated global marketplace.  Nevertheless, this is an important perspective that will require a new paradigm for how we think about postsecondary education.  Furthermore, it is clear that no one, no government, is in control of the higher-education industry.  Instead it responds to forces of the marketplace.  Universities will have to learn to cope with the competitive pressures of this marketplace while preserving the most important of their traditional values and character.

Evolution or Revolution?

     In spite of the growing awareness of the changing nature of our world, many within the academy still believe that change will occur only at the margins of higher education.  They see the waves of change lapping on the beach as just the tide coming in, as it has so often before.  They stress the role of the university in stabilizing society during a period of change rather than leading those changes.  This too shall pass, they suggest, and demand that the university hold fast to its traditional roles and character.  And they will do everything within their power to prevent change from occurring.
     Yet, history suggests that the university must change and adapt in part to preserve these traditional roles.  It is true that many, both within and outside the academy, believe that significant change must occur not simply in the higher education enterprise but in each and every one of our institutions. Most of these people see change as an evolutionary, incremental, long-term process, compatible with the values, cultures, and structure of the contemporary university. 
     There are a few voices, however, primarily outside the academy, who believe that both the dramatic nature and compressed time scale characterizing the changes of our times will drive not evolution but revolution.  They have serious doubts about whether the challenges of our times will allow such gradual change and adaptation.  They point out that there are really no precedents to follow.  Some even suggest that long before reform of the educational system comes to any conclusion, the system itself will collapse.
     The forces driving change in higher education, both from within and without, may be far more powerful than most people realize.  It could well be that both the pace and nature of change characterizing the higher education enterprise both in America and worldwide will be considerably beyond that which can be accommodated by business-as-usual evolution.  As one of my colleagues put it, while there is certainly a good deal of exaggeration and hype about the changes in higher education for the short term—meaning five years or less—it is difficult to stress too strongly the profound nature of the changes likely to occur in most of our institutions and in our enterprise over the longer term—a decade and beyond.
     While some colleges and universities may be able to maintain their current form and market niche, others will change beyond recognition.  Still others will disappear entirely.  New types of institutions—perhaps even entirely new social learning structures—will evolve to meet educational needs.  In contrast to the last several decades, when colleges and universities have attempted to become more similar, the years ahead will demand greater differentiation.  There will be many different paths to the future.
     So what might we expect over the longer term for the future of the university?  It would be impractical and foolhardy to suggest one particular model for the university of the 21st Century.  The great and ever-increasing diversity characterizing higher education in America makes it clear that there will be many forms, many types of institutions serving our society.  But there are a number of themes, which will almost certainly factor into at least some part of the higher education enterprise. 

  • Learner-centered:  Just as other social institutions, our universities must become more focused on those we serve.  We must transform ourselves from faculty-centered to learner-centered institutions.
  • Affordable:  Society will demand that we become far more affordable, providing educational opportunities within the resources of all citizens.  Whether this occurs through greater public subsidy or dramatic restructuring of our institutions, it seems increasingly clear that our society—not to mention the world—will no longer tolerate the high-cost, low productivity paradigm that characterizes much of higher education in America today.
  • Lifelong Learning:  In an age of knowledge, the need for advanced education and skills will require both a willingness to continue to learn throughout life and a commitment on the part of our institutions to provide opportunities for lifelong learning.  The concept of student and alumnus will merge.  Our highly partitioned system of education will blend increasingly into a seamless web, in which primary and secondary education; undergraduate, graduate, and professional education; on-the-job training and continuing education; and lifelong enrichment become a continuum.
  • Interactive and Collaborative:  Already we see new forms of pedagogy:  asynchronous (anytime, anyplace) learning that utilizes emerging information technology to break the constraints of time and space, making learning opportunities more compatible with lifestyles and career needs; and interactive and collaborative learning appropriate for the digital age, the plug-and-play generation.
  • Diverse:  Finally, the great diversity characterizing higher education in America will continue, as it must to serve an increasingly diverse population with diverse needs and goals.

We will need a new paradigm for delivering education to even broader segments of our society, perhaps to all of our society, in convenient, high quality forms, at a cost all can afford.   Fortunately, today's technology is rapidly breaking the constraints of space and time.  It has become clear that most people, in most areas, can learn and learn well using asynchronous learning, that is, "anytime, anyplace, anyone" education.  Lifetime education is rapidly becoming a reality, making learning available for anyone who wants to learn, at the time and place of their choice, without great personal effort or cost.  With advances in modern information technology, the barriers in the educational system are no longer cost or technological capacity but rather perception and habit.
     But even this may not be enough.  Perhaps we should instead consider a future of "ubiquitous learning"—learning for everyone, every place, all the time.  Indeed, in a world driven by an ever-expanding knowledge base, continuous learning, like continuous improvement, has become a necessity of life.
     Rather than "an age of knowledge," we could instead aspire to a "culture of learning," in which people are continually surrounded by, immersed in, and absorbed in learning experiences.  Information technology has now provided us with a means to create learning environments throughout one's life.  These environments are able not only to transcend the constraints of space and time, but they, like us, are capable as well of learning and evolving to serve our changing educational needs.  Higher education must define its relationship with these emerging possibilities in order to create a compelling vision for its future as it enters the next millennium.

The Challenges to Engineering Education

     Let us now turn to the subject of engineering education, within the context of the rapid changes in our society and the institutions that serve it.  Study after study has suggested that dramatic change is necessary in engineering education.  There have been dozens of conferences and reports, major programs such as the NSF Engineering Coalitions and Systemic Initiatives efforts, and hundreds of efforts by individual engineering schools.  Even professional societies have called for reform, e.g., through the new Engineering Criteria 2000 requirements of the Accreditation Board on Engineering and Technology (ABET). 
     Despite these efforts, many today believe that engineering education remains trapped in a mid-20th Century paradigm (or perhaps even a late 19th Century paradigm, if my archeological discoveries about similarity between early engineering curricula and today's offerings are correct).  We continue to provide a form of engineering education, which, while familiar from our own educational experiences, is increasingly irrelevant to the changing needs of a profession—not to mention a society—that is already far beyond our universities.  Let me list some of the more apparent issues and concerns:

The Changing Nature of Engineering Practice

     Today, engineering practice is evolving rapidly in response to a rapidly changing world.  The shifting nature of national priorities from defense to economic competitiveness, the impact of rapidly evolving information technology, the use of new materials and biological processes—all have had deep impact on engineering practice.  Put another way, the shift of our society from guns to butter, from transportation to communication, from atoms to bits, means that today's engineering students will spend most of their careers coping with challenges and opportunities vastly different from those most currently practicing engineers—or currently teaching faculty—have experienced. 
     While engineers are expected to be well grounded in the fundamentals of science and mathematics, they are increasingly expected to acquire skills in communication, teamwork, adaptation to change, and social and environmental consciousness.  It is also clear from this perspective that engineering education simply has not kept pace with this changing environment.  It is only a slight exaggeration to say that our students are currently being prepared to practice engineering in a world that existed when we, as their faculty, were trained a generation or two ago.  They are not being prepared for the 21st Century.

From Specialization to Integration

     The intellectual activities of the contemporary university are partitioned into increasingly specialized and fragmented disciplines.  Perhaps reflecting the startling success of science in the 20th Century, most disciplines are reductionist in nature, focusing teaching and scholarship on increasingly narrow and specialized topics.  While this produces graduates of great technical depth, it is at a certain sacrifice of a broader, more integrated education.  This is particularly true in science-based disciplines such as engineering.  The old saying is not far off the mark, "A Harvard graduate knows absolutely nothing about absolutely everything.  An MIT graduate knows absolutely everything about absolutely nothing!"
     We must question the value of narrow specialization at a time when engineering practice and engineering systems are becoming large, more complex, and involving components and processes from widely dispersed fields.  Many believe that the most important intellectual problems of our time will not be addressed through disciplinary specialization but rather through approaches capable of integrating many different areas of knowledge—through "big think" rather than "small think".
     Ironically enough, the essence of engineering practice is the process of integrating knowledge to some purpose.  Unlike the specialized analysis characterizing scientific inquiry, engineers are expected to be society's master integrators, working across many different disciplines and fields, making the connections that will lead to deeper insights and more creative solutions, and getting things done.  Thus, engineering education is under increasing pressure to shift away from specialization to a more comprehensive curriculum and broader educational experience in which topics are better connected and integrated.

Learning for Life

     As the knowledge base in most engineering fields continues to increase at an ever more rapid rate, the engineering curriculum has become bloated with technical material, much of it already obsolete.  Most undergraduate engineering programs have already become almost five years in length for most students.  Even with this increasing technical content, most engineers will spend many months if not years in further workplace training before they are ready for practice.  Furthermore, the effort to include the new technical knowledge in many fields, while retaining as well much of the old, has squeezed out other important curriculum content in areas.  For example, at the University of Michigan, the humanities and social sciences component of the undergraduate curriculum has dropped to less than twenty credit hours, with as low as two credit hours of free electives in some engineering majors.
     We simply have to accept the fact that it is no longer possible (if it ever was) for an engineering student to learn all they need to know during their undergraduate studies.  Acquiring the array of technical knowledge and experience is a lifetime goal and requires a personal commitment to continual learning.  An undergraduate engineering education should be viewed as only the initial launch for a career, designed to place the student in a lifetime orbit of learning.

The Professional Degree

     As the growth of technical knowledge accelerates and the undergraduate engineering curriculum becomes more bloated and strained with new technical content, it becomes ever more apparent that it is simply no longer possible to regard the baccalaureate degree as sufficient for professional practice.  Today, engineering is one of the very few professions that require only an undergraduate degree for professional status.  Most other knowledge-intensive professions such as law, medicine, and even business, utilize graduate programs built upon a diversity of undergraduate majors.  Little wonder that the status of engineers lag somewhat behind those of other professionals with more advanced education.
     The inadequacy of the baccalaureate degree for professional practice is becoming apparent to employers as well.  There is an increasing trend to hire graduates at the masters or even Ph.D. level for technical work, while relying upon baccalaureate engineering graduates for supporting services such as sales and technical support.  Although study after study has recommended that the masters degree become the accepted route into the engineering practice, this continues to be resisted both by the profession and the academy.

Curriculum Reform

     There is little doubt that the current sequential approach to engineering education, in which the early years are dominated by science and mathematics courses with engineering content deferred to the upper-class years, discourages many capable students.  Students have little opportunity to find out what engineering is all about until late in their undergraduate studies.  It is not unusual to find students wandering into our counseling and placement offices in their senior year, still trying to find out what they are majoring in and what they can do with an engineering degree.  Compounding this is the fragmentation of the current curriculum, consisting of highly specialized and generally unconnected and uncoordinated courses, whose relationship to one another and to engineering education is rarely explained.  Although everyone agrees that the undergraduate curriculum should focus on the fundamentals, few can agree on just what content is truly fundamental. 
     While the rigor of the scientific and mathematics foundation of modern engineering is important, it must be augmented by the broader contextual and integrative approach characterizing engineering practice.  Students must gain experience not only in solitary analysis but also in group work and hands-on "design-build-operate" projects.  We must strive to integrate real design and process understanding into the educational system.  Above all, we must challenge our students to think, to create, and to understand excellence.

Shifting Careers

     In today's world of change, most graduates will find themselves frequently changing not only jobs, but entire careers.  We already find that only about fifty percent of engineering graduates will enter technical careers, and after five years, about half of these will have moved into other areas such as management or sales.  Put another way, most engineering graduates of today will find themselves in engineering practice for only a relatively short period, if at all.
     Yet the increasing importance of technology to our world has made an engineering degree an excellent preparation for many other careers and professions:  business, law, medicine, consulting, and government service, to name only a few.  This poses a particular challenge to engineering educators, since they still focus primarily on educating students for the engineering profession. 
     Instead, as Roland Schmitt has noted, we must enlarge the very concept of the engineer to cover a wider range of human activities than every before.  Engineering educators must begin by realizing that it is their duty to educate the leaders of our society as well as to educate the professional engineer.  We should develop and promote a new kind of engineering education as a form of "liberal education" for the 21st Century.  This will require new objectives and new curricula, some radically different that those of today because of a radically different objective:  educating not simply professional engineers but a new breed of graduates with an engineering-based, liberal education.

Diversity

     America's population is changing rapidly.  Today roughly eighty-five percent of the new entrants to the workforce are minorities, women, or immigrants.  It is becoming ever more apparent that the strength of our engineering workforce will be dependent upon our ability to provide these underrepresented groups with the opportunity for an engineering education. 

The Faculty

     Engineering faculties are almost unique among those of professional schools since they generally have little experience or activity in professional practice.  The strong research focus of most engineering schools has led to a cadre of strong engineering scientists, able at generating new knowledge but relatively inexperienced in professional practice.  Furthermore, engineering faculty are judged and rewarded by criteria appropriate to science faculty.  Indeed, professional practice is not only absent in promotion and reward criteria, but frequently discouraged.  The faculty reward system recognizes teaching, research, and service to the profession, but it gives little recognition for developing a marketable product or process or designing an enduring piece of the nation's infrastructure.
     It would be hard to imagine a medical school faculty comprised only of biological scientists rather than practicing physicians or music school faculty comprised only of musicologists rather than performing artists.  Yet such detachment from professional practice and experience is the norm in engineering education.

The Responses Thus Far

     Engineering educators, professional societies, and federal funding agencies such as the National Science Foundation have not been insensitive to these concerns.  Following an intensive dialog among engineering deans, professional societies, and the Accreditation Board of Engineering and Technology has significantly restructured its criteria for accreditation of undergraduate engineering education.  The new Engineering Criteria 2000 includes, among other elements, criteria which stress the important of an engineering graduate's ability to:

  • Apply knowledge of science, mathematics, and engineering
  • Design and conduct experiments and analyze data
  • Design a system, component, or process to meet desired needs
  • Function on multi-disciplinary teams
  • Identify, formulate, and solve engineering problems
  • Understand professional and ethical responsibility
  • Communicate effectively
  • Understand the impact of engineering solutions in a global/social context
  • Engage in life-long learning
  • Exhibit a knowledge of contemporary issues
  • Use the techniques, skills, and modern engineering tools necessary for engineering practice.

The new ABET criteria also allow greater flexibility on the part of engineering schools to innovate and experiment with new approaches to engineering education.
     Many engineering schools have taken important steps to address concerns about engineering education.  For example, the University of Michigan's College of Engineering is moving to implement a new Curriculum 2000 with the mission of preparing graduates to begin a lifetime of technical and professional creativity and leadership in their chosen fields.  Michigan has chosen to identify three quite distinct educational paths:

  • For students wishing to enter engineering practice at highest level, provide a combined BSE/MSE and BSE/MEng path that can be completed in ten semesters.
  • For students wishing to enter practice with only entry-level preparation or pursue graduate work in alternative fields, will provide BSE and BS paths that can be completed in eight semesters.
  • For students interested in advanced graduate study, will provide a Research Honors BSE path characterized by significant research experience during the final two years

Some of the more significant objectives of the new effort include:

  • Adopting a four-course, four-SCH/course, eight semester conceptual model for all BSE and BS engineering curricula
  • Requiring all majors to have at least twelve hours of free electives
  • Offering a common curriculum for all first-year students so that it would not be necessary to decide on a major prior to the second year
  • Introducing a new first-year course, Engineering 100, that includes both project work and technical communication
  • Instituting a program of Communication Across the Curriculum with at least three credit hours of technical communication
  • Requiring that environmental issues and professional ethics be included implicitly in curriculum

The National Science Foundation has also played an important role in the modernization of the engineering curriculum.  As the science and engineering education activities of the NSF were restored during the late 1980s after devastating cuts earlier in the decade, engineering education had a high priority.  Not only were programs launched encouraging curriculum innovation, but a broader set of initiatives aimed at systemic change were launched such as the Engineering Coalitions Program.  Furthermore, a broad range and studies and workshops were sponsored to better define the nature of the "new engineering education" appropriate for the 21st Century.  
     These studies were remarkably consistent in the attributes they recommended for the new breed of engineering graduates.  All agreed that system change in engineering education would require a concurrent change from the predominant engineering school academic culture based on compartmentalization of knowledge, individual specialization, and a research-based reward structure to one that values integration as well as specialization, teamwork as well as individual achievement, and educational research and innovation as well as research in the engineering sciences.

These studies suggested a new set of goals for engineering education:

  • To offer a broad liberal education that provides the diversity and breadth needed for engineering
  • To prepare graduates for entry into careers and further study in both the engineering and nonengineering marketplace
  • To develop the motivation, capability, and knowledge base for lifelong learning

This will require a very major change in the engineering curriculum.  To some degree, it will require modernizing the science and mathematics instruction, e.g., recognizing that discrete rather than continuous mathematics is the foundation of the digital age, that biology and chemistry are rapidly becoming more important than physics, that new materials and processes have made obsolete much of the traditional curriculum.  Beyond these technical changes, the NSF studies recognized that the new engineering curriculum must reflect a broad range of concerns, including environmental, political, social, international, and legal and ethical ramifications of decisions.  Although the technical component would continue to be the core of an engineering education, the economic, political, social, and environmental context of engineering practice needs to be explicitly addressed. 

The skill set of the new engineering will be:

Engineering science (analysis)

  • Systems integration (synthesis)
  • Problem formulation as well as problem solving
  • Engineering design
  • The ability to realize products
  • Facility with intelligent technology to enhance creative opportunity
  • Ability to manage complexity and uncertainty
  • Teamwork (sensitivity in interpersonal relationships)
  • Language and multicultural understanding
  • Ability to advocate and influence
  • Entrepreneurship and decision making
  • Knowledge integration, education, and mentoring

Beyond that, engineering education should move away from the current dominance of classroom-based pedagogy to more active learning approaches that engage problem-solving skills and team building.  Bordogna recalls the old Chinese proverb:

      I hear and I forget.
      I see and I remember.
      I do and I understand.

This is apt indeed for engineering education.  As a recent NSF workshop put it, the ubiquitous lecture is the bane of true learning, especially in observation-based, hands-on fields such as engineering.  The lecture-dominated system encourages a passive learning environment, a highly compartmentalized (lecture-sized) curriculum, and worst of all, instills neither the motivation nor the skills for life-long learning.  The dependence on the standard lecture must be diminished with emphasis given instead to discovery-oriented learning.  We must create discovery-oriented learning environments that capitalize on the full power of new communication, information, and visualization technologies."
     Undergraduate engineering programs can no longer ignore the fact that they simply cannot provide all the necessary knowledge for graduates to remain competitive throughout their careers.  Content-based learning alone must not drive engineering education.  The primary aim should be instead to instill a strong knowledge of how to learn, while still producing competent engineers who are well-grounded in engineering science and mathematics and have a understanding of design in the social context.  Engineering schools must educate the student for a lifetime of learning rather than just for their initial job.  Students must learn how to learn, and they must be able to assess their skills and educational needs throughout their many careers.  As Peter Drucker puts it, "We are redefining what it means to be an educated person.  Traditionally an educated person was someone who had a prescribed stock of formal knowledge.  Increasingly an educated person will be someone who has learned how to learn and who continues to learn throughout his or her lifetime."

Why Is Change So Slow?

     Despite this broad effort, change in engineering education has been modest, as reflected in the tone of frustration in the recent remarks of Bill Wulf, President of the National Academy of Engineering:  "We have studied engineering reform to death.  While there are differences among the reports, the differences are not great.  Let's get on with it!  It is urgent that we do!"
     Who is holding back change?  Professional societies and accreditation agencies such as ABET?  No, we have seen that they have become important forces of change.
     What about industry?  To be sure there is still a good deal of myopia among the recruiters that visit our placement office, all too often reinforcing very narrow definitions of student majors and abilities.  Yet at high levels of management, there is strong awareness of the need for a broader form of engineering education.  In a recent survey of CEOs conducted by the Business Higher Education Forum, it was found that the qualities valued most highly in graduates were not specific technical knowledge or skills but rather:

  • The ability to communicate well
  • A commitment to lifelong learning
  • The ability to adapt to an increasingly diverse world
  • The ability not only to adapt to change but to actually drive change

What about the academy?  To be sure, change is sometimes a four-letter word on university campuses.  It is sometimes said that universities change one grave at a time.  Judging from my comparison of the engineering curriculum of a century ago, even this may be too optimistic for engineering education.  In fact, engineering educators do tend to be very conservative with regard to pedagogy, curriculum, and institutional attitudes.  This conservatism produces a degree of stability (perhaps inflexibility is a more apt term) that results in a relatively slow response to external pressures. 
     For the past several decades, the emphasis of engineering education has been focused on the scientific foundation of engineering knowledge.  In part this had to do with the impact of modern science on technology.  But it was also due to the culture of the research university, in which engineering faculty were evaluated based on their performance in fundamental research rather than engineering practice.  Many believe this emphasis on research also has eroded the quality of teaching in engineering schools.  In fact, a recent conference of young faculty suggested that most engineering schools not only fail to support adequately but also outright discourage faculty achievements in teaching, instructional scholarship, and public service.  Tenure and promotion criteria do not encourage faculty to aspire to broad scholarly achievements, especially innovation, nor to contributions to public understanding.

How Can We Accelerate Change?

     In the spirit of stimulating debate and thought, let me suggest a few more Draconian actions designed both to shake up and transform engineering education:

Eliminate all specialized engineering majors

The ever more narrow specialization among engineering majors is driven largely by the reductionist approach of scientific analysis rather than the highly integrative character of engineering synthesis.  It may be appropriate for basic research, but it is certainly not conducive to the education of contemporary engineers nor to engineering practice.  Although students may be stereotyped by faculty and academic programs—and perhaps even campus recruiters—as electrical engineers, aerospace engineers, etc., they rapidly lose this distinction in engineering practice.  Today's contemporary engineer must span an array of fields, just as modern technology, systems, and processes.
     Perhaps it is time to go even further and simply abandon the concept of an undergraduate engineering major and instead provide a general engineering curriculum, much as in other professions such as medicine, law, and business.  Like these professions, one could leave specialization until later, provided either through graduate study or on-the-job training. 
     In fact, one might conjecture that in a future characterized by lifelong learning, perhaps engineering will rapidly evolve along the lines of other learned professions and shift professional education and training entirely to the graduate level, eliminating the undergraduate engineering degree altogether.  There are strong reasons to suspect that a broad, liberal education is just as important for engineering practice as it is for other professions such as medicine and law.   (Here one could also make the case for significantly greater technical and scientific content in the contemporary liberal arts curriculum.)

Shift away from the classroom to more suitable forms of pedagogy

Although science and engineering are heavily based on laboratory methods, in fact they are usually taught through classroom lectures coupled with problem-solving exercises.  Contemporary engineering education stresses the analytic approach to solving well-defined problems so familiar from science and mathematics—not surprising, since so many engineering faculty members received their basic training in science rather than engineering.
     To be sure, design projects required for accreditation of engineering degree programs are introduced into advanced courses at the upper-class level.  Yet design and synthesis are quite small components of most engineering programs.
     Clearly those intellectual activities associated with engineering design—problem formulation, creativity, innovation—should be introduced throughout the curriculum.  This will require a sharp departure from classroom pedagogy and solitary learning methods.  Beyond team design projects, engineering educators might consider adopting the case method approaches characterizing business and law education.  More use might be made of internships as a formal part of the engineering curriculum, whether in industry or perhaps even in the research laboratories of engineering faculty where engineering design is a common task.

Attract more practitioners into engineering education

It is absolutely essential to broaden the engineering faculty to include practitioners.  One approach would be to work with industry to persuade and allow senior engineering staff to accept faculty appointments.  In fact, many retired engineers would make ideal faculty members, bringing their wealth of experience in engineering practice not only to the students but to the reshaping of the current science-driven culture of engineering schools.  Of course, this would require a very significant restructuring of the faculty promotion and reward systems.  It might even lead to the elimination of tenure, at least in some components of engineering education.  But the mix of practitioners and scholars has been both accepted and constructive in most other professional schools—medicine, law, business, architecture, and the fine arts.  It seems high time to bring engineering education into line.

Broaden the perspective of engineering education

As we noted earlier, engineering educators should be challenged to devise an engineering-based "liberal education" for students of the 21st Century.  Engineering principles and modes of thought should be the centerpiece of what the liberally educated person should know in the age of knowledge that is our future.  We should produce graduates for all careers—from industry to law to government—with an education attuned to the issues and challenges of a knowledge-driven society, many of which have dominant technical themes.

Concluding Remarks

     We have entered a period of significant change in higher education as our universities attempt to respond to the challenges, opportunities, and responsibilities before them.  This time of great change, of shifting paradigms, provides the context in which we must consider the changing nature of the university and academic programs such as engineering education.
     Much of this change will be driven by market forces—by a limited resource base, changing societal needs, new technologies, and new competitors.  But we also must remember that higher education has a public purpose and a public obligation. Those of us in higher education must always keep before us two questions:  "Who do we serve?" and "How can we serve better?"  And society must work to shape and form the markets that will in turn reshape our institutions with appropriate civic purpose.
     From this perspective, it is important to understand that the most critical challenge facing most institutions will be to develop the capacity for change.  We must remove the constraints that prevent us from responding to the needs of rapidly changing societies, remove unnecessary processes and administrative structures, and question existing premises and arrangements.  Universities should strive to challenge, excite, and embolden all members of their academic communities to embark on what should be a great adventure for higher education.
     Certainly the need for higher education will be of increasing importance in our knowledge-driven future. Certainly, too, it has become increasingly clear that our current paradigms for the university, its teaching and research, its service to society, its financing, all must change rapidly and perhaps radically.  The real question is not whether higher education will be transformed, but rather how . . . and by whom.  If the university is capable of transforming itself to respond to the needs of a culture of learning, then what is currently perceived as the challenge of change may, in fact, become the opportunity for a renaissance in higher education in the years ahead.


CAUTION: OUTCOMES ASSESSMENTS IN USE

George A. Hazelrigg

     As we move into the new millennium, outcomes assessments are being adopted as the cornerstone of our engineering education continuous-improvement process. They comprise a key element of the ABET 2000-2001 Criteria, with many advocates of outcomes assessments touting both their simplicity and promise. Myriad papers have appeared on the subject of their conduct and efficacy. Yet, assessments are neither simple to conduct nor obvious in their use, and few people, if any, ever validate their application. So, it is clearly time to step back a bit and take a more critical look at the conduct and use of assessments.
     Already, the tone I have set suggests that there may be at least complexities associated with the use of outcomes assessments, if not downright problems. And, indeed, this is the case. To see how problems can arise, well take a very brief look at the mathematics of an assessment. In the typical assessment, a set E of j entities exist, each with k attributes, a. Through an assessment, we seek to abstract information from this field and map it onto assessment measures, m. For example, the set E might consist of graduating students, and a measure might be the percentage of those students whose Graduate Record Exam score is above a certain number. In this case, the score each student obtains is an attribute of that student. In all practical situations, the dimesionality of m is far less than the product jk, and thus the measures represent a condensation of the original data such that the measures are uniquely determined by the data, but the data are not uniquely determined by the measures. In mathematics, this is referred to as a many-to-one mapping.
     Now suppose that we elect to use such measures as indicators in a decision making process.  Mathematically, we wish to choose one alternative, si , from among the set of alternatives, S, such that our utility of the choice, u(m,X) is maximized, where m are the measures in question and X are other factors, the inference being that m clearly influences our choice. Already, you might suspect that, because of the loss of information in going from the raw attribute data to the measures, m, strange results can occur. This is indeed the case. The surprise, however, is the frequency with which strange results occur, and the extent of their consequence. The danger lies in the fact that improper or naive use of assessments can lead us to very poor choices indeed. A simple example will illustrate this point.
     The faculty at Omega Institute of Technology have been trying to improve teaching techniques for the presentation of fundamental concepts in engineering. Two rather promising teaching techniques, Methods B and C, have been proposed as alternatives to the present approach, which we will call Method A. It is decided to test these techniques on the incoming Freshman class of 342 students. The class is divided into three groups of 114 each, and each group is taught three different topics, each with a different one of the three techniques. The teaching techniques are rotated among the groups, so that every student experiences every technique, and every technique is used on every topic. After each topic is covered, a standardized test is administered, and the students grades are recorded. For simplicity, let us assume that the three groups are perfectly sampled, so that there is no sampling error, and all students test scores are completely reflective of the knowledge that they gained from their studies so that the assessments are true and accurate. Thus, any problems that arise are strictly the result of the assessment method itself.
     The question asked is, which teaching method is most efficacious? Efficacy is measured in Method A Method B Method C terms of test scores. After testing the students, the faculty scanned and aggregated the test scores to determine which teaching method yielded the highest scores. The results are shown in Figure 1.  Clearly, of the three methods tested, more students do better with Method C. The outcomes have been assessed and, of the three methods tested, the students taught by Method C get the highest grades. The choice to adopt Method C is now well rooted in the assessment, and this then is the choice taken.

20

25

30

35

40 Percent

Figure 1 Percent of students for which each method yields the highest test scores.

What could be more straight forward than an assessment such as this? To be sure, Method C is the best method among the three tested. What could possibly be wrong? We get some interesting insights when we look at an underlying student population from which such a result as that given in Figure 1 might derive. Recognizing that different teaching methods work better for different students, the table below shows one possible student population that would yield exactly the results of Figure 1.

Relative Efficacy of Methods

Number of Students

A>B>C

99

A>C>B

0

B>A>C

75

B<C<A

42

C>A>B

75

C>B>A

51


Given this underlying population, lets now compare the different teaching methods by pairs.  It is easy to see that more students, 174 out of 342 or 51 percent, get a higher grade when taught by Method A than when taught by Method B. Also, more students get higher grades when taught by Method A than Method C. Further, 216 out of the 342 students or 63 percent score higher when taught by Method B than Method C. In fact, more students would get higher grades if taught by Method A, the current method, than either Methods B or C. Surprisingly this result is completely compatible with the results of Figure 1. Furthermore, more students would get higher grades if taught by Method B than Method C. So, examination of the underlying population provides exactly the opposite insights as our original assessment results. Based on their assessment, OIT has elected to adopt a teaching method that will actually result in students getting lower grades. 
     There is a very subtle yet crucial difference between the results of Figure 1 and those obtained by pairwise comparison of the methods. The results of Figure 1 address the question, if all students are taught by all three methods, A, B and C, what fraction of students will obtain their highest grades for each method? But the pairwise comparison addresses the question, if all students are taught by only one method, which method will result in more students getting better grades? The surprise is that, although these questions differ only very subtly, the correct conclusions for each are diametrically opposite.
     Of course, the student population from which we obtained the above example is just one possible population that gives the result of Figure 1. There are many others, and they can give quite different pairwise conclusions. So, without asking exactly the right question and studying the underlying population, the assessment is hardly better than flipping a coin. But the problem is that most assessment methods, such as the seemingly considered one taken by OIT, never retain the necessary data on the underlying population. So, in fact, the validity of their assessment could not be ascertained.
     Are these problems commonplace, or have I merely constructed a rare pathological case? I contend that there exists ample evidence that these cases are the rule, not the exception. For the example given above, and with overly conservative assumptions about the data, almost 70 percent of the possible data sets cause a paradox of some sort. Indeed, the case given above is relatively well behaved. In this case there is a best method, namely a method for which more students get higher grades than for any other method. For many underlying populations, there will be no choice for which there is not a better choice. For example, it may be the case that Method A is better than Method B, Method B is better than Method C, and Method C is better than Method A. Thus, every choice is inferior to some other choice (for extensive discussions on this possibility, refer to the literature on Arrows Impossibility Theorem for which Kenneth Arrow won the Nobel Prize in economics).
     Problems with assessments have been a major topic of concern and research in the social sciences for at least 300 years. The interested reader will find an extensive literature survey at the following web site:
http://www.maxwell.syr.edu/maxpages/faculty/jskelly/biblioho.htm. More recently, mathematician Donald Saari, recently Northwestern University now at the University of California at Irvine, has studied and published on these topics at length (see the referenced biblio).  The interested reader is strongly urged to look at this literature. There is far too little space in this magazine to do this topic justice. Yet, conducted without adequate expertise, outcomes assessments are dangerous at best. And I have touched upon only one of the significant concerns here.
     As a conclusion, I would offer the following warning: CAUTION, OUTCOMES ASSESSMENTS IN USE.

George Hazelrigg is the National Science Foundation Program Director for Design and Integration Engineering .  The views expressed above are strictly those of Dr. Hazelrigg, and they do not necessarily reflect the views of either the National Science Foundation or the Federal Government.