Portfolios have been widely used in elementary and secondary schools for more than a decade. Engineering education's current interest in using portfolios of student work has been driven by the adoption of the Accreditation Board for Engineering and Technology's Engineering Criteria 2000, in which portfolios are mentioned as one way to document and assess student outcomes.

Our assessment planning team reasoned that the use of electronic portfolios (dubbed "RosE-Portfolios") would greatly reduce the storage and access disadvantages of the portfolio system and provide students an opportunity to document their learning in a familiar multimedia format. The RosE-Portfolios were also deemed an efficient and cost-effective method of collecting and accessing student materials.

Rose-Hulman faculty and staff members designed and developed the electronic portfolios with an eye toward how they were to be used by both faculty members and students. We also identified the following primary design requirements:

• Ease of use
• Ability to archive student material in multimedia format
• Allow searching by multiple criteria
• Permit students to update and replace materials
• User access online, anytime
• Faculty ratings automatically logged and aggregated
• Provide students with feedback online
• Student submissions focused on institute-defined learning outcomes

The RosE-Portfolio system is Web-based, allowing students to access their portfolio using their network username and password. The Oracle database provides all the archiving and search functions necessary to meet the design specifications, including a feature that automatically compresses submitted documents, reducing the required data storage space. We are currently considering providing the students with their electronic portfolios on CD-ROM when they graduate.

Special screens developed for faculty raters and faculty advisors allow searching using multiple criteria, such as learning outcome goal and performance criteria. For the purpose of rating submissions, students will be identified by number only to ensure objectivity. Faculty advisors will have access only to the work of their advisees, and they can search by outcome goal for all advisees or on any subset of performance criteria. They can also review the entire portfolio for any given advisee, and, by monitoring submission dates, quickly determine whether advisees are keeping their RosE-Portfolios current.

Students had 10 weeks to complete the project, then participated in an extensive assessment of the pilot project experience that included both written questionnaires and focus groups. The volunteers submitted more than 130 files to the RosE-Portfolios.

Six faculty evaluators spent two days reviewing and rating the student-submitted materials. The reviewers had information explaining the concept of holistic scoring and the use of rubrics to rate student work. The primary purpose of the session was to determine the clarity of the written materials and instructions for faculty raters and the ease of holistic scoring and applying the assessment rubrics. We were also interested in finding out whether the faculty raters believed that the use of the RosE-Portfolio would produce the type of information from students that would help us to evaluate and improve our programs.

More than 50 percent of the students considered submitting evidence of learning outcomes that were a result of activities outside the classroom. As sophomores, these students thought the most difficult objectives to document for seniors would be those related to ethical and professional responsibility and an understanding of contemporary issues' interrelationships with mathematics, science, and engineering. Students were optimistic that the use of the RosE-Portfolio process would provide Rose-Hulman with information that would help to improve programs, and more than 50 percent of the sophomores in the pilot project indicated that they would like to continue to use the RosE-Portfolio.

Faculty members. Raters reported that they found the system reliable and easy to use, but noted that some of the performance criteria were not clearly written and some criteria were too complex. Faculty members proposed rewording some of the criteria for clarity. Examining student submissions was deemed generally enlightening in regard to a wide range of student abilities.

What's in it for students? And what will motivate students to keep their portfolios current and take the project seriously? The system provides opportunities for students to customize their RosE-Portfolio Web site, and encourages them to use their portfolios as they seek internships, co-ops, or employment after graduation.

The Career Services office, which has been involved in the development of the RosE-Portfolio, is hopeful that students will "hotlink" the best of their portfolio work to an online resume built into their portfolios. This will provide recruiters an opportunity to view a student's writing samples, design projects, entries documenting their understanding of the importance of a global perspective, ethics, and so on, in a multimedia format.

What's in it for faculty members? The RosE-Portfolio system is student-driven, eliminating the need for faculty members to be responsible for the collection of student material for submission. Faculty advisors will have access to their advisees' RosE-Portfolios for the purpose of reviewing their progress if needed. Academic advisors will receive periodic reports on the status of their advisees' RosE-Portfolios. For example, if a student has not submitted any materials in his/her RosE-Portfolio for a period of time, the advisor will receive an electronic communication asking him or her to encourage the student to submit materials before registering for classes in the upcoming quarter.

In addition to the advantages for individual student advising, information can be made available to academic departments by sorting the results of the faculty ratings by department. This process will be automated and will provide quality assessment data that departments can use to validate their own assessment efforts.

Because the RosE-Portfolio can be accessed, viewed, and rated from anywhere in the world where one could access the Web, the potential of involving our National Board of Advisors, alumni, and other constituent groups in the rating of portfolios is very real. The value of the input of our external constituents on the quality of student work would provide a perspective that is not  currently available.

Are your computing facilities up to snuff?
Although Rose-Hulman's extensive computing support may be atypical, we believe that most engineering education programs provide adequate computing access to support an electronic portfolio system.

What is the primary purpose of the data collection?
Portfolios can be used to assess the growth of a student over time, or to take a "showcase" approach where students are told to submit material that represents their "best" work. Instead of assessing individual students, portfolios can be sampled to determine whether or not program or institutional student outcomes are being met. If a showcase approach is selected, snapshots of an electronic portfolio taken over time can document student progress.

How will you assess the material submitted in the portfolios?
Keeping in mind the admonition that portfolios are not an end in themselves, you should have a clear plan for categorizing and assessing the material that is relevant to your desired student outcomes. Because of the potential time commitment to evaluate portfolios for a number of outcomes, establish a clear plan on the rubrics to be used. Think ahead about what an assessment report might contain, including the results of the portfolio assessment.

If you are planning to use portfolios for program assessment, the answers to the following questions will guide you:

How much data can you handle?
If your program has a small number of students, it may be desirable to assess every portfolio. For a large program, however, this may be impractical or impossible, in which case you should incorporate a statistical sampling mechanism to ensure that part of every student's portfolio is evaluated. In this example the primary purpose of the portfolio is to assess programs and not individual students, but a well-constructed portfolio process would enable faculty members to assess individual student skills and knowledge as well.

Similarly, you must decide if you intend to assess every goal and performance criterion every year or semester. Evaluators often use the expression "E=MC2". This represents not Einstein's law of relativity but "Evaluation = Measurement X Common sense squared." If you have 11 student outcomes that you are going to measure (e.g., EC2000), each with an average of six performance criteria, and 100 students in your program, you have potentially 6,600 assessment data points. Common sense would dictate that you must develop an assessment plan based on sound sampling methodology. Sample your portfolios for potential problem areas and develop your assessment schedule to maximize your ability to identify areas for improvement early in the process.

When evidence indicates that students are having difficulty demonstrating a desired outcome at the appropriate level, you can improve the processes designed to promote that outcome and make new assessments. Outcomes targeted for improvement can be assessed more frequently than those that students consistently meet. An electronic portfolio system can automate the potentially tedious search and sampling process.

Gloria Martin Rogers is vice-president for institutional resources, and Julia Williams is assisstant professor of English and coordinator of technical communication at Rose-Hulman Institute of Technology.