Integrating Computer Ethics into the Computer Science Curriculum**
Keith Miller
CS 3: Introduction to Computer Systems
An important objective in CS 3 is to familiarize students with the finely detailed low-level control available with assembler programming. Another important point is the difficulty of controlling the complexity of a large assembler program, even with the addition of a macro facility.
In 1986, at least two people died and at least one other was maimed after receiving excessive radiation from a linear accelerator radiation machine called the Therac 25 (Joyce, 1986). The first death occurred after a treatment on March 21. Technicians and doctors carefully examined the equipment and the incident, and concluded the machine was safe. On April 11, however, a second patient was given what proved to be a fatal overdose from the machine. After this incident, the problem with the machine was finally discovered: a bug in the assembler language program which controlled the machine.
The Therac 25 has two modes: x-ray mode and electron beam mode. In x-ray mode, a very high powered beam strikes a heavy metal plate; that plate gives off x-rays which are focused on the patient. In electron beam mode, the plate is retracted and a much lower powered beam of electrons is focused directly on the patient. A technician controls the Therac 25 using a PDP-ll minicomputer. An assembler program on the PDP interprets the technician’s commands, and controls the radiation machine as a peripheral. The software includes two different methods for correcting a mistake when entering commands: retyping the command completely, or using the up-arrow to edit the mistaken command. Under one particular set of circumstances, when the technician used the up-arrow edit to change from the x-ray mode to the electron mode, the assembler program retracted the heavy metal plate (a correct action) but did not lower the power of the beam (an incorrect omission). Thus, the high power beam was focused directly on the patient, delivering a lethal dose of radiation. Whenever this situation occurred, a sensor detected the large amount of radiation and flashed a warning on the monitor, “MALFUNCTION 54.” The significance of this warning was not understood until after April 11.
The particular circumstances which led to MALFUNCTION 54 were rare. In tens of thousands of treatments over a period of years, the manufacturer claims that no such incidents had occurred prior to March, 1986. Perhaps, because of this spotless safety record, both the owners of the machines and the manufacturer assumed that some extraordinary event had caused the first injury, and therefore the 11 installations that owned Therac 25 units continued using them until after the second deadly accident.
What aspects of assembler programming make testing particularly difficult? What are the advantages of assembler programming? What kinds of tasks do you think are appropriate for assembler language programming, and which tasks are not? Testing is always incomplete; what are the ethical implications of this when programming, especially when using assembler? Malfunction 54 killed and maimed; what responsibility do the following people have for the accidents: the assembler programmer; the manager in charge of the programming; the manufacturing company that produced the entire machine; the hospitals that purchased the machine; the technicians who ran the machine; the engineers who tested the machine after the first accident but before the last.
Go to: CS 4: Introduction to Computer Organizations
Home > Teaching
Resources > Teaching Computer
Ethics > Integrating Computer Ethics into the Computer Science Curriculum
HOME | IN
THE NEWS | RESEARCH
RESOURCES
TEACHING RESOURCES | STUDENT
RESOURCES | LINKS
The Research Center on Computing & Society
at Southern Connecticut State University
501 Crescent Street | New Haven, CT 06515
Director: (203) 392-6790 | e-mail: webmaster@computerethics.org
© 2000 – 2004 – Research Center on Computing & Society