Integrating Computer Ethics into the Computer Science Curriculum**
Keith Miller
CS 8: Organization of Programming Languages
Formal specifications and proofs of correctness have influenced programming language development dramatically. Correctness proofs examine agreement between a formal specification and its implementation.
This fictional case study is based on a scenario presented by Nancy Leveson (Leveson, 1986).
Following a detailed English specification from a systems analyst, a programmer produces code that controls safety features in a nuclear power plant. One part of the specification states:
Whenever one of the plant sensors discovers a potentially dangerous situation, the task monitoring these sensors should shut down all plant systems. When plant personnel have rectified the situation that caused the exceptional sensor condition, the program will allow a manual override that will restart the plant systems.
The programmer tests the code, and installs it at the plant. The systems analyst views the programmer’s test results, and attests to the correctness of the program. The program is installed and runs for six months without incident.
One component of the nuclear power plant controls fuel rods and flow of water into the reactor in order to regulate the temperature of the reaction. One of the sensors in the reactor has a hardware failure, and gives a false, abnormally high reading. The program controlling safety immediately shuts down all reactor systems. Unfortunately, at the moment the sensor hits, the valve for the cooling water had just begun to open because the temperature was starting to rise in the reactor; insufficient water gets to the reactor because of the shut down, and the temperature continues to rise. While the sensor in the reactor is being replaced, the reactor overheats, and some radioactive steam is emitted into the atmosphere as the pressure builds up. A technician notices the problem and, even though it is against safety procedures, manually overrides the safety system so that the cooling water valve opens. The communities surrounding the plant are aroused both by the radioactivity that was released and the potential disaster that was narrowly avoided.
The systems analyst in this case blames the nuclear physicist who signed off on his specifications. The physicist blames the systems analyst for overlooking the obvious. Both the analyst and the physicist claim that the programmer should have done more testing, The communities around the plant want the plant shut down.
Who is responsible for correctness? In this case, discuss the responsibilities of the physicist, the systems analyst, and the programmer. Is it possible that everyone can do a good, thorough job and still allow a dangerous situation to occur? Discuss any weaknesses of formal proofs of correctness; what ethical implications are there when correctness proofs are impractical or suspect? Discuss methods by which errors like the one described above can be caught before they become dangerous. If these methods require additional investments of time and money, who should pay the additional costs? Do you think that the communities around the plant would be willing to pay extra for power to ensure safer power plants? Do you think that communities far away from any plant should pay the same additional costs?
Home > Teaching
Resources > Teaching Computer
Ethics > Integrating Computer Ethics into the Computer Science Curriculum
HOME | IN
THE NEWS | RESEARCH
RESOURCES
TEACHING RESOURCES | STUDENT
RESOURCES | LINKS
The Research Center on Computing & Society
at Southern Connecticut State University
501 Crescent Street | New Haven, CT 06515
Director: (203) 392-6790 | e-mail: webmaster@computerethics.org
© 2000 – 2004 – Research Center on Computing & Society