What is Computer Ethics?*
James H. Moor
There is an important fact about computers. Most of the time and under most conditions computer operations are invisible. One may be quite knowledgeable about the inputs and outputs of a computer and only dimly aware of the internal processing. This invisibility factor often generates policy vacuums about how to use computer technology. Here I will mention three kinds of invisibility which can have ethical significance.
The most obvious kind of invisibility which has ethical significance is invisible abuse. Invisible abuse is the intentional use of the invisible operations of a computer to engage in unethical conduct. A classic example of this is the case of a programmer who realized he could steal excess interest from a bank. When interest on a bank account is calculated, there is often a fraction of a cent left over after rounding off. This programmer instructed a computer to deposit these fractions of a cent to his own account. Although this is an ordinary case of stealing, it is relevant to computer ethics in that computer technology is essentially involved and there is a question about what policy to institute in order to best detect and prevent such abuse. Without access to the program used for stealing the interest or to a sophisticated accounting program such an activity may easily go unnoticed.
Another possibility for invisible abuse is the invasion of the property and privacy of others. A computer can be programmed to contact another computer over phone lines and surreptitiously remove or alter confidential information. Sometimes an inexpensive computer and a telephone hookup is all it takes. A group of teenagers, who named themselves “the 414s” after the Milwaukee telephone exchange, used their home computers to invade a New York hospital, a California bank, and a government nuclear weapons laboratory. These break-ins were done as pranks, but obviously such invasions can be done with malice and be difficult or impossible to detect.
A particularly insidious example of invisible abuse is the use of computers for surveillance. For instance, a company’s central computer can monitor the work done on computer terminals far better and more discreetly than the most dedicated sweatshop manager. Also, computers can be programmed to monitor phone calls and electronic mail without giving any evidence of tampering. A Texas oil company, for example, was baffled why it was always outbid on leasing rights for Alaskan territory until it discovered another bidder was tapping its data transmission lines near its Alaskan computer terminal.
A second variety of the invisibility factor, which is more subtle and conceptually interesting than the first, is the presence of invisible programming values. Invisible programming values are those values which are embedded in a computer program.
Writing a computer program is like building a house. No matter how detailed the specifications may be, a builder must make numerous decisions about matters not specified in order to construct the house. Different houses are compatible with a given set of specifications. Similarly, a request for a computer program is made at a level of abstraction usually far removed from the details of the actual programming language. In order to implement a program which satisfies the specifications a programmer makes some value judgments about what is important and what is not. These values become embedded in the final product and may be invisible to someone who runs the program.
Consider, for example, computerized airline reservations. Many different programs could be written to produce a reservation service. American Airlines once promoted such a service called “SABRE.” This program had a bias for American Airline flights built in so that sometimes an American Airline flight was suggested by the computer even if it was not the best flight available. Indeed, Braniff Airlines, which went into bankruptcy for awhile, sued American Airlines on the grounds that this kind of bias in the reservation service contributed to its financial difficulties.
Although the general use of a biased reservation service is ethically suspicious, a programmer of such a service may or may not be engaged in invisible abuse. There may be a difference between how a programmer intends a program to be used and how it is actually used. Moreover, even if one sets out to create a program for a completely unbiased reservation service, some value judgments are latent in the program because some choices have to be made about how the program operates. Are airlines listed in alphabetical order? Is more than one listed at a time? Are flights just before the time requested listed? For what period after the time requested are flights listed? Some answers, at least implicitly, have to be given to these questions when the program is written. Whatever answers are chosen will build certain values into the program.
Sometimes invisible programming values are so invisible that even the programmers are unaware of them. Programs may have bugs or may be based on implicit assumptions which don’t become obvious until there is a crisis. For example, the operators of the ill-fated Three Mile Island nuclear power plant were trained on a computer which was programmed to simulate possible malfunctions including malfunctions which were dependent on other malfunctions. But, as the Kemeny Commission which investigated the disaster discovered, the simulator was not programmed to generate simultaneous, independent malfunctions. In the actual failure at Three Mile Island the operators were faced with exactly this situation – simultaneous, independent malfunctions. The inadequacy of the computer simulation was the result of a programming decision, as unconscious or implicit as that decision may have been. Shortly after the disaster the computer was reprogrammed to simulate situations like the one that did occur at Three Mile Island.
A third variety of the invisibility factor, which is perhaps the most disturbing, is invisible complex calculation. Computers today are capable of enormous calculations beyond human comprehension. Even if a program is understood, it does not follow that the calculations based on that program are understood. Computers today perform, and certainly supercomputers in the future will perform, calculations which are too complex for human inspection and understanding.
An interesting example of such complex calculation occurred in 1976 when a computer worked on the four color conjecture. The four color problem, a puzzle mathematicians have worked on for over a century is to show that a map can be colored with at most four colors so that no adjacent areas have the same color. Mathematicians at the University of Illinois broke the problem down into thousands of cases and programmed computers to consider them. After more than a thousand hours of computer time on various computers, the four color conjecture was proved correct. What is interesting about this mathematical proof, compared to traditional proofs, is that it is largely invisible. The general structure of the proof is known and found in the program and any particular part of the computer’s activity can be examined, but practically speaking the calculations are too enormous for humans to examine them all.
The issue is how much we should trust a computer’s invisible calculations. This becomes a significant ethical issue as the consequences grow in importance. For instance, computers are used by the military in making decisions about launching nuclear weapons. On the one hand, computers are fallible and there may not be time to confirm their assessment of the situation. On the other hand, making decisions about launching nuclear weapons without using computers may be even more fallible and more dangerous. What should be our policy about trusting invisible calculations?
A partial solution to the invisibility problem may lie with computers themselves. One of the strengths of computers is the ability to locate hidden information and display it. Computers can make the invisible visible. Information which is lost in a sea of data can be clearly revealed with the proper computer analysis. But, that’s the catch. We don’t always know when, where, and how to direct the computer’s attention.
The invisibility factor presents us with a dilemma. We are happy in one sense that the operations of a computer are invisible. We don’t want to inspect every computerized transaction or program every step for ourselves or watch every computer calculation. In terms of efficiency the invisibility factor is a blessing. But it is just this invisibility that makes us vulnerable. We are open to invisible abuse or invisible programming of inappropriate values or invisible miscalculation. The challenge for computer ethics is to formulate policies which will help us deal with this dilemma. We must decide when to trust computers and when not to trust them. This is another reason why computer ethics is so important.
Dartmouth College
Go to: Computer Ethics in the Computer Science Curriculum
Home > Teaching Resources > Teaching Computer Ethics > What is Computer Ethics?
HOME | IN
THE NEWS | RESEARCH
RESOURCES
TEACHING RESOURCES | STUDENT
RESOURCES | LINKS
The Research Center on Computing & Society
at Southern Connecticut State University
501 Crescent Street | New Haven, CT 06515
Director: (203) 392-6790 | e-mail: webmaster@computerethics.org
© 2000 – 2004 – Research Center on Computing & Society