Computing Security and Human Values – User-View Systems Requirements – The Research Center on Computing & Society

Text-only Version

Peter G. Neumann

3. User-View Systems Requirements

There are numerous security-relevant expectations that people may have of a particular computer system, such as the following:

Preservation of human safety and general personal well-being in the context of computer-related activities. Computer systems in numerous disciplines (transportation, medical, utilities, process control, etc.) are increasingly being called upon to play a key role in life-critical operations.
Observance of privacy rights, proprietary interests, and other expected attributes. People should be notified when they are being subjected to unusual monitoring activities, and should be given the opportunity to observe and correct erroneous personal data.
Prevention against undesired human behavior. This includes malicious acts such as sabotage, misuse, fraud, compromise, piracy, and similar antisocial acts. It also includes accidental acts that could have been prevented.
Prevention against undesired system behavior, such as hardware or software induced crashes, wrong results, untolerated fault modes, excessive delays, etc.
Balancing the rights of system users against the rights of system administration, particularly with respect to resource usage and monitoring.

These requirements are intertwined with value-related issues in a variety of ways, including some related to human foibles in system design, development, operation, and use, and some related to misplaced trust in systems – e.g., excessive or inadequate.

4. System Security Requirements

The above human-motivated requirements are typically related to computer system requirements, such as the following:

System security requirements, both functional and behavioral. Computer systems should dependably enforce certain agreed-upon system and application security policies such as system integrity, data confidentiality, data integrity, system and application availability, reliability, timeliness, human safety with respect to the system, etc., as needed to enforce or enhance the socially relevant requirements listed in the previous section.

5. Expectations on Human Behavior

There are also numerous security-relevant expectations that system designers and administrators may wish to make of people involved in particular computer systems and applications. At one extreme are reasonable expectations on supposedly cooperative and benign users, all of whom are trusted within some particular limits; at the other extreme is the general absence of assumptions on human behavior-admitting the possibility of “Byzantine” human behavior such as arbitrarily malicious or deviant behavior by unknown and potentially hostile users. A few of the most important expectations are the following. It is convenient to consider both forms of human behavior within a common set of assumptions, with benign behavior treated as a special case of Byzantine behavior.

Nonspecific expectations relevant across the spectrum of users, e.g., cooperative and uncooperative, remote and local, authorized and unauthorized. Sensible security policies must be established and enforced, with default access attributes that support the user’s needs and the administrators’ demands for controllable system use.
User security requirements on generally cooperative users. Even in the presence of friendly users, benignness assumptions are risky, particularly in light of masqueraders and accidents. In relatively constrained or non-hostile environments, it may be reasonable to make some simplifying assumptions, e.g., that there are no external penetrators (as in a classified system that has no external access and only trusted users), and that the likelihood of malicious misuse by authorized users is relatively small, and then to make appropriate checks for deviations.
User security assumptions on potentially uncooperative users. Designing for Byzantine human behavior is an extremely difficult task, just as it is for Byzantine fault modes. In a totally hostile environment, it may be necessary to assume the worst, including arbitrary malice by individuals and possible collusion among collaborating hostile authorized users, as well as unreliability of hardware.

Back to the top

Go to: 6. Design/Implementation Concerns

Home > Research Resources > Computer Security > Computer Security and Human Values

The Research Center on Computing & Society
at Southern Connecticut State University
501 Crescent Street • New Haven, CT 06515
Director: (203) 392-6790 • e-mail: webmaster@computerethics.org

© 2000 – 2007 – Research Center on Computing & Society