Why Cryptosystems Fail
--- Anderson, 1994

Summary by Emre Kiciman et al.

 Problem:

   Crypto experts at a research lab or security company design a crypto product
   that assumes a certain attack model (cryptanalysis by a technically savvy
   bad guy). This product adheres to strict standards set by the crypto
   community.  However, when deployed in the real world, it fails, repeatedly.
   Why? 

 Analysis:

   Most security failures are not due to crypto product failure but due
   to implementation and management errors:

   (a) It should be noted that a crypto "product" is not a "solution"
   for the industry/company deploying it; only a part of it.
   Designing, implementing and maintaining a solution is a complex process
   in which several people (managers, consultants, programmers, maintenance
   staff, etc.) get to interact with the system 'from the inside'. Any of them
   can insert/detect weaknesses in the system to spoof/break it.  The paper
   provides numerous examples to support this point.

   (b) There are no accepted guidelines for designing and implementing
   security solutions. They are sometimes designed and implemented
   by people who are incompetent. The design+implementation is sometimes
   not scrutinized by a third-party. Once the solution is deployed, it
   is managed by incompetent managers who do not understand security.
   Some companies have no 'security team' at all.  Finally, some crypto
   products are tricky and complex to use. In short, there is
   no "quality management" in the design/implementation/maintenance process.

   (c) Once a system is broken, the owners of the system try to hide it.
   Thus, crypto system designers get virtually no feedback. As a consequence,
   they continue to design their system assuming the same old threat model.
   There is no "learning"/"feedback" mechanism, as is found, for example,
   in the airlines industry.

 Solution:

   Ross Anderson claims that as more people become aware of the problems
   that cryptosystems face, the security business will experience a paradigm
   shift: the focus will change from building and selling 'certified products'
   to an engineering discipline concerned with "quality control" processes
   within the organization deploying a security solution. The paper presents
   some guidelines for designing such quality control processes.

 -------------------------------------------------------------------------

 Interesting:

   Ross defines a spectrum ranging from "reductionist" solutions
   as exemplified by railways, to "holistic" solutions as employed by airlines.
   The railway system puts all intelligence in the system, thereby deskilling
   human operators. The system has control. The airlines puts all intelligence
   in the human operators. The system is there to aid the operators, who are
   always in control. Ross says that current view of compuer systems security 
   is reductionist. This ought to change: we ought to learn from the aviation
   industry.