|
|
|
|
|
|
Do not read the title again. I'll repeat it for you: "Software does not fail."
"Everything fails," people say. "Hey, especially software."
Whatever their intelligence, their knowledge, their experience, these people have one thing in common.
They are not sophisticated.{HyperNote 2}
Software either works or it does not work.
To be said to work, software must do what it is supposed to do. A specification determines what software is supposed to do -- and by implication what software is not supposed to do.{HyperNote 3}
Software that does not work has not failed. It did not work in the first place.
To be said to fail, software would have to work in the first place then not work in the second place. Software does not do that.{HypeNote 4}
Neither Moth nor Dust
If software ever worked, it will always work. Always.
People think that software worked in the first place and then stopped working.
They are not sophisticated.
Hardware Fails
To work, software must run, which means commanding hardware to do what hardware is supposed to do. Software, then, depends upon hardware. And hardware...
Hardware Always Fails
It is possible for hardware to work and then stop working.
It is more than possible; it is certain.
No question about it, when hardware works and then stops working, that's a failure.
Repair Applies to Failures
A hardware failure can be fixed by replacement of the hardware in whole or in part. Replacement by identical hardware, in fact. That's what 'repair' is.
Since software does not fail, 'repair' has no meaning.
If software does not comply with the specification, replacing it in whole or in part with identical software will not make it comply with the specification. Replacing software that does not work with redesigned software that does work is not 'repair.' {HyperNote 5}
Reliability Applies to Failures
Reliability is a term which has meaning only for hardware not software. Reliability is defined as the probability of not failing within a given period of time.
Software does not fail within any period of time. The statistical expression 'mean time between failures' (MTBF) has no meaning for software, inasmuch as the inverse, 'failure rate,' is zero for software.{HyperNote 6}
Concrete Versus Abstract
Hardware is 'hard,' which is to say concrete, not abstract. Hardware can be made, in some sense harder, more reliable. Hardware can be...
Harder the better, presumably. But never hard enough. Sooner or later, hardware fails.
By the way, humans are hardware.
Software, being abstract, is -- well, 'soft.' No reason to make software hard, though.
Truth be known: It is because of its softness that software does not fail.
Nothing Else Is Software
There is nothing softer than software. Other things may be as abstract as software but hardly softer.
A 'procedure,' for example, is abstract and therefore as soft as software.
Software is a procedure but not all procedures are software. In fact no other procedures are software.
Software runs on hardware; other procedures run on a special kind of hardware: humans.
People like to call books and films and videos "software."
They are not sophisticated.
The Specification
The specification for either hardware or software is abstract and therefore soft. The specification does not do what software does, however. So the specification cannot be said to work or not work.
The specification, which is derived from functional requirements, can be dichotomized many ways:
To go from one to another, the specification itself would need to be 'revised.' Or the functional requirements would need to be 'amended.'
One Dichotomy
Only one dichotomy applies to software: either it works or it does not work, which is determined by whether the software complies with the specification. Same for hardware.
A failure, however, takes hardware from working to not working; conversely, repair takes hardware from not working to working.
For the record, then, complying with a specification that is...
...does not mean failure of either hardware or software.{HyperNote 8}
Objects
Accessing an object in memory is called 'reading.' Assigning a value to an object in memory is called 'writing.' Software does something else, too.
Internal States
Software references its 'internal state' and changes its 'internal state.'
Hardware does what software does, but hardware does something else, too. Hardware fails.
Cases
To be said to work, software must comply with the specification in all 'cases.' Not some, all.
A 'case' is merely a combination of input values.
Now, depending on its 'internal state,' software generally does something different with a given combination of input values. A given case, then, is not sufficient to determine what software does.
Sequences
Each internal state is reached as the result of software being acted upon by a sequence of input values -- cases. Software changes its internal state whenever necessary to comply with the specification. Which means that a sequence of cases is necessary to determine what software does.
Necessary but not sufficient -- unless the sequence begins with the software in a known state such as 'reset' and the sequence is long enough.{HyperNote 10}
The statement that software must be designed to work in all sequences of cases is general enough to accommodate software's ability to reference and change its internal state.
Doing the Right Thing
Software necessarily does the same thing in each sequence of cases -- the right thing. If what the software does in a given sequence of cases complies with the specification, the software is said to work -- for that sequence of cases.
Not doing what it is supposed to do in a given sequence of cases does not mean the software has failed. Instead, that software did not work in the first place.
Not only that but the specification must determine software's response to a class of inputs called 'interrupts,' which are distinguished by the fact that an 'interrupt' can directly change software's internal state -- but not without the authorization by software. The specification must also determine software's treatment of the 'non-maskable interrupt' and 'reset.'
Nothing can be left to chance, here.
Specification in Code
To do what it does, software must run on hardware, which means commanding the hardware on which it runs.
Commands are codes, and 'code' results from a procedure called design, which is what 'software engineers' do.
The specification is not code, and more than one code may comply with the specification. Each code can be designed differently and produce a different sequence of internal states.
A given code can be different from another code that works and still work.
In other words, a specification determines what software is supposed to do but not how the software is supposed to do it. A specification must only be explicit enough to determine what values software will assign to output objects in each sequence of cases -- the external consequences of each sequence.
But a specification does not determine the sequence of internal states. If a specification did that, the specification would be more than a specification. It would be code.{HyperNote 12}
A sophisticated person would describe 'design' as a process not a procedure, the latter implying the power to determine. Code is not determined by a procedure.
Design is a rigorous process guided by knowledge.{HyperNote 11}
|
|
|
|
The expression 'quality assurance' was initially coined for software. Today the term also applies to hardware.
Quality assurance differs from 'quality control,' which applies only to hardware -- specifically to hardware manufacturing. Being abstract, software is not, in the conventional sense, manufactured.
Quality assurance means proving that a given code works. Which may not be easy. But it is worth the effort, since, once proven to work in all sequences of cases, software does not fail.
To be said to work, quality assurance must prove that software works. More to the point, quality assurance must not allow software that does not work to be said to work.
The previous sentence deserves to be read again.
Quality assurance is a procedure and therefore soft -- as soft as software. It either works or it does not work.
Quality assurance is not a matter of degree. It is all or none. And 'all' can be plenty.
That depends on the complexity of software. Complexity is indeed a matter of degree. Degrees, plural.
Each line of code applies to some number of cases and sequences of cases for which the software must be proven to work.
The quantity of cases and sequences of cases can be exceedingly large. But not infinite. Quality can in principle always be assured.
Testing: Myth and Reality
Testing is the best way to assure quality, some people say. They are not sophisticated.
Whereas proving that software works in a given sequence of cases can indeed be accomplished by testing, testing one sequence may or may not prove that the software works in another sequence.
If the software does not do what it is supposed to do in a given test, the software does not work. On the other hand, if the software does do what it is supposed to do in a given test, the software may or may not work.
Quality assurance requires all possible tests. That may not be practical.
A typical 'test' would be prepared as a sequence of input objects that are systematically caused to act upon the software. The sequence of values assigned by the software to the output objects would then be observed. The output sequence, having been independently predicted, would need to be compared with what the software actually outputs.
Tests often do relate to each other, though.
Cleverness: The Myth
A clever selection of tests might make quality assurance by testing practical.
In the previous sentence, 'might' appears in the predicate. That's because 'clever' appears in the nominative.
The clever selection of a test from all possible tests can be exceedingly difficult. More difficult, in fact, than designing the code. That testing is not the best way to assure quality can be demonstrated for software of any complexity.
Even simple software.
Simple Example
Consider the following simple specification:
Many practical applications rely on such a specification. The input object may represent...
...the output object in each case being used to allow or to limit some action.
Software does not get much simpler, and testing that software would not be exceedingly difficult.
Trivial Sequence
There are merely two 'internal states' required to fulfill the specification in the Simple Example: Call them 'on' and 'off,' which in conjunction with the input object determine the value of the output object to be one and zero.
After 101 tests, then, the software would not have been proven to work. The software may, in fact, be utterly ignoring the input values.
After 102 tests, the software has not been proven to work. There are still 154 more tests to go. {HyperNote 14}
Cleverness: The Reality
For the Simple Example, tests do indeed relate to each other. There are four sets of cases:
For quality-assurance-by-testing, sets numbers 1 and 2 are the most interesting, since 256 cases must be tested. Still, it is tempting to suppose that a 'clever' selection of cases will make quality-assurance-by-testing practical. There seems to be no need to bother with input values much above or below the dividing line, 100. The idea would be to concentrate testing in the range of, say, 90 to 110.
That will not work. Here's why.
The Rational Software Engineer
Assume there to be a rational software engineer who, given the specification for the Simple Example, would yawn and then design his or her software to perform a simple arithmetic comparison between the binary value of the input byte and a constant value of 100.
A rational software engineer can make a mistake in the coding, though.
For example, he or she might have neglected to code the software to treat the input object as an unsigned integer. If so, then for all tests of input values higher than 127, the software would interpret the binary number as negative.{HyperNote 15}
For simplicity, let us set aside the need for a separate detection of a zero value on the input in order to switch the internal state from 'on' to 'off.'
Accordingly, the software does not work.
Finding Out the Hard Way
For that particular mistake in code, then, a test having an input value larger than 127 would prove that the software does not work. On the other hand, quality assurance based only on a 'clever' selection of tests near the value 100 would allow software with a design mistake to be said to work.
Then one of the untested input values comes along and poof:
Hardly the preferred way to find out that the software -- what? -- did not work.
Quality Assurance Did Not Fail
Some people will say it was the quality assurance that failed.
They are not sophisticated.
Quality assurance does not fail. For the Simple Example, what was demonstrated is...
...by proving that the software did not work.{HyperNote 17}
Quality assurance did not work for a few tests -- and then stop working. That would be the evidence of failure. Instead, quality assurance did not work, period.
Cleverness: The Impossibility
If performing all possible tests is not practical, then quality-assurance-by-testing will work if and only if the selected tests are the ones that do not allow software that does not work to be said to work.
The 'clever' selection of tests to assure quality can be exceedingly difficult.
A sophisticated person will say that it would be impossible to select tests that assure quality without knowledge of the code. Some people will say that 'improbable' is a better word, but software deserves more stringent treatment:
Quality assurance is not a matter of probability.
Knowing the specification is not enough, as we shall see.
The Reasonable Software Engineer
Consider the same Simple Example that was coded by the rational software engineer in the hands of a different person -- a reasonable software engineer. After yawning, he or she designs the software to use a table look-up procedure.
Rather than arithmetically comparing the binary value of each input byte to a constant 100, the software is coded to treat the byte as a 'pointer' which references an object in memory called a 'bit-map.'
As above, a separate detection of a zero value on the input would be needed to switch the internal state from 'on' to 'off,' but let us for simplicity set that requirement aside.
There are reasons for the reasonable software engineer to adopt this more general design approach. A 'translator,' for example, would have called for a byte as the output, which is best coded as a table-lookup.
Note that the code is different, but the specification is the same.
Let us again postulate that quality assurance is to be achieved by testing -- without knowledge of the code, only the specification. As before, there are 256 possible tests. The problem would be to select fewer than all possible tests and still assure quality.
If so, for tests of input values greater than 127, the pointer would point outside the table and there's no telling what the value of the output bit might be --including the correct value, of course. During testing. Later, after a revision to the software, tah-dum, those out-of-bounds references -- to objects having nothing whatever to do with the software segment at hand -- will contain different and therefore incorrect values. It happens.
There are 256 ways to make one such mistake. Any of them will make the software not work.
Quality-assurance-by-testing would not work unless the selected tests happen -- by pure chance -- to include the one or more in which the software does not work. After the software is put into use and the untested input value arises, the consequences would be attributed to a software failure by some people -- and to quality assurance failure by other people.
A sophisticated person would say...
The tests that might have proven that the software does not work in the first place were not performed, so the software was said to work.{HyperNote 18}
The Simple Example is not uncommon and the coding errors postulated above are not far-fetched. The 'software age' is with us. What ever shall we do? {HyperNote 19}
|
|
|
|
A sophisticated persons will say that testing does not assure quality unless all possible tests are performed.
That's what is meant by the aphorism, "You cannot test quality into something."
In software, complexity makes testing all possible cases not practical, and making a 'clever' selection of tests that will assure quality without knowledge of the code is not possible. The next sentence may need to be read more than once.
With enough knowledge of the code, testing is not necessary.{HyperNote 20}
First Principles Apply
Setting aside quality-assurance-by-testing, a sophisticated review of First Principles would include...
The human conducting the code walk-through can make an 'undersight,' of course, much as a software engineer designing software can make a mistake. {HyperNote 21}
If there is no undersight but not all the mistakes are found, the Code Walk-Through did not work.
Unlike quality-assurance-by-testing, the Code Walk-Through cannot be automated much.
Unlike quality-assurance-by-testing, the Code Walk-Through is mind labor. Like designing software.
Soft Things Made Soft
Quality assurance by the Code Walk-Through can be exceedingly difficult. But no more difficult than designing the software itself. And far less difficult than testing all possible sequences of cases.
Consider the Simple Example again. Only a few lines of code were needed to meet the specification.
Performing the Code Walk-Through of a few lines would surely be less difficult than quality-assurance-by testing, which means...
Then too, many of these steps are vulnerable to human error -- undersights.
Managing Complexity
Some people will say that complexity makes the Code Walk-Through not practical.
They are not sophisticated.
If the code is too complex to walk-through, it is too complex to design -- and too complex to test, since without knowledge of the code, quality assurance requires testing all sequences of cases, which is not practical.
The sophisticated person knows that complexity must be managed.
Managing Practicality
One way to do that is to partition the code into 'modules.' A specification must then be prepared for what each module is supposed to do and for what all the modules together are supposed to do.
Unfortunately, quality-assurance-by-testing requires preparation of sequences of cases for each module and some method to intercept the consequences -- 'stubs.' All software modules need to be tested together, which means that testing all possible sequences of cases will not be practical, even with modular software.
On the other hand, if the code is modular, the Code Walk-Through can indeed more readily assure quality of software.
The practical becomes more practical.
The Sophisticated Code Walk-Through
The sophisticated Code Walk-Through must include procedures...
A human must perform all of those procedures -- and more -- with no undersights.The consequent software may still not work.
Whatever the ultimate result, though, software does not fail.
HyperNotes
See for example Pressman, Roger S., Software Engineering (1987, McGraw-Hill) or von Mayrhauser, Anneliese, Software Engineering (1990, Harcourt Brace Jovanovich). As for hearsay references, suffice it to say that forcible expressions of these sentiments have reached my ears from a hundred sources since the mid-fifties. {Return}
My license for bluntness here is appropriated from Ralph Waldo Emerson (1803-1882).
"Speak what you think in words as hard as cannon-balls..." {Return}
Permit me to quote William Makepeace Thackeray (1811-1863):
"For the wicked are wicked indeed and they go astray and they fall and they come to their just deserts, but who can tell the mischief that the very virtuous do!"
Words well suited to lamenting the unintended consequences of features gratuitously designed into software. {Return}
Some people will say that not working in the first place is just another expression for 'failure.'
They are not sophisticated.
An immense distinction is being presented here. We need to restore the verb 'fail' to its full meaning. One might substitute 'break'-- but only for hardware. Nobody says software breaks. {Return}
Terms selected here are somewhat arbitrary but intended to serve clarity by virtue of consistency, thus one might...
For the last one, somebody who was obviously not sophisticated long ago appropriated the term 'maintain' for software.
Software 'maintenance' in every meaningful respect...
...does not even faintly resemble hardware maintenance. {Return}
Unhelpful indeed are statements such as are found in Siewiorek, Daniel P. and Swarz, Robert S., The Theory and Practice of Reliable System Design (1982, Digital Press, Bedford, MA):
"A failure causing a [computer] crash may be the result of either hardware or software failure."
The passage goes on to attribute software 'failures' to "the introduction of new features" or "previously undetected faults." {Return}
Ironically, hardware gets effectively harder when more of its functions are given over to software, as in embedded controllers or, in the extreme, the 'reduced instruction set computer' (RISC).
The term 'firmware' means software that resides in read-only- memory. Nota bene, firmware is as 'soft' as software. {Return}
The distinction being drawn is between 'software' and 'hardware,' the latter can be taken to mean a 'computer,' 'processor,' 'controller,' whatever.
Other words have been eliminated altogether as not relevant to the present scope 'program' and 'subprogram,' 'routine' and 'subroutine.' The word 'system' long ago got stretched beyond its elastic limits and lost its strength.
Professional 'programmers' were replaced by 'software engineers' in the mid-seventies; the other kind became 'hackers.' As a term of disparagement, 'hacker' has two meanings: 'cybumblers' and 'cyburglers.' {Return}
The sophisticated term 'object,' which is general in the extreme, gives renewed evidence of a linguistic struggle dating back to the dawn of the 'software age' and the realization that calling 'software engineering' 'computer science' is tantamount to calling 'mechanical engineering' 'automobile science.' {Return}
Not all cases can arise in real life, thus we have the term 'don't-care' for those cases.
The sophisticated person cares about those cases: first making sure that they cannot indeed arise and second making sure that if they do arise, the software works anyway. {Return}
In Cultural Literacy (Random House, 1988), E. D. Hirsch frames a compelling picture of the knowledge-bound character of all cognitive skills. {Return}
The term 'code' is taken here to mean 'source code,' which is what software engineers produce.
Development software -- compiler or assembler -- produces 'object code,' which is what actually commands hardware. The distinction is not germane to the present subject.
Unless the development software does not work. {Return}
The word 'byte' may be the purest neologism of the 'software age,' giving evidence of the forlorn and tardy impact of the 'computer science' on the language we speak. I dare to use the term here without explanation. {Return}
The 256 test cases in the Simple Example are not too many to do, even by hand. A minor tweak of the example, though, to 16 bits on the input will change that.
Automating the testing with a 'driver' means, for the Simple Example, writing more code to do the testing than the code in the first place. Another tweak or two, and the number of test cases -- even for the Simple Example -- will exceed the number of subatomic particles in the Milky Way Galaxy.
The issue is, of course, Sir Karl Popper's Principle of Non-Falsifiability. {Return}
The leftmost digit of a binary number of a given length can be treated either as a binary bit or as the sign of a binary number one bit shorter in length.
In byte-land, the former interpretation will support a range of values from 0 through +255, the latter, values from -128 through +127. {Return}
A compendium of the most sensational consequences of -- and misguided attributions to -- software failures appears in Leonard Lee's The Day the Phones Stopped (1991 Donald T. Fine, Inc), which inspired the present polemic. The book is listed by the Library of Congress under the category 'Computer software -- Reliability.' {Return}
Most people like to say that if software works in some cases but does not work in other cases, the software has a 'bug' in it. Which is more meaningful than saying that the software failed. But not by a lot. The software did not get a bug in it. The software engineer made a mistake.
A software engineer bemoaning a bug in his or her software is tantamount to a tennis player lamenting flaws in his or her own backhand -- hey, or concentration.
A few people like to say that if software works in some cases but does not work in other cases, the software has a 'glitch' in it. Which is less meaningful than saying that the software failed. By a lot.
When 'glitch' gets into a sophisticated dictionary, it will be given a synonym, 'intermittent.' {Return}
One handy software engineering tool is the 'debugger,' which is a misnomer tantamount to calling a 'magnifying glass' an 'insecticide.' {Return}
The Simple Example appeared as a portion of a real-life specification, which called for the software to use the binary value of the input byte to select an object from memory for reading.
However inconclusive, testing of software will continue to be practiced until the sun flickers from the sky.
Software 'bureaucracies' brim with testing. And what is known as SVVP (Software Validation and Verification Plan, an orthodoxy wherein both Vs are mandated: The first testing that the software is correct according to the specification and the second testing that the specification is correct. {Return}
The word 'oversight' like 'cleave' can cut both ways ("split apart" or "cling together").
A sophisticated writer might use 'undersight' to denote the latter. {Return}
|
|
|
|
|