Difference between revisions of "Benchmarking, performance testing, load testing, stress testing, etc."

From Helpful
Jump to: navigation, search
m (Timing inaccuracies)
m (Fuzz testing)
(5 intermediate revisions by the same user not shown)
Line 15: Line 15:
  
 
===Unit testing===
 
===Unit testing===
Often essentially "is this part behaving sanely on its own".  
+
Often essentially "is this part behaving sanely on its own, according to the tests I've thought up".  
  
Typically for small pieces of code - often usually functions, behaviour within classes, etc.
+
Typically for small pieces of code - often functions, behaviour within classes, etc.
  
  
Line 25: Line 25:
  
 
===Integration testing===
 
===Integration testing===
Often essentially "does this interact sanely with existing bits"
+
Often essentially "does this interact sanely with all the existing parts"
  
 
Tests the combination and interaction of units of code.
 
Tests the combination and interaction of units of code.
Line 36: Line 36:
 
but you suspect might be altered, e.g. because it is often-touched code.  
 
but you suspect might be altered, e.g. because it is often-touched code.  
  
Useful to avoid regressing to older bugs, as well as emergence ofsimilar bugs.
+
Useful to avoid regressing to older bugs, as well as emergence of similar bugs.
 +
 
 +
Makes sense for parts that may be rewritten, may be precarious in their interactions.
 +
Often introduced after a specific bug was found, to ensure it won't be introduced again later.
  
 
===Acceptance testing===
 
===Acceptance testing===
Line 43: Line 46:
 
which is often its functional design.
 
which is often its functional design.
  
Which can involve any type of test and scale of testing, though in many cases is relatively basic.
 
  
 +
Which can involve any type of test and scale of testing, though in many cases is relatively basic.
  
 
===Fuzz testing===
 
===Fuzz testing===
  
Fuzz testing, or fuzzing, feed in what is often largely random data, hence the name.
+
Fuzz testing, a.k.a. fuzzing, feed in what is often largely random data, hence the name.
  
 
If software does something other than complain about bad input,
 
If software does something other than complain about bad input,
Line 55: Line 58:
  
  
Used in security context, but also in some extended tests for robustness.
+
Used in security reviews, but also in some extended tests for robustness.
  
  

Revision as of 13:40, 15 August 2019

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)


The many more names for tests. More than most of us can remember or define on the spot.

And some are fuzzily defined, or treated as interchangeable in some contexts.


Code testing

Unit testing

Often essentially "is this part behaving sanely on its own, according to the tests I've thought up".

Typically for small pieces of code - often functions, behaviour within classes, etc.


They are also useful in regression testing

Most unit tests written don't really test possible interactions.

Integration testing

Often essentially "does this interact sanely with all the existing parts"

Tests the combination and interaction of units of code.

Regression testing

Often essentially "is this still working as it always was?"


Tests that should always be true (because things build on it), but you suspect might be altered, e.g. because it is often-touched code.

Useful to avoid regressing to older bugs, as well as emergence of similar bugs.

Makes sense for parts that may be rewritten, may be precarious in their interactions. Often introduced after a specific bug was found, to ensure it won't be introduced again later.

Acceptance testing

Often essentially "are we living up to the formal requirements in that document over there?", which is often its functional design.


Which can involve any type of test and scale of testing, though in many cases is relatively basic.

Fuzz testing

Fuzz testing, a.k.a. fuzzing, feed in what is often largely random data, hence the name.

If software does something other than complain about bad input, it may reveal cases it's not considering, and e.g. the presence of exploitable buffer overflows, injection vectors, ability to DoS, etc.


Used in security reviews, but also in some extended tests for robustness.


See also:

On code coverage

Load, performance, stress testing

Stress testing, recovery testing

Common pitfalls in benchmarking

Measuring the overhead

Measuring your cache instead

Timing inaccuracies

Microbenchmarking

Caring about users

Usability testing

Accessibility testing

See also

http://en.wikipedia.org/wiki/System_testing