n South African Computer Journal - Truth in advertising : reporting performance of computer programs, algorithms and the impact of architecture and systems environment

Volume 2010, Issue 46
  • ISSN : 1015-7999
  • E-ISSN: 2313-7835



The level of detail and precision that appears in the experimental methodology section computer science papers is usually much less than in natural science disciplines. This is partially justified by different nature of experiments. The experimental evidence presented here shows that the time taken by the same program varies so significantly on different CPUs that without knowing the exact model of CPU, it is difficult to compare the results. This is placed in context by analysing a cross-section of experimental results reported in the literature. The reporting of experimental results is sometimes insufficient to allow experiments to be replicated, and in some case is insufficient to support the claims made for the algorithms. Comparing the computational costs of two programs by running them on different computers - even with similar labels and nominal ratings − can be very misleading. New standards for reporting on algorithms results are suggested.

Loading full text...

Full text loading...


Article metrics loading...


This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error