Testing Strategies for Heuristics: Part 1 - An Overview of Software Testing

Chris Sharpe

## Keywords

[Software Testing, Matlab]

Unreviewed

# 1. Tutorial Introduction

In this set of Tutorials the aim is to introduce and demonstrate how one may go about testing code implementations of heuristics optimisation techniques. No matter skilled a programmer one is some form of bug will get introduced at some point during the creation of a program. There is no question about this. Once this fact is established the next thing to do is think of how to develop a program with testing very much at the forefront of the effort, so bugs, errors, and faults are spotted and eliminated as soon as possible.

We first discuss which general test strategies are appropriate for heuristics and show through an example from a proper experiment how they are applied.

Matlab is the specific langauge used for the code example, but ideas presented on testing strategies are universal in computer programming.

The Tutorial is divided thus,

1. Test Strategies for Heuristics Part 1: An Overview of Software Testing
2. Test Strategies for Heuristics Part 2: Test Identification for the General Algorithm
3. Test Strategies for Heuristics Part 3: Implementaion Example with Threshold Accepting and CAPM

# 2. Software Testing Definition

There are many definitions out there, and all subtley different. In the interests of keeping this simple (but hopefully not simplistic) here it is in general. Please refer to the links provided for a more indepth view on the concepts and ideas of software testing.

A Definition: Software Testing is the process of discovering whether a program does what is is expected to do. The operative word is discovery: it is in activity, hopefully through careful and thorough planning, that one comes upon a problem.

From Thought to Expression: Testing a program is quite unlike testing a physical entity, which you can bash, bang, and break. One implements an idea, something abstract and complex mapped to a simple set of instruction of a computer langauge. No matter how well the original idea is specified - say. a highly formal mathematical model - from the moment one starts tapping away at the keyboard, tanslating the idea into an executable program, a bug will be introduced at some point. And even when these are ironed out, a combination of events in an executable program may also cause quite unexpected results. The key is to be aware of such and know how to deal with it.

# 3. Testing in context of Research

There are many case studies available to read, usually horror stories, where poor on non exisiting testing as occured on major software projects. IFor example, if you search for Therac-25, an engineering project for a radiation therapy machine, you get a report of the catastrophic extreme of such neglect. However, these are most examples are for large software projects consisting of tens of thousand of lines of code. A researcher they may say, "Well, my experiment uses a few hundred lines, and mainly uses the inbuilt libraries - I'll hack away at things and it'll sort itself out more or less."