Show understanding of the methods of testing available and select appropriate data for a given method

12.3 Program Testing and Maintenance

Learning Objective (linked to Assessment Objectives)

Show understanding of the methods of testing available and select appropriate data for a given method.

• AO1 – Knowledge & Understanding: define test levels, testing approaches and data‑selection techniques.
• AO2 – Application: analyse a specification and decide which testing level, approach and data‑selection technique is most suitable.
• AO3 – Evaluation: design test artefacts (test cases, test plan) and justify the choices made.

1. Why Test?

• Detect faults before the software is released.
• Confirm the program meets its specification.
• Give confidence that future changes will not introduce new errors.
• Testing is continuous – it starts during development and continues throughout maintenance.

2. Fault‑Avoidance vs Fault‑Exposure

Fault‑avoidance techniques are applied during design and coding; fault‑exposure techniques are applied during testing.

3. Types of Errors

4. Debugging Checklist (AO3 – Evaluation)

1. Read the error message or stack trace carefully.
2. Re‑produce the fault with the smallest possible test case.
3. Dry‑run the algorithm on paper (trace values step‑by‑step).
4. Insert temporary print / log statements to display variable values.
5. Use an IDE debugger: set break‑points, step through code, inspect watches.
6. Check assumptions (input ranges, initialisation, pre‑conditions).
7. After fixing, re‑run all relevant tests (regression).

5. Levels of Testing (AO2 – Application)

• Unit (Component) Testing – tests a single module or function in isolation.
• Integration Testing – checks interactions between combined units.
• System Testing – validates the complete, integrated system against the specification.
• Acceptance Testing – performed by the client/end‑user to confirm fitness for purpose.
• Regression Testing – re‑runs previously successful tests after any change.
• Maintenance Testing – testing carried out during corrective, adaptive, perfective or preventive maintenance (see §9).

6. Testing Approaches

Decision‑Tree for Choosing an Approach (exam‑friendly)

1. Do you have access to the source code?
   

   • No → Black‑Box
   • Yes → go to 2

2. Is the aim to prove structural correctness (e.g., 100 % statement coverage) or to find hidden logic errors?
   

   • Structural correctness → White‑Box
   • Hidden logic errors or limited time → Grey‑Box

7. Data‑Selection Techniques (AO2 – Application)

8. Mapping Test Levels to Data‑Selection Techniques

9. Coverage Metrics (White‑Box) – AO3

Metrics show how much of the program’s structure has been exercised.

• Statement Coverage – % of executable statements run.
  

  Statement Coverage = (Executed statements ÷ Total statements) × 100 %

• Branch (Decision) Coverage – % of true/false outcomes taken.
  

  Branch Coverage = (Executed decision outcomes ÷ Total decision outcomes) × 100 %

• Path Coverage – % of possible execution paths traversed (usually impractical for large programs).

Generating coverage in the exam languages:

• Java – IDE plugins such as JaCoCo or Cobertura.
• Python – coverage.py (run coverage run … then coverage report).
• VB.NET – Visual Studio’s built‑in Code Coverage tool (Test → Analyze Code Coverage).

10. Maintenance Types and Associated Testing (AO2)

11. Test Strategy & Test Plan (AO3 – Design & Evaluation)

11.1 Test‑Strategy Elements

• Objectives – what we aim to prove (fault detection, compliance, performance).
• Scope – which components, features and interfaces are in‑scope / out‑of‑scope.
• Approach – black‑box, white‑box or grey‑box; manual vs automated.
• Risk Assessment – high‑risk areas receive deeper testing.

11.2 Test‑Plan Template

12. Selecting an Appropriate Testing Method – Step‑by‑Step (AO2)

1. Identify the required testing level (unit, integration, system, acceptance, regression, maintenance).
2. Determine knowledge of the code:

   • No internal knowledge → Black‑Box.
   • Source code available → White‑Box (or Grey‑Box if only parts are known).

3. Match the nature of the input domain to a data‑selection technique (see §7).
4. Consider cost‑benefit and risk:

   • Safety‑critical modules → aim for high statement and branch coverage.
   • Low‑risk modules → representative black‑box tests may be sufficient.

5. Document the test plan** and ensure entry/exit criteria are met before moving to the next level.

13. Worked Example – Choosing Test Data for int grade(int mark)

Specification

• Valid range: 0 ≤ mark ≤ 100
• Grades:

  • 0–39 → “F”
  • 40–49 → “D”
  • 50–59 → “C”
  • 60–69 → “B”
  • 70–100 → “A”

13.1 Technique 1 – Equivalence Partitioning + Boundary‑Value Analysis (Black‑Box)

13.2 Technique 2 – Decision Table (Black‑Box, useful for complex rules)

Conditions: mark < 0, 0 ≤ mark ≤ 39, 40 ≤ mark ≤ 49, 50 ≤ mark ≤ 59, 60 ≤ mark ≤ 69, 70 ≤ mark ≤ 100, mark > 100. Actions: return “Invalid”, “F”, “D”, “C”, “B”, “A”.

From the decision table we select at least one representative value for each rule – e.g., -5 (Rule 1), 20 (Rule 2), 45 (Rule 3), 55 (Rule 4), 65 (Rule 5), 85 (Rule 6), 150 (Rule 7).

13.3 What the Example Demonstrates

• How to choose two different techniques for the same specification (requirement of the syllabus).
• How each technique leads to a clear set of test cases that can be recorded in a test case table (AO3).
• How the chosen technique matches the testing level – here unit testing with black‑box data selection.

14. Summary Checklist (AO3)

• Identify the testing level → choose the matching data‑selection technique(s).
• Decide on the testing approach (black/white/grey) using the decision‑tree.
• Generate test cases (include at least two techniques where required).
• Apply appropriate coverage metrics for white‑box testing.
• When code changes, run regression tests and update the test plan.
• Document each test case: description, input data, expected result, actual result, pass/fail.
• During maintenance, map the maintenance type to its typical testing activities (see §10).