Home
PreviousNext
1. Introduction1.1. Course Structure1.2. Learning Goals1.3. Course Lecturers1.4. Topics1.5. Bibliography1.6. Tools1.7. Evaluation1.8. Agreements
2. Course Project2.1. Scope2.2. Rules of the Game2.3. Important Dates2.4. Requirements2.5. Tips
3. Git 1013.1. Introduction3.2. Installing Git3.3. Basic shell commands3.4. Git commands3.5. Ignoring files3.6. Branches3.7. Remote repositories
4. How Java Works4.1. Introduction4.2. Java Architecture4.3. Variables and Parameters4.4. Java Memory Model
5. Exception Handling5.1. Introduction5.2. Exception Types5.3. Declaring Exceptions Thrown by Methods5.4. Throwing Exceptions5.5. Custom Exceptions5.6. Accessing the Stack Trace
6. Inheritance6.1. Introduction6.2. Interfaces6.3. Method Overriding6.4. Multiple Inheritance6.5. Abstract Classes6.6. Method Overloading6.7. Polymorphism6.8. The Object Class6.9. Casting
7. Generic Programming7.1. Introduction7.2. Generic Methods7.3. Bounded and Unbounded Type Parameters7.4. Generic Types
8. Arrays8.1. One Dimensional Arrays8.2. Multidimensional Arrays8.3. The Arrays class8.4. Varags8.5. Limitations
9. Java Collections Framework9.1. Introduction9.2. The Collection Interface9.3. Lists9.4. Sets9.5. Queues9.6. The Collections Class9.7. Maps9.8. Choosing My Collection
10. Lambda Expressions10.1. Introduction10.2. What is a Lambda Expression?10.3. Functional Interfaces10.4. Local Variables in Lambda Expressions10.5. Standard Functional Interfaces10.6. Sorting with Lambdas
11. Streams11.1. Introduction11.2. Pipelines11.3. IntStream11.4. Optionals11.5. Operations
12. Maven12.1. Introduction12.2. Compiling Java12.3. Getting Started12.4. Coordinates12.5. Repositories12.6. Project Object Model12.7. Dependencies12.8. Standard Directory Layout12.9. Lifecycles and Phases12.10. Wrapper12.11. Archetypes12.12. Maven and Git12.13. Common Maven plugins
13. Testing13.1. Introduction13.2. Getting Started with JUnit 513.3. Basics of JUnit 513.4. JUnit Lifecycle API13.5. Expected Exceptions13.6. Timeouts13.7. Nested Tests13.8. Repeating Tests13.9. Third-party Assertion Libraries13.10. FIRST Properties of Good Tests13.11. What to Test: The Right-BICEP13.12. Boundary Conditions: The CORRECT Way
14. Regular Expressions14.1. String Manipulation14.2. Writing Regular Expressions14.3. Using Regular Expressions in String Methods14.4. Regex API
15. Design Patterns15.1. Instructions15.2. What is a Design Pattern?15.3. Gang of Four Design Patterns
16. I/O16.1. File manipulation16.2. I/O Streams16.3. Reading: FileReader and BufferedReader16.4. Reading: Scanner16.5. Reading: Files16.6. Writing Text Files
17. Serialization17.1. Introduction17.2. CSV17.3. JSON17.4. Reading and writing JSON with Jackson17.5. Binary serialization
18. Multithreading18.1. Concurrency18.2. Processes & Threads18.3. Multithreading in Java18.4. Thread Synchronization18.5. Thread Signaling
13. Testing
Programming Project 2021/22

13.1. Introduction

The purpose of tests

Why do we test our software?

  1. Software systems are pervasive in our lives. Even small bugs have relevant impacts.
  2. Software is often used in mission critical scenarios where failure is unacceptable.
  3. Software defects cost money
    • to the users, and
    • to the developers!
  4. We want to confidently release a new version of our software.
  5. We want to document how our system should behave.

Testing terminology

Product specification: an agreement among the software development team that defines the product they are creating, detailing what it will be, how it will act, what it will do, and what it won't do.

Bug:

  • a problem in the software,
  • or a deviation from the product specification.
  • Also known as defect, fault, error, problem, anomaly...
  • May be big or small, intended or unintended.

For the purposes of this course, a software bug occurs when one of the following is the case.

  1. The software doesn't do something that the product specification says it should do.
  2. The software does something that the product specification says it shouldn't do.
  3. The software does something that the product specification doesn't mention.
  4. The software doesn't do something that the product specification doesn't mention but should.
  5. The software is difficult to understand, hard to use, slow, or—in the software tester's eyes—will be viewed by the end user as just plain not right.

A bug example

Let us exemplify bugs using a calculator.

  • Specification:

    • The calculator should perform correct addition, subtraction, multiplication, and division operations.
    • The calculator should not crash if multiple buttons are pressed simultaneously.

  • Bugs:

    • If you press + and nothing happens (1).
    • If you press 2 and 3 at the same time and the calculator stops responding (2).
    • The calculator does square root (3).
    • The calculator gives wrong results when the battery is low (4).
    • The buttons are too small (5).

Why do bugs occur?

Here is the result of a an assessment of bug causes.

Most bugs can be traced to the product specification because the specification

  • is not written,
  • is poorly written,
  • keeps changing, or
  • is not properly communicated.

Another relevant portion can be attributed to design because it may be

  • not well thought out,
  • rushed,
  • poorly communicated, or
  • changing repeatedly.

The third main category is only traceable to the mistakes in the code that maybe due to

  • complex nature of the software,
  • schedule pressure,
  • simple mistakes, or
  • poor documentation.

The cost of bugs

The costs of fixing bugs grow logarithmically with time:

Testing

Definition: The activity to verify if a software or a portion of a software behaves according to its specification.

Types of testing:

  • By approach:
    • white-box x black-box
    • static x dynamic
  • By level:
    • unit x integration x system x acceptance
      • Unit testing: the smallest testable parts of an application, called units, are individually and independently scrutinized for proper operation.
      • Integration testing: individual software modules are combined and tested as a group.
      • System testing: how various components of an application interact together in the full.
      • Acceptance testing: a quality assurance (QA) process that determines to what degree an application meets end users' approval.
  • By aspect:
    • functional x non-functional
      • Functional testing is a type of testing that seeks to establish whether each application feature works as per the software requirements.
      • Non functional testing is a type of software testing that verifies non functional aspects of the product, such as performance, stability, usability, and security.
        • Performance testing is the practice of evaluating how a system performs in terms of responsiveness and stability under a particular workload.
        • Usability testing refers to evaluating a product or service by testing it with representative users.
        • Security testing is a process intended to reveal flaws in the security mechanisms.
  • By automation:
    • automated x manual

Unit tests

A unit test is:

  • automated,
  • designed to verify a small piece of code,
  • fast,
  • isolated,
  • typically white-box.

A small piece of code usually refers to a method or a class.

Fast is subjective, but you should aim for a test suite that runs in a few seconds.

Unit tests themselves should be run in isolation from each other, so that you run tests in parallel, sequentially, or in any order.

Unit tests are not suitable for testing complex user interface or component interaction.

From: Vladimir Khorikov, Unit Testing Principles, Practices, and Patterns (2020). Manning Publications. Unit Testing Principles, Practices, and Patterns

Benefits of unit testing

  • Consolidates the specification

    Before we start with coding a component we must try to determine what the component must do.

    The act of building test cases at the start helps to clarify the expected behavior of the component. We build the test case for the possible inputs and the possible outputs. If we are unable to come up with a test, it means that the specifications are not clear enough and require revision.

  • Early error detection

    Unit tests are proof of working code. They are executed in every build and can detect failures at the first instance.

    Unit tests can detect not only coding bugs but flaws in product specifications as well. A unit test demonstrates progress; thus, as soon as a component is complete, it can be demo-ed to the stakeholders to find gaps, if any. The sooner a bug is uncovered, the cheaper it is to fix.

  • Supports maintenance

    Unit tests help in understanding the intended behavior in development cycles without being distracted by the actual code.

    Product specifications evolve over time with changes that lead to development cycles. In each of these cycles, the team has to understand how the existing code works before team members can make any changes. A well-written unit test suite helps to keep the development in focus for the team.

  • Improves design

    Unit tests make us think in terms of the expected input and the expected output.

    Unit tests are the first client of the code being tested. They uncover various issues that a client can face while interfacing with the code being tested. Unit tests can help in classifying responsibility boundaries for the components. That can improve product specifications by exposing gaps in the interface design.

  • Product documentation

    Unit tests describe how a piece of code works—that is, the expected output for a given input.

    They always describe the latest state of a specification, as they are kept in sync with the code changes.

Test coverage

The percentage of code which is tested by unit tests.

Criteria:

  • Statement
  • Branch
  • Function
  • Line

It helps you to identify which portions of your code have been tested.

  • 100% test coverage is unrealistic.
  • Test coverage is not the whole picture.

Testing frameworks

There are several testing frameworks available for Java, the most popular ones being:

In this course, we use JUnit 5.

PreviousNext