ECE366 - Lesson 12
Testing, Software Engineering Ethics
Instructor: Professor Hong
## Types of Testing
- Unit testing - smallest pieces possible
- System testing - bigger part, complete API
- Integration testing - connecting different systems
- Acceptance testing - end user testing
- Performance testing - speed of program
- Regression testing - testing everything again
- Security testing - spot weaknesses in security
- Load testing - large loads / lots of hits
- End-to-end testing - entire system
## TDD
- Test Driven Development
- Written before implementation
- Tests will fail first and then succeed after writing code
- Forces developer to think about requirements
- Not skipped due to time pressure
- Bugs caught early
- Still need system and integration tests
## Advantages of Unit Testing
- Validating smallest units of software
- Find bugs easily and early
- Save time and money in the long run
- Forcing developers to write better and cleaner code
## JUnit
- Unit testing framework for Java
- Part of xUnit series
- Enables automated unit testing
## JUnit and Maven
- Create a new Maven project
- Add Maven dependency
```
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.9.2</version>
<scope>test</scope>
</dependency>
</dependencies>
```
## Your First Test
src/main/java/Code.java
```
public class Code {
public String sayHello() {
return "Hello world!";
}
}
```
## Your First Test
test/java/CodeTest.java
```
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
public class CodeTest {
@Test
public void testSayHello() {
Code code = new Code();
assertEquals("Hello world!", code.sayHello());
}
}
```
## Bank Account
src/java/BankAccount.java
```
public class BankAccount {
private double balance;
private double minimumBalance;
public BankAccount(double balance, double minimumBalance) {
this.balance = balance;
this.minimumBalance = minimumBalance;
}
public double getBalance() {
return balance;
}
public double getMinimumBalance() {
return minimumBalance;
}
public double withdraw(double amount) {
if(balance - amount > minimumBalance) {
balance -= amount;
return amount;
} else {
throw new RuntimeException();
}
}
public double deposit(double amount) {
try {
Thread.sleep(5);
} catch (InterruptedException e) {
e.printStackTrace();
}
return balance += amount;
}
}
```
## Bank Account Tests
test/java/BankAccountTest.java
```
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import java.time.Duration;
import static org.junit.jupiter.api.Assertions.*;
@DisplayName("Test BankAccount class")
public class BankAccountTest {
@Test
@DisplayName("Withdraw 500 successfully")
public void testWithdraw() {
BankAccount bankAccount = new BankAccount(500, -1000);
bankAccount.withdraw(300);
assertEquals(200, bankAccount.getBalance());
}
@Test
@DisplayName("Deposit 500 successfully")
public void testDeposit() {
BankAccount bankAccount = new BankAccount(400, 0);
bankAccount.deposit(500);
assertEquals(900, bankAccount.getBalance(), "Unexpected value, expected 900");
}
}
```
- DisplayName gives the tests a better name/description
## Assertions
- Check the outcome of the test
- If the assertion fails, the test fails
- Assertions class in the org.junit.jupiter.api package
- Many methods and overrides
- Add isActive and holderName
```
private boolean isActive = true;
private String holderName;
public boolean isActive() {
return isActive;
}
public void setActive(boolean active) {
isActive = active;
}
public String getHolderName() {
return holderName;
}
public void setHolderName(String holderName) {
this.holderName = holderName;
}
```
## Additional Properties for BankAccount
src/java/BankAccount.java
```
private boolean isActive = true;
private String holderName;
public boolean isActive() {
return isActive;
}
public void setActive(boolean active) {
isActive = active;
}
public String getHolderName() {
return holderName;
}
public void setHolderName(String holderName) {
this.holderName = holderName;
}
```
## Additional BankAccount Tests
tests/java/BankAccountAssertionTest.java
```
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import java.time.Duration;
import static org.junit.jupiter.api.Assertions.*;
import static org.junit.jupiter.api.Assertions.assertTimeout;
public class BankAccountAssertionsTest {
@Test
@DisplayName("Withdraw will become negative")
public void testWithdrawNotStuckAtZero() {
BankAccount bankAccount = new BankAccount(500, -1000);
bankAccount.withdraw(800);
assertNotEquals(0, bankAccount.getBalance());
}
@Test
@DisplayName("Test activation account afer creation")
public void testActive() {
BankAccount bankAccount = new BankAccount(500, 0);
assertTrue(bankAccount.isActive());
}
@Test
@DisplayName("Test set holder name")
public void testHolderNameSet() {
BankAccount bankAccount = new BankAccount(500, 0);
bankAccount.setHolderName("Chris");
assertNotNull(bankAccount.getHolderName());
}
@Test
@DisplayName("Test that we can't withdraw below the minimum")
public void testNoWithdrawBelowMinimum() {
BankAccount bankAccount = new BankAccount(500, -1000);
assertThrows(RuntimeException.class, () -> bankAccount.withdraw(2000));
}
@Test
@DisplayName("Test no exceptions for withdraw and deposit")
public void testWithdrawAndDepositWithoutException() {
BankAccount bankAccount = new BankAccount(500, -1000);
assertAll(() -> bankAccount.deposit(200), () -> bankAccount.withdraw(450));
}
@Test
@DisplayName("Test speed deposit")
public void testDepositTimeout() {
BankAccount bankAccount = new BankAccount(400, 0);
assertTimeout(Duration.ofSeconds(1), () -> bankAccount.deposit(200));
}
// assertEquals, delta, message
// fail(); // fails the test
}
```
## Assumptions
- Setting a condition for executing a test
- If the assumption is met, the test will be executed
- If the assumption is not met, the test won't be executed
- Assumptions are in the org.junit.jupiter.api package
## Assumptions Test
tests/java/BankAccountAssumptionsTest.java
```
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assumptions.*;
public class BankAccountAssumptionsTest {
@Test
@DisplayName("Test activation account after creation")
public void testActive() {
BankAccount bankAccount = new BankAccount(500, 0);
assumeTrue(bankAccount != null);
assumeFalse(bankAccount == null);
assumingThat(bankAccount == null, () -> assertTrue(bankAccount.isActive()));
// doesn't abort the test and continues
assertTrue(bankAccount.isActive());
}
}
```
## Test Order
- Without specifying, we cannot predict the order of the tests
- Usually, this isn't a problem as the tests should be independent
- If you do need to order your tests, you can leverage the Order annotation
## Ordered Execution Tests
tests/java/BankAccountOrderedExecutionTest.java
```
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import static org.junit.jupiter.api.Assertions.assertEquals;
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class BankAccountOrderedExecutionTest {
static BankAccount bankAccount = new BankAccount(0,0);
@Test
@Order(2)
public void testWithdraw() {
bankAccount.withdraw(300);
assertEquals(200, bankAccount.getBalance());
}
@Test
@Order(1)
public void testDeposit() {
bankAccount.deposit(500);
assertEquals(500, bankAccount.getBalance());
}
}
```
## Nested Tests
- Used to control the relationship between tests
- Useful when you have feature separation or when code is organized around a method or feature
## Nested Tests
tests/java/BankAccountNestedTest.java
```
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class BankAccountNestedTest {
@Test
@DisplayName("Withdraw 500 successfully")
public void testWithdraw() {
BankAccount bankAccount = new BankAccount(500, -1000);
bankAccount.withdraw(300);
assertEquals(200, bankAccount.getBalance());
}
@Test
@DisplayName("Deposit 400 successfully")
public void testDeposit() {
BankAccount bankAccount = new BankAccount(400, 0);
bankAccount.deposit(500);
assertEquals(900, bankAccount.getBalance());
}
@Nested
class WhenBalanceEqualsZero {
@Test
@DisplayName("Withdrawing below minimum balance: exception")
public void testWithdrawMinimumBalanceIs0() {
BankAccount bankAccount = new BankAccount(0, 0);
assertThrows(RuntimeException.class, () -> bankAccount.withdraw(500));
}
@Test
@DisplayName("Wtihdrawing below minimum balance: negative balance")
public void testWithdrawMinimumBalanceNegative1000() {
BankAccount bankAccount = new BankAccount(0, -1000);
bankAccount.withdraw(500);
assertEquals(-500, bankAccount.getBalance());
}
}
}
```
## Dependency Injection
- Less tightly coupled classes
- No need to manually create instances everytime
- No need to create new BankAccount(0,0) any more
## Parameter Resolver
tests/java/BankAccountParameterResolver.java
```
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.ParameterContext;
import org.junit.jupiter.api.extension.ParameterResolutionException;
import org.junit.jupiter.api.extension.ParameterResolver;
public class BankAccountParameterResolver implements ParameterResolver {
@Override
public boolean supportsParameter(ParameterContext parameterContext, ExtensionContext extensionContext) throws ParameterResolutionException {
return parameterContext.getParameter().getType() == BankAccount.class;
}
@Override
public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext) throws ParameterResolutionException {
return new BankAccount(0, 0);
}
}
```
## Dependency Injection Test
tests/java/BankAccountDITest.java
```
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import static org.junit.jupiter.api.Assertions.assertEquals;
@ExtendWith(BankAccountParameterResolver.class)
public class BankAccountDITest {
@Test
@DisplayName("Deposit 500 successfully")
public void testDeposit(BankAccount bankAccount) {
bankAccount.deposit(500);
assertEquals(500, bankAccount.getBalance());
}
}
```
## Repeated Tests
tests/java/BankAccountRepeatedTest.java
```
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.RepeatedTest;
import org.junit.jupiter.api.RepetitionInfo;
import org.junit.jupiter.api.extension.ExtendWith;
import static org.junit.jupiter.api.Assertions.assertEquals;
@ExtendWith(BankAccountParameterResolver.class)
public class BankAccountRepeatedTest {
@RepeatedTest(5)
@DisplayName("Deposit 400 successfully")
public void testDeposit(BankAccount bankAccount) {
bankAccount.deposit(500);
assertEquals(500, bankAccount.getBalance());
}
@RepeatedTest(5)
@DisplayName("Deposit 400 successfully")
public void testDepositRepetitionInfo(BankAccount bankAccount, RepetitionInfo repetitionInfo) {
bankAccount.deposit(500);
assertEquals(500, bankAccount.getBalance());
System.out.println("Nr: " + repetitionInfo.getCurrentRepetition());
}
}
```
Great for repeating tests multiple times.
## Parameterized Tests
tests/java/BankAccountParameterizedTest.java
```
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.RepeatedTest;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvFileSource;
import org.junit.jupiter.params.provider.CsvSource;
import org.junit.jupiter.params.provider.EnumSource;
import org.junit.jupiter.params.provider.ValueSource;
import java.time.DayOfWeek;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
@ExtendWith(BankAccountParameterResolver.class)
public class BankAccountParameterizedTest {
@ParameterizedTest
@ValueSource(ints = {100, 400, 800, 1000})
@DisplayName("Deposit successfully")
public void testDeposit(int amount, BankAccount bankAccount) {
bankAccount.deposit(amount);
assertEquals(amount, bankAccount.getBalance());
}
@ParameterizedTest
@EnumSource(value = DayOfWeek.class, names = {"TUESDAY", "THURSDAY"})
public void testDayOfWeek(DayOfWeek day) {
assertTrue(day.toString().startsWith("T"));
}
@ParameterizedTest
@CsvSource({"100, Mary", "200, Richard", "150, Ted"})
//@CsvFileSource(resources = "details.csv", delimiter = ',')
public void depositAndNameTest(double amount, String name, BankAccount bankAccount) {
bankAccount.deposit(amount);
bankAccount.setHolderName(name);
assertEquals(amount, bankAccount.getBalance());
assertEquals(name, bankAccount.getHolderName());
}
}
```
- Run test several times with different parameters
## Timeout Tests
tests/java/BankAccountTimeoutTest.java
```
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Timeout;
import org.junit.jupiter.api.extension.ExtendWith;
import java.time.Duration;
import java.util.concurrent.TimeUnit;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTimeout;
// @Timeout(value=500, unit=imeUnit.MILLISECONDS)
@ExtendWith(BankAccountParameterResolver.class)
public class BankAccountTimeoutTest {
@Test
@Timeout(value=500, unit= TimeUnit.MILLISECONDS)
public void testDepositTimeoutAssertion(BankAccount bankAccount) {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
bankAccount.deposit(300);
assertEquals(300, bankAccount.getBalance());
}
@Test
public void testDepositTimeoutAnnotation(BankAccount bankAccount) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
bankAccount.deposit(300);
assertTimeout(Duration.ofMillis(500), () -> {
Thread.sleep(10);
});
}
}
```
## Parallel Execution Setup
tests/resources/junit-platform.properties
```
junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.config.strategy=dynamic
```
## Parallel Execution
tests/java/BankAccountParallelExecutionTest.java
```
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.parallel.Execution;
import org.junit.jupiter.api.parallel.ExecutionMode;
import static org.junit.jupiter.api.Assertions.assertEquals;
@Execution(ExecutionMode.CONCURRENT)
@ExtendWith(BankAccountParameterResolver.class)
public class BankAccountParallelExecutionTest {
@Test
@DisplayName("Deposit 500 successfully")
public void testDeposit(BankAccount bankAccount) {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
bankAccount.deposit(500);
assertEquals(500, bankAccount.getBalance());
}
@Test
@DisplayName("Deposit 500 successfully")
public void testDeposit2(BankAccount bankAccount) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
bankAccount.deposit(500);
assertEquals(500, bankAccount.getBalance());
}
@Test
@DisplayName("Deposit 500 successfully")
public void testDeposit3(BankAccount bankAccount) {
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
bankAccount.deposit(500);
assertEquals(500, bankAccount.getBalance());
}
}
```
## Before and After Test
tests/java/BankAccountBeforeAndAfterTest.java
```
import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.assertEquals;
@TestInstance(TestInstance.Lifecycle.PER_CLASS) // can use this for BeforeAll/AfterAll w/o static
public class BankAccountBeforeAndAfterTest {
static BankAccount bankAccount;
@BeforeAll
//@BeforeEach // fn must not be static
public void prepTest() {
System.out.println("Hi");
bankAccount = new BankAccount(500, 0);
}
@Test
public void testWithdraw() {
bankAccount.withdraw(300);
assertEquals(200, bankAccount.getBalance());
}
@Test
public void testDeposit() {
bankAccount.deposit(500);
assertEquals(1000, bankAccount.getBalance());
}
@AfterAll
//@AfterEach // fn must not be static
public void endTest() {
System.out.println("Bye");
}
}
```
## Conditional Tests
tests/java/BankAccountConditionalTest.java
```
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.*;
public class BankAccountConditionalTest {
@Test
@EnabledOnOs({OS.MAC})
public void testMac() {
}
@Test
@EnabledOnOs({OS.WINDOWS})
public void testWindows() {
}
@Test
@EnabledOnJre({JRE.JAVA_16})
public void testJRE() {
}
@Test
@DisabledOnJre({JRE.JAVA_16})
public void testNoJRE16() {
}
}
```
Generally not good practice
## Disable a Test
tests/java/BankAccountDisabledTest.java
```
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import static org.junit.jupiter.api.Assertions.assertEquals;
@ExtendWith(BankAccountParameterResolver.class)
public class BankAccountDisabledTest {
@Test
@Disabled("Temporarily disabled due to maintenance")
@DisplayName("Deposit 500 successfully")
public void testDeposit() {
BankAccount bankAccount = new BankAccount(400, 0);
bankAccount.deposit(500);
assertEquals(900, bankAccount.getBalance(), "Unexpected value, expected 900");
}
}
```
## Mockito
- A way to mock functions/systems
pom.xml
```
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<version>2.23.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>3.8.0</version>
<scope>test</scope>
</dependency>
```
## Calculate Methods
src/main/java/CalculateMethods.java
```
public class CalculateMethods {
public double divide(int x, int y) {
return x / y;
}
}
```
## Mockito Test
test/java/CalculateMethodsMockitoTest.java
```
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import static org.junit.jupiter.api.Assertions.assertEquals;
@ExtendWith(MockitoExtension.class)
public class CalculateMethodsMockitoTest {
@Mock
CalculateMethods calculateMethods;
@BeforeEach
public void setupMocks() {
Mockito.when(calculateMethods.divide(6,3)).thenReturn(2.0);
}
@Test
public void testDivide() {
assertEquals(2.0, calculateMethods.divide(6,3));
}
}
```
## Surefire Reports
mvn.pom
```
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.2</version>
</plugin>
</plugins>
</build>
```
```
mvn surefire-report:report
```
## Jacoco
pom.xml
```
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.9</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
```
```
mvn jacoco:report
```
## Best Practices
- Keep it simple
- Test the actual unit of code
- Clear naming and stick to naming conventions
- Low cyclomatic complexity - the fewer codes in path, the better
- Shouldn't have implementation in code
- Deterministic tests - you should always have the same result
## Software Engineering Ethics
## Software Engineering Ethics
[https://ethics.acm.org/code-of-ethics/software-engineering-code/](https://ethics.acm.org/code-of-ethics/software-engineering-code/)
## Final Project Requirement
- Take 10 items from the software engineering ethics and describe how you followed them in your project
- 2.5 points / 25 points