Data-Driven Testing & API Integration

Data‑Driven Testing & API Integration

A Comprehensive Guide for Java Automation Testers (Beginner → Advanced)

Why this chapter?
• Data‑driven testing decouples test data from test logic, boosting maintainability.
• API integration is the backbone of modern SaaS, micro‑services, and mobile back‑ends.
• Combining the two gives you a powerful, repeatable, and scalable test harness.


1. Fundamentals

TopicCore IdeaWhy it mattersTypical Tools
Read Data from CSV/ExcelLoad external data sets into Java objectsKeeps data out of code, easier to updateOpenCSV, Apache POI, Commons CSV
Map Data to Test CasesParameterise tests with data setsEliminates boilerplate, supports edge‑case coverageTestNG DataProvider, JUnit @Parameterized
Invoke REST APIs with RestAssuredSend HTTP requests and receive responsesProvides a fluent API for HTTP verbs, headers, authRestAssured, JSON‑Path, JsonSchemaValidator
Validate API ResponsesAssert status codes, payloads, performanceGuarantees contract complianceHamcrest, JsonSchemaValidator, RestAssured Filters
Persist Results to DatabaseStore test outcomes for audit & analyticsEnables traceability, regression analysisJDBC, HikariCP, Hibernate

Industry Insight – In 2025, the Java ecosystem remains the #1 platform for enterprise services (Oracle Java). Test automation frameworks like Selenium, Appium, and AI‑driven solutions are still built on top of Java. The ability to read data from CSV/Excel and persist results to a database is a common requirement across SaaS, e‑commerce, and banking domains.


2. Implementation

Below we walk through real code for each sub‑topic. All snippets are self‑contained and can be copied into a Maven or Gradle project.

2.1 Read Data from CSV/Excel

2.1.1 CSV – Using OpenCSV

// pom.xml dependency
<dependency>
  <groupId>com.opencsv</groupId>
  <artifactId>opencsv</artifactId>
  <version>5.8</version>
</dependency>
import com.opencsv.CSVReader;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;

public class CsvDataReader {

    public static List<String[]> readCsv(String filePath) throws IOException {
        List<String[]> rows = new ArrayList<>();
        try (CSVReader reader = new CSVReader(new FileReader(filePath))) {
            rows = reader.readAll();
        }
        return rows;
    }
}

Tip – Skip the header row by rows.subList(1, rows.size()).

2.1.2 Excel – Using Apache POI

// pom.xml dependency
<dependency>
  <groupId>org.apache.poi</groupId>
  <artifactId>poi-ooxml</artifactId>
  <version>5.2.3</version>
</dependency>
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import java.io.FileInputStream;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

public class ExcelDataReader {

    public static List<Map<String, String>> readExcel(String filePath, String sheetName) throws IOException {
        List<Map<String, String>> data = new ArrayList<>();

        try (FileInputStream fis = new FileInputStream(filePath);
             Workbook workbook = new XSSFWorkbook(fis)) {
            Sheet sheet = workbook.getSheet(sheetName);
            Iterator<Row> rowIterator = sheet.iterator();

            // first row is header
            Row headerRow = rowIterator.next();
            List<String> headers = new ArrayList<>();
            headerRow.forEach(cell -> headers.add(cell.getStringCellValue()));

            while (rowIterator.hasNext()) {
                Row row = rowIterator.next();
                Map<String, String> rowMap = new HashMap<>();
                for (int i = 0; i < headers.size(); i++) {
                    Cell cell = row.getCell(i);
                    String value = cell.getCellType() == CellType.STRING
                                   ? cell.getStringCellValue()
                                   : String.valueOf(cell.getNumericCellValue());
                    rowMap.put(headers.get(i), value);
                }
                data.add(rowMap);
            }
        }
        return data;
    }
}

Edge‑Case – Handle merged cells, hidden rows, or cells with formulas.


2.2 Map Data to Test Cases

2.2.1 TestNG DataProvider

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class ApiTest {

    @DataProvider(name = "csvData")
    public static Object[][] csvDataProvider() throws IOException {
        List<String[]> rows = CsvDataReader.readCsv("src/test/resources/testdata.csv");
        return rows.stream().map(r -> new Object[]{r[0], r[1], r[2]}).toArray(Object[][]::new);
    }

    @Test(dataProvider = "csvData")
    public void testCreateUser(String userId, String name, String email) {
        // Build request body from parameters
        String payload = String.format("{\"userId\":\"%s\",\"name\":\"%s\",\"email\":\"%s\"}", userId, name, email);
        RestAssured
            .given()
                .header("Content-Type", "application/json")
                .body(payload)
            .when()
                .post("/users")
            .then()
                .statusCode(201)
                .body("userId", equalTo(userId));
    }
}

Professional Tip – Use @Factory for generating test classes dynamically when you have a very large data set.

2.2.2 JUnit 5 – Parameterized Tests

@ParameterizedTest(name = "Create user {index}")
@MethodSource("userProvider")
void testCreateUser(String userId, String name, String email) {
    // same as above
}

Industry Use‑Case – Parameterised tests are ideal for end‑to‑end flows where the same sequence of API calls must be executed with different user profiles.


2.3 Invoke REST APIs with RestAssured

2.3.1 Basic Setup

import io.restassured.RestAssured;
import io.restassured.response.Response;

RestAssured.baseURI = "https://api.example.com";
RestAssured.basePath = "/v1";

2.3.2 GET with Path & Query Parameters

Response response = RestAssured
    .given()
        .pathParam("userId", 12345)
        .queryParam("includeDetails", true)
    .when()
        .get("/users/{userId}")
    .then()
        .statusCode(200)
        .extract()
        .response();

2.3.3 POST with JSON Body & Auth

String payload = "{\"title\":\"API Test\",\"body\":\"Testing integration\"}";

Response resp = RestAssured
    .given()
        .auth().preemptive().basic("user", "pass")
        .header("Content-Type", "application/json")
        .body(payload)
    .when()
        .post("/posts")
    .then()
        .statusCode(201)
        .extract()
        .response();

Advanced Filter – Capture request/response logs for debugging.

RestAssured.filters(new RequestLoggingFilter(), new ResponseLoggingFilter());

2.4 Validate API Responses

2.4.1 Status & Body Assertions

resp.then()
    .statusCode(200)
    .body("data.id", equalTo(12345))
    .body("data.name", equalTo("John Doe"));

2.4.2 JSON Schema Validation

import io.restassured.module.jsonschema.JsonSchemaValidator;

resp.then()
    .assertThat()
    .body(JsonSchemaValidator.matchesJsonSchemaInClasspath("post_schema.json"));

Edge‑Case – Validate that the response contains optional fields only when a flag is set. Use hasItem or hasKey.

2.4.3 Performance Check

resp.then()
    .time(lessThan(2000L)); // 2 seconds

Industry Insight – In 2025, API performance is as critical as functional correctness, especially for real‑time services.


2.5 Persist Results to Database

2.5.1 JDBC with HikariCP (Connection Pooling)

// pom.xml
<dependency>
  <groupId>com.zaxxer</groupId>
  <artifactId>HikariCP</artifactId>
  <version>5.0.1</version>
</dependency>
import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.SQLException;

public class ResultPersistor {

    private static HikariDataSource ds;

    static {
        HikariConfig cfg = new HikariConfig();
        cfg.setJdbcUrl("jdbc:mysql://localhost:3306/testdb");
        cfg.setUsername("root");
        cfg.setPassword("root");
        ds = new HikariDataSource(cfg);
    }

    public static void saveTestResult(String testName, boolean passed, long durationMs) throws SQLException {
        String sql = "INSERT INTO test_results (test_name, passed, duration_ms, run_at) VALUES (?,?,?,NOW())";
        try (Connection conn = ds.getConnection();
             PreparedStatement ps = conn.prepareStatement(sql)) {
            ps.setString(1, testName);
            ps.setBoolean(2, passed);
            ps.setLong(3, durationMs);
            ps.executeUpdate();
        }
    }
}

Advanced – Batch insert for millions of rows, or use INSERT … ON DUPLICATE KEY UPDATE for idempotent logging.

2.5.2 Persist via JPA (Spring Data)

If you already use Spring, you can create an @Entity and use CrudRepository.

@Entity
@Table(name = "test_results")
public class TestResult {
    @Id @GeneratedValue
    private Long id;
    private String testName;
    private Boolean passed;
    private Long durationMs;
    private LocalDateTime runAt;
}

Industry Use‑Case – Persisting results into a central Test Results DB enables dashboards and trend analysis.


3. Advanced Topics

TopicWhat to MasterWhy it MattersSample Code
Parallel Data‑Driven ExecutionTestNG parallel mode, @DataProvider(parallel=true)Cuts execution time@DataProvider(parallel = true)
Dynamic Data GenerationFaker, Java Faker, or custom generatorsReduces test maintenanceFaker faker = new Faker(); faker.name().fullName();
GraphQL IntegrationRestAssured with graphql endpointHandles modern APIsRestAssured.given().body("{\"query\":\"{ user(id:1){name} \"}")
Custom Filters & LoggingRestAssuredFilter to capture request/responseHelps debugging flaky testsnew RequestLoggingFilter()
Performance & Load TestingJMeter integration, RestAssured + ExecutorServiceValidates API under loadExecutorService exec = Executors.newFixedThreadPool(50)
Mocking & Contract TestingWireMock, MockServerGuarantees API contractWireMock.stubFor(get(urlEqualTo("/users/123")).willReturn(aResponse().withBody(...)))
Database Transaction Management@Transactional with Spring, rollback on failureKeeps test DB clean@Transactional @Rollback
CI/CD IntegrationJenkins, GitHub Actions, Azure PipelinesAutomates test runs.yml files with mvn test

Professional Insight – In 2025, AI‑driven test frameworks are emerging. They can auto‑generate test data, detect flaky tests, and even suggest refactoring. Incorporating data‑driven and API tests early will make your automation suite future‑proof.


4. Real‑World Applications

DomainTypical API UseData‑Driven BenefitExample
E‑commerceProduct catalog, order placement, paymentLoad multiple product IDs from CSVValidate 200 OK for each product
BankingAccount balance, fund transferExcel sheet of account IDs, amountsEnsure 201 Created for each transfer
HealthcarePatient records, appointment schedulingCSV of patient IDs, doctor IDsVerify 404 for non‑existent patient
IoTDevice registration, telemetryExcel with device IDs, firmware versionsValidate 202 Accepted for each telemetry upload

Case StudyAcme Bank integrated data‑driven API tests into their nightly pipeline. They read a 10,000‑row CSV of account IDs, invoked GET /accounts/{id}, and persisted status codes into a MySQL table. Over 3 months, they detected a regression that returned 500 for a subset of accounts.


5. Exercises

#DescriptionSkill FocusDeliverable
1Read CSV – Create a CsvReader that reads a users.csv and prints each row.CSV parsingJava file
2Excel Mapping – Read an orders.xlsx file and map each row to a Order POJO.POJO mappingJava file
3Data‑Driven TestNG – Write a TestNG test that reads products.csv and verifies GET /products/{id} returns 200.DataProviderJava test class
4RestAssured POST – Implement a test that posts a new user, validates 201, and asserts the returned userId.API POSTJava test
5Response Validation – Validate JSON schema for a GET /posts endpoint.JSON schemaJava test + JSON schema file
6Persist to DB – Create a method that inserts test results into a MySQL table.JDBCJava class
7Parallel Execution – Run the product test in parallel mode.ParallelismTestNG XML
8Mock Server – Use WireMock to stub a GET /customers/{id} endpoint and test failure scenarios.MockingJava test + WireMock config
9Performance Test – Use RestAssured + ExecutorService to send 100 concurrent requests to /search and assert average latency < 300ms.Load testingJava code
10Dashboard – Build a simple Java Swing app that reads the results table and displays a bar chart of pass/fail counts.Data visualizationJava UI

Bonus – Combine all above into a mini‑framework that can be executed from the command line: java -jar test-runner.jar.


6. Summary & Take‑aways

  1. Decouple data from logic – CSV/Excel + DataProviders keep tests maintainable.
  2. Use RestAssured – It’s the de‑facto HTTP client for Java tests; learn its filters & JSON schema validation.
  3. Persist results – Store into a DB for traceability; use connection pooling for performance.
  4. Edge cases – Handle missing data, API timeouts, schema changes, and authentication failures.
  5. Advanced – Parallel execution, dynamic data, mock servers, and performance testing are next steps.
  6. Real‑world – Most enterprises rely on these patterns; mastering them opens doors to roles like Automation Engineer, Test Architect, and DevOps.

Final Thought – In 2025, the most valuable skill is building a reusable, data‑driven, API‑centric test framework that can evolve with your product. Start small, iterate, and keep the code clean. Happy testing!