API Testing – What to Validate

Has this above scenario ever happened with you. Some might say yes, some might say no. But in all situations, we have to take a pause to think – what are the key things that can be validated when you’re testing an API. It’s a valid question that I think the interviewer would use to gauge an interviewee how much in-depth they have tested the API’s and what different scenarios they can think of validating.

Let me try to accumulate these into some specific categories – but keep this in mind that this list is not exhaustive – this can change and extend depending upon the usage, the application and the design of the API’s.

What to Validate

Now let’s see one by one what things we can validate.

Response Code

One of the first and foremost things that we need to check is the response status code. This will vary depending upon what you’re test case is. Verify that the API returns appropriate HTTP status codes (e.g. 2xx, 4xx, 5xx, etc.) for different scenarios.

Response Time

Time is the essence in this world. Everybody wants to have the output of their work quickly. So the time in which an API returns a response is very very important. So one of the key things to validate in API Testing is the API response time. Verify that the API responds in a reasonable amount of time, ensuring that performance meets expectations. And that expectations has to be set via a SLA that needs to be discussed when you’re creating a test strategy for an API test. Discuss this with you PM, that what should be a defined SLA for an API.

Response Format

The format in which response comes is also a key factor to validate when testing API’s. Verify that the API returns responses in the expected format (e.g. JSON, XML, etc.), with correctly formatted data.This can be done by either validating the data format using the content-type header or by validating the sanctity of the data. If you want to know more about the content-type header, read this article I published some time ago.

Authentication

Broken authentication is one of the prime reasons of applications getting hacked now a days. So ensuring that the api authentication is proper is one of the key things to validate when testing API’s. Verify that the API requires proper authentication and authorization, ensuring that only authorized users can access protected resources.Whatever be the auth used – basic, bearer or Oauth2.0, be sure to test around scenarios that might uncover some issues in API authentication.

Input Validation

In most cases, API will take some input and then return necessary output. Verify that the API properly validates input data, ensuring that invalid data is not accepted or processed. If the input data needs to be sanitised or encoded before being sent as payload, be sure to test those scenarios too.

Error Handling

Not every time will the API return response that is expected. An API may return a error response in many conditions – incorrect auth, resource crunch, invalid input data, or the server being busy or being unavailable to process the response. So verify that the API properly handles errors, returning informative error messages to help troubleshoot issues. Ensuring that the error response is descriptive enough for user to understand, but should hide critical implementation details is a very test to do when testing API’s.

A good example of error message is like this

{
    "error": true,
    "message": "Sorry, we couldn't process your request. Please try again later or contact support for assistance."
}

While a not so good error message is like this

{
    "error": {
        "message": "An error occurred while processing your request.",
        "code": 500,
        "stackTrace": "java.lang.NullPointerException at com.example.api.MyController.doSomething(MyController.java:25) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at ..."
    }
}

Data Integrity And Validation

Verify that the API properly stores and retrieves data, ensuring that data is not corrupted or lost. This may be a very critical test for api’s that handle encrypted, financial or encoded data that is sent across a network. Data that is being sent and recieved, both should be tested for correct data sanity.

One way of doing that is by validating that the data confirms to a given JSON Schema. If the output is deviating from a given schema, then we can debug to see if there are bugs that may creep into system due to this.

JSON Schema gives a range of options to define in the schema so that you can use those to verify the data format and the data that is being sent/received. You can read about some of the options here and how they can be used.

Header Validation

API’s contain header information – both in the request and the response. It is always a good idea to validate the header that are sent in request or received in response, using appropriate tests.

Suppose you have an API endpoint /users that returns a list of users. The API is expected to return a Content-Type header with the value application/json and a Cache-Control header with the value public, max-age=60. So you can test that in Postman or using DSL like Rest Assured

 public void testContentTypeHeader() {
        given()
        .when()
            .get("/users")
        .then()
            .statusCode(200)
            .contentType("application/json");

Another example test – Let’s say you have an API – /file-upload, where the functionality is to upload a file. You can add tests in your test suite that the content-type header in this scenario, should not be application/json – it should be either multipart/form-data something like that based on the file type you’re uploading.

Security

Security is one of the key consideration when designing an API. So it should be also key consideration when testing them. You can verify that the API follows best practices for secure communication (e.g. using HTTPS), protecting sensitive data. You can do simple checks like checking for HTTPS -> HTTP traffic. Validate if the no-referrer-when-downgrade header is available and if so, the HTTPS->HTTP traffic should return either an error, a warning or the sensitive information should be masked etc.

You can also use solutions like Pynt to include security checks in your Postman collection. Adding a video by Naveen Automation Labs for reference

Rate Limiting

If your API’s are publicly exposed ( or even if not), it is a good idea to test for rate limiting. Rate Limiting and API throttling allows you to ensure that the API in question is not abused. I wrote about this in my blog and you can refer to it. Always test for this when your api’s are either public or are upstream to any downstream micro service.

Concurrency

Verify that the API is capable of handling multiple requests simultaneously, without affecting performance or functionality. This becomes critical in case of API’s that need to handle large volumes of data – suppose a Spotify API, or a Google Maps API. This is one of the things that needs to be tested for ensuring that the API performance doesn’t degrade over certain load.

Compatibility

Compatibility testing in API development involves verifying that the API functions as expected across different environments, devices, operating systems, and browsers. This ensures that the API is accessible and usable by a wide range of clients, including desktop and mobile applications, web browsers, and other software systems.

Versioning

It generally happens that over a period of time, your team would add some new features, remove some, and refine others. At that time, it is a good idea for API versioning if you’re trying to include breaking changes or if you’re trying to add/remove some functionalities in the current API.

In this scenario, there might be teams that would still need to use your old api’s. So tests around new and old versions of api is very necessary.Verify that the API supports versioning, ensuring that changes to the API do not break existing integrations.

Here is an example of API versioning:

Let’s say we have an API that provides information about books, including their titles, authors, descriptions, and cover images. The API has the following endpoint:

GET /books

This endpoint returns a list of all books available in the API. Now let’s say we want to add a new field to the book data called publication_date.

To avoid breaking any existing clients that are already using the /books endpoint without the publication_date field, we can create a new version of the API with a new endpoint that includes the new field:

GET /v2/books

This new endpoint is the same as the original /books endpoint but with a version number v2 added to the URL. Clients that want to use the new publication_date field can now use this new endpoint, while clients that don’t need the field can continue using the original endpoint without any changes.

Caching

Caching is an integral part of the web API’s in today’s world. When testing API’s it is always a good idea to test for the cache, implemented if any in the API. You can verify that the API uses caching where appropriate, improving performance by reducing the number of requests to the server.

How to do that?

I wrote an article regarding how the cache-control header can be used and interpreted in the API’s. You can read it and then use tests to validate that.

Pagination

Verify that the API supports pagination where appropriate, allowing users to retrieve large amounts of data in manageable chunks. pagination tests in api. Pagination tests in APIs are important to ensure that the API returns the correct subset of data based on the requested page and page size.

Suppose you have an API endpoint /users that returns a list of users with pagination support. The API supports two query parameters: page and pageSize, which determine the page number and page size, respectively.

Using a DSL like Rest-Assured, we can write tests like this

import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;

public class PaginationTest {

    @Test
    public void testPagination() {
        // Send a request for the first page with 10 items
        given()
            .queryParam("page", 1)
            .queryParam("pageSize", 10)
        .when()
            .get("/users")
        .then()
            .statusCode(200)
            .body("size()", equalTo(10));

        // Send a request for the second page with 10 items
        given()
            .queryParam("page", 2)
            .queryParam("pageSize", 10)
        .when()
            .get("/users")
        .then()
            .statusCode(200)
            .body("size()", equalTo(10));

        // Send a request for a different page size
        given()
            .queryParam("page", 1)
            .queryParam("pageSize", 20)
        .when()
            .get("/users")
        .then()
            .statusCode(200)
            .body("size()", equalTo(20));

        // Send a request with invalid parameters
        given()
            .queryParam("page", -1)
            .queryParam("pageSize", "foo")
        .when()
            .get("/users")
        .then()
            .statusCode(400)
            .body("message", containsString("Invalid parameters"));

        // Verify pagination headers
        given()
            .queryParam("page", 1)
            .queryParam("pageSize", 10)
        .when()
            .get("/users")
        .then()
            .statusCode(200)
            .header("X-Total-Count", equalTo("100"))
            .header("Link", containsString("rel=\"first\""))
            .header("Link", containsString("rel=\"next\""))
            .header("Link", containsString("rel=\"last\""));
    }
}

We use the body() method to verify the response body, such as the size of the returned list.We also use the header() method to verify the pagination headers in the response, such as the X-Total-Count header and the Link header that provides links to the previous, next, first, and last pages.

Analytics and Logging

Verify that the API collects and logs data about usage, performance, and errors, providing insights into how the API is being used and how it can be improved. API logs are very important criteria to evaluate the usefulness, and the errors that may come in the future or if there are any error if present.

So it would be a good test to include in your test suite, that the API is sending proper logs, to maybe a central logging mechanism like Splunk, or ELK.

I think this would be a good measure of the validations that you need to include in your API tests. Again, this is not exhaustive, and will differ from API to API. So it’s always a good idea to have a good testing strategy defined for your API tests, and include as much as different scenarios when testing. I would recommend reading this list and coming up with more things that can be validated in API’s. Feel free to add in comments.