Chapter 2: Record and replay HTTP requests and responses to create API mocks

Mocking and simulating APIs using service virtualization

If you are new to the subject have a look at Chapter 1: Getting started with stubbing, mocking and service virtualization. If you know why and how to use mocking, simulating APIs and service virtualization, continue reading to learn how to do it for HTTP APIs. We will use mocking, simulating and service virtualization terms here interchangeably.

Useful Terms To Know

To fully grasp what we will be covering you should have a basic knowledge of software testing and understand what the following terms mean:

API (Application Programming Interface): Describes how frontend, middleware, backend and third-party systems communicate. For example, front-end systems will communicate via APIs with middleware and backend systems, while backend systems will communicate via APIs with third-party systems, etc. You can use service virtualization to simulate APIs.
HTTP: A format (protocol) of communication in which systems exchange information on the internet. HTTP defines the format of requests and responses that systems will use to communicate with each other. Here is an excellent introduction to HTTP if you would like to learn more.
SOAP: a format for sending and receiving messages that can be used on top of HTTP. HTTP is used to transfer almost any type of data; SOAP is used to transfer objects. Here is a great introduction to SOAP if you would like to learn more.

Two systems communicating with HTTP

Let us focus on a typical communication pattern where two systems communicate with HTTP request and responses:

System A will send HTTP request to system B
System A will wait until System B processes the request and responds
System B will send a HTTP Response to System A

Now let us explore how we can mock and simulate System B.

Mocking and simulating HTTP responses

Now that we have two systems communicating with HTTP, we can explore how we could test those systems. What happens if we wanted to test System A, but System B is temporarily unavailable? Or, what happens if System B does not exist yet at all?

Testing issues when the system is not available

In these scenarios, we can simulate System B using Traffic Parrot so that System A can still be tested. Traffic Parrot will receive the HTTP requests and send HTTP responses, pretending to be System B.

Consuming HTTP requests and generating HTTP responses using Traffic Parrot

This will allow us to test System A in isolation in both typical and hypothetical situations. You will be able to simulate HTTP responses that are expected to be seen in production environments, but also non-typical error responses or failure scenarios that would usually be hard to reproduce. You can do that by configuring Traffic Parrot to respond with response messages of your choice.

Using Traffic Parrot to mock a SOAP API

Let's go through an example of how to use Traffic Parrot to simulate a system that is available via a HTTP SOAP API. Our system under test is a UV Index application that is used to fetch UV Index for a given postcode. The app connects to the UV Index HTTP SOAP API provided by the United States Environmental Protection Agency (EPA) located at https://iaspub.epa.gov/ and displays the details of the UV Index status to the user.

UV Index application communicates via HTTP SOAP XML with the EPA API

In our example, we will assume that the EPA UV Index SOAP API (EPA API) is available, but we would like to simulate different types of responses from that API in our testing to understand how our UV index application works in hypothetical scenarios. For example, we would like to know how it behaves for different values of the UV index or incorrect data returned by the API.

Testing issue with setting up third-party test data

For example, if we wanted to test UV Index values like 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 and 10, we would have to find postcodes where the UV Index is exactly equal to that value when we run our tests. Searching for those postcodes would be time-consuming, and also depending on the day we might not be able to find a postcode that would give us the UV Index we want. If we wanted to test how our application behaves when the EPA API returns incorrect UV values like -5 or 'blah' there is no way to do that.

Fortunately, we can use Traffic Parrot to simulate the EPA API and return any values we need. We will create those simulators (also called mocks or virtual services) by recording HTTP SOAP traffic. We will use Traffic Parrot to record the SOAP request and response between the UV Index application and the EPA API. Then, we will modify the recorded response to simulate different value of the UV Index.

Simulating or mocking SOAP HTTP responses with Traffic Parrot

To do the recording, we will:

point Traffic Parrot recorder at the EAP API
point UV index application at Traffic Parrot

Recording SOAP HTTP responses using Traffic Parrot

To do the replay, we will stop the recorder and the SOAP response Using Traffic Parrot to simulate systems communicating with HTTP SOAP messages should be sent back whenever a matching SOAP request message will arrive.

Replaying recorded SOAP HTTP responses using Traffic Parrot

Recording HTTP SOAP

To do a recording we will need:

Java version 8 or above. You can download it here.
The UV Index application (our system under test). You can download it here (sources are available on GitHub).
We will also need Traffic Parrot

To record the traffic between the UV Index application and the EPA API using Traffic Parrot follow these steps:

Download and extract Traffic Parrot.
Download UV Index application and extract it
Start UV Index application by double-clicking the start.cmd script on Windows (or running start.sh on Linux)
If you now click on the "Get" button, the application will send a SOAP request to the EPA API with the given postcode and wait for a SOAP response.

Once it receives the response, it will show the result to you in the text area as shown below.
Now its time to record those SOAP responses. Start Traffic Parrot and open a web browser, navigate to http://localhost:8080 and open the HTTP record page:
Fill in the recording URL by typing in the address of the EPA server (https://us-central1-traffic-parrot-production.cloudfunctions.net), and click "Start recording".
Traffic Parrot is now recording any requests to http://localhost:8081 and forwarding them to https://iaspub.epa.gov
Now point the UV Index application instead of the real EPA API by changing the URL and the port number in the UV Index application. And click "Get".
Now, you should be able to see the response from Traffic Parrot.
When you clicked the "Get" button the UV Index application sent a request to Traffic Parrot, which forwarded it to the EPA API and recorded the response. You should be able to see that recorded request to response mapping now:
You can stop the recording now by clicking the "Stop recording" button. When the recording is stopped, Traffic Parrot will expect to receive requests on http://localhost:8081 and will return the recorded responses.
Click the "Get" button several times. All responses you see are coming from Traffic Parrot, not the real EPA API. We can now test the UV Index application in isolation without having to rely on the third-party EPA API servers.
Now we can simulate different values of UV Index and see how the application handles them. Click the edit mapping button in Traffic Parrot to open the edit mapping form:
You can see the recorded request and response. Click on the index VALUE in the response section to change its value.
Change the value to 7 and click "OK" and then "Save".
If you click the "Get" button the UV Index Application, you will see the index is now 7! So, we were able to test a hypothetical situation and simulate a given value of the index. You can also see that the text is now yellow, not green. That is because an index of 7 is very high and you should take care!
Now, lets test what happens if the index is 9. Change the value to 9 in Traffic Parrot and press the "Get" button again. The text is now in red!
We can also test index is outside of the range of valid values. Valid values are positive integer numbers. If you change the value to -5 you, will see the text is black, and the application says that index value is -5. We might have to consult this requirement with the designers of the application, but it should probably validate the UV index values and report an issue with the third-party service if the value is outside the correct range.
We can also test invalid index values. Change the value of the index to "blahh" and see what happens. We are getting an error message, but we can see that it is not descriptive. We should probably talk to the designers of the application and confirm how should we report issues with the third-party API because the current behaviour might be confusing to the users.
We also have a homework for you! There is a bug in the UV Index application. The correct colours for UV Index are defined in the UV Index Wikipedia page. Can you find the bug with text colour in the UV Index Application?
This way, using Traffic Parrot, we were able to mock (simulate) the EPA API and test different values of the UV Index.

Simulating dynamic mock responses

Have a look at the second half of the following video tutorial to see how to generate dynamic SOAP messages.

Then, you can familiarize yourself with the Dynamic Responses Tutorial which contains more details on different options of generating dynamic responses.

Chapter 2: How to record and replay HTTP SOAP requests and responses to create API mocks (API simulators)