Tutorial: Record and replay HTTP requests and responses

For a basic introduction to HTTP record and replay with examples have a look at Record and replay HTTP requests and responses to create API mocks.

HTTP virtual service architecture

A tester uses a web browser to access the console. The console manages the virtual service. The system under test (application under test) connects directly to the virtual service on different ports.
screenshot

Simulators

What are simulators?

Simulators are virtual services that mimic the behaviour of a well known HTTP service. For example, simulating complex cloud based REST services that have stateful behaviour.

Amazon SQS

The Amazon SQS simulator can be enabled by setting the property trafficparrot.virtualservice.amazon.sqs.enabled=true.

To connect to the simulator, connect to Traffic Parrot on the http://localhost:8081/aws/sqs endpoint. The access key and secret key provided may contain any value. You can use the existing AWS SDK to connect.

Traffic Parrot currently supports the following SQS commands:
  • GetQueueUrl
  • SendMessage
  • ListQueues
  • ReceiveMessage
  • DeleteMessage
  • PurgeQueue

Traffic Parrot also has a web console to view outstanding messages and receipt handles:

Import and export HTTP mappings

How to export

First, go to HTTP in the top navigation bar and then click Export.

Click the button to download a ZIP file that contains all of the HTTP mappings in the WireMock 2.x format.

You can also export programmatically using the Traffic Parrot Management APIs.

How to import

First, go to HTTP in the top navigation bar and then click Import.

Click the button and select the file(s) to import using the file picker. Traffic Parrot will display the mappings that were successfully imported in the table below.

You can also import programmatically using the Traffic Parrot Management APIs.

Supported formats

Traffic Parrot has support for the following formats to import:
  • WireMock 2.x mappings in ZIP format
  • Swagger 1.x and 2.x (both YAML and JSON)
  • OpenAPI 2.x and 3.x (both YAML and JSON)
  • RAML 0.8

Dynamic responses

When importing from a specification format such as Swagger or OpenAPI, it is possible to use dynamic responses in the example responses in the specification.

Dynamic responses typically result in usage of the {{...}} notation that is not valid JSON or YAML. In these cases, you will need to specify the response as an escaped string when importing from JSON or YAML based formats such as Swagger and OpenAPI.

You can find examples of dynamic OpenAPI responses in the examples project.

Add/Edit HTTP mappings

Usage

First, go to HTTP in the top navigation bar and then click Add/Edit.

Fill in the Request/Response fields and click Save to configure a mapping.

You can also select from the HTTP skeleton dropdown which will populate the Request/Response fields for that skeleton.

After saving the mapping, it will appear in the list of mappings.

Clicking the edit button will allow you to edit an existing mapping.

Priority

The request priority can be set in order to set up a preference order for matching mappings. This works in the same way that priority works in WireMock.

The highest priority value is 1. If two or more mappings both match a request, the mapping with the higher priority will be used to provide the response. The default priority is 5.

This can be useful, if you want a "catch-all" mapping that returns a general response for most requests and specific mappings on top that return more specific responses.

Proxy responses for request passthrough

Traffic Parrot supports responding with a HTTP response including headers, body and status code, from another HTTP server.

This can be used to forward the requests you don't have mappings for to another endpoint.

You can use this to provide a default response for unmatched requests:

  1. Set a low priority like 10 for the mapping with the proxy response
  2. Leave default priority for other non-proxy response mappings
  3. Now any unmatched requests will return a response from the proxy mapping

Webhooks or callbacks

Traffic Parrot can also be configured to send an additional HTTP request after the mock response has been returned. This is called HTTP webhooks or callbacks.

It is possible to enable the callback based on a script evaluation condition. For example, this can be used to only send the callback if the mock response contains a particular success code.

Request matcher script

Another way to match HTTP requests is with a request matcher script. If the expression evaluates to true, then the request is matched, otherwise it is not matched.

For example, Handlebars templating can be used to match a request header using a regular expression:

HTTP skeletons

HTTP skeletons offer a quick way to generate a template of a mapping that matches an endpoint defined in an OpenAPI specification file. We may also add support for other specifications e.g. RAML in the future.

In order to edit the list of elements on the HTTP skeletons dropdown on the Add/Edit page you will need to place OpenAPI specifications into the file trafficparrot-x.y.z/openapi configuration directory. JSON and YAML OpenAPI files are supported.

Alternatively, you can use the button to import OpenAPI files. If you import a file with the same name as a file that was previously imported, it will be overwritten.

SOAP

Record SOAP and generate dynamic responses

Traffic Parrot helps with SOAP and XML mocking by providing an XML editor, the matchesXML request body matcher and dynamic response helpers such as xPath and xPathList.

Please see the video below for a full demo of recording SOAP requests and responses, and then generating dynamic responses.

Please download Traffic Parrot and the sample UV Index SOAP application and follow the demo in the video below.

If you cannot see a video frame below please download it here

Recording HTTP

Configuration

If your system requires the use of an outbound HTTP proxy (e.g. in a corporate environment) you will need to make sure Traffic Parrot was started using those proxy settings so that it can correctly record. See the configuration guide HTTP proxy settings.

If you need to record a system with a certificate that is not trusted by default, you will need to specify the trust store properties. See the configuration guide Outbound HTTPS certificates.

If you need to record a system that requires a client certificate for authentication, you will need to specify the key store properties. See the configuration guide Outbound HTTPS certificates.

Usage

First, navigate to the recording page by clicking HTTP in the top navigation bar and then click Record.

In order to record traffic to a URL, simply enter the Recording URL and click Start recording.

All traffic received by the Traffic Parrot virtual service will be proxied to the host in the Recording URL and recorded as mappings.

Clicking the edit button will allow you to edit the recorded mapping.

Existing mappings

If there are existing mappings present before a recording starts, these mappings will be used to return responses instead of recording a new response.

For example, if there is a mapping with path equal to /example then any traffic to path /example will return a response from the existing mapping and traffic to other paths will still be recorded.

Filter by URL path

If the Recording URL includes a path, only traffic to this path will be recorded, however all traffic will still be proxied.

For example, if the Recording URL is set to http://example.com/aSampleResource then only traffic to paths staring with /aSampleResource will be recorded and all other traffic will be proxied to http://example.com without saving mappings.

Filter by Content-Type

In order to record only specified content types go to trafficparrot.properties file and change the value of the property:
trafficparrot.virtualservice.recordOnlyContentTypesContaining=TP_RECORD_ALL_CONTENT_TYPES
to include content types you would like to record, for example:
trafficparrot.virtualservice.recordOnlyContentTypesContaining=application/javascript,application/xml

Recording multiple APIs

The Recording URL supports an advanced syntax to allow recording multiple APIs simultaneously. For example:

Record multiple HTTP URLs in Traffic Parrot

The path is used to determine which API should be used to provide the recorded response. The expression .* is used as a wildcard matcher.

In the example above:
  • /general/something will route to http://127.0.1.1:8090
  • /general/specific/something will route to http://127.0.1.1:8091
  • /something will route to http://127.0.1.1:8092

Recording headers for matching

In order to record HTTP request headers that can be later used for matching request go to the HTTP->Record page, click on "Advanced parameters" and fill in "Record request headers for matching" text area. It should contain a list of headers, one per line, for example:
screenshot

HTTP request matchers

When Traffic Parrot receives a request, it will try to simulate the system it is replacing by sending back a response to the client that sent the request. To decide which response to send, it will go through all the request to response mappings it has available to find the response to be returned. For more details how request matching works, see Request matching.

There are several matchers available to match HTTP requests, depending on the attribute.

Request URL matchers
Matcher name Matcher Id Description
equal to urlEqualTo Check that the whole url (including query parameters, etc.) of the request received is equal to the url specified in the mapping
matches regex urlMatching Check that the url of the request received matches the regexp specified in the mapping
path equal to urlPathEqualTo Check that the url path of the request received is equal to the path specified in the mapping
path matches regex urlPathMatching Check that the url path of the request received matches the path specified in the mapping
Request HTTP method (verb) matchers
Matcher name Description
ANY When matching a request, do not pay attention to the request method
GET Check that the method of the request received is GET
POST Check that the method of the request received is POST
PUT Check that the method of the request received is PUT
HEAD Check that the method of the request received is HEAD
OPTIONS Check that the method of the request received is OPTIONS
DELETE Check that the method of the request received is DELETE
CONNECT Check that the method of the request received is CONNECT
TRACE Check that the method of the request received is TRACE
Request headers
Valid values of request headers is a new line separated list of header keys and values, for example:
Accept: text/html
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
.
Request body matchers
Matcher name Matcher Id Description
any any Any request body will match.
equal to equalTo Check that the received request body is equal to the request body specified in the mapping
contains contains Check that the received request body contains the sequence of characters specified in the mapping
does not contain doesNotContain Check that the received request body does not contain the sequence of characters specified in the mapping
matches regex matches Check that the received request body matches the regexp specified in the mapping
does not match regexp doesNotMatch Check that the received request body does not match the regexp specified in the mapping
equal to JSON equalToJson Check that the received request body is JSON and that it is equal to the request body JSON specified in the mapping
matches JSON matchesJson

Check that the received request body matches (allowing for special wildcard tokens) JSON specified in the mapping.

Tokens allowed:
  • {{ anyValue }} - matches any value
  • {{ anyNumber }} - matches any whole or decimal number
  • {{ anyElements }} - matches any number of sub-elements (child-nodes)
For example a "matches JSON" request body matcher:
{
  "name": "{{ anyValue }}",
  "lastName": "{{ anyValue }}",
  "age": "{{ anyNumber }}",
  "children": "{{ anyElements }}"
}
will match a request body:
{
  "name": "Bob",
  "lastName": "Smith",
  "age": 37,
  "children": [{"name": "sam"}, {"name": "mary"}]
}
matches JSONPath matchesJsonPath Check that the received request body is JSON and that it matches JSONPath specified in the mapping. For example, if we use the following expression as the request body matcher
$[?(@.xyz.size() == 2)]
it will match this request body:
{"xyz":[{"a":true}, {"b":false}]}
but will NOT match this one:
{"xyz":["a":true, "b":false, "c":true]}
For more examples see the request matching documentation.
equal to XML equalToXml Check that the received request body is XML and that it is equal to the request body XML specified in the mapping
matches XML matchesXml

Check that the received request body matches (allowing for special wildcard tokens) XML specified in the mapping.

Tokens allowed:
  • {{ anyValue }} - matches any value
  • {{ anyNumber }} - matches any whole or decimal number
  • <tp:AnyElements/> - matches any number of sub-elements (child-nodes)
For example a matches XML request body matcher:
<example>
  <name>{{ anyValue }}</name>
  <age>{{ anyNumber }}</age>
  <children><tp:AnyElements/></children>
</example>
will match a request body:
<example>
  <name>Sam</name>
  <age>29</age>
  <children><child name="bob"/></children>
</example>
matches XPath matchesXPath Check that the received request body is XML and that it matches XPath specified in the mapping. For example, if we use the following expression as the request body matcher
/xyz[count(abc) = 2]
it will match this request body:
<xyz><abc/><abc/></xyz>
but will NOT match this one:
<xyz><abc/></xyz>

Virtualizing identical URLs

When using Traffic Parrot to virtualize multiple HTTP services at the same time, you may encounter namespace issues if the services have name clashes in the context path.

For example, you may have two services that both have a GET /api/resources endpoint, and one might be hosted at http://service1/api/resources and the other at http://service2/api/resources.

The Traffic Parrot virtual service http://localhost:8081/api/resources would have two mappings associated with it.

There are a number of ways to deal with this situation, one of which is detailed below.

Using custom headers

One solution to having the same name resource URL /api/resources for different services is to make a change the system under test and start sending a custom header when communicating with those services. For example, a request to http://service1/api/resources could include a HTTP header Service-Name: service1, and a request to http://service2/api/resources could include a HTTP header Service-Name: service2. That way the HTTP requests are different and Traffic Parrot can map them to different responses.

If you would like to capture those headers during a recording, you need to tell Traffic Parrot to record those headers.

You can also add those headers manually by editing the mapping.

Configuration

See configuration in the user guide.

Typical environments

See typical environments in the user guide.

Proof of concept with on-premises installation at a large enterprise

See proof of concept in the user guide.

Dynamic responses and custom extensions

Old version warning!

This documentation is for an old version of Traffic Parrot. There is a more recent Traffic Parrot version available for download at trafficparrot.com

Browse documentation for recent version