JMS vs. Native IBM® MQ

Traffic Parrot supports IBM® MQ brokers via JMS and native IBM® MQ APIs. If you are connecting to the IBM® MQ broker via JMS, please use JMS in Traffic Parrot as well and follow the JMS IBM® MQ instructions. If you are using native IBM® MQ APIs, please follow the instructions below.

IBM® MQ virtual service architecture

A tester or developer uses a web browser to access the console. The console manages the virtual service. The system under test (application under test) connects directly to the virtual service on different ports.

Here is an example of how that could look like for a scenario where the virtual service is replaying messages to an external queue.

Mocking IBM MQ architecture Virtual service replaying IBM® MQ messages to and from a queue

Configuration

Define connections

To record from or replay to brokers you will need to tell Traffic Parrot how to connect to them.

Those connections are displayed in the dropdown menus on the record and replay panels, for example: screenshot

The "Internal broker" connection is available in the dropdown by default when you choose to use an internal broker in the Broker panel.

To define a new connection that will be available in the dropdown in the record and replay panels:
  1. Open ibm-mq-connections.json file located in the main Traffic Parrot directory, using your favourite text editor
  2. Add the new connections (make sure the connectionId field has a unique value!). An example content of the file, with 2 connections defined:
    [
      {
        "connectionId": "1",
        "connectionName": "Test Payments Broker",
        "connectionData": {
          "ibmMqVersion": "IBM_MQ_7_5",
          "hostname": "mqserver.example.com",
          "port": 1415,
          "queueManager": "PAYMENT.QM",
          "channel": "PAYMENT.SVRCONN",
          "username": "payuser",
          "password": "paypassword222",
          "useMQCSPAuthenticationMode": false,
          "readConnectionsToOpen": 1,
          "writeConnectionsToOpen": 1,
          "sslCipherSuite": null,
          "sslPeerName": null
        }
      },
      {
        "connectionId": "2",
        "connectionName": "Local docker MQ 9",
        "connectionData": {
          "ibmMqVersion": "IBM_MQ_9",
          "hostname": "localhost",
          "port": 1414,
          "queueManager": "QM1",
          "channel": "DEV.APP.SVRCONN",
          "username": "app",
          "password": "",
          "useMQCSPAuthenticationMode": false,
          "readConnectionsToOpen": 1,
          "writeConnectionsToOpen": 1,
          "sslCipherSuite": null,
          "sslPeerName": null
        }
      }
    ]
  3. Supported values of ibmMqVersion are IBM_MQ_7_5, IBM_MQ_8 or IBM_MQ_9
  4. Set the value of useMQCSPAuthenticationMode to true if you require MQCSP authentication mode when connecting (e.g. for passwords longer than 12 characters)
  5. Save the file
  6. Refresh the page in Traffic Parrot to reload the dropdown list.

In the current Traffic Parrot version you edit connections directly in the ibm-mq-connections.json file. In near future you will be able to do it via the Web UI as well.

Change the location of ibm-mq-connections.json
If you would like to change the location of the ibm-mq-connections.json file, change the value of property
trafficparrot.virtualservice.ibmMqConnectionsUrl=classpath:ibm-mq-connections.json
to for example
trafficparrot.virtualservice.ibmMqConnectionsUrl=file:/home/john/git/project/trafficparrot-ibm-mq-connections.json
This can be useful if you would like to version control it with your application source code.
Required JARs

To connect to IBM® MQ you need jar files provided by IBM that will allow Traffic Parrot to establish connections with MQ.

Before you proceed please read these instructions to double check your actions are inline with the supported way to install WebSphere MQ Java jar files.

If you are running IBM® MQ version 8 and 9 you will need:
  • com.ibm.mq.allclient.jar
If you do not have the required JAR files yet, follow these instructions to get a copy:

If you have any issues with obtaining the jar files please contact us.

Copy those files to trafficparrot-x.y.z/lib/external and restart Traffic Parrot.

Startup replay script
The following property allows to specify a replay IBM® MQ script that will be run on startup
trafficparrot.ibmmq.start.queue.replay.on.startup.script=classpath:start-ibmmq-queue-replay-on-startup.txt
The script is in the same format as the one used in the web user interface.
Connect to the queue manager via SSL/TLS channels

To connect Traffic Parrot Native IBM MQ to an SSL/TLS channel, you need to configure the cipher suite and provide the certificates.

To configure the cipher suite, you need to update the connection definition in the ibm-mq-connections.json file, for example below we set the sslCipherSuite to TLS_RSA_WITH_AES_128_CBC_SHA:
[
  {
    "connectionId": "1",
    "connectionName": "Local Docker MQ 9",
    "connectionData": {
      "ibmMqVersion": "IBM_MQ_8",
      "hostname": "localhost",
      "port": 2414,
      "queueManager": "QM1_V8",
      "channel": "DEV.APP.SVRCONN.SSL",
      "username": "app",
      "password": "",
      "useMQCSPAuthenticationMode": false,
      "readConnectionsToOpen": 1,
      "writeConnectionsToOpen": 1,
      "sslCipherSuite": "TLS_RSA_WITH_AES_128_CBC_SHA",
      "sslPeerName": "OU=TP IBM MQ"
    }
  }
]

In the example above we are also configuring the sslPeerName which allows us to validate the server certificate DN.

To provide the server and client certificates, we will use a trust store and key store. The trust store should contain all certificates, be it CA or direct public keys to validate the IBM MQ queue manager certificate. The key store should include a client certificate that when requested by the IBM MQ queue manager SSL channel is signed by the approved issuer. The key store should also contain a private key associated with the client certificate.

You can configure both of these in the jvm.opts file by adding at the bottom:
-Djavax.net.ssl.trustStore=certificates/ca-chain.jks
-Djavax.net.ssl.trustStorePassword=trafficparrot
-Djavax.net.ssl.keyStore=certificates/mq-client.jks
-Djavax.net.ssl.keyStorePassword=trafficparrot
-Dcom.ibm.mq.cfg.useIBMCipherMappings=false

For the example above to work, you will need to create the certificates/ca-chain.jks and certificates/mq-client.jks JKS files with the required certificates in them.

Below you can find sample content of the trust store and the key store. In our example, the trust store contains a certificate authority chain of 2 certificates. The intermediate ca was used to sign both client and the IBM MQ queue manager certificates. The key store contains the client private key and a certificate signed by the intermediate ca.

trafficparrot@trafficparrot-pcs:~/Downloads/tp/trafficparrot-linux-x64-jre$ keytool -list -keystore certificates/ca-chain.jks
Enter keystore password:
Keystore type: jks
Keystore provider: SUN

Your keystore contains 2 entries

intermediateca, 13-Jul-2020, trustedCertEntry,
Certificate fingerprint (SHA1): 2E:FB:4F:8A:44:89:CD:FB:EE:25:5B:78:D0:D8:9B:89:A8:1E:1C:9E
rootca, 13-Jul-2020, trustedCertEntry,
Certificate fingerprint (SHA1): 13:D1:98:09:59:66:C7:64:F0:78:C3:8C:C1:C2:68:9D:76:F0:CF:34


trafficparrot@trafficparrot-pcs:~/Downloads/tp/trafficparrot-linux-x64-jre$ keytool -list -keystore certificates/mq-client.jks
Enter keystore password:
Keystore type: jks
Keystore provider: SUN

Your keystore contains 1 entry

1, 13-Jul-2020, PrivateKeyEntry,
Certificate fingerprint (SHA1): 21:9B:F6:A4:12:B0:31:1E:D6:29:B5:D3:C4:19:6F:ED:E4:C3:50:C2
Performance monitoring

You can enable additional logging that displays total processing time of messages.

To enable set property:
trafficparrot.ibmmq.monitorPerformance=true
You should then see INFO log lines that contain "Total processing time", for example:
2020-08-25 18:32:54,046 INFO  DEV.QUEUE.1-replay-responses-ibm-mq-message-scheduler-0 Request message '414D5120514D312020202020202020202E3C455F02FFE524' was received on '2020-08-25T16:32:54.004Z' from 'QUEUE:DEV.QUEUE.1'. Request message putDateTime is '2020-08-25T16:32:53.990Z'. Response message '414D5120514D312020202020202020202E3C455F02FEE524' was sent on '2020-08-25T16:32:54.045Z' to 'QUEUE:DEV.QUEUE.2'. Total processing time 41ms
You can increase the logging level by setting it to DEBUG or TRACE, change trafficparrotserver.log4j.properties (last line in the file):
log4j.category.com.trafficparrot.messaging.ibmmq.connection.monitor.ReportingIbmMqPerformanceMonitor=DEBUG

Recording and replaying IBM® MQ

Introduction

As with HTTP recording Traffic Parrot, during recording incoming and outgoing messages are matched up to provide mappings. Then upon playback receipt of a matching incoming message will trigger generation of an outgoing message.

Before you proceed to recording or replaying messages make sure you have installed the required libraries (required JARs) so that Traffic Parrot can connect to the broker.

Message matching

Traffic Parrot can form mappings in a number of ways.

  • Time: pair messages in the order they happened in time
  • Correlation ID: pair messages with matching correlation id

The default is to use time based matching. This can be changed in the advanced parameter section of the record page.

Configuration steps overview

Please see the diagram below for an overview of the steps required for configuring recording of messages between existing systems.

MQ recording configuration steps overview

Recording and replaying IBM® MQ queues using an external broker

Traffic Parrot does not support running IBM® MQ internally. You will need to use an external broker.

You can use one of the following:
  • You probably already have a test environment where there is a IBM® MQ broker available. You can use that IBM® MQ instance to set up your virtual service queues, which we have described below.
  • You can run a developer edition of IBM® MQ using Docker on your laptop, for more information see IBM® MQ for Developers
This diagram shows how two production systems connect:
Systems communicating via queues

One system generates messages onto a queue; another system consumes these messages and puts responses onto a second queue, which the first system consumes. If our goal is to test the system-under-test in isolation, we must record these interactions in order to replay them.

Recording IBM® MQ queues using an external broker

We will create extra queues on the existing broker that will be used by the virtual service. Then we will reconfigure the system-under-test to connect to these queues instead of the original ones. Traffic Parrot will move messages between these queues and the original queues, recording and creating mappings as it does so.

screenshot
Steps to perform a queue recording using an external broker
  1. Add the queues that will be used by the virtual service to the broker. In the example above you would add to the existing broker queues "Virtual Service Request Queue" and "Virtual Service Response Queue".
  2. Define connections that will be used for recording
  3. Start Traffic Parrot
  4. Go to the IBM® MQ record page
  5. Select "External" in the "Broker" panel
  6. Select "Queue"
  7. Choose the connections to brokers, and specify the queue name where messages will be received and responses sent. Follow the diagram Configuring recording queue messages using an external broker which will guide you how to populate those fields.
  8. Click "Turn on"
  9. Reconfigure System A to connect to the virtual service queues
  10. Execute a test case via System A to allow Traffic Parrot to record requests and responses. They will appear on the Messages and Mappings lists on the web page as shown on Mappings and messages lists during recording image.
  11. Click "Turn off"
screenshot

As the system-under-test generates messages they are listed in the bottom table 'Current recording session'. As the second system generates responses to these messages, they are also listed at the bottom, but in addition mappings are generated in the 'Mappings' table showing incoming and outgoing messages that Traffic Parrot has paired up.

Mappings and messages lists during a IBM® MQ recording

Replaying IBM® MQ queues using an external broker

To replay the recorded mappings we will need the same virtual service queues in place as shown on the image below.

Replaying queue messages using an external broker
Steps to perform a queue replay using an external broker
  1. Add the queues that will be used by the virtual service to the broker. In the example above you would add to the existing broker queues "Virtual Service Request Queue" and "Virtual Service Response Queue" (Note - depending on your broker and it's configuration these queues may be auto-created for you when the software first attempts to connect.)
  2. Start Traffic Parrot
  3. Go to the IBM® MQ replay page
  4. Select "External" in the "Broker" panel
  5. Select "Queue"
  6. De-select "Topic"
  7. Choose the connections as shown on the Configuring recording queue messages using an external broker diagram.
  8. Click "Turn on"
  9. Reconfigure System A to connect to the virtual service broker
  10. Execute a test case via System A to allow Traffic Parrot to consume a request message and send a response message.
  11. Click "Turn off"
Configuring replaying queue messages using an external broker

Recording and replaying IBM® MQ model queues using an external broker

Traffic Parrot does not support running IBM® MQ internally. You will need to use an external broker.

You can use one of the following:
  • You probably already have a test environment where there is a IBM® MQ broker available. You can use that IBM® MQ instance to set up your virtual service queues, which we have described below.
  • You can run a developer edition of IBM® MQ using Docker on your laptop, for more information see IBM® MQ for Developers
This diagram shows how two production systems connect when using model queues:
MQ model queues communication

One system obtains a temporary queue based on a model queue. It puts messages onto a request queue with the ReplyToQ set to the temporary queue. Another system consumes these messages and puts responses onto the temporary queue that was in the ReplyToQ field, which the first system consumes. If our goal is to test the system-under-test in isolation, we must record these interactions in order to replay them.

Recording IBM® MQ model queues using an external broker

We will create an extra request queue on the existing broker that will be used by the virtual service. Then we will reconfigure the system-under-test to connect to this request queue instead of the original one. Traffic Parrot will move messages between the virtual and original request queue, recording and creating mappings as it does so.

Both the system-under-test and the virtual service will use a model queue that exists on the existing broker. This model queue will be used to create a dynamic response queue. Traffic Parrot will move messages between the virtual and original dynamic queue, recording and creating mappings as it does so.

MQ model queues recording

Steps to record are similar to the usual recording steps but the response queues selected should be the same model queue name that exists in the external broker. In addition, the advanced parameter "Model response queues" should be set to "Yes":

Advanced recording parameters

Replaying IBM® MQ model queues using an external broker

To replay the recorded mappings we will need the same virtual service queues in place as shown on the image below.

MQ model queues replay

Steps to replay are the same as the usual replay steps. Response messages will be sent to the queue mentioned in th ReplyToQ field of the request message.

Recording and replaying IBM® MQ topics using an external broker

Recording and playback of topics using an external broker is less intrusive than queues because Traffic Parrot can subscribe to a topic just like any other application and receive messages, without affecting the delivery of those messages to other applications.

This diagram shows how the production systems connect:

Systems communication with topics

The system-under-test generates messages onto a topic which are received by a number of other systems. One of these systems generates responses onto a different topic (which are received by a number of systems, one of which is the system-under-test).

Recoding IBM® MQ topics using an external broker

We will configure Traffic Parrot to connect to the request and response topics and record the messages appearing on both, generating mappings as it goes, as shown on diagram Recording topics using an external broker

Recording topics using an external broker
Steps to perform a topic recording using an external broker
  1. Define IBM® MQ connections that will be used for recording
  2. Start Traffic Parrot
  3. Go to the IBM® MQ record page
  4. Select "External" in the "Broker" panel
  5. Select "Topic"
  6. Choose the connections to brokers, and specify the topic names where messages will be received and responses sent. Follow the diagram Configuring recording topics using an external broker which will guide you how to populate those fields.
  7. Click "Turn on"
  8. Reconfigure System A to connect to the virtual service queues
  9. Execute a test case via System A to allow Traffic Parrot to record requests and responses. They will appear on the Messages and Mappings lists on the web page as shown on Mappings and messages lists during recording image.
  10. Click "Turn off"
Configuring recording topics using an external broker
As the system-under-test generates messages they are listed in the bottom table 'Current recording session'. As the second system generates responses to these messages, they are also listed at the bottom, but in addition mappings are generated in the 'Mappings' table showing incoming and outgoing messages that Traffic Parrot has paired up.
Mappings and messages lists during IBM® MQ topic recording

Replaying IBM® MQ topics using an external broker

When replaying to an external broker Traffic Parrot will subscribe to the request topic and publish to the response topic as shown in the diagram Replaying topic messages using an external broker.
Replaying topic messages using an external broker
Steps to perform a topic replay using an external broker
  1. Start Traffic Parrot
  2. Go to the IBM® MQ replay page
  3. Select "External" in the "Broker" panel
  4. Select "Topic"
  5. De-select "Queue"
  6. Choose the connection as shown on the Configuring replaying topics messages using an external broker diagram.
  7. Click "Turn on"
  8. Disconnect System B from the broker so it does not produce response messages to the response topic
  9. Execute a test case via System A to allow Traffic Parrot to consume a request message and send a response message.
  10. Click "Turn off"
Configuring replaying topics messages using an external broker

Recording multiple queues

If you would like to record multiple queues at the same time, please use the recording script. See recording using a script for more details.

Recording using a script

You can use the "Script" tab to record advanced setups of multiple queues on multiple queue managers.
Recording using a script
Here is an example script to record multiple queues at the same time.
#
# This is a sample comment
#
QueueManager:'Local Docker MQ 9'
ProxyRequestQueue:'PROXY_PROCESS_PAYMENT'
LiveRequestQueue:'PROCESS_PAYMENT'
LiveResponseQueue:'PAYMENT_PROCESSESED'
ProxyResponseQueue:'PROXY_PAYMENT_PROCESSESED'


#
# This is a sample comment
#
QueueManager:'Local Docker MQ 9'
ProxyRequestQueue:'PROXY_CREATE_ORDER'
LiveRequestQueue:'CREATE_ORDER'
LiveResponseQueue:'ORDER_CREATED'
ProxyResponseQueue:'PROXY_ORDER_CREATED
You can also use the another script format with syntax:
#
# Order and payment processing
#
Record request messages from connection 'Local Docker MQ 9' and queue 'PROXY_PROCESS_PAYMENT' and proxy to 'AWS Docker MQ 8' and queue 'PROCESS_PAYMENT', and record response messages from connection 'AWS Docker MQ 8' and queue 'PAYMENT_PROCESSESED' and proxy to 'Local Docker MQ 9' and queue 'PROXY_PAYMENT_PROCESSESED'.
Record request messages from connection 'Local Docker MQ 9' and queue 'PROXY_CREATE_ORDER' and proxy to 'AWS Docker MQ 8' and queue 'CREATE_ORDER', and record response messages from connection 'AWS Docker MQ 8' and queue 'ORDER_CREATED' and proxy to 'Local Docker MQ 9' and queue 'PROXY_ORDER_CREATED'.
Record request messages from connection 'Local Docker MQ 9' and queue 'PROXY_MAKE_SHIPMENT' and proxy to 'AWS Docker MQ 8' and queue 'MAKE_SHIPMENT', and record response messages from connection 'AWS Docker MQ 8' and queue 'ORDER_SHIPPED' and proxy to 'Local Docker MQ 9' and queue 'PROXY_ORDER_SHIPPED'.

#
# Compute statistics
#
Record request messages from connection 'Local Docker MQ 9' and queue 'PROXY_COMPUTE_USER_STATS' and proxy to 'AWS Docker MQ 8' and queue 'COMPUTE_USER_STATS', and record response messages from connection 'AWS Docker MQ 8' and queue 'USER_STATS' and proxy to 'Local Docker MQ 9' and queue 'PROXY_USER_STATS'.

Replaying to multiple queue managers

If you would like to replay multiple queues to multiple queue managers at the same time, please use the replay script. See replaying using a script for more details.

Replaying using a script

You can use the "Script" tab to replaying advanced setups of multiple queues on multiple queue managers.
Recording using a script
Here is an example script to replay multiple queues on multiple queue managers at the same time:
#
# QM1 queue replay
#
RequestQueueManager:'Request QM1'
ResponseQueueManager:'Response QM1'
RequestQueueNames:'REQ_1_A','REQ_1_B'

#
# QM2 queue replay
#
RequestQueueManager:'Request QM2'
ResponseQueueManager:'Response QM2'
RequestQueueNames:'REQ_2_A','REQ_2_B'

Request message matchers

When Traffic Parrot receives a request message, it will try to simulate the system it is replacing by sending back a response message on the response queue or topic. To decide which response message to send, it will go through all the request to response mappings it has available to find the response to be sent. For more details how request matching works, see Request matching.

Request body matchers
Matcher name Matcher Id Description
any any Any request body will match.
equal to equalTo Check that the received request message body is equal to the request body specified in the mapping
contains contains Check that the received request message body contains the sequence of characters specified in the mapping
matches regex matches Check that the received request message body matches the regexp specified in the mapping
does not match regexp doesNotMatch Check that the received request message body does not match the regexp specified in the mapping
equal to JSON equalToJson Check that the received request message body is JSON and that it is equal to the request body JSON specified in the mapping
matches JSON matchesJson

Check that the received request message body matches (allowing for special wildcard tokens) JSON specified in the mapping.

Tokens allowed:
  • {{ anyValue }} - matches any value
  • {{ anyNumber }} - matches any whole or decimal number
  • {{ anyElements }} - matches any number of sub-elements (child-nodes)
For example a "matches JSON" request body matcher:
{
  "name": "{{ anyValue }}",
  "lastName": "{{ anyValue }}",
  "age": "{{ anyNumber }}",
  "children": "{{ anyElements }}"
}
will match a request body:
{
  "name": "Bob",
  "lastName": "Smith",
  "age": 37,
  "children": [{"name": "sam"}, {"name": "mary"}]
}
matches JSONPath matchesJsonPath Check that the received request message body is JSON and that it matches JSONPath specified in the mapping. For example, if we use the following expression as the request body matcher
$[?(@.xyz.size() == 2)]
it will match this request body:
{"xyz":[{"a":true}, {"b":false}]}
but will NOT match this one:
{"xyz":["a":true, "b":false, "c":true]}
equal to XML equalToXml Check that the received request message body is XML and that it is equal to the request body XML specified in the mapping
matches XML matchesXml

Check that the received request message body matches (allowing for special wildcard tokens) XML specified in the mapping.

Tokens allowed:
  • {{ anyValue }} - matches any value
  • {{ anyNumber }} - matches any whole or decimal number
  • <tp:AnyElements/> - matches any number of sub-elements (child-nodes)
For example a matches XML request body matcher:
<example>
  <name>{{ anyValue }}</name>
  <age>{{ anyNumber }}</age>
  <children><tp:AnyElements/></children>
</example>
will match a request body:
<example>
  <name>Sam</name>
  <age>29</age>
  <children><child name="bob"/></children>
</example>
matches XPath matchesXPath Check that the received request message body is XML and that it matches XPath specified in the mapping. For example, if we use the following expression as the request body matcher
/xyz[count(abc) = 2]
it will match this request body:
<xyz><abc/><abc/></xyz>
but will NOT match this one:
<xyz><abc/></xyz>

Edit IBM® MQ mappings

Introduction

On the following IBM® MQ pages, the edit button will allow you to edit an existing mapping:
  • Add/Edit
  • Record
  • Replay
Traffic Parrot supports the following MQ message formats:
  • MQFMT_STRING
  • MQFMT_NONE
  • MQFMT_RF_HEADER_1 and MQFMT_RF_HEADER_2
Currently, editing header values is not yet possible and only the message body can be edited. Please contact support@trafficparrot.com to be notified of when edit functionality is ready.

Response delay

Traffic Parrot supports postponing the delivery of a MQ message. This can be useful to better simulate a more realistic scenario where the responding system does not send a response message immediately after receiving the request message.

Use the Edit delay screenshot field on the edit mapping panel to specify the delay in milliseconds.

Priority

The request priority can be set in order to set up a preference order for matching mappings.

The highest priority value is 1. If two or more mappings both match a request, the mapping with the higher priority will be used to provide the response. The default priority is 5.

This can be useful, if you want a "catch-all" mapping that returns a general response for most requests and specific mappings on top that return more specific responses.

Text messages

Traffic Parrot is able to record and replay messages that could contain text in a variety of formats e.g. JSON, XML or plain text.

Edit MQ text mapping screenshot
Supported features
  • Edit the response body
  • Match requests using the request body matchers (e.g. JsonPath, contains, regexp)
  • Use priority to set up a preference order for matching mappings

Proof of concept with on-premises installation at a large enterprise

See proof of concept in the user guide.

Dynamic responses and custom extensions