Sunday, November 27, 2011

BDD With Soap UI

The tool
Soap UI is a testing tool, primarily aimed at testing web services. This blog post is by no means a commercial for the tool, it is only a recap of my experiences with it. We use it extensively where I work, and it seems a good fit for both developers, testers and analysts.

One might argue that Soap UI is a narrow tool, only suitable for testing web services. That might be true, but breaking your architecture down into services is not limiting at all. It is a common way of enabling composability. Therefore I argue that having a tool that targets services in your BDD tool belt is not at all bad. Since its focus lies in services, much of the wiring is yours for free.

I assume that you know a little bit about Soap UI, at least its core constructs. If not, try the tool. There is a free version located at the Smartbear site. I will however point out that I am using the Pro version, which is not free. If you plan to do serious testing with Soap UI, buy that version.

Now this is not a post on BDD. Actually, I am quite new to the concept. But I strongly recommend reading the "Introducing BDD" post by Dan North or the "Specification by example" book by Gojko Adzic.  For the scope of this post, knowing the concept of Given-When-Then is quite enough. For the remainder of this post I will recklessly use the term BDD, even though you might as well use specification by example or other similar terms.

Mapping BDD to Soap UI
First we identify three common building blocks used when building software with BDD:
  • Feature
    The actual feature we are building, such as Login, Withdrawal or PublishBlogPost
    In Soap UI, this maps well to Test suite. For this particular feature, we will have a suite of tests running. 
  • Scenario/Example
    Examples of using the feature. Some will be "happy", like Succesful login. Some will be "sad", like FailedLogin or NoConnectionWithBank. 
    This maps well to the TestCase of SoapUI. We will build our suite using several test cases presenting different usages of the feature.
  • Step
    Now this is where we find the "Given-When-Then".
    Soap UI gives us the test steps that can be used for this level, even if we sometimes have to be a bit "clever" to keep it intact. 
An example
Lets assume that we have an authentication service in our architecture. Lets assume that it enables users to login and be authenticated. Yes - I am using the textbook example of login, but what else is business-agnostic like that...?

So, our first feature is Login. Let's create a test suite for it:

The next step is to identify a scenario and what gives more business value than "Succesful login"? We'll create a test case for it:

Finally lets create the test steps for this example:

There we are! Let's examine the steps in more detail.

The Given 
We use this step to setup the "Pam" user in the system. This might be a database operation, a set of service calls or something similar. I always try to create as much of the context from scratch so that I am not relying to much on data still being in the database/system. What I found quite common is to use the step type "Run test case". This gives us the possibility of reusing code between cases. I also use properties of this test case to configure this particular instance to setup a user named "Pam". If the test case gives me data back, such as a user id, I might use that to drive further testing.

The When
In a service-oriented architecture this will most certainly be a service call. We'll call a service using some parameters, such as UserId = "Pam". This was the only way I used Soap UI in the past. 
However, there is a tricky part involved. The normal way of things is to make assertions against the actual web service request, in the request test step. But that would mean that we would assert in our when step, which would break our BDD flow. Read on please...

The Then
This is where we want the assertions. We want the test case to fail in one of the then clauses, or even better - not fail... 
To enable this, we will use a groovy script step to assert against the response of the when step.  This is quite simple to do using the "get data" feature of Soap UI. And you don't need to know Groovy to do it!

Create a new groovy test step. Right-click in the code window and select "get data". Select the "When Pam logs in...." test step and the response.
This will bring up a window to select a property from the response. You will then be prompted to define a variable name. In our case, since we are using the "AuthenticationToken" property, lets call it something simple like authenticationToken.

This will bring in code to the window on the form of 
def authenticationToken = context.expand( '${WHEN Pam logs...

What the code actually says to the right is not that important at this point. You now have the returned token i a variable. Lets now assert that it holds the correct value.

assert authenticationToken != null, 'No authentication token was returned'

The assert statement holds three parts. The assert keyword, the condition (in our case that the token is not null) and a message that will be displayed if the condition is not met. This statement will give us the same type of assertion errors that a normal assert against a web service call gives us. 

I'll admit that this requires extra work and that it might not seem as straight-forward as the normal approach. But since it is the key to make the Given-When-Then syntax happen I am willing to accept the extra work. And its not that hard. More importantly, once you start to use groovy scripts in your tests, a whole new range of possibilities presents itself.

Also, it is not always required. Sometimes your then step will just be another service call that you can assert against in the normal way. The when might be a request to do something to the system and the then another call to make sure stuff happened. Using the groovy script approach is only needed when the evidence of then occurring is only located in the response.

Once we have these constructs in place it is easy to add more steps into the test, using "and". 
We can use this to add extra context in the given, perform multiple actions in the when or assert many things in the then. In my example I added an extra JDBC step to ensure that an audit of the login was stored in the database. 

In conclusion
Making this happen with Soap UI did not only satisfy my technical ego, it really opened my eyes. I am a passionate TDD guy, but my unit tests are not supposed to be understood by the "non-dev" crowd. Since a broader audince where I work are familiar with using tools like Soap UI, I find it very useful to communicate both requirements and the current state of things.

Another key takeaway for me is that the tool you select for implementing BDD or Specification by example is really not that important. You might use SpecFlow, FitNesse or your own custom code. At our company we already had invested time into learning Soap UI and that made it a practical choice. The tool is not important, the process of specifying requirements in the shape of test is.

I know this is not a full-fledged guide or tutorial but I felt really inspired when this easy mapping became apparent to me. I will dig further into both BDD and Soap UI since they are both great additions to my tool-belt. Whatever I find - I'll share.


  1. Very Nice blog it would surely help lot of Soap UI learners to know about the BDD methodology.

    Thanks a lot for putting this in your blog

  2. Simple yet interesting and effective.
    Will try in one of my projects :)

    1. Thanks for your comment! An updated verson of this post can be found at

  3. how do i get get data" feature of Soap UI. Its only available in SoapUI Pro..?? Is there any possibilities to make that happen?

  4. Hi

    I like your blog and love to share that i have created a framework in similar fashion which include all four condition Given - When - Then And Given - When - Then And..... (goes on in same way) in a single test case as required I mean might be not require AND in some test case.

    I agree that approach connect everyone in the project for example dev easy to understand what we execute and where it got fail and why, more over it also connect with Business team to give a clear picture.

    @Rohit: Yes "getData" option is available only in soapui pro for groovy step but in freeware soapui you can right the code to get data from SOAP step. For more you can

    Saurabh Gupta