BDD With Soap UI

The tool
Soap UI is a testing tool, primarily aimed at testing web services. This blog post is by no means a commercial for the tool, it is only a recap of my experiences with it. We use it extensively where I work, and it seems a good fit for both developers, testers and analysts.

One might argue that Soap UI is a narrow tool, only suitable for testing web services. That might be true, but breaking your architecture down into services is not limiting at all. It is a common way of enabling composability. Therefore I argue that having a tool that targets services in your BDD tool belt is not at all bad. Since its focus lies in services, much of the wiring is yours for free.

I assume that you know a little bit about Soap UI, at least its core constructs. If not, try the tool. There is a free version located at the Smartbear site. I will however point out that I am using the Pro version, which is not free. If you plan to do serious testing with Soap UI, buy that version.

BDD
Now this is not a post on BDD. Actually, I am quite new to the concept. But I strongly recommend reading the "Introducing BDD" post by Dan North or the "Specification by example" book by Gojko Adzic.  For the scope of this post, knowing the concept of Given-When-Then is quite enough. For the remainder of this post I will recklessly use the term BDD, even though you might as well use specification by example or other similar terms.

Mapping BDD to Soap UI
First we identify three common building blocks used when building software with BDD:

• Feature
  The actual feature we are building, such as Login, Withdrawal or PublishBlogPost
  In Soap UI, this maps well to Test suite. For this particular feature, we will have a suite of tests running. 
• Scenario/Example
  Examples of using the feature. Some will be "happy", like Succesful login. Some will be "sad", like FailedLogin or NoConnectionWithBank. 
  This maps well to the TestCase of SoapUI. We will build our suite using several test cases presenting different usages of the feature.
• Step
  Now this is where we find the "Given-When-Then".
  Soap UI gives us the test steps that can be used for this level, even if we sometimes have to be a bit "clever" to keep it intact. 

An example

Lets assume that we have an authentication service in our architecture. Lets assume that it enables users to login and be authenticated. Yes - I am using the textbook example of login, but what else is business-agnostic like that...?

So, our first feature is Login. Let's create a test suite for it:

The next step is to identify a scenario and what gives more business value than "Succesful login"? We'll create a test case for it:

Finally lets create the test steps for this example:

There we are! Let's examine the steps in more detail.

The Given 

We use this step to setup the "Pam" user in the system. This might be a database operation, a set of service calls or something similar. I always try to create as much of the context from scratch so that I am not relying to much on data still being in the database/system. What I found quite common is to use the step type "Run test case". This gives us the possibility of reusing code between cases. I also use properties of this test case to configure this particular instance to setup a user named "Pam". If the test case gives me data back, such as a user id, I might use that to drive further testing.

The When

In a service-oriented architecture this will most certainly be a service call. We'll call a service using some parameters, such as UserId = "Pam". This was the only way I used Soap UI in the past. 

However, there is a tricky part involved. The normal way of things is to make assertions against the actual web service request, in the request test step. But that would mean that we would assert in our when step, which would break our BDD flow. Read on please...

The Then

This is where we want the assertions. We want the test case to fail in one of the then clauses, or even better - not fail... 

To enable this, we will use a groovy script step to assert against the response of the when step.  This is quite simple to do using the "get data" feature of Soap UI. And you don't need to know Groovy to do it!

Create a new groovy test step. Right-click in the code window and select "get data". Select the "When Pam logs in...." test step and the response.

This will bring up a window to select a property from the response. You will then be prompted to define a variable name. In our case, since we are using the "AuthenticationToken" property, lets call it something simple like authenticationToken.

This will bring in code to the window on the form of 

def authenticationToken = context.expand( '${WHEN Pam logs...

What the code actually says to the right is not that important at this point. You now have the returned token i a variable. Lets now assert that it holds the correct value.

assert authenticationToken != null, 'No authentication token was returned'

The assert statement holds three parts. The assert keyword, the condition (in our case that the token is not null) and a message that will be displayed if the condition is not met. This statement will give us the same type of assertion errors that a normal assert against a web service call gives us. 

I'll admit that this requires extra work and that it might not seem as straight-forward as the normal approach. But since it is the key to make the Given-When-Then syntax happen I am willing to accept the extra work. And its not that hard. More importantly, once you start to use groovy scripts in your tests, a whole new range of possibilities presents itself.

Also, it is not always required. Sometimes your then step will just be another service call that you can assert against in the normal way. The when might be a request to do something to the system and the then another call to make sure stuff happened. Using the groovy script approach is only needed when the evidence of then occurring is only located in the response.

And...

Once we have these constructs in place it is easy to add more steps into the test, using "and". 

We can use this to add extra context in the given, perform multiple actions in the when or assert many things in the then. In my example I added an extra JDBC step to ensure that an audit of the login was stored in the database. 

In conclusion

Making this happen with Soap UI did not only satisfy my technical ego, it really opened my eyes. I am a passionate TDD guy, but my unit tests are not supposed to be understood by the "non-dev" crowd. Since a broader audince where I work are familiar with using tools like Soap UI, I find it very useful to communicate both requirements and the current state of things.

Another key takeaway for me is that the tool you select for implementing BDD or Specification by example is really not that important. You might use SpecFlow, FitNesse or your own custom code. At our company we already had invested time into learning Soap UI and that made it a practical choice. The tool is not important, the process of specifying requirements in the shape of test is.

I know this is not a full-fledged guide or tutorial but I felt really inspired when this easy mapping became apparent to me. I will dig further into both BDD and Soap UI since they are both great additions to my tool-belt. Whatever I find - I'll share.