Since I discovered how well BDD concepts fit into SoapUI, I have been trying to spread those ideas and get more people to write tests in the Given-When-Then syntax. It can be a big step but I believe it is worth it. Tests become so much more than tests, they become documentation and provide an easy way in for someone trying to understand what the system actually does.
Recently I have been doing a lot of exciting work in SoapUI Pro. I have created some custom test steps, guided by Ole Lensmar's excellent blog post, that made SoapUI more usable in more scenarios. I'll go in to that in another post, but for now I'll just say that SoapUI has now become my primary testing tool for many projects.
The point of documenting something about your system is to have someone else understand it. Even if I am definitely in the target audience (how well do you remember what you wrote last week...?) the main point is to make the documentation understandable by other stakeholders. That's where I think my idea of BDD in SoapUI started to crumble. I have sent several screen shots from SoapUI to customers, but that's not really a professional approach. I have also opened up SoapUI projects to explain, but in those cases the tool sort of comes in my way. Many stakeholders don't care about what tool I use. They just want the information, preferably as short and concise as possible.
Enter SoapUI reporting
Recently I discovered the reporting capabilities of SoapUI. The built-in reports are mostly concerned with displaying test results, performance and coverage. They are all good things, but what I was after was a report that could be used as documentation of the system, not a test result report. I am interested in red and green test cases, customers will assume that the system is working.
What I wanted was a specification that only contained my BDD features and scenarios with their given-when-then steps and some additional information. Plain and simple - I thought. I did have to battle both the SoapUI object model, my lacking Java skills and JasperReports, but I think I won. After two nights of battle, I have ended up with something that is not yet perfect, but shows some great promise!
An example
Lets say you have created a test suite for a feature in your system. I will use the "withdraw cash" example, from Dan Norths excellent intro to BDD, again and possibly exhaust it even further beyond its limit...
Basically, this project now contains one feature with two scenarios. One happy flow and one where the customer is rejected. Tests are written with the Given-When-Then syntax. And yes, they're all fake.
To be even more expressive, I have added a description to the test suite "Customer withdraws cash".
Now, I could give this screen-shot to the customer, but it would not be very...cool. It contains a lot of noise and may lead to discussions things like "why is there no Security Tests?" or "What do the symbols before test step names mean, why are some starred and some not...?". So, instead I'll open the project by double-clicking the "ReportingDemo" node, click the "Test suites" tab and then the "Create report" button:
If I select my report, and click ok - This is what happens:
Magic! Instead of a screen-shot, I can now hand this document to my customer. Away with all the noise, in with a clear and concise specification! The report actually iterates over all the test suites, test cases and test steps in my project. It adds some custom formatting, such as the bold given-when-then. It also finds the description of each suite and adds it to the report. It is actually dead simple. But that's what I think is so brilliant. This is exactly what I see in those SoapUI test suites, but for some stakeholders it is not that visible.
They don't need to know how I implemented the tests, just that I did and that they define the system that we agree on. And the best part is that this is not an external document, it's the actual tests - only formatted for the intended audience.
I think this is pretty great. How I did it?
Given that this blog post is already too long
When I consider adding the actual "how I did it" section
Then I'll put that in a blog post of its own
And put some pressure on myself to actually write it
And come off looking quite geeky writing this in Gherkin
Until then!
