Saturday, July 28, 2012

A tester's nightmare: wrong testdata

Imagine the following situation:
A team of IT-professionals are working hard to get a GO for a release of a module of their online CRM software. The architect designed the module, the developer creates the module and last, but not least, the tester has to do an end-to-end test to see if the whole chain is still working after the module is incorporated into the software.
As always, the end-to-end test is the finishing test, so the tester is on the critical path of the project. Better said, he has only 2 days to test and give a sound test advise of whether a GO/NO GO should be given.
Naturally, the tester prepared this test from the beginning.
While executing the testcases something weird happens: data is not migrated from the miodule to the other parts of the SaaS-landscape.
Awkward, because with other testcases this does not appear.

The tester thinks it's a bug, because the only difference with other testcases is the situation it wants to test and the expected result does not occur, so hee, it must be a bug. It's almost D-day and the tester has to go further with testing.
So, the tester documents the bug and communicates it to the team so the developer can pick it up and the tester continues the testing. Luckily it was not a showstopper.
The next day (the last day of testing!!) the developer comes to the tester and says it's not a bug, it's wrong testdata, which the database could not process.
After testing with the correct testdata the expected situation occurred. Shoot, almost a day lost due to bad testdata.

What can we learn from this?

A lot of you would say the tester screwed up here,he's responsible, but is he?

In the beginning of the story I described the project and its team.
And that's exactly where the responsibility lies, the team!

This Single Point of Failure (wrong testdata) would not have occurred if other teammembers would have reviewed the testcases and the used testdata.
Then the rotten apple in the testdata would have been removed early in the process, even before the testing started.

A tester reviews documents and tests software for a living, but we are also human beings and we too make mistakes.
I am now a tester for 8 years and every now and then I use bad testdata.
But I learned to get my testcases reviewed and diminish the risk of using bad testdata,stall the test process and unintentionally get the project in the danger zone.

Bottom line, when you are testing in a very short time period, make sure the whole team is aware of the importance of testing and let them review your testcases, so the described situation above does not occur.

Tuesday, July 17, 2012

Testing IAM: it's not only the identity that counts

For a month now I'm busy with testing an Identity and Access Management (IAM) solution build for a Dutch Energy company.
As you already might know by reading my blog, testing IAM is not only testing if someone can log in and out of an application.
No, there's more.
IAM stands for Identity and Access Management.
Identity management, a part of computer security, is about the management of identities and their authentication, authorization,and permissions within or across system and enterprise boundaries.
Access Management also relates to authentication and authorization, but includes Access Control too, which is perhaps a better word for it.
Next to this, Access Control is about measures such as physical devices (biometrics, encryption) and monitoring by humans and automated systems. Trends like Bring Your Own Device (BYOD) and 'Het Nieuwe Werken' (work anywhere any place any time) makes Access Control (and Identity Management) challenging:
People want to work with their devices perfectly in a secure and interoperable way.

This not only affects IAM, but also the testing of IAM: more and more complex testcases which has to be generated and executed in a short time.
These are just some thoughts from me. As you might have seen on Twitter or FaceBook I have started a TestingSaaS-FaceBook page where I want to interact with my blog readers and develop a social network dedicated to sharing knowledge on testing SaaS applications with a special emphasis on identity and security.

Why FaceBook?
I saw with blogging it was most of the time a one-to-one interaction, this social network could be an interaction where all blogreaders are involved and knowledge can be shared. So, if you have thoughts about testing IAM, drop by at my TestingSaaS- FaceBookpage and let's get this network started.

Sunday, July 8, 2012

The real deal: testing enterprise identity and access manangement

After two years of weekendtesting identity protocols like UMA I am given the opportunity to show my testing skills in identity management in an international enterprise. And it's not only identity management I have to test, it's IAM: Identity and Access Management. A difference I'll explain in a future post.

Multiple applications, multiple users and multiple devices (BYOD) and all in an international context: that's a challenge I want to take.

I will have to test the whole chain, so it will not be only to check if a user can log in. No, it demands a good functional and technical understanding of the chain and its points of failures. And I found out, it has to be good from the beginning.

It is a chain test, so no stubs or drivers are allowed and the testdata has to be fed in the enterprise personnel management system based on SAP HR. SAP I always wanted to learn, so here's my chance.

This will be an adventurous job, with every day different things in IAM to test: authentication, authorization, IAM-software, SAP, BYOD, Role Based Access Management are just a few. Did I already say I'll have to test this on my own with a lot of stakeholders and little information?

Enough challenges to conquer: Who dares wins!

The next posts I will share my experiences and give you an insight in the life of an IAM-tester.

Tomorrow it's Monday again, mucho trabajo!!