Wednesday, November 18, 2009

Testing SaaS, all parties included?

This week I was at a meeting in The Hague where a SaaS-solution in e-government (GovUnited) was discussed by people from science, companies and the Dutch government.
The idea is to develop and maintain a standardized website for Dutch cities, which can be customized per city for its particular needs. The maintenance will be done from a central place in the Netherlands and the cooperating cities (the customers) will pay the service-provider GovUnited a yearly fee for development and maintenance of their website.
Next to this, GovUnited can act as a intermediate between the cities and other parties, like e-payment services (eg.Ogone) or other government services and facilitate the connection between both e-services.

This makes it for me as a testprofessional interesting, because with all these different parties involved, who is solely responsible for the quality of the SaaS-product?
If the website is running, but one of the links to another party (like Ogone) is malfunctioning, who is responisble for this, Ogone or GovUnited? Or perhaps even the party hired to develop the website?
Each party can develop and test their component of the SaaS-product, but who is responsible for testing the SaaS-product as a whole. This multisystem integration test must be considered in the development + maintenance and can't be just be planned and executed at the end of development because if things get wrong then (and most times it will) it gets nasty and dirty for all parties involved.

So, a careful planning of development and test should be made between all stakeholders to ensure the deadline can be made with possible risks taken care for.


Joel Montvelisky said...

I see in this project 2 main areas that make it really interesting to test.

The first one is not connected to it being SaaS, and it is the fact that you need to test a Platform that will most certainly be fully customizable in order to fit the needs of each city.
On this account the tests are extremely varied and challenging, since you need to foresee as much variety as you can with regards to how each city will customize its site and how their users will access it and use (or misuse it as is many times the case)

The second area is the SaaS itself, where as you wrote you need to take into account all the infrastructure issues that may come from being responsible for hosting and maintaining this service for all your direct users (in this case the cities).

I've had significant experience on testing SaaS systems lately since we are developing a SaaS Test Management System called PractiTest, and many of my challenges as a tester were to adapt to all the new risks and tasks that are exclusive or at least augmented on these kind of systems. I wrote a summary of what we're doing on my blog.

Good Luck with your tests, it surely sounds like a fun challenge!

Cordny Nederkoorn said...

Hi Joel,

many thanks for the feedback.
Practitest seems a very nice SAAS solution for end to end testmanagement.
I will keep an eye on that :-)

Good luck too!