Testing SaaS, a necessity for both vendor and client

Last week I read Phil Wainewright's blog about a SaaS application with a very serious security breach.
Now you might think: 'What has that do do with testing SaaS applications?'.
Well, just read this part of his story and you will know:

'I suspect the root of the problem in Sage’s case was an unthinking assumption that Aqualogic was such an established Web platform that basic security would just be built in as standard. This is typical of the blind-leading-the-blind nature of the on-premise software model, in which customers blithely believe that vendors have built everything they’ll need into the platform, while vendors naively assume that anything they’ve missed will be easily spotted and corrected by customers during the implementation process. It’s bad enough when it results in catastrophic roll-outs at just a single company, but when the application is being deployed as a service to multiple downstream customers, a far higher duty-of-care is required, because the risk exposure is massively amplified.'

If Mr. Wainewright hunch is correct, this shows it is again all about communication between a software (read SaaS)vendor and client(s). Both parties rely so much on each other's testing process, blindfolded for both testing processes, believing everything is covered and 'ok'.

This example is bad for SaaS-marketing, but it is not the fault of the SaaS but a typical mistake of communication between vendor and client and also a risky time-to market damaging all parties connected to the SaaS-application.
A solution for such a mistake? YES!
Get rid of the barrier between the testing teams of client and developer and let both testing teams develop a strategy how to plan their tests and who covers what.
This narrows down the time to test because each party knows what they and the other testteam have to test and when to test.
This will allow a more efficient test process,covering all risks to be tested in a shorter time. Creating this way a shorter time to market enabling a better economic position for both SaaS vendor and client, giving SaaS a best practice.
It's a waste when innovation does not succeed due to bad communication.

Model based testing and SaaS, an example

In one of my earlier posts, I discussed the possibility of using model based testing (MBT) as a methodology for testing SaaS-applications.
I already discussed MBT is possible at a system test-level, not at acceptance test-level. MBT for complex software systems like SaaS is an area still evolving, and it could be a good idea what the current possibilities are.

Let's say a crack testteam from the fictional (!) company 'Einstein 2.0' has been assigned to do a system integration test or SIT for the company's SaaS-solution: 'ERP On Demand!'.

But first a short introduction to 'ERP On Demand!'.
This innovative product is a ERP-suite designed as an online ERP-dashboard for the enduser with all the benefits of web2.0(!):
By using the dashboard the enduser has secure access to its various ERP-resources (eg. CRM, HRM) through the internet and can change its settings by choosing from various modules given by 'ERP On Demand!'

The online dashboard 'ERP On Demand!' is for use as a service provided by 'Einstein 2.0' to customers on demand.
Inplementation of the software is not necessary, a good internet connection is enough, enabling the application to be used by the customer effectively from day 1.
For all this, the customer has to pay a monthly fee to the software vendor 'Einstein 2.0' so it's licensed to use 'ERP On Demand!' serviced by the vendor with the latter obliged to give 24h. secure service and maintenance.

This obligation is very essential for the testteam of 'Einstein2.0': 'ERP On Demand! should be online 24 hours a day with excellent performance and high security.
This addresses one of the issues associated with SaaS: how to deliver a safe B2B-application through the internet 24 hours a day??

From a tester's point of view these issues are nonfunctional: performance and security.
A model based testing approach could be an option, next to the available loadtesting and security testmethods.

The next weeks I will discuss this MBT-approach for performance and security testing of 'ERP On Demand!' in my blog.
Feel free to share your thoughts with me about the testing method MBT for SaaS.