Testing SaaS, a necessity for both vendor and client

Last week I read Phil Wainewright's blog about a SaaS application with a very serious security breach.
Now you might think: 'What has that do do with testing SaaS applications?'.
Well, just read this part of his story and you will know:

'I suspect the root of the problem in Sage’s case was an unthinking assumption that Aqualogic was such an established Web platform that basic security would just be built in as standard. This is typical of the blind-leading-the-blind nature of the on-premise software model, in which customers blithely believe that vendors have built everything they’ll need into the platform, while vendors naively assume that anything they’ve missed will be easily spotted and corrected by customers during the implementation process. It’s bad enough when it results in catastrophic roll-outs at just a single company, but when the application is being deployed as a service to multiple downstream customers, a far higher duty-of-care is required, because the risk exposure is massively amplified.'

If Mr. Wainewright hunch is correct, this shows it is again all about communication between a software (read SaaS)vendor and client(s). Both parties rely so much on each other's testing process, blindfolded for both testing processes, believing everything is covered and 'ok'.

This example is bad for SaaS-marketing, but it is not the fault of the SaaS but a typical mistake of communication between vendor and client and also a risky time-to market damaging all parties connected to the SaaS-application.
A solution for such a mistake? YES!
Get rid of the barrier between the testing teams of client and developer and let both testing teams develop a strategy how to plan their tests and who covers what.
This narrows down the time to test because each party knows what they and the other testteam have to test and when to test.
This will allow a more efficient test process,covering all risks to be tested in a shorter time. Creating this way a shorter time to market enabling a better economic position for both SaaS vendor and client, giving SaaS a best practice.
It's a waste when innovation does not succeed due to bad communication.

Model based testing and SaaS, an example

In one of my earlier posts, I discussed the possibility of using model based testing (MBT) as a methodology for testing SaaS-applications.
I already discussed MBT is possible at a system test-level, not at acceptance test-level. MBT for complex software systems like SaaS is an area still evolving, and it could be a good idea what the current possibilities are.

Let's say a crack testteam from the fictional (!) company 'Einstein 2.0' has been assigned to do a system integration test or SIT for the company's SaaS-solution: 'ERP On Demand!'.

But first a short introduction to 'ERP On Demand!'.
This innovative product is a ERP-suite designed as an online ERP-dashboard for the enduser with all the benefits of web2.0(!):
By using the dashboard the enduser has secure access to its various ERP-resources (eg. CRM, HRM) through the internet and can change its settings by choosing from various modules given by 'ERP On Demand!'

The online dashboard 'ERP On Demand!' is for use as a service provided by 'Einstein 2.0' to customers on demand.
Inplementation of the software is not necessary, a good internet connection is enough, enabling the application to be used by the customer effectively from day 1.
For all this, the customer has to pay a monthly fee to the software vendor 'Einstein 2.0' so it's licensed to use 'ERP On Demand!' serviced by the vendor with the latter obliged to give 24h. secure service and maintenance.

This obligation is very essential for the testteam of 'Einstein2.0': 'ERP On Demand! should be online 24 hours a day with excellent performance and high security.
This addresses one of the issues associated with SaaS: how to deliver a safe B2B-application through the internet 24 hours a day??

From a tester's point of view these issues are nonfunctional: performance and security.
A model based testing approach could be an option, next to the available loadtesting and security testmethods.

The next weeks I will discuss this MBT-approach for performance and security testing of 'ERP On Demand!' in my blog.
Feel free to share your thoughts with me about the testing method MBT for SaaS.

OpenID, an utopia for SAAS?

My last blog entry was about testing SSO. Now I want to discuss another access method for web-based applications: OpenID.
If you would believe the open source community, OpenID simplifies your online experience by using one single username for different websites. This is also the most important difference with SSO.
With SSO you have access to different sites by 1 single action of giving your username and password. With OpenID you log in with your username (OpenID URL) and password at one site and still have to enter your username to access other sites. This mandatory logging in with your username is less user-friendly than the single action signing-in of SSO, but can also be safer, because a person has always to fill in his username to have access to the websites of the SAAS-application.
Then why is OpenID not a common thing in the SAAS-community?
First, there is the security threat of phishing. For a B2B environment this can be a disaster. But there is another more simple explanation why OpenID is not so common used with SAAS.
Because OpenID is a single set of credentials to all sites supporting it.
That means all OpenID-credited sites have 1-to-1 agreements with each other and that also the different OpenID-providers are active on the same sites with each OpenID-provider authenticating access to the sites by users with other OpenID-provider.
This race for federated login is going on right now.
Maybe nice for B2C, but when considering B2B there are other issues to look after. Using OpenID, you have access to different websites using 1 URL.
In a B2B environment (SAAS) an employee has access to different websites, but he is restricted in his access due to the role(s) he has in the company. An employee can have many roles, making it impossible to squeeze all these different roles and corresponding access rights into 1 URL. Or the directory of OpenID should be coupled to a built-in authorization module from the clients intranet as I previously discussed in my previous post on SSO.
But why then use OpenID and not the SSO-solution Distal?

You might already guessed I am not so fond of using OpenID as a means of access management for SAAS-applications. I think more work should be done here to avoid phishing and authorization-problems. For the last problem, Microsoft claims to have the solution by combining its Windows Cardspace with OpenID. But that leads again to a manufacturarID and not a OpenID. Though I like to see the development of OpenID and Windows CardSpace, which seems to me a challenge to test considering the scenarios Microsoft gives here.

In my opinion, considering authentication, OpenID is, in comparison to SSO, still far away from becoming a suitable access management method for SAAS.
OpenID still has a lot of issues to attend which are less in SSO.
When looking at authorization, both access management methods lack a proper and safe authorization-module, so external solutions have to be sought in for instance built-in intranet authorization modules (federated SSO) or Windows Cardspace (OpenID).
So it's unfortunately not only testing SSO or OpenID, SAAS is a 'network' and should be tested as such!

SSO

SSO permits a user to login with a single action of user authentication and authorization(see also definition SSO Open Group).
So, with 1 action 2 different processes are executed: authentication and authorization. This makes it a very important access control mechanism in the identity management system of a company. A software tester who tests SSO should therefore split his testcase SSO in two testcases: SSO authentication and SSO authorization.
Otherwise he can't establish the origin of the defect when the SSO does not function properly.
To split the testcase SSO we have to look at the process of SSO and the parties contributing to this process.
First the process of SSO. Kjell Backlund from Emillion was very helpfull by sharing his knowledge on SSO and identity management with me.
He sees 2 possible SSO-configurations available for SAAS: web SSO and federated SSO.
With the help of SAML,an XML-based standard built in the SOA message , Web SSO is able to give an authentication message from one site (where the user is logged in) to the second site so the user can also log in on the second site. SAML can also function beyond the intranet where the user works on, because it is a webstandard.
Next to the user, XML and internet, 2 other parties are needed: the service provider and the identity provider.
Google Apps is using SAML this way in about 8-10 steps between the 3 parties. These steps can be evaluated by testing the different occuring SAML parsings and responses between the URLs of the parties. This is an elaborate process, and a lot of teststeps have to be taken.

One thing you have to take in mind:
SAML is just the messenger of the authentication information (it is XML), it does not perform authentication, nor authorization.It transports information from authentication authorities (eg. Active Directory) allowing identifying by for instance passwords or even biometrics.
This only shows the website the user is authenticated to log in.
A seperate rules engine, provided by the client, is necessary to evaluate attributes in the SAML-message if the user is authorized to enter the website.
This shows the different steps taken by SAML to tackle authorization and authentication.
Emillion has an alternative by ways of federated SSO, named Distal, where identity provider software can be eliminated from the process by server side scripting (ASP, Lotusscript, JSP) in the already available intranet of the SAAS customer.
From a testpoint of view this simplifies the testprocess because one of the parties is eliminated from the process.

But what about authorization and authentication when using Distal?
Distal can be integrated in web applications, application platforms and identity and access management solutions.The authentication messages can be built in the server side scripting of the clients intranet.
Authorization is more difficult, because it is a built-in feature of the clients intranet and with URLS this is difficult to tackle.
So, when considering Distal, a tester has a simpler testprocess considering authentication, because the identity provider is eliminated, although for testing authorization the tester needs the built-in authorization module from the clients intranet.

Concluding, SSO is a practical access control-method for SAAS-applications and different configurations are applicable.
For me, as a tester, Emillions Distal is a favorite because of its simpler test process for authentication. Considering authorization, this is still an issue for the SAAS client because the service provider has difficulties tackling the authorization rules by URLs.

Feel free to share your thoughts with me on this subject. I am very interested in the way Microsoft deals with SSO.
In my next blog-entry I will discuss the testing of another access management method for SAAS: OpenID

Access Control and SAAS: a comparison of OpenID and SSO

A SAAS-application can be seen as a B2B-ecosystem of different stakeholders.
For usability every stakeholder should have a method of access control allowing him to gain access to the different areas of the SAAS-ecosystem.
How is this possible and what are the risks?

Here authorization and authentication play a key role. As is seen on Wikipedia authorization (deciding whether to grant access) is a separate concept to authentication (verifying identity), and usually dependent on it.

Both concepts can be seperately tested in a SAAS-application.
In my next blog-entries I will illustrate this by comparing two ways of access control for a SAAS-application: OpenID and SSO
Both access-control mechanisms are different and have to be tested differently.
The key question here is: Can a user login in a webapplication, which acts as a access control-gateway, and have access to other registered member-webapplications without being prompted or causing errors?
And, not less important, when this user logs out of the system, does he or she still have any access to the other member-webapplications?

See you on my next blog-entry which will discuss the testing of SSO .

For now, good luck with making a quality SAAS-application!

And don't forget, feedback on my blog-posts are welcome.