For me, 2012 was a year full of challenges, setting up ways to meet peers (Facebook) and gaining experience in online identities, software testing and compliance .
This I want to continue in 2013.
Next year I want to deepen my security knowledge about online sharing protocols like UMA, OAuth2 , OpenID and my adventures (work experience, conference meetings) will be highlighted.
Also, I will continue to follow the news on privacy , big data and compliance and blog about this to express my views on these subjects, which will be combined in 2013.
Papers will be written, conferences will be visited and no worries, software and protocols will be tested.
All thoroughly done to give you a quality up to date repository about testing Software as a Service, with a flavor of online identities.
See you all in 2013 on Facebook, TestingSaaS-blog and Twitter!
And perhaps in real life too!!
Monday, December 31, 2012
Tuesday, September 25, 2012
Mobile payment ecosystems: a challenge for compliance testing
When you have read my blog lately you know I am very interested in compliance and software testing, especially for SaaS, NFC and mobile payments.
Since Google initiated Google Wallet I kept a close eye on what the compliance institutions were planning to do to develop testing programs for the emerging mobile payment ecosystems.
Especially because this is a new payment ecosystem, still unknown to the many merchants, acquirers and customers who are going to have to deal with it.
That it is susceptible to malicious attacks is illustrated by the POS (point of sale/checkout) attacks at different merchant locations in the USA, like Subway and Penn Station.
As always, the criminals know the weaker spots of the payment ecosystem the quickest and the compliance institutions are lagging behind, but have to react.
The compliance institute which is publicly expected to design countermeasures is the PCI Security Standards Council (PCI SSC). And recently it has 'done' that.
The PCI SSC has provided clarifications how every organisation (which stores, processes or transmits creditcard/debitcard data) should comply with the already devised PCI Data Security Standard (PCI DSS). A standard which should not be unknown to you if you read my former blog posts.
However, it are still clarifications about the PCI DSS, no update is planned yet until 2013.
Why clarifications?
Well, The entity which executes the compliance validation annually(!) is dependent on the amount of volume of transactions involved. If the volumes are small a Self-Assesment Questionnare (SAQ) is used. If the volumes are large, a Qualified Security Assessor (QSA) tests if the stakeholder is PCI DSS compliant.
When you read the PCI DSS you will encounter a lot of text, but no real details about how to test mobile payment ecosystems effectively (by QSA or through SAQ) resulting in an inadequate coverage. Especially what was part of the to be tested ecosystem (the 'scope') was insufficiently explained.
These and other points of concern are now written down in the Summary of 2012 Feedback for the PCI DSS and Payment Application Data Security Standard (PA-DSS). This document is a description of the international (!) feedback given by the PCI SSC stakeholders (merchants, acquirers, QSAs and payment software vendors) to the PCI SSC regarding PCI DSS v.2.0 and PA-DSS.
Regarding PCI DSS the feedback suggestions were mainly about the already mentioned need for scope guidance, a more detailed description of requirements, a more simplified SAQ and an update on password requirements. The last one is, regarding the current changes in identity management, an effective step in actualizing PCI DSS procedures.
Now I know what you're thinking: 'PCI SSC stakeholders, USA, is this also important for the European mobile payment system?'
Yes, it is. It's not a U.S. standard, it's a global standard, also affecting Europe, Africa and the Asia-Pacific (mobile) payment environments. The standard is a result of aligned policies from different U.S. Credit and Debitcard companies like MasterCard and VISA.
If these guidelines are not met annually, non -compliant merchants and acquirers meet its consequences: up to $500.000 fines and litigation costs, degradation to lower level PCI compliance and lower brand reputation and consumer confidence in the long term.
So, it is not amazing merchants and acquirers are giving feedback on guidelines they must comply to, but do not know how to comply to.
Let's see how the feedback from the PCI SSC stakeholders will result in a more testable, usable and , hopefully, more qualitative better PCI DSS standard.
Since Google initiated Google Wallet I kept a close eye on what the compliance institutions were planning to do to develop testing programs for the emerging mobile payment ecosystems.
Especially because this is a new payment ecosystem, still unknown to the many merchants, acquirers and customers who are going to have to deal with it.
That it is susceptible to malicious attacks is illustrated by the POS (point of sale/checkout) attacks at different merchant locations in the USA, like Subway and Penn Station.
As always, the criminals know the weaker spots of the payment ecosystem the quickest and the compliance institutions are lagging behind, but have to react.
The compliance institute which is publicly expected to design countermeasures is the PCI Security Standards Council (PCI SSC). And recently it has 'done' that.
The PCI SSC has provided clarifications how every organisation (which stores, processes or transmits creditcard/debitcard data) should comply with the already devised PCI Data Security Standard (PCI DSS). A standard which should not be unknown to you if you read my former blog posts.
However, it are still clarifications about the PCI DSS, no update is planned yet until 2013.
Why clarifications?
Well, The entity which executes the compliance validation annually(!) is dependent on the amount of volume of transactions involved. If the volumes are small a Self-Assesment Questionnare (SAQ) is used. If the volumes are large, a Qualified Security Assessor (QSA) tests if the stakeholder is PCI DSS compliant.
When you read the PCI DSS you will encounter a lot of text, but no real details about how to test mobile payment ecosystems effectively (by QSA or through SAQ) resulting in an inadequate coverage. Especially what was part of the to be tested ecosystem (the 'scope') was insufficiently explained.
These and other points of concern are now written down in the Summary of 2012 Feedback for the PCI DSS and Payment Application Data Security Standard (PA-DSS). This document is a description of the international (!) feedback given by the PCI SSC stakeholders (merchants, acquirers, QSAs and payment software vendors) to the PCI SSC regarding PCI DSS v.2.0 and PA-DSS.
Regarding PCI DSS the feedback suggestions were mainly about the already mentioned need for scope guidance, a more detailed description of requirements, a more simplified SAQ and an update on password requirements. The last one is, regarding the current changes in identity management, an effective step in actualizing PCI DSS procedures.
Now I know what you're thinking: 'PCI SSC stakeholders, USA, is this also important for the European mobile payment system?'
Yes, it is. It's not a U.S. standard, it's a global standard, also affecting Europe, Africa and the Asia-Pacific (mobile) payment environments. The standard is a result of aligned policies from different U.S. Credit and Debitcard companies like MasterCard and VISA.
If these guidelines are not met annually, non -compliant merchants and acquirers meet its consequences: up to $500.000 fines and litigation costs, degradation to lower level PCI compliance and lower brand reputation and consumer confidence in the long term.
So, it is not amazing merchants and acquirers are giving feedback on guidelines they must comply to, but do not know how to comply to.
Let's see how the feedback from the PCI SSC stakeholders will result in a more testable, usable and , hopefully, more qualitative better PCI DSS standard.
Saturday, July 28, 2012
A tester's nightmare: wrong testdata
Imagine the following situation:
A team of IT-professionals are working hard to get a GO for a release of a module of their online CRM software.
The architect designed the module, the developer creates the module and last, but not least, the tester has to do an end-to-end test to see if the whole chain is still working after the module is incorporated into the software.
As always, the end-to-end test is the finishing test, so the tester is on the critical path of the project. Better said, he has only 2 days to test and give a sound test advise of whether a GO/NO GO should be given.
Naturally, the tester prepared this test from the beginning.
While executing the testcases something weird happens: data is not migrated from the miodule to the other parts of the SaaS-landscape.
Awkward, because with other testcases this does not appear.
The tester thinks it's a bug, because the only difference with other testcases is the situation it wants to test and the expected result does not occur, so hee, it must be a bug. It's almost D-day and the tester has to go further with testing.
So, the tester documents the bug and communicates it to the team so the developer can pick it up and the tester continues the testing. Luckily it was not a showstopper.
The next day (the last day of testing!!) the developer comes to the tester and says it's not a bug, it's wrong testdata, which the database could not process.
After testing with the correct testdata the expected situation occurred.
Shoot, almost a day lost due to bad testdata.
What can we learn from this?
A lot of you would say the tester screwed up here,he's responsible, but is he?
No!
In the beginning of the story I described the project and its team.
And that's exactly where the responsibility lies, the team!
This Single Point of Failure (wrong testdata) would not have occurred if other teammembers would have reviewed the testcases and the used testdata.
Then the rotten apple in the testdata would have been removed early in the process, even before the testing started.
A tester reviews documents and tests software for a living, but we are also human beings and we too make mistakes.
I am now a tester for 8 years and every now and then I use bad testdata.
But I learned to get my testcases reviewed and diminish the risk of using bad testdata,stall the test process and unintentionally get the project in the danger zone.
Bottom line, when you are testing in a very short time period, make sure the whole team is aware of the importance of testing and let them review your testcases, so the described situation above does not occur.
Tuesday, July 17, 2012
Testing IAM: it's not only the identity that counts
For a month now I'm busy with testing an Identity and Access Management (IAM) solution build for a Dutch Energy company.
As you already might know by reading my blog, testing IAM is not only testing if someone can log in and out of an application.
No, there's more.
IAM stands for Identity and Access Management.
Identity management, a part of computer security, is about the management of identities and their authentication, authorization,and permissions within or across system and enterprise boundaries.
Access Management also relates to authentication and authorization, but includes Access Control too, which is perhaps a better word for it.
Next to this, Access Control is about measures such as physical devices (biometrics, encryption) and monitoring by humans and automated systems.
Trends like Bring Your Own Device (BYOD) and 'Het Nieuwe Werken' (work anywhere any place any time) makes Access Control (and Identity Management) challenging:
People want to work with their devices perfectly in a secure and interoperable way.
This not only affects IAM, but also the testing of IAM: more and more complex testcases which has to be generated and executed in a short time.
These are just some thoughts from me.
As you might have seen on Twitter or FaceBook I have started a TestingSaaS-FaceBook page where I want to interact with my blog readers and develop a social network dedicated to sharing knowledge on testing SaaS applications with a special emphasis on identity and security.
Why FaceBook?
I saw with blogging it was most of the time a one-to-one interaction, this social network could be an interaction where all blogreaders are involved and knowledge can be shared.
So, if you have thoughts about testing IAM, drop by at my TestingSaaS- FaceBookpage and let's get this network started.
Sunday, July 8, 2012
The real deal: testing enterprise identity and access manangement
After two years of weekendtesting identity protocols like UMA I am given the opportunity to show my testing skills in identity management in an international enterprise.
And it's not only identity management I have to test, it's IAM: Identity and Access Management. A difference I'll explain in a future post.
Multiple applications, multiple users and multiple devices (BYOD) and all in an international context: that's a challenge I want to take.
I will have to test the whole chain, so it will not be only to check if a user can log in. No, it demands a good functional and technical understanding of the chain and its points of failures. And I found out, it has to be good from the beginning.
It is a chain test, so no stubs or drivers are allowed and the testdata has to be fed in the enterprise personnel management system based on SAP HR. SAP I always wanted to learn, so here's my chance.
This will be an adventurous job, with every day different things in IAM to test: authentication, authorization, IAM-software, SAP, BYOD, Role Based Access Management are just a few. Did I already say I'll have to test this on my own with a lot of stakeholders and little information?
Enough challenges to conquer: Who dares wins!
The next posts I will share my experiences and give you an insight in the life of an IAM-tester.
Tomorrow it's Monday again, mucho trabajo!!
Multiple applications, multiple users and multiple devices (BYOD) and all in an international context: that's a challenge I want to take.
I will have to test the whole chain, so it will not be only to check if a user can log in. No, it demands a good functional and technical understanding of the chain and its points of failures. And I found out, it has to be good from the beginning.
It is a chain test, so no stubs or drivers are allowed and the testdata has to be fed in the enterprise personnel management system based on SAP HR. SAP I always wanted to learn, so here's my chance.
This will be an adventurous job, with every day different things in IAM to test: authentication, authorization, IAM-software, SAP, BYOD, Role Based Access Management are just a few. Did I already say I'll have to test this on my own with a lot of stakeholders and little information?
Enough challenges to conquer: Who dares wins!
The next posts I will share my experiences and give you an insight in the life of an IAM-tester.
Tomorrow it's Monday again, mucho trabajo!!
Monday, April 16, 2012
UMA Interop Testing at European Identity Conference 2012
Kantara Initiative's User-Managed Access WorkGroup (UMAWG)will reach a new milestone this month.
The UMAWG will be present at the European Identity Conference on April 17th in Munich (today).
It's mission: to show some UMA real-world examples during the Kantara Initiative Summit , which is chaired by my good friend Joni Brennan.
These examples include SMARTAM.org and a UMA-based app by Fraunhofer AISEC.
Before the UMA-show UMAnitarians have been busy with interop testing of the mentioned UMA examples.
Between 12.00 and 13.30 the UMAWG will share it's latest status, it's heritage with Oauth and OpenIDConnect and the status of the current implementations.
Unfortunately I can't be present today, but through blogs and tweets I will support my fellow UMAnitarians in answering questions and giving info on interop testing.
The UMA Interop won't be finished this day, it's just the beginning.
Because OpenID has a very good wiki on OpenID interop testing, OSIS, the UMAWG asked OSIS for help with setting up a UMA interop wiki.
With the help of the OSIS folks the UMAWG will make an effort to start interop testing of all available UMA implementations.
UMAnitarians and other UMA-interested people are invited to take a look at the OSIS-wiki and start interop testing their
UMA-based apps the OSIS-way.
Exciting times ahead for the UMAWG.
No worries, the quality of UMA is my gig, bugs are NOT allowed!
The UMAWG will be present at the European Identity Conference on April 17th in Munich (today).
It's mission: to show some UMA real-world examples during the Kantara Initiative Summit , which is chaired by my good friend Joni Brennan.
These examples include SMARTAM.org and a UMA-based app by Fraunhofer AISEC.
Before the UMA-show UMAnitarians have been busy with interop testing of the mentioned UMA examples.
Between 12.00 and 13.30 the UMAWG will share it's latest status, it's heritage with Oauth and OpenIDConnect and the status of the current implementations.
Unfortunately I can't be present today, but through blogs and tweets I will support my fellow UMAnitarians in answering questions and giving info on interop testing.
The UMA Interop won't be finished this day, it's just the beginning.
Because OpenID has a very good wiki on OpenID interop testing, OSIS, the UMAWG asked OSIS for help with setting up a UMA interop wiki.
With the help of the OSIS folks the UMAWG will make an effort to start interop testing of all available UMA implementations.
UMAnitarians and other UMA-interested people are invited to take a look at the OSIS-wiki and start interop testing their
UMA-based apps the OSIS-way.
Exciting times ahead for the UMAWG.
No worries, the quality of UMA is my gig, bugs are NOT allowed!
Saturday, February 4, 2012
UMA back to the future
The last 2 years I have done Some UMAnitarian work: Co-developing and evangelizing the User-Managed Access protocol.
My input: specify testcases and devise a interoperability testplan.
I also blog, tweet and FaceBook about UMA, counting THE retweets, blog and faceBook-likes & reads UMA is doing a good job.
Online UMA-marketing, I love it!
Back to testing.
Last year the UMA-specs were delivered to the IETF. A UMA-milestone, but also a wakeup-call for me.
Specs were changed, so my 2010 testcases were outdated.
Next to this, The UMA workgroup aka UMAWG, works along with OpenIDConnect for interoperability and changes in spec were necessary.
Also, more organizations got interested in implementing UMA, so it was back to the drawingboard.
However, waiting for a moment when less spec-changes were going on was wiser.
This moment has arrived and for the last two weeks UMA filled my evenings again: pseudocoding with megamugs of coffee.
Hopefully next week a draft can be sent to the UMAWG for review.
Also promising is the interoperability testing of The OpenIdConnect group (OSIS).
This interests me because of the re-usability of testmethods.
Well, that's my UMAnitarian work in a nutshell.
If this interests you, the UMAWG will be holding a UMA tweetchat, focusing on development, specs, interoperability and implementations of the UMA-protocol.
UMA tweetchat
8 February 2012
9-10am Pacific time
Tweethandle: #umachat
Ok, back to the specs for some good ol' pseudocoding
My input: specify testcases and devise a interoperability testplan.
I also blog, tweet and FaceBook about UMA, counting THE retweets, blog and faceBook-likes & reads UMA is doing a good job.
Online UMA-marketing, I love it!
Back to testing.
Last year the UMA-specs were delivered to the IETF. A UMA-milestone, but also a wakeup-call for me.
Specs were changed, so my 2010 testcases were outdated.
Next to this, The UMA workgroup aka UMAWG, works along with OpenIDConnect for interoperability and changes in spec were necessary.
Also, more organizations got interested in implementing UMA, so it was back to the drawingboard.
However, waiting for a moment when less spec-changes were going on was wiser.
This moment has arrived and for the last two weeks UMA filled my evenings again: pseudocoding with megamugs of coffee.
Hopefully next week a draft can be sent to the UMAWG for review.
Also promising is the interoperability testing of The OpenIdConnect group (OSIS).
This interests me because of the re-usability of testmethods.
Well, that's my UMAnitarian work in a nutshell.
If this interests you, the UMAWG will be holding a UMA tweetchat, focusing on development, specs, interoperability and implementations of the UMA-protocol.
UMA tweetchat
8 February 2012
9-10am Pacific time
Tweethandle: #umachat
Ok, back to the specs for some good ol' pseudocoding
Subscribe to:
Posts (Atom)