What is Certify?

In the 2006 to 2007 timeframe, Worksoft made the strategic decision to tailor its automated software testing platform for the SAP experience. This well may be the tipping point for this relatively small company. They are gaining a lot of traction with some pretty major SAP customers. Here in St. Louis, there are already several Certify customers (here’s hoping a Worksoft Certify User’s Group will be in the making).

Among the testing tools that we looked at, Certify most closely matched our testing strategy – at least what we hoped our strategy would be going forward. Certify came across as having the advantage of allowing your business process experts (BPX’s) create and maintain the test cases for their areas of responsibility. Other testing platforms talked about Quality and Testing Engineers, and other dedicated resources that we were not prepared to secure.

Has Certify lived up to the ease of use we expected? Not quite. Let me qualify that. This is certainly a tool that the average BPX can become proficient in, however, there is a learning curve, and this bike requires a lot of practice to get good at riding it. We are finding that as BPX’s use the system more regularly, they are finding more uses for it in their daily work. They are dreaming up additional test cases and variations that give us much broader testing coverage than we’ve had in the past. Some of them actually like it . Certify is a complex and quirky tool that is made all the more complex because of the complexity of the patient – SAP. If SAP weren’t so complex, Certify would seem a lot simpler.

Does Certify Deliver the Replicator Experience?

Yes…once you’ve got the test cases built. That is the mountain you must climb to reach Valhalla. Our BPX’s are able to do Sales Tax testing on 50 states in a tiny fraction of the time that it was taking them in the past. We have people creating 300 line item PO’s with the push of a button. This ties up the computer for 20 minutes, by this was an hour+ of manual work before, and heaven help the person who flubbed up the downstream testing and requested another 300 line item PO.

The creation of test cases is not to be taken lightly, especially if you’re running some bolt-ons in your SAP environment. Hopefully we are blazing the trail for future Certify customers (thank you cards are accepted ) in the area of bolt-ons. We are running a number of them such as GuiXT, and it seems that these frequently have quirks that Certify is not prepared for. Worksoft has been very responsive in applying resources to help us through these problems. These usually result in a work around or in some cases a hot fix that gets applied to your clients.

Gaining user acceptance is a major feat, but it is definitely a battle worth fighting, no matter which testing platform you go with. The payoff is huge for your testing strategy. We have had the most success by dedicating 2 resources to the test case ramp-up phase. All of our BPX’s are busy with daily life in SAP, so adding the Certify learning curve and test case creation to their plates was not going to really work. We’ve had our BPX’s identify test cases to be created. Our Certify experts are scheduling time to work directly with the BPX’s to get these created. We’ve seen a complete turn around once we provided these resources to our BPX’s…knock on wood

A List of Issues to Consider

Here are some items we are/have been dealing with:

1. General testing strategy – what, when, where, why, and how to test our system…and how to document that
2. User acceptance
3. Test data management – no testing system will work with bad or unpredictable data
4. Test case maintenance
5. Possible Integration with SAP Solution Manager – Worksoft is building in Solution Manager integration…we are considering what that looks like and how we would use it, but haven’t gotten very far on this item
6. Tracking test case creation – I created a web application that lets us track the status of our test cases (are they built yet? do they work? are they scheduled to be created? when?). I’ve talked with Worksoft about the need for something like this. Whether you have a dedicated department of testers doing the creation or whether you have distributed that responsibility to your BPX’s, you will need some mechanism to track and manage your progress.
7. Integrating test results with your Support/Development Ticket system – whether you use Remedy or some other ticket system, it would be great to be able to document the Certify testing in that system…automagically

That’s the short list…

Final Words

Should you use Certify? Well, that’s begging for the age old consultant answer…I won’t even say it. Suffice it to say that Certify is a solid product, gaining some real traction among SAP users, and it’s getting better and better every day. The leap from Certify 7 to Certify 8 was like Windows 3.1 to Windows 95 (maybe that’s a bad analogy…forgive me, but you get the point). You should seriously consider how you want to test your system. Look at the alternatives on the market, and pick the best for you. We’re giving Certify a tentative 2 thumbs up. We’re still early on in our use, so anything else would be rash. Worksoft are interested in being a successful company, and they realize that there is really only one path to that…making their customers successful. We can attest to that.

Cheers!

Byron