Sahi Best Practices for Large Projects

mjmckay

For large projects that will require complex tests and involve teams of people in the creation of the tests, can community members suggest any best practices in the design of the Sahi test architecture to maximize code reuse and the maintainability of the test code? Are there any pitfalls to avoid?

Your experiences and wisdom would be appreciated.

Mike.

jimschilling

First, design and write an automation test framework around Sahi. I have written a lightweight JavaScript library around Sahi that is utilized by our automation engineers. All Sahi functions have a corresponding library function. For example, we use XLIB.click() instead of _click. All our functions have the same argument signature as corresponding Sahi function. The wrapper functions allow us to add other functionality - logging, exception handling, validating existence/visibility without need to alter calling script.

Second, organize your scripts into some type of logical hierachy. We use the concept of Suites, Scenarios, and Test Flows. A suite can map to multiple scenarios. A scenario can map to multiple test flows. The test flows are effectively reusable components for scenarios. Along these same lines, we associate data with Scenarios and elements with test flows.

Third, consider different data/element repository formats. We use a series of different CSV formats to handle different types of raw data and different types of testing.

Fourth, convention and standards. Impose strict naming/coding standards for your team. Don't allow deviation. Long term maintenance is a must for successful automation. Be verbose and use self describing function names, variable names, etc. There are many good articles, books, blogs, and code that talk about the importance of this topic.

Fifth, create a custom results reporter. The Sahi log is good, but isn't practical if all you want is the result for each suite, scenario, and test flow.

SP

Can you please give more explanation on how we can create a custom results reporter in Sahi?

bharathi

Thanks for your information

jimschilling

Here's an example of what you do for reporting into a file formatted as an XML fragment (not a true XML document).

/**
 * namespace XLIB
 * function reportTestSuiteInfo
 * public
 * description
 * param {String} pTestSuite
 * param {String} pAppUnderTest
 * returns {Void}
 * exception {Void}
 */
XLIB.reportTestSuiteInfo = function(pTestSuite, pAppUnderTest) {

    var timestamp = XLIB.dateToString("yyyyMMMddhhmmss", null, "-", ":", "T");

    if (typeof(pTestSuite) == "string" && typeof(pAppUnderTest) == "string")
    {
        var $result = "<test-suite name=\"" + pTestSuite + "\" runid=\"" + XLIB._runUUID + 
           + "\" application=\"" + pAppUnderTest + "\" framework=\"Sahi\" timestamp=\"" 
           + timestamp + "\"/>\n";
         
        try {
           _writeFile($result, XLIB._xmlResultsFilename);  
        } catch (e) {
           XLIB.logException(e);
        }
   }
};


/**
 * namespace XLIB
 * function reportTestCaseResult
 * public
 * description
 * param {String} pTestCaseName
 * param {String} pWorkFlowName
 * param {Boolean} pResult 
 * returns {Void}
 * exception {Void}
 */
XLIB.reportTestCaseResult = function(pTestCaseName, pWorkFlowName, pResult) {   
   
   if (typeof(pTestCaseName) == "string" && typeof(pWorkFlowName) == "string" && 
       typeof(pResult) == "boolean")
   {
       var timestamp = XLIB.dateToString("yyyyMMMddhhmmss", null, "-", ":", "T");
       
       var $result = "<test-case-result runid=\"" + XLIB._runUUID + "\" timestamp=\"" + timestamp 
           + "\" name=\"" + pTestCaseName + "\" workflow=\"" + pWorkFlowName
           + "\" result=\"";
       
       if (pResult) {
           $result += "Pass";
       } else {
           $result += "Fail";
       }
       $result = $result + "\"/>\n";
       
       try {
           _writeFile($result, XLIB._xmlResultsFilename);
       } catch (e) {
           XLIB.logException(e);
       }
   }
};

RESULTS FILE CONTENTS:

<test-suite runid="66179f69-8289-489f-8013-da8e4920a45c" name="MyTestSuite" application="My Application v1.03" framework="Sahi" timestamp="2011-JUL-07T08:54:42"/>
<test-case-result runid="66179f69-8289-489f-8013-da8e4920a45c" timestamp="2011-JUL-07T08:54:42" name="Test Case 101" workflow="CreateSomething" result="Pass"/>
<test-case-result runid="66179f69-8289-489f-8013-da8e4920a45c" timestamp="2011-JUL-07T08:55:08" name="Test Case 102" workflow="EditSomething" result="Pass"/>

jimschilling

With the XML fragment, you can write a routine in JavaScript, Java, or other language to create a true XML document from the fragment or multiple fragments. Then you can create an XSL transform that can be used to create HTML output, text output, or other format.

The decision to use XML fragments is based on the idea that whether or not the test run completed, failure that resulted in abnormal termination, or the user stopped playback - the XML fragment will be still contain a valid fragment. I have seen other automation tools log/report in XML, but if some type of premature end of the run occurred, the XML was corrupted because it couldn't properly close all XML tags. HP's QTP is really good at corrupting the results.

globalworming

Hey guys.

Today I took some inspiration from this thread and some time to write down my thoughs on Sahi. I would really appreciate if you would have a look at my post and leave a comment about what I might see wrong, what items I should add and so on.

http://wiredworm.blogspot.de/2012/04/checklist-automate-tests-with-sahi.html

It is pretty unstructured an will be improved as I get more input and some insight, how someone without any Sahi experience would set up Sahi for testing a huge application with a large QA team involved.

When the post reaches a certain quality, I will link it from the Customer Stories and Testimonials area.