Online-selenium-trainings.webs.com



Selenium Tutorial : Testing Strategies

| |

[pic]

9. Testing Strategies

Selenium is able to test the webapp from a user perspective and so you can evaluate all of its aspects : navigation, controls, rendering, etc This broadness is a source of misbehaviors: you may be tempted to test all at once in the same tests. This is a bad idea you’ll end up with:

• A bloated and unmaintainable test infrastructure

• Test suites too slow to be exploitable

• Too much coverage is a source of brittleness

9.1. What Kinds Of Tests?

9.1.1. Acceptance tests

Acceptance testing (also named Exploratory Testing) means that your application conforms to the required level of quality and functionality the client expects

• In practice, it means that the business processes are implemented and functional according to specifications: it’s functional testing

• This is the kind of tests you should do with Selenium

• If you’re Agile, you can base your tests on your user stories (if you use that)

• This scenario integrates well with continuous integration frameworks and tools

9.1.2. Unit testing

Dan Fabulich, who is one of the creators of Selenium RC proposed another way to use Selenium: UI unit-testing

• The idea is to unit test your controls at compile/deploy time (in an ant or rake task for instance)

• It increases test coverage compared to acceptance testing alone while limiting the size of the test infrastructure

• Each component (for instance a calendar control) is isolated in a dumb html file on disk for high-speed tests

• Ajax calls are redirected to a dummy backend injected in the page by overriding the window.XmlHTTPRequest object

Figure 8. Unit testing UI components with Selenium RC

[pic]

1. Developer writes test-cases and targets html files embedding the control to unit-test

2. Selenium RC interprets the test-cases and commands the browser

3. Instead of targetting a webserver, the tests targets files on disk

4. The pages contains a script overwriting the XmlHTTPRequest in order to redirect backend calls to a dummy backend object

5. The dummy backend answers with whatever data is needed to test the control

6. Selenium RC evaluates the control' state

|[pic]|Note |

| |Some testing frameworks also provide server-side testing (eg: rspec view-testing in Rails) |

9.2. How Should You Test Your Apps

9.2.1. Testing For Missing Elements

• During your acceptance tests, you should test the presence of the elements before testing their functionality:

• describe "Google Search" do

•

• it "can find Selenium on Google" do

• page.open ""

• page.title.should eql "Google"

• page.type "q", "Selenium seleniumhq"

• page.click "btnG"

• page.value("q").should eql("Selenium seleniumhq")

• page.text?("").should be_true

• page.title.should eql("Selenium seleniumhq - Google Search")

• page.element?("link=Cached").should be_true

• end

•

end

• Could be improved in:

• describe "Google Search" do

•

• it "can find Selenium on Google" do

• page.open ""

• page.title.should eql "Google"

• page.element?("q").should be_true # we test the "q" field is present

• page.type "q", "Selenium seleniumhq"

• page.element?("btnG").should be_true # same for btnG

• page.click "btnG"

• page.value("q").should eql("Selenium seleniumhq")

• page.text?("").should be_true

• page.title.should eql("Selenium seleniumhq - Google Search")

• page.element?("link=Cached").should be_true

• end

•

end

This can seems trivial on this example but when your tests fails, you’ll immediately know what went wrong instead of trying to figure out if the problem is related to the data used for the test (the search text here) or to the element itself.

9.2.2. Navigation

• A frequent yet trivial source of errors is navigation:

o Broken links: links that contains a typo

o Missing pages: pages not available for whatever reason

• You could leverage the get_all_links method of the API to to that (assuming your links have ids):

• it "has all links working" do

• ...

• links = page.get_all_links()

• links.each do | id |

• if not id.empty?

• linkText = page.js_eval("this.browserbot.findElement('" + id + "').innerHTML") [pic]

• page.click("id=#{id}")

• page.title.include?(linkText).should be_true [pic]

• page.go_back [pic]

• page.title.should eql "Index"

• end

• end

end

public void testLinks() {

...

String[] links = selenium.getAllLinks();

foreach(String id : links) {

if(id != null) {

String linkText = selenium.getEval("this.browserbot.findElement('" + id + "').innerHTML;");

selenium.click(id);

verifyTrue(selenium.getTitle().contains(linkText));

selenium.goBack();

verifyEqual(selenium.getTitle().contains("title of the first page");

}

}

}

|[pic] |We get the link text |

|[pic] |We base our test on the assumption that the link’s text is contained into the title of the page they’re |

| |reffering to |

|[pic] |We go back to index.html |

|[pic]|Note |

| |The example code is at code/ruby/working_links.rb |

9.2.3. Dynamic Elements

Sometimes, you do not have control over your elements' ids (when using a web framework generating a list from a database for instance). In these cases:

• You can use xpath, dom or a structure-based locator (prefer CSS)

• Use the capabilities of the Selenium API with Selenium RC to handle loops, conditions etc

• Use the capability of Selenium to use JavaScript (see example above)

9.3. Best Practices

9.3.1. General rules

• Test-suites size

As a general rule, you should try to keep your test-suites small; they’ll execute faster and you’ll find bugs more easily:

o Test one application feature per suite

o Use one test-case per user-facing bug

• Testing time

All your Selenium tests should not take more than 10 minutes to run for a given application:

o Parallelize your tests

o Test only what you need

9.3.2. Locators

• Avoid text-matching pattern

o Instead of testing the presence of a text pattern, look for the element (or its id) containing that text (and THEN eventually the value)

o With i18n, your text-based locators are useless!

o Changes in the text breaks your tests

o What if the same text appears twice in the same page?

• Globbing and Regexps are here for you!

o Helps with generated ids

o You have access to all the features of JavaScript’s regexp implementation

9.3.3. Ajax

• Ajax calls don’t trigger a page load event and currently selenium detects only these events

• You could use click followed by a pause(time) but:

o It’s unreliable: you can only assume the call is complete by the time limit

o It doesn’t tell you the call had the assumed effect

• What you shouldg use are waitFor...(timeout) statements

o Every accessor has a waitFor flavor as well as a complementary one: waitForElementPresent has waitForElementNotPresent

o For most complex logic, you can use waitForCondition(script,timeout) that takes a javascript and keeps executing it until it evaluates to true or timeout is reached

o In ruby, use the js_eval(script, timeout) binding

Example of waitForCondition statement (Ruby). 

describe "Google Search" do

it "can display suggestions on the search bar" do

page.open "/"

page.title.should eql "Google"

page.element?("q").should be_true

page.type_keys "q", "Selenium" #[pic]

script = "var result;" +

"page = selenium.browserbot.getCurrentWindow().document;" + #[pic]

"var suggestionBox = page.getElementsByClassName('gac_od');" + #[pic]

" if (suggestionBox.visibility == 'hidden' && suggestionBox == null) {" + #[pic]

" result = false;" +

"}" +

"else {" +

" result = true;" +

"}" +

"result;"

page.js_eval(script, 30000).should be_true

page.element?("class=gac_a").should be_true #[pic]

end

end

Example of waitForCondition statement (Java). 

package ....

import ....

public class TestGoogleSearch extends SeleneseTestCase {

...

public void testGoogle() {

selenium.open("/");

verifyEquals(selenium.getTitle(), "Google");

verifyTrue(selenium.isElementPresent("q"));

selenium.typeKeys("q", "Selenium");

String script = "var result;" +

"page = selenium.browserbot.getCurrentWindow().document;" + #[pic]

"var suggestionBox = page.getElementsByClassName('gac_od');" + #[pic]

" if (suggestionBox.visibility == 'hidden' && suggestionBox == null) {" + #[pic]

" result = false;" +

"}" +

"else {" +

" result = true;" +

"}" +

"result;";

verifyEquals(selenium.getEval(script),"true");

verifyTrue(selenium.isElementPresent'class=gac_a")); #[pic]

}

}

|[pic]|We use type_keys instead of type to simulate keyboard typing instead of directly setting |

| |the value of the field |

|[pic]|The javascript executes in the context of Selenium Core so the JS’s window object refers |

|[pic]|to the monitor window. This call allows us to get the actual application window |

|[pic]|We retrieve the suggestion box element by its class name |

|[pic]| |

|[pic]|If the tag contains a visibility: hidden attribute, then the box is invisible |

|[pic]| |

|[pic]|Checking there is at least one element in the suggestion box |

|[pic]| |

|[pic]|Note |

| |the example code is at code/ruby/google_suggestions.rb |

9.3.4. Globbing & Regular Expressions

You can improve the flexibility of locators using these modificators:

9.3.4.1. Globbing

• Is used using the following syntax: glob:pattern

• Globbing uses wildcard-characters to match text with a pattern.

• It is a simple form of regular expression

• * represents any number of characters

• ? represents only one character

Table 2. Globbing example

|Pizza Test |

|… |

|assertTextPresent | |glob:*pizza* |

|… |

9.3.4.2. Regular Expressions (Regexps)

• Is used by adding the regexp prefix to your pattern: regexp:pattern

• Selenium uses the JavaScript implementation of regular expressions (Perl-based):

o any character matches its litteral representation: abc will match abc

o [ starts a class, which is any number of characters specified in the class

▪ [a-z] will match any number of lowercase alphabetical characters without spaces: hello, pizza, world

▪ [A-Z] does the same with uppercase characters

▪ [a-zA-Z] will do the same with either uppercase of lowercase characters

▪ [abc] will match either a, b or c

▪ [0-9] will match numeric characters

▪ ^ negates the character class is just after [: [^a-z] will match all but lowercase alphabetic characters

▪ \d, \w and \s are shortcuts to match respectively digits, word characters (letters, digits and underscores) and spaces

▪ \D, \W and \S are the negations of the previous shortcuts

o . matches any single characters excepts line breaks \r and \n

o ^ matches the start of the string the pattern is applied to: ^. matches a in abcdef

o $ is like ^ but for the end of the string: .$ matches f in abcdef

o | is equivalent to a logical OR: abc|def|xyz matches abc, def or xyz

▪ | has the lowest priority so abc(def|xyz) will match either abcdef or abcxyz

o ? makes the last character of the match optional: abc? matches ab or abc

▪ ? works in a greedy way: it will include the last character if possible

o * repeats the preceding item at least zero or more times: ".*" matches "def" and "ghi" in abc "def" "ghi" jkl

o *? is the lazy star: matches only "def" in the previous example

o + matches the previous item at least once or more times.

o {n} will match the previous item exactly n times: a{3} will match aaa

▪ {n,m} will match the previous item between n and m times with m >= n and is greedy so it will try to match m items first: a{2,4} will match aaaa, aaa and aa

▪ {n,m}? is the same but in a lazy way: will start by matching at least n times and increase the number of matchs to m

▪ {n,} will match the previous item at least n times

o Common regexps:

▪ \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3} matches an ip adress but will also match 999.999.999.999.

▪ (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?) will match a real ip adress. Can be shortened to: (?:\d{1,3}\.){3}\d{1,3}

▪ [A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4} matches an email adress

|[pic]|Note |

| |As JavaScript depends on the browser’s implementation, some regexps' features might not work (eg: \d doesn’t work on Firefox) |

9.3.4.3. Strict matching

• exact:string

o Will match a text verbatim, bypassing wildcards

o exact:*pizza* will match *pizza* and not my delicious pizza

|[pic]|Note |

| |if no prefix is given, Selenium will handle your pattern as a glob by default |

9.3.5. Security

• Use real SSL certificates

o Self-signed certificates triggers warnings on the browser

o Selenium can’t interact with these warnings: your tests will fail

o If you can’t have certificates signed by a CA, you can use the custom browser profile feature (Firefox only)

9.4. UI-Elements:

• In order to consolidate your tests, you should separate the test logic from the page description

• A model of one or more pages is created and contains the locators that targets your elements

• The test logic works against this model

• UI elements provides a way to abstract the elements of your pages into more meaningful locators

• You first need to create a pageset that will match the urls with your models

• The pagesets are injected in Selenium using the user-extensions.js file

• A pageset represents a page or a set of pages with common elements: it’s a template

• var map = new UIMap(); //[pic]

•

• map.addPageset({

• name: 'orders', //[pic]

• description: 'the elements common to the orders management pages', //[pic]

• pagePrefix: 'orders/', //[pic]

• pathRegexp: '(add|edit|view)order\\.html' //[pic]

• });

•

• map.addPageset({

• name: 'invoices',

• description: 'the elements common to all invoice management pages',

• paths: ['invoice.html', 'invoicereports.html'], //[pic]

• paramRegexps: { //[pic]

• action: '^(add|edit|view)$', //[pic]

• id: '^[0-9]{3}$' //[pic]

• }

});

|[pic]|We have to initialize a new UIMap object first - mandatory |

|[pic]|The name of the pageset - mandatory |

|[pic]|The description of the pageset - mandatory |

|[pic]|Optionally, you can give a prefix to the path of all included urls |

|[pic]|You can match the urls concerned by the pageset with a regular expression OR |

|[pic]|You can use an array of pages. Either one of these methods is mandatory |

|[pic]|It is also possible to match url parameters |

|[pic]|The parameter action has either one the values add, edit or delete |

|[pic]|The id parameter is a 3-digit number |

• Once we have the page set, we can add element mappings to our pages ( reffered to as UI-Elements)

• map.addElement('allPages', { //[pic]

• name: 'about_link', //[pic]

• description: 'link to the about page', //[pic]

• locator: "//a[contains(@href, 'about.php')]" //[pic]

• });

• // usage:

• // ui=allPages::about_link()

•

• map.addElement('allPages', {

• name: 'form_element',

• description: 'element from the register form by label map',

• args: [ //[pic]

• {

• name: 'label', //[pic]

• description: 'the form element to retrieve', //[pic]

• getDefaultValues: function() { //[pic]

• return keys(this._labelMap);

• }

• ],

• _labelMap: { //[pic]

• 'Name': 'user',

• 'Email': 'em',

• 'Password': 'pw'

• },

• getLocator: function(args) { //[pic]

• var label = this._labelMap[args.label]; //(11)

• return '//form/tr[contains(.,' + label.quoteForXpath() + ')'; //(12)

• }

• // usage:

• // ui=allPages::form_element(label=Name)

•

});

|[pic] |This UI-Element will be associated with the allPages pageset |

|[pic] |The name of the UI-Element - mandatory |

|[pic] |The description of the UI-Element - mandatory |

|[pic] |The locator used to match the element - mandatory unless getLocator() is defined |

|[pic] |We need to give the list of arguments accepted by the UI-Element (referred to as UI-Argument) |

|[pic] |The name of the UI-Argument - mandatory |

|[pic] |The description of the UI-Argument - mandatory |

|[pic] |A function of numerical or string values or an array of string or numerical values - use defaultValues if the default |

| |values set is static (e.g: defaultValues: ['a', 'b', 'c'] - mandatory |

|[pic] |A local variable (here an associative array). All local variables should start with _ and are accessible within the scope |

| |of the UI-Element with this |

|[pic] |You can also define the locator mapping as a JavaScript function to make it dynamic - mandatory unless locator is defined |

|(11) |We get the value associated by the key passed as an argument (label) |

|(12) |The getLocator function returns a string representing the locator |

9.5. Rollups

• A rollup rule is a set of Selenium commands that are grouped into one single command

• Rollups are expanded at runtime but a set of commands can be reduced into a rollup using the commandMatcher property or the getRollup method

• Rollup rules are defined in the user-extension.js file and are usable by both IDE and RC via the rollup command

• A rollup rule needs at least 4 mandatory components:

o name: the name of the rollup rule

o description: a description used for documentation

o A command matcher property/method: commandMatcher or getRollup(): this property allows the rollup engin to automatically infer a rollup from a list of commands. If the commands match the matcher’s definition, then the commands are reduced into a rollup

o A list of commands to execute: expandedCommands or getExpandedCommands()

• Optionally, there is also:

o An UI-Argument list: args. If none of your commands uses parameters, it is optional. They follow the same rule as with UI-Elements

o A pre and post property: is used to document the rollup rule

o An alternateCommand property: defines an alternate name for the first matched command. For instance, alternateCommand: 'clickAndWait' applied to a rollup will allow the RollupManager to build a rollup rule even if the first command of the commandMatcher property is click

• An example rollup rule:

• var myRollupManager = new RollupManager();

•

• myRollupManager.addRollupRule({

• name: 'navigate_to_subtopic_article_listing'

• , description: 'drill down to the listing of articles for a given subtopic from the section menu, then the topic itself.'

• , pre: 'current page contains the section menu (most pages should)'

• , post: 'navigated to the page listing all articles for a given subtopic'

• , args: [

• {

• name: 'subtopic'

• , description: 'the subtopic whose article listing to navigate to'

• , exampleValues: ['Food','Health','Technology','Economy']

• }

• ]

• , commandMatchers: [

• {

• command: 'clickAndWait'

• , target: 'ui=allPages::section\\(section=topics\\)'

• // must escape parentheses in the the above target, since the

• // string is being used as a regular expression. Again, backslashes

• // in strings must be escaped too.

• }

• , {

• command: 'clickAndWait'

• , target: 'ui=topicListingPages::topic\\(.+'

• }

• , {

• command: 'clickAndWait'

• , target: 'ui=subtopicListingPages::subtopic\\(.+'

• , updateArgs: function(command, args) {

• // don't bother stripping the "ui=" prefix from the locator

• // here; we're just using UISpecifier to parse the args out

• var uiSpecifier = new UISpecifier(command.target);

• args.subtopic = uiSpecifier.args.subtopic;

• return args;

• }

• }

• ]

• , getExpandedCommands: function(args) {

• var commands = [];

• var topic = subtopics[args.subtopic];

• var subtopic = args.subtopic;

• commands.push({

• command: 'clickAndWait'

• , target: 'ui=allPages::section(section=topics)'

• });

• commands.push({

• command: 'clickAndWait'

• , target: 'ui=topicListingPages::topic(topic=' + topic + ')'

• });

• commands.push({

• command: 'clickAndWait'

• , target: 'ui=subtopicListingPages::subtopic(subtopic=' + subtopic + ')'

• });

• commands.push({

• command: 'verifyLocation'

• , target: 'regexp:.+/topics/.+/.+'

• });

• return commands;

• }

});

|[pic]|Note |

| |A sample user-extensions.js file is included in the Selenium IDE extension and is accessible from Firefox at |

| |chrome://selenium-ide/content/selenium/scripts/ui-map-sample.js |

9.6. Lab 5: UI-Elements And Rollups

Using the sample user-extension.js provided with Selenium IDE:

• Build an user-extensions.js file targeting the google search page. Only two elements are needed: the search input box (id=q) and the google search button (id=btnG)

• Refactor the pizza test of Lab 1 using UI-Elements

• Group the commands into a rollup rule that take the search string as an argument

• Bonus: implement this test with RSpec and make it run against Selenium RC

|[pic] |Tip |

| |use the -userExtensions argument when launching the server |

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download