Selenium is a multifunctional tool for automating ...



CS576

Leana Whitman

Analysis of the Selenium application

Selenium is a multifunctional tool for automating functional testing. Acceptance testing of web applications usually consists of manual tasks such as opening the browser, clicking on different web controls and links, typing text in the input fields and so on. These tasks are usually described in the test cases. If done manually, they are very time-consuming and prone to human error. That is where the tool such as Selenium comes in handy. Automating processes of development and testing allows complicated test suites to be executed several times a day which in turn allows developers to catch any problems sooner rather then later.

There are many other tools available for testing web applications. However, Selenium stands out because of its ability to simulate user-browser interaction. Selenium can open a browser and perform tasks as though it was a user. It is relatively simple to use, and allows both programmers and non-programmers to write test scripts. The execution of the Selenium test scripts can be completely automated and be a part of continuous integration.

Another important quality that makes Selenium so attractive to the developer/Quality Assurance community is its compatibility. It can be run in different web browsers (Firefox, IE, Safari, Opera and others) and on different operating systems (Windows, OS X, Linux, Solaris, and others). Tests can be created programmatically using any of the following programming languages: C#, Java, Perl, PHP, Python, Ruby and others.

There are several tools in the Selenium toolset. Selenium IDE is very simple to use and doesn’t require programming skills. Therefore, it can be used by non-technical users. This tool can be easily used for most static websites. Some testing tasks are more complex and require the use of Selenium-RC for programming logic. One of the examples of a case where Selenium-RC would be a better choice, is a web application where you need to iterate the list to test each item. Iteration is not supported by Selenium IDE. Another tool that is less frequently used is Selenium Core, which doesn’t support iteration or the switch between http and https protocols. The use of Selenium Core is therefore discouraged by Selenium community.

Static pages can easily be tested with Selenium IDE and the use of basic Selenium commands. Selenium provides a rich set of commands which are the core of a so-called “Selenese” pseudo-language. All of the commands can be divided into three categories: actions (support user actions such as clicking or selecting), accessors (verify the state of the application and store the results in variables), and assertions (verify the state of the application, i.e. “verify the title of the page is X”). The simplest tests can be created by running Selenium IDE and interacting with the web-site. As Selenium IDE starts, by default it starts recording a test script (this feature can be disabled). As the user starts clicking different elements of the page, such as links, buttons, checkboxes, each action is recorded in the test script. The testing of static web pages can include:

- Opening the URL – the command is openWindowAndWait

- Verifying or asserting that particular text is present on the page – the command used is either verifyTextPresent or assertTextPresent

- Verifying or asserting that particular element is present on the page – commands verifyElementPresent and assertElementPresent respectively. Selenium is a very flexible tool and allows the user to verify that an element exists in many different ways. One can verify that an element exists by providing an HTML tag that surrounds an element, by providing an ID of an element or by providing a name attribute of an element. In cases when element’s ID or name cannot be used, XPATH can be used to get the exact location of an element. In addition, user can check if the link exists. For example:

|verifyElementPresent |link=Masterpiece Portraiture |

Many commands have suffix “AndWait”. For example. openWindowAndWait or clickAndWait. clickAndWait will tell Selenium to open a new page and wait until the page is fully loaded. All of the Selenium scripts are saved in the HTML file where all tests are contained in the table with three columns. The columns correspond to Selenium command, target and value. For example,

verifyTitle

Commercial Photography and Design, Reading, Berks Pa

Commercial Photography and Design, Reading, Berks Pa

Other basic commands that can be used while testing a static page are verifyText, verifyTable, waitForPageToLoad, etc.

One problem that I have noticed while testing with Selenium IDE was with using the command openWindowAndWait. I noticed that even though a page seems to be fully loaded in the browser, when this command is executed, I get a “timed out” error: “[error] Timed out after 30000ms”. Subsequent tests fail as well. Other users in online forums reported similar problems and relate them to Selenium not being able to always properly handle loading of elements on pages with iFrames and pop-up windows.

As I mentioned earlier Selenium IDE does not support flow control. Iterations are supported only through Selenium RC using programming logic. Selenese does not support conditional statements. The commands can be useful in many cases, but if one needs to test dynamic pages and perform iterations, Selenese falls short. However, there is an extension that can be installed on top of Selenium IDE which provides flow control support. It can be downloaded here: . A simple example of how iterations can be done in Selenium IDE is shown in the following script:

label

loopPages

clickAndWait

link=Next

gotolabel

loopPages

When I tried to iterate through Google results pages with Selenium IDE, the following message was diplsayed on the Google page: “We're sorry...

... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.” This can be resolved in Selenium RC by inserting a random pause value between iterations.

Another issue I noticed with Selenium IDE was that I couldn’t run several instances of Firefox/Selenium IDE. The workaround for this is to create multiple user profiles and start each Firefox instance with a unique profile from the command line (start firefox.exe -no-remote -profile).

Selenium IDE cannot be used with a browser other then Firefox. If a user needs to perform cross-browser testing, Selenium RC needs to be used.

Selenium RC is a more powerful tool then Selenium IDE because it uses the power of a programming language, and can use a variety of programming languages. Whenever there is a need for conditional statements, iteration, logging and reporting results, database testing, etc. Selenium RC should be used. Selenium RC consists of two major components: Selenium Server and client libraries. Selenium Server can launch and kill the browser, execute any Selenese commands, execute an HTML file containing Selenium test script, and act as an HTTP proxy. It receives the Selenese commands from the testing application, executes them and sends the results back to the application. Client libraries provide an API between the programming language and the Selenium Server. Each Selenium command corresponds to a programming function that can be used to send the command to the server.

Selenium RC is very useful in testing dynamic web pages. It allows for conditional statements, such as checking if an element is present before accessing that element. If the element is not present, the appropriate message can be outputted, and the rest of the tests can continue to run. JavaScripts can be executed from the test application, which comes in handy when a page is populated with elements with no static IDs. The getEval method can be used to execute a java script which would get the IDs of the elements, and then the Selenium commands can be executed on those elements. In the example with (1 Dress, 1 Woman, 1World gallery), we could get an array of IDs for images displayed on the page, loop through that array and perform different tests on each image. The clickAndWait command could be used to test if the new page is loaded with a particular image in the center of the page, mouseOver – for testing that the color image was swapped with the black & white version of it, mouseOut – for testing that the image gets restored to the color version, verify alternative text, width and height of the images using getAttribute function, etc.

Some problems can arise when testing web applications that use Ajax technology. The intent of the Ajax technology is to make web pages more responsive to user actions. When a user clicks on a link or enters something on a form, instead of whole page being reloaded, only small amounts of data are requested from the server and refreshed on the page. Some tests will fail due to asynchronous nature of Ajax. For example, if clicking a button is supposed to change the value in the input field, with Ajax it doesn’t happen immediately. If the user has a test where the click command is used and then verifyValue, the verifyValue command will take place immediately after click command and the test will fail. When using clickAndWait instead, Selenium will wait for the page to reload. Since the whole page doesn’t get reloaded with Ajax, the test times out. Using pause command is good in some cases especially when we want to test non-functional requirements such as responsiveness of the application in certain environments. However, when testing UI controls in an Ajax application, using the pause command is not a good solution. The application under test might perform differently in different environments. Its responsiveness depends on many factors that we are not able to control. In some cases setting pause to 10000 might work, but in others it will fail. Selenium supports this by providing a command waitForValue. It makes Selenium wait until expected value appears and then continues with the test. There are many waitFor commands available in Selenium: waitForText, waitForTitle, waitForElementPresent, waitForVisible and so on.

Selenium has some limitations due to security issues. For example, it does not support testing of uploading a file. The JavaScript security model in browsers makes uploading files very difficult. It is by default not possible to have JavaScript fill in a path in a respective input field. There are several workarounds for this problem. One of them is to set the configuration option in Firefox called signed.applets.codebase_principal_support to true which allows non-signed scripts to request higher privileges.Selenium must request higher privileges. To enable typing into a file field the UniversalFileRead privilege needs to be enabled. It needs to be done in function Selenium.prototype.doType in the file selenium-api.js. This solution will work only on Firefox and to enable uploading of local files only. Another workaround is described at . Here Selenium is used in combination with AutoIt, which is another web testing tool, to upload the files.

Selenium is a great tool that can be used not just for acceptance testing but also for regression testing. Well-written tests will detect any broken functionality while running regression tests. With some web applications it is very important to run the same tests with various data. Using the same data over and over again might not detect problems. There are several ways to provide fresh data every time one tests. An external csv file could be created with values that need to be tested, or random values can be passed to tests. In the case, collections of values, which will be used for selecting a random value, need to be created. Unique values can be created using either timestamps or sequential counters. First, one would ensure that the tests run with no problems with small sets of data which is hard-coded. If the tests do what they are supposed to do with that data, one would start adding more and more variety to the values being fed to tests to ensure testing the application from different angles. If a programming language is used to test the application (not in Selenium IDE), then parameterizing tests can be done using native feature of the language. For example, if we need to ensure that a different user name is entered into the username field on a login form, we can create an array with as many items as we want, and randomly pick values from it using Math.random and Math.floor functions. In Selenium IDE the Math.random and Math.floor functions could be used in the following way:

type

q

javascript{'username'+Math.floor(Math.random()*100000)}

The array can always be modified later to fit our needs. If the number of items in the array is too big, then we might decide to go with another option – creating an external file which would hold the data. This option is especially convenient when we don’t want to make code changes in order to change the values to be tested. Those with no knowledge of programming could successfully run tests with various data sets using this method. When the uniqueness achieved by randomly selecting values from a pre-defined collection is not good enough, or when we need many more data sets for testing than we can possible record in the array or add to the file, timestamps can be appended to the string to give our data the required uniqueness. The following is the example for using a timestamp in Selenium IDE:

type

q

javascript{'user'+new Date().getTime()}

Parameterizing tests reduces the amount of duplicity contained in testing scripts and therefore increases maintainability of the testing application. While creating test cases, one always has to keep in mind how many changes he or she will need to do when the program’s user interface changes. How many scripts will need to be revised in order to test the program accurately? One should always account for changes and write the tests so that they don’t have to be revised every time something in the software changes. Just like in OO programming, anything that varies needs to be encapsulated. If we know that the sequence of commands might change in the future, and the failure or success of the test depend on it, we should encapsulate the sequence. If we know that the structure of the table to be tested might change in the future, we should write our code in a way so that we can change it without revising each single test case.

Good testing practices make a huge impact on the quality of the application. Software testing phase is one of the most important phases in the software development life cycle. There are many different tools available for writing automated tests. Selenium is one of the most popular ones. It has many features that make this tool attractive to the QA community. It does have its limitations as well, but workarounds can be found for almost any limitation. A variety of extensions and plug-ins are available to extend its functionality.

It is compatible with many programming languages which makes it easy to use. Its support for multiple browsers and platforms, and the fact that it is an open source tool which is free to use makes it a good choice for both experienced testers and novices.

References:

1) Selenium

2) "Improving the Maintainability of Automated Test Suites" by Cem Kaner (paper presented at Quality Week 97)

3) "Totally Data-Driven Automated Testing" by Keith Zambelich

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download