Is Selenium worth the pain?
- Share
-
- |
Read later
My Reading List
Is Selenium worth the pain? Atlassian developer Nick Menere has asked that very question on the Atlassian Developer Blog. Selenium is a test tool for web applications that runs directly in a browser. In his blog post Menere looks at the roadblocks found while trying to use Selenium to test two new Ajax features of JIRA 3.10. The roadblocks included:
What do you mean by "production" ?
Selenium is a testing tool, by its very nature it doesn't get used in production.
Are you suggesting that developers should write Selenium tests for the purposes of development, but not run them as a regular test suite?
If so, then that seems like a very strange position to take.
I think that what Kumal means is that atm you cannot rely on these tests. You can definitely write and run them periodically, but you shouldn't bet your release on just the fact that your Selenium tests passed. At least this is the way I've read his message.
./alex
--
.w( the_mindstorm )p.
________________________
Alexandru Popescu
Senior Software Eng.
InfoQ TechLead&CoFounder
14
Discuss
Key Events
It turns out selenium.type(...) doesn't actually simulate a user typing into an input box. Go figure. So we typed in the text and triggered everykey stroke seperately - selenium.keyDown(...); selenium.keyPress(...); selenium.keyUp(...);. It worked!! in Firefox... In other browsers, it would now would print every character twice.
Timings
..Due to our non-selectable items (section labels etc.), we wanted to test that some elements attributes do not change. We fire a mouse event on the element, and then use an xpath locator to check that the element's attribute did not change ... it needed a slight pause between the event fire and test ... This was a very common fix for a lot of the things that would break - "Add/increase the pause".
Mouse Positioning Issues
To make things a little nicer for our users we decided to put in an autoscroll feature.... After putting in/increasing numerous pauses, I still couldn't get them to pass. Why would scrolling the screen cause the tests to fail? We fire events on selected elements - not co-ordinates. The build kept on breaking......I saw the mouse positioned directly in the middle of the screen, when the browser window scrolled, the mouse was positioned over one of the non-selectable items ... How to fix? We installed xwarppointer, to enable us to move the mouse pointer through bash, and tucked it away in a little corner.
Overall Menere learned a number of lessons. The most important of which was that the Selenium client is not a user.
Rate this Article
- Editor Review
- Chief Editor Action
Submit
Submit
Hello stranger!
You need to Register an InfoQ account or Login or login to post comments. But there's so much more behind being registered.Get the most out of the InfoQ experience.
Tell us what you think
Community comments
Selenium pains
by
Abbas Raza
Posted
Selenium is good
by
J House
Posted
Develop Selenium!
by
Kunal Shah
Posted
Re: Develop Selenium!
by
Tim Vernum
Posted
Re: Develop Selenium!
by
Alex Popescu
Posted
Yes - is it worth the pain
by
Reg Whitton
Posted
Re: Yes - is it worth the pain
by
Reg Whitton
Posted
URL for Blog Article
by
Eric Pugh
Posted
Re: URL for Blog Article
by
Deborah Hartmann
Posted
Try Sahi
by
Greg Willits
Posted
Re: Try Sahi
by
Alex Popescu
Posted
Selenium Woks!
by
Alvaro Gareppe
Posted
Re: Selenium Woks!
by
Judit Raj
Posted
Chickenfoot another contender
by
Alex Popescu
Posted
Selenium pains
by
Abbas Raza
We have been going through similar pains with Selenium. Sometimes Selenium fails without any deterministic reason. We could figure some issues within our test usage. However, there are other failures which are totally random and do not give any clue. Running from command prompt with maven resulted in test failures at different places than the ones that appear when tested from within eclipse. Arggh.
Selenium is good
by
J House
While I agree Selenium can be a pain to work with against ajax enabled pages, I personally think it is a great product
- it's free!!!
- it is not even on version 1 yet (0.9.0), can only get better
- API's exist for multiple programming languages
by the way the 'Atlassian Developer Blog' link in the article is busted - prefixed with 'link:'
- it's free!!!
- it is not even on version 1 yet (0.9.0), can only get better
- API's exist for multiple programming languages
by the way the 'Atlassian Developer Blog' link in the article is busted - prefixed with 'link:'
Develop Selenium!
by
Kunal Shah
Selenium is great! But not so great to put it into production. Developers can use it randomly to assist them. It fits with agile practices, but can not run reliably in production yet.
I am not sure why the development is so slow (no updates since dec 2006). With Web2.0 applications, so many languages and browsers, (functional) testing on a regular basis is necessary. I would suggest to develop Selenium without going into conclusions (like pain).
I am not sure why the development is so slow (no updates since dec 2006). With Web2.0 applications, so many languages and browsers, (functional) testing on a regular basis is necessary. I would suggest to develop Selenium without going into conclusions (like pain).
Re: Develop Selenium!
by
Tim Vernum
But not so great to put it into production
What do you mean by "production" ?
Selenium is a testing tool, by its very nature it doesn't get used in production.
Are you suggesting that developers should write Selenium tests for the purposes of development, but not run them as a regular test suite?
If so, then that seems like a very strange position to take.
Re: Develop Selenium!
by
Alex Popescu
Are you suggesting that developers should write Selenium tests for the purposes of development, but not run them as a regular test suite?
If so, then that seems like a very strange position to take.
I think that what Kumal means is that atm you cannot rely on these tests. You can definitely write and run them periodically, but you shouldn't bet your release on just the fact that your Selenium tests passed. At least this is the way I've read his message.
./alex
--
.w( the_mindstorm )p.
________________________
Alexandru Popescu
Senior Software Eng.
InfoQ TechLead&CoFounder
Yes - is it worth the pain
by
Reg Whitton
It is worth it.
I have used HttpUnit for a few years and that had some advantages. The best was that once you have written your test, you could start it on a great many threads and knock seven bells out of your server. This is not so easy with Selenium as it requires one instance of FireFox, IE or another to simulate one user.
However, writting the test in the first place is much easier with Selenium IDE, and keeping it upto date as the UI changes is quite easy too. With HttpUnit I quite often had to revert to debugging my unit test to see what was in the page that had been returned by the server.
OK using Selenium with AJAX isn't a smooth ride yet. I think selenium.type() will set the value of the node in the DOM instead of simulating key presses. However, one of the really nice things about Selenium is where it doesn't quite do what you need, it is very easy to extend. Create your own user-extensions.js, put the following in it, and set up the extensions file in the options:
You should now have a typeKeys command that individually enters the keys into the target. This can drive each browser differently, while keeping your unit test unchanged.
I have used HttpUnit for a few years and that had some advantages. The best was that once you have written your test, you could start it on a great many threads and knock seven bells out of your server. This is not so easy with Selenium as it requires one instance of FireFox, IE or another to simulate one user.
However, writting the test in the first place is much easier with Selenium IDE, and keeping it upto date as the UI changes is quite easy too. With HttpUnit I quite often had to revert to debugging my unit test to see what was in the page that had been returned by the server.
OK using Selenium with AJAX isn't a smooth ride yet. I think selenium.type() will set the value of the node in the DOM instead of simulating key presses. However, one of the really nice things about Selenium is where it doesn't quite do what you need, it is very easy to extend. Create your own user-extensions.js, put the following in it, and set up the extensions file in the options:
Selenium.prototype.doTypeKeys = function(locator, value) {
for(var i=0; i<value.length; i++){
var key = value.charAt(i);
if(browserVersion.isIE){
// whatever you need to do for IE
}
else { /* Mozilla? */
this.doKeyDown(locator, key);
this.doKeyPress(locator, key);
this.doKeyUp(locator, key);
}
}
};
>
You should now have a typeKeys command that individually enters the keys into the target. This can drive each browser differently, while keeping your unit test unchanged.
Re: Yes - is it worth the pain
by
Reg Whitton
On thinking about it - you probably shouldn't do both a keyPress and a keyUp. You should only need keyDown and a keyUp, any onKeyPress event handler would still be fired. May be this why you get double key strokes in other browsers to FireFox.
Try Sahi
by
Greg Willits
sahi.co.in/ -- from what I have seen, it's pretty good at behaving like a user
Re: Try Sahi
by
Alex Popescu
I must confess that at first glance Sahi looks impressive. Thanks for sharing.
./alex
--
.w( the_mindstorm )p.
________________________
Alexandru Popescu
Senior Software Eng.
InfoQ TechLead&CoFounder
./alex
--
.w( the_mindstorm )p.
________________________
Alexandru Popescu
Senior Software Eng.
InfoQ TechLead&CoFounder
Re: URL for Blog Article
by
Deborah Hartmann
First link in this item now fixed, thanks for pointing it out.
Selenium Woks!
by
Alvaro Gareppe
I've been working with selenium since last year and it´s great for web test.
It´s true that ajax have been a problem for selenium but the extensibility allows anything.
The posibility of implemet new commands have allowed me, and my company, to solve the problems implementing personalized comands.
Selenium: It works... and if it doesn´t you could make it work!
It´s true that ajax have been a problem for selenium but the extensibility allows anything.
The posibility of implemet new commands have allowed me, and my company, to solve the problems implementing personalized comands.
Selenium: It works... and if it doesn´t you could make it work!
Chickenfoot another contender
by
Alex Popescu
Another contender may be: Chickenfoot. Funny name, isn't it? :-).
./alex
--
.w( the_mindstorm )p.
________________________
Alexandru Popescu
Senior Software Eng.
InfoQ TechLead&CoFounder
./alex
--
.w( the_mindstorm )p.
________________________
Alexandru Popescu
Senior Software Eng.
InfoQ TechLead&CoFounder
