The quality engineering has gained the popularity as same way as software design and development and sometime more than some of the development technologies; as it is required for most of the application /systems. As being most recognized company in Quality engineering, we observed that most of the companies will have quality engineering department of their own even-though the software development is out sourced to other vendors.
Now a days, software test engineering (QA) is not just limited to test case managements with tools like QMetry etc. and test execution manually. But there are many areas like test automation, performance / load testing, security testing, SOA testing and code review with testing etc. being introduced and widely used by each departments at different level. In all of this, the test automation is very much popular as it reduces the regression test duration plus easily integrated with continuous build systems etc. to result in best ROI over the time.
There are many automation tools available which can be used by any manual test engineer for regression tools like Selenium IDE, Sahi etc.. Still once you start the automation, there will be some more demand from your QA Manager / Test Lead / Project manager / CTO about test results, debugging, reusability, continuous integration, maintainability, execution from cloud etc. and that will required you to develop the framework using scripting language / programming language.
Amongst big group of automation tools, selenium is the only tool which supports many programming languages ranging from simple like HTML to complex like Ruby/Java/C# to automate the test bed of your web application. As selenium supports variety of languages for automation, manier get confused to select the suitable language for automation. Mainly the selection of laungage depends on many different crieterias as well. Therefore to overcome from this delta situation, I tried to get input from different big players in automation world about the preferred language and based on which crieteria they select that etc. We had run the poll for same for quite a month long time and able to get good information as descried in following sections of this post.
Overall Result:
We can see that JAVA got around 77% votes from total 194 votes worldwide. The HTML, PHP and Javascript etc. was not part of selection but few peoples are using this for their test automation.
We have got some language selection criteria based on the feedback during this poll are:
- SUT development environment
- Open Source Tool
- Material/libraries available for language
- Performance and Transaction Time
- Easy to code, IDE
- Language exposure to automation engineer
Etc.
If we have to rate these criteria’s, then the first point will win the race and most of has voted based on this criteria.
Brainstorming: Based on first preference and poll data, can we think that JAVA is most popular in development as well!!
The consolidated feedback and related information is as below.
1. SUT development environment
Many time, we (auto architects) think to use the language as same as the application under test development platform/language. Because it help us to use the same tests as unit tests and helps us for customization etc.
“I prefere Python because I like this language and I used if for wide range of tasks before I start automation with Selenium. Python has everything what I need for any purposes. But Current project based on PHP, so one of requirement was that automation will also based on PHP” — how can you leave your preference.
Helps in unit testing — “ Being able to write Selenium tests in JUnit allows me to do end-to-end testing of our server-side Java application and browser clients. So I can call into a backend Java application(handy for me when doing test setup and cleanup) and control the browser from the tests. I can also easily take automated screenshots of the browser view.”.
Further from other expert, “It really depends on what your goals are. Personally I prefere using the language the app is written in. Of the benefits its nice to be using the same language and tool chain the developers are using to make collaboration and integration points simpler.”
Manish says, “I’m working in a product that uses Selenium RC with C# and codedUI and I’m enjoying it and the reason of choosing c# is the same app written in .Net”
More specific quote “For your tests to be economically maintainable, they need to be in a Turing-complete language (e.g. not HTML-based). That said, assuming either: 1. There’s no dedicated QA team and testing is an integral function of Development or 2. there’s a dedicated QA team that works closely with development, the only responsible choice for your scripts is in the language of the application under test. Otherwise, cooperation with Development for test case issue resolution will be difficult as not all members may be familiar with the script language.”
2. Open source Tool
We all know that why JAVA won the race, because it is open source.
“Eclipse with JAVA and selenium is free, however .NET IDE is not free. There is an open source .net IDE called mono that could be used with selenium;”
Ratankumar adds “ I’d prefer JAVA, as it is also open source and easily available for download.”
3. Material/Libraries available for language
The availability of the documentation / samples / tutorials can matter the selection of the language for your selenium (others as well) automation projects very well because not all test automation engineers / architectures will have strong programming language experience. So this type of supporting material helps them for quick start. Selenium is the very best automation platform which gives us the wide variety of language selection for automation and which make ease for the new people to quick start with the help of this kind of supporting material.
“I favor Perl because I can use the martial from CPAN.org to build a framework. I have lot of modules that I can interface to the selenium api” — Perl selection for automation based on this criteria.
Kshitij describes in same line “ I prefer java/junit with Eclipse, due to easy available solution on web and most other plugins and technology for automation is available in Java.. So its easy to integrate with selenium.”
C# Documents seems ahead of all, “ C# has a ton of documentation and tools that can complement your script development efforts.”
Do you think same “ There’s much less user support for Perl, but having that extensibility opens up many avenues. I’m looking forward to checking out the Webdriver Perl bindings.”
4. Performance and Transaction times
Sometimes we think on this before starting the automation development.
Selenium can perform well with Java. Do you agree with “Selenium is built on JAVA and runs through a JVM. So I suggest using the language that selenium was created in.”
5. Easy to code, IDE
Definitely this is the easiest criteria as well, but test automation sometimes required customization in results presentation, looping based on data etc. So I am not sure that whatever language we will define as “easy to code” will be good selection for test automaton. Still we can find some other supporting interfacing language etc. like HTML is very easy so try to use Java Script extensions with this will definitely help you to keep it easy to code plus some level of customization. Let me describe same thing in users’ words: “I prefer html tests/cases with selenium server + js extentions + Hudson”.
Also, IDE does matter in the selection as it helps us to make our coding with easy due to context sensitive help, auto complete etc. C# IDE Visual Studio, Java IDE Eclipse etc. are good IDEs and is important point in language selection decision making process.
“If you have access to the VisualStudio IDE you cant go wrong with that”. I like this.
6. Language exposure to automation engineer
Sometimes we do not disclose or mention this criteria, but I think it is always get hidden consideration for language selection.
Are you thinking same? “I personally prefer Java because that’s the language I’m most familiar with. However, if the project and its timeline provided the opportunity for my learning curve, I would love to try out some other languages — it for nothing more than to see what differences there are.”
Anand is asking us with explanation, “Depends on which language you are already comfortable in, isn’t it? If you are comfortable in JAVA you don’t learn PHP to write selenium tests. As far as I know, most of the bindings provides the same features. And thanks to people behind Selenium for making this possible !”
In short, all points has their own pros and cons but this poll has created a platform to help someone to ask related queries to other who using same language.
At last, thanks to all for voting and commenting. We will come up with something similar and useful topic again!!
courtesy:
- Selenium Automation User Group
- Apexon Selenium Test Automation Framework
- All the linked in users who voted and commented on poll : “Language preferred for Selenium Test Automation Project? Write in comment for other languages.”
Thanks and Regards,
Akhilkumar Patel
Waiting for your suggestions and queries!!
