This is how watir recorder performed for the following elements. The result is not very satisfying, but i still prefer to use the tool. It saves a lot of your time. At least it can recognize the most essential html elements.

HTML Field	Recognized?	Comment
text input	Yes
button	Yes
links	Yes
checkbox	Yes
dropdown list	Yes
radio button	Yes
frame	No	Unable to recognize objects inside
iframe	No	Unable to recognize opjects inside
optgroup	No
option	No
textarea	No

I am required to automate a portion of a .NET application to cut down the time spent in smoke testing. I have used Selenium RC successfully on other projects that run on both Internet Explorer and Firefox. I used Selenium IDE for firefox to record user actions and Selenium RC under eclipse to tune my script and add logic, switching and DB Access blocks. And JUnit of course to create test cases.

There's a little problem: the application is a .Net back office application that is always run on Internet Explorer by internal clients. Because of that, the developer haven't ever paid attention to browser compatibility. And as a result, the application runs terribly on Firefox. And it's not possible to use Selenium IDE to record actions and then run the scripts on IE.

Theoritically, it's possible to code all your scripts by hands using Selenium and run them on IE with Selenium RC, but who has time to do that?

I've heard about Watir before, but never used it. Now i am experimenting with it and i like what i see. Watir Recorder will seamlessly save your time in recording user actions. Here's how a recorded ruby script will look like using Watir Recorder (click to enlarge):


Or you may prefer the text version:

require 'watir'
include Watir
require 'test/unit'
class TC_recorded < Test::Unit::TestCase
def test_recorded
ie = IE.new
ie.goto('http://www.google.com/webhp?sourceid=navclient&ie=UTF-8')
ie.link(:url, 'http://www.google.com/advanced_search?hl=en').click
ie.text_field(:name, 'as_q').set('testsquad test automation blog')
ie.select_list(:name, 'num').select_value('10')
ie.select_list(:name, 'num').select_value('20')
ie.select_list(:name, 'lr').select_value('lang_en')
ie.button(:value, 'Advanced Search').click
end
end

Up from this point you can copy your base script to a ruby (*.rb) file and add whatever logic, switching and DB access blocks to extend and parametrize your test. Today, Watir Recorder has been able to recognize text boxes, buttons, links and combo boxes for non .Net applications. As i said, i am still experimenting and i will post more results in the coming days.

Please share your experience! :)

Updates: Google Test Automation Conference is coming. Check the details on Google Testing Blog

A TestSquad reader sent me an email asking:

Is there a verification process to help validate that application developers comply to test automation requirements during their application development?
Usually we just give the requirements and expect application developers to comply but when you are doing the automation scripts already, some things are still skipped. I am looking for a process that will help them comply.
e.g. automation requirement: name tables as xxxx

I have to admit that answering such a question is not easy. How can you guarantee that development activity is compliant with a subsequent test automation? How does the developer's code affect the success of your test automation?

1 month ago, i was considering Selenium RC to automate a Java web application. Unfortunately and for a reason, automation was not introduced till a late stage in the project. So as soon as we started recording scripts, a problem emerged: the development team did not give an "id" attribute to the numerous html elements. And this forced us to use some workaround; unnecessary if if the QA automation effort was introduced from day one. This is relatively a simple scenario that could be solved using workaround, but i am sure the impact can much more severe for complex projects.

So a corollary answer for the original question is "continuous collaboration between developers and testers from day one in the SDLC" and that naturally leads us to Agility.


I like to use LinkedIn for tough questions. Here's the best answer i got from Greg Akins

I don't think there is an easy answer to this. One issue is
that there is a responsibility for the developer to not only create
features which are requested by the customer/business, but also to
ensure that the features are are created in a way that supports the
test process. It's my experience that the incentive to do the latter
is somewhat limited.

However, having spent time writing and designing Selenium and FIT
tests on smallish team, there was a long learning period associated
with getting the team to understand how the application needed to be
designed to support testing. I'd definitely suggest working side by
side with your developers, and iterating frequently while you're still
expecting frequent "failures" in the dev/test process

I would love to her your personal experience :)

Some people may get confused when first introduced to Selenium RC or Selenium Grid. The best answer to this is what i read on Selenium users group by Haw-Bin:

It's possible, but it's not a great usage of Selenium. One reason is that actually starting that many browsers will create a lot of overhead on the machines they are running on. You'll end up wasting a lot more computing power than you have to.

IBM Rational Robot

• Smart object recognition
• Stable recording and playback. Robot solves the problem of custom objects encountered in AppWorks by the ability to add those object to a class map.
• IBM Rational TestManager is necessary to view detailed tests logs.
• Datapools supported
• Support distributed testing and sharing test scripts.
• Very simplified scripting using SQABasic; a simplified scripting language. Other scripting languages are possible (C++ and Java) with an aid from TestManager
  
• IBM Rational Robot Authorized User License + SW Subscription & Support 12 Months (D53NDLL) $ 4,676.00
• IBM Rational Robot Floating User License + SW Subscription & Support 12 Months (D52ZFLL) $ 9,009.00

HP (Mercury) QuickTest Professional

• Smart object recognition.
• Stable recording and playback
• Detailed reporting
• VB scripting
• Scripts can be shared, but HP (Mercury) Quality Center is necessary.
  
• Datapools supported\HP (Mercury) Quality Center is necessary to share scripts.
• Pricing: Information Not available. I will update it soon!
  

Disclaimer: This post is related to IBM Rational Functional Tester With Java. It needs some slight modifications to work with other tools.

Sometimes, your automated test script needs to do some strange stuff. Like accessing the database, extracting some value and using this value in subsequent U.I steps in your script.

Suppose that you have hundreds of scripts that needs the following block to access the DB and read a certain value:

try{
Connection db_connection = DriverManager.getConnection("jdbc:odbc:DBNAME",
"username","password");

Statement stm2 = db_connection.createStatement();
stm2.executeQuery("select * from TableName");

}
catch( SQLException x ){

JOptionPane.showMessageDialog(null,
x.getLocalizedMessage().toString(),"Error",
JOptionPane.ERROR_MESSAGE);

}

What if the username and password is changed after a period of time for a certain reason. This means that you need to modify all your scripts and change the credentials. And this could be a pain and very time consuming.

A good practice is creating a class that will prompt the user to enter the credentials and validates them before running the script. This will save time and headache in subsequent releases.

The following is the code for the class with the proprietary information removed:

import javax.swing.JOptionPane;
import resources.SendValidUNandPWHelper;

import com.rational.test.ft.*;
import com.rational.test.ft.object.interfaces.*;
import com.rational.test.ft.object.interfaces.siebel.*;
import com.rational.test.ft.script.*;
import com.rational.test.ft.value.*;
import com.rational.test.ft.vp.*;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;

/**
* Description : Functional Test Script
* @author Ashehadeh
*/
class SendValidUNandPW extends SendValidUNandPWHelper
{
/**
* Script Name : CheckDBDialog
* Generated : Feb 19, 2008 8:31:06 AM
* Description : Functional Test Script
* Original Host : WinNT Version 5.1 Build 2600 (S)
*
* @since 2008/02/19
* @author Ashehadeh
*/
/**
* @param args
*/





public String[] sendValidCredentials(){

String[] result=null;
String[] k =validateInput();

while (k==null){
k= validateInput();
}

return k;


}


//check if the provided credential can access the DB
public static String[] validateInput(){

String[] result=null;
String[] k =isDBonline();

if (k==null){
result =null;
System.exit(0);

}

else {


try{
Connection db_connection = DriverManager.getConnection("jdbc:odbc:DBNAME",k[0],k[1]);

Statement stm2 = db_connection.createStatement();
stm2.executeQuery("select * from TABLEName");
result =k;

}
catch( SQLException x ){

JOptionPane.showMessageDialog(null,
"Please Enter correct Uasername/Password!","Error",
JOptionPane.ERROR_MESSAGE);
/* JOptionPane.showMessageDialog(null,
x.getLocalizedMessage().toString(),"Error",
JOptionPane.ERROR_MESSAGE);*/

result=null;

}

}
return result;
}


//check if the db is online from the first place

public static String[] isDBonline(){

String[] result=null;
String[] k =isOneEmpty();


if (k==null){
result =null;
System.exit(0);


}


//now check for DB connectivity
else {

try{

Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");

result =k;

}



catch(ClassNotFoundException c){
JOptionPane.showMessageDialog(null,
"DB Unavailable!","Error",
JOptionPane.ERROR_MESSAGE);
result=null;
System.exit(0);

}

}
return result;
}


//is one of the fields empty? if so, exit
public static String[] isOneEmpty(){

String[] k=showDialog();
String[] result;

// if one of the username or passwords are empty, exit.
if (k[0]==null || k[1].equals("") ||
k[1].equals("") || k[1]==null){

JOptionPane.showMessageDialog(null,
"Error. Fill the two fields. Click ok to exit!","Error",
JOptionPane.ERROR_MESSAGE);

result=null;
System.exit(0);

}


//if both of them are not empty continue
else{
result=k;

}

return result;

}

//prompt the user for username and password
public static String[] showDialog() {

String DBUsername=JOptionPane.showInputDialog(null,
"Enter DB username:", "Input", JOptionPane.QUESTION_MESSAGE);
String DBpassword=JOptionPane.showInputDialog(null,
"Enter DB password:", "Input", JOptionPane.QUESTION_MESSAGE);
String[] UPArray={DBUsername,DBpassword};
return UPArray;

}



}

I had some time lately to get my hands dirty with .Net 2008 . The tool as described works under the http layer. So it’s designed to test hard core web functionalities. The tool is not a good choice when it comes to creating verification points and verification of JavaScript client side actions details.

• The tool could be used to verify the core functionality (Not the UI) of the Web application. For example if you have a web application which is notorious for crashes. This tool is so perfect for detecting this whenever a new release is out.

• Recording and playback is so easy and stable.
  

• System requirements are relatively acceptable if compared with other tools. My machine is 2MB in memory. The tool is running smoothly on it.

• Playback only displays actions that are associated with Http requests. It will not display javascript user actions. However those actions can be simulated using coding. Click Microsoft Web Test Authoring and Debugging Techniques article for details.
  

• Only VB and C# coding are supported.
  

This tool may not give you all what we need, but it has some great features for web application functional testing.

The following article is a great resource:

http://msdn2.microsoft.com/en-us/library/ms364082(VS.80).aspx

I am in the process of evaluating .net 2008 Tester Edition. my evaluation revolves around automation. I will post results soon.

Today i passed IBM Test 842 for RFT. The exam was not easy. You have to be very familiar with the tool. The questions difficulty varies very much from a question to another. And a knowledge in regular expressions helps a lot.
I cant' really say more since you have to sign on non-disclosure agreement prior to the exam. However, you can study the online course, which you have to buy of course or take the instructor led course.







What i really didn't like is that the online course does not guarantee you to answer all exam questions, since it's explicitly mentioned in some chapters that you have to take the instructor led course if you want to know a certain procedure. I think this is not fare :(

There's an easier way if you want to pass the exam. It's by looking at some virtual exams engines like pass4sure. You may achieve a 99% score, but you'll guarantee your failure in the first job interview.

Just for the record: I personally think that certifications are ridiculous. :)

One of the challenges for a test automation newbie is the change of mindset; from a developer to tester and vice versa. I encountered this the first time I wrote my very first test automation script. I did some manual exploratory testing using mouse and keyboard, I visualized a certain repeatable intact scenario, recorded mouse and keyboard interactions, then played back the script and a neat automated test report popped with the passed and failed results, and I couldn't be happier :)


As I finished my delicious launch, an email popped with an unpleasant content: Please test the functionality when the bla bla bla is not bla bla bla. Now wait, recording will not do me this. The only way I can do it is by writing my own new java class. So I rolled my sleeves up and started coding.


Hours and hours passed, and noticed that my thinking is dangerously drifting into a developer mindset. I remember some testing guru saying : "A good developer is not a good tester", but am I not a developer now? How can I develop a good automation script and keep in mind my third eye sharp and active?


Through practice, I learned that in order to write a good automation script, and at the same time, the automation script tests it's intended functionality, you need to learn to change the way you think and to keep remembering that you are after all a tester and not a developer.

While writing an automation script that involves coding intervention, think as a developer and keep yourself contained in the quality theme. This is manifested when you run this script against your application. At this moment, abandon your developer mindset and think only about the test results.


This may seem easily said, and yes it is, but through practice an oxymoron of test developer can be achieved.

It's common among test engineers to mix between performance testing and stress testing. It's worth to mention that the difference between the two lies in how they are executed:

Performance testing (aka load testing) is testing that is performed, from one perspective, to determine how fast some aspect of a system performs under a particular workload. [source]. A very simple analogy is by imagining yourself holding a stopwatch while your work colleague is performing certain navigations and mouse clicks on a web application. Your job here is to take time measures for the transactions. Then compare them to the required figures.

The purpose of performance testing is to make sure that the system is running according to the required response time performance specifications under a particular load. And to reveal the system time bottlenecks and thresholds.

The challenge in performance testing is to achieve a workload for the testing environment that is a close match to that of production environment.

---

Stress testing is a form of testing that is used to determine the stability of a given system or entity. It involves testing beyond normal operational capacity. [source]. The application under test is subjected to extraordiary loads that may cause it to crash. In more details, it's intended to exhaust the system hardware resources (memory, storage).

The purpose of stress testing is to fortify the system against unusual workloads or attacks such as bots, DOS or DDOS attacks.



References:
Automated Software Testing

([1-9]|1[012])/([1-9]|[12][0-9]|3[01])/0[0-9]$

This will verify if the date is mM/dd/yy, where m is optional. Example: 9/21/07 or 11/21/07

I have encountered a problem while using RFT. I needed the script to capture a value of an object and stores it in a string. It turned out that getPropertyFromMap and getProperty work not exactly the same.

getProperty will capture the most recent property value of the specified object. While
getPropertyFromMap will capture the property value at the time the object was added to the map.

The GTAC videos are finally online. In GTAC, you have the brightest of the brightest in the testing industry talking about the latest about automation testing. I will have to spare some time to watch them all.

We are using RFT to test a terminal based application running on an IBM Iseries
machine.For certain testing scenarios we need to access the database from within then script.Here's how you do it (Note that we are running RFT version 6 and eclipse 3.0 ) :

1- Activate RFT

2- Go To “Help” in the menu bar choose “Software Updates” then choose “find and install”.

3- Then choose “Search for new features to install and press next”.

4- Press on “New Remote Site” in the name type “Eclipse Update” and in the URL put this URL http://update.eclipse.org/updates/3.0/.

5- Do step 4 again and in the name type “Quantum” and in the URL put http://quantum.sourceforge.net/update-site.

6- Check them both and press the next button.

7- Install all the “GEF” and “Graphical Editing Framework” and the “Graphical Editing Framework SDK” that doesn’t produce dependencies error.

8- Install All the Quantum features

9- After you have completed the above steps open windows control panel à administrative tools à Data Sources (ODBC) .

10- In the System DSN tab, press on the add button, choose the iSeries Access ODBC Driver press finish.

11- In the Data Source Name type DB Name, then in the Server tab change the SQL Default Library to SLSQADATA and in the Packages Tab change the Package Library to SLSQADATA then press ok.

12- In the RFT go to the Windows menu in the menu bar à open perspective à others and choose Quantum DB.

13- When the Quantum Browser Loads press on the Add Bookmark.

14- Choose JDBC-ODBC Bridge press next and fill in the user id and password you use for your DB connection and in the Data Source field enter db name.

15- Press next and enter a name for the bookmark then press next leave the default choice of schema then click finish.

Now the connection is established between RFT and the Database Using a JDBC-ODBC bridge between the DB2 layer using the iSeries Navigator Driver and the RFT thus establishing the full connection.

I used this script to test if a table in the database is empty or not. I am using IBM RFT

boolean found;
ResultSet result;

try{

Connection db_connection = DriverManager.getConnection("jdbc:odbc:DBNAME","USERNAME","PWD");
Statement stm2 = db_connection.createStatement();

result=stm2.executeQuery("your query");
found = result.next();

if (!(found))
logTestResult("Test Failed",false);
else
logTestResult("Test Passed",true);
}

catch( SQLException x ){
logError(x.getLocalizedMessage().toString());
}

Reference: http://www.rgagnon.com/javadetails/java-0116.html

While browsing Google London Test Automation Conference 2006 topics, a cool tool drew my attention. It's a collaborative open source project of Jason Huggins, a test engineer at Google.

The tool offers:

• Easy record and playback
• Intelligent field selection will use IDs, names, or XPath as needed
• Autocomplete for all common Selenium commands
• Walk through tests
• Debug and set breakpoints
• Save tests as HTML, Ruby scripts, or any other format
• Support for Selenium user-extensions.js file
• Option to automatically assert the title of every page

I tried the Firefox extension and it was just awesome. What i really liked the most is its light weight and its support of many scripting languages. So why don't you get your hands dirty and try it out. :)

Jason Huggins did a dedicated presentation on the tool back in 2006: