Bug tracking made simple

Bug tracking tool is a backbone of all the processes in the company. It is a tool which is used by every engineer / tester and management in the company.

We cannot recruit a tester / engineer without the knowledge of bug tracking tool. An engineer needs to know everything around the the tool fields / reports. I will try to cover following topics in the blog:

1. Fields of a bug
2. Test Reports

See the image below - It has all the fields that are required to file a bug: 

• Reporter - Person who has filed the bug. So if you have filed the bug it will be your email Id in the reporter field.
• Product - Every company has the list of products. Reporter needs to select the product from the product list in which the bug was found.
• Version - Whenever you file a bug you need to remember the version of the build that you are testing. 
• Component - Is it on UI or  server or any other component ?  Remember the component is not the feature. 
• Severity - Impact of a bug on the system.it can have the value:
• Critical
• Major
• Normal
• Minor
• Priority - How quick a bug needs to be resolved? It depend on the impact of the bug on the business.
• OS - Operating system
• Target milestone - In which release the bug fix will be targeted. 
• Initial state - state of the bug. When you file the bug its state is New.
• AssignTo - Next owner of the bug.
• CC - People you want to be aware of the issue.
• Summary - One line description 
• Description - It will have the steps to reproduce the bug , expected output, actual output and if anything extra needs to be provided to the developer it needs to be mentioned in the bug.
• Attachment - Any screenshot or log file that you feel will help the team to understand the issue better.
• Depends on : If there is a parent bug and you want the present bug to be dependent on that bug give the parent bug Id in the depend on bug.
• Blocks : This field will help you in tracking the bugs on a feature. This needs to be explained in detail. Lets take a scenario : You are a feature owner and you need to track the bugs for a feature . Here are the steps:
• File a bug on a feature.
• You will get the bug ID - e.g. 21234
• Now every bug that you file on the same feature in the blocks field of the bug give the bug Id 21234.
• So all the bugs will be blocked by the parent bug 21234.
• If you want to see the bugs filed on that feature you can click on the bug id 21234 and it will show you the entire list of bugs filed on that feature.

Test Reports : Think of the type of reports that a person is interested in :

1. # bugs distributed on the basis of status e.g. bugs under fixed, bugs under open, bugs under reopen,bugs under closed etc.
2. # bugs distributed on the basis of status and then on priority. e.g. if there are 100 bugs open we will definitely like to know the priority of the open bugs.

 I will keep writing around bug tracking tool but right now I need to leave. I would be happy to help and make my students successful.

Test plan made simple

I am copying the data from some other sites which I found useful around a test plan. Please go through the video and the content available below:

Here is also a link to study risk management - http://inderpsingh.blogspot.in/2011/10/risk-management-in-software-testing.html

Both the links will help you to understand and prepare for test plan. You could further download a sample test plan from Sample Test plan.

 

 

 

Master the test cases

Its a way you think. We expect an engineer to think of all possible scenarios for a feature / application. If you need to write the test cases you need to be very clear with the scenarios that you need to test. Rather you should be a master in thinking of the scenarios.

Here's is an approach that you should take while thinking for the functional test cases:

1. Get the feature that you want to test.
2. Think of the possible ways that a person can use that feature.
3. Cover every possible case under the scenario.

Let me explain with the help of an example:

If you need to test the notepad, most of the engineers will say - check if it opens or not, all the buttons are working , I should be able to type etc... Great! But does this approach guarantee the coverage? Answer is No.

Take this approach - Break the notepad under following heads:

1. File --> New
2. File --> Open
3. etc..

You can create a new file from following possible scenarios:

• New
• If its a blank file.
• file is saved
• file is not saved
• If file is not blank.
• file is saved
• files is not saved

Now if you need to write a test case you know the direction to follow - e.g. Create a new file if its blank and saved, create a new file if its blank and not saved, create a new file from the file with data and saved, create a new file with data and not saved. This way you can ensure the coverage of the test cases

Monday, 11 June 2012

Integration testing made simlpe

Definition - In Integration Testing, individual software modules are integrated logically and tested as a group.

A typical software project consists of multiple software modules, coded by different programmers.  Integration testing focuses on checking data communication amongst these modules.

Hence it is also termed as ‘I & T’ (Integration and Testing), ‘String Testing’ and sometimes ‘Thread Testing’.

Approaches/Methodologies/Strategies of Integration Testing:

The Software Industry uses variety of strategies to execute Integration testing , viz.

•  Big Bang Approach :
•  Incremental Approach: which is further divided into following
•  Top Down Approach
•  Bottom Up Approach
•  Sandwich Approach - Combination of Top Down and Bottom Up  

Big Bang Approach: Here all component are integrated together at once, and then tested.

Advantages: Convenient for small systems.

Disadvantages: -

         Incremental Approach: -  In this approach, testing is done by joining two or more modules that are logically related. Then the other related modules are added and tested for the proper functioning. Process continues until all of the modules are joined and tested successfully.
This process is carried out by using dummy programs called Stubs and Drivers. Stubs and Drivers do not implement the entire programming logic of the software module but just simulate data communication with the calling module.
Stub: Is called by the Module under Test.
Driver: Calls the Module to be tested.
Incremental Approach in turn is carried out by two different Methods:

• Bottom Up
• Top Down

Latent Bug

Def 1 : A bug which is not fixed and is already uncovered in earlier versions/releases/rounds of testing and is still present.

Def 2: Bugs that remain dormant or unhidden are latent bugs. These bugs are not found until one or more releases of the product.

Fuzz Testing

Fuzz testing or fuzzing is a software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of a computer program. The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leaks. Fuzzing is commonly used to test for security problems in software or computer systems.

Smoke / Sanity made simple

Here is another video that I think can be used to understand sanity / smoke. Enjoy the video:

Video Transcript with Key Takeaways Highlighted:

• Consider a scenario, when after fixing defects of integration testing , the system is made available to the testing team for system testing
• You look at the initial screen , system looks fine and delay system test execution for the next day since you have other critical testing requirements to attend to
  Next day say you plan to execute the scenario login > View Balance > Transfer 500 > logout . The deadline is 4 hours.
• You begin executing the scenario , enter a valid login id , password, click the login button and boom you are taken to a blank screen with absolutely no links , no buttons & no where for you to proceed with succeeding steps of your scenario
• This is not a figment of any imagination but a very practical condition which could arise due to developer negligence, time pressures , test environment configuration & instability
• To fix this developer requires atleast 5 hours & deadline would be missed
• In fact , none of your team members would be able to execute their respective scenarios , since view balance is start point to perform any other operation and the entire project will be delayed
• Had you checked this yesterday itself, the system would been fixed by now and you were good for testing
• To avoid such situation sanity also know SMOKE  testing is done to check critical functionalities of the system before its is accepted for major testing. Sanity testing is quick and is non- exhaustive. Goal is not to find defects but to check system health

Traceability Matrix

This is the best video to study the Traceability Matrix.

Video Transcript with Key Takeaways Highlighted:

• Consider a scenario where, the client changes the requirement , something so usual in the practical world and adds a Field Recipient name to the functionality. So now you need to enter email id and name both to send a mail
• Obviously you will need to change your test cases to meet this new requirement
• But , by now your test case suite is very large and it is very difficult to  trace the test cases affected by the test cases
•  Instead , if the requirements were numbered  and were referenced in the test case suite it would have been very easy to track the test cases that are affected. This is nothing but Traceability
•  The traceability matrix links a business requirement to its corresponding functional requirement right up to the  corresponding test cases.
•  If a Test Case fails, traceability helps determine the corresponding functionality easily .
•  It also helps ensure that all requirements are tested.

In Other words.  A traceability matrix is a document, usually in the form of a table, that show the relationship between Test Requirement and test cases. It consist req_id, feature_id, function_id, Test cases_id.

 

QTP frameworks made simple - interview questions

Now what is a TEST Automation Framework ?
A set of guidelines like coding standards , test-data handling , object repository treatment etc... which when followed during automation scripting produce beneficial outcomes like increase code re-usage , higher portability , reduced script maintenance cost etc. Mind you these are just guidelines and not rules; they are not mandatory and you can still script without following the guidelines. But you will miss out on the advantages of having a Framework.

What are the various Automation Frameworks available?

1. Linear Scripting
2. The Test Library Architecture Framework.The Data-Driven Testing Framework.
3. The Keyword-Driven or Table-Driven Testing Framework.
4. The Hybrid Test Automation Framework.

Lets look at them in detail -

Linear Scripting - Record & Playback - It is the simplest of all Frameworks and also know as "Record & Playback".In this Framework , Tester manually records each step ( Navigation and User Inputs), Inserts Checkpoints ( Validation Steps) in the first round . He then , Plays back the recorded script in the subsequent rounds.

Advantages

• Fastest way to generate script
• Automation expertise not required
• Easiest way to learn the features of the Testing Tool

Disadvantages

• Little reuse of scripts
• Test data is hard coded into the script
• Maintenance Nightmare

The Test Library Architecture Framework. - It is also know as "Structured Scripting" or "Functional Decomposition". In this Framework , test scripts are initially recorded by “Record & Playback” method. Later, common tasks inside the scripts are identified and grouped into Functions. These Functions are called by main test script called Driver in different ways to create test cases.

Advantages

• Higher level of code reuse is achieved in Structured Scripting as compared to “Record & Playback”
• The automation scripts are less costly to develop due to higher code re-use
• Easier Script Maintenance

Disadvantages

• Technical expertise is necessary to write Scripts using Test Library Framework.
• More time is needed to plan and prepare test scripts.
• Test Data is hard coded within the scripts

The Data-Driven Testing Framework.-  In this Framework , while Test case logic resides in Test Scripts, the Test Data is separated and kept outside the Test Scripts.Test Data is read from the external files (Excel Files, Text Files, CSV Files, ODBC Sources, DAO Objects, ADO Objects) and are loaded into the variables inside the Test Script. Variables are used both for Input values and for Verification values. Test Scripts themselves are prepared either using Linear Scripting or Test Library Framework.

Advantages

• Changes to the Test Scripts do not affect the Test Data
• Test Cases can be executed with multiple Sets of Data
• A Variety of Test Scenarios can be executed by just varying the Test Data in the External Data File

DisAdvantages - More time is needed to plan and prepare both Test Scripts and Test Data

The Keyword-Driven or Table-Driven Testing Framework.- The Keyword-Driven or Table-Driven framework requires the development of data tables and keywords, independent of the test automation tool used to execute them . Tests can be designed with or without the Application. In a keyword-driven test, the functionality of the application-under-test is documented in a table as well as in step-by-step instructions for each test.

There are 3 basis components of a Keyword Driven Framework viz. Keyword , Application Map , Component Function.

What is a Keyword ? - Keyword is an Action that can be performed on a GUI Component. Ex . For GUI Component Textbox some Keywords ( Action) would be InputText, VerifyValue, VerifyProperty and so on..
What is Application Map? - An Application Map Provides Named References for GUI Components. Application Maps are nothing but “Object Repositry’

What is Component Function? - Component Functions are those functions that actively manipulate or interrogate GUI component. An example of a function would be click on web button with all error handling , enter data in a Web Edit with all error handling. Component functions could be application dependent or independent.

Advantages

• Provides high code re-usability
• Test Tool Independent
• Independent of Application Under Test, same script works for AUT (with some limitations)
• Tests can be designed with or without AUT

Disadvantages

• Initial investment being pretty high, the benefits of this can only be realized if the application is considerably big and the test scripts are to be maintained for quite a few years.
• High Automation expertise is required to create the Keyword Driven Framework.

The Hybrid Test Automation Framework.-  As the name suggests this framework is the combination of one or more frameworks discussed above pulling from their strengths and trying to mitigate their weaknesses. This hybrid test automation framework is what most frameworks evolve into over time and multiple projects. Maximum industry uses Keyword Framework in combination of Function decomposition method.

Other Frameworks worth a mention are
Test Modularity Framework -  In this framework common task in test script are grouped together as Modules. Ex:Using Actions in QTP use can create a Modualr Scripts

‘Sample Script for Login
SystemUtil.Run "flight4a.exe","","","open"

Dialog("Login").WinEdit("Agent Name:").Set "test123"

Dialog("Login").WinEdit("Password:").Set "Mercury"

Dialog("Login").WinButton("OK").Click

‘End of Script

Now you can call this Action in the main script as follows -

RunAction ("Login[Argument]", oneIteration)