Risk analysis

Risk Analysis


In this tutorial you will learn about Risk Analysis, Technical Definitions, Risk Analysis, Risk Assessment, Business Impact Analysis, Product Size Risks, Business Impact Risks, Customer-Related Risks, Process Risks, Technical Issues, Technology Risk, Development Environment Risks, Risks Associated with Staff Size and Experience.


Risk Analysis is one of the important concepts in Software Product/Project Life Cycle. Risk analysis is broadly defined to include risk assessment, risk characterization, risk communication, risk management, and policy relating to risk. Risk Assessment is also called as Security risk analysis.


Technical Definitions:


Risk Analysis: A risk analysis involves identifying the most probable threats to an organization and analyzing the related vulnerabilities of the organization to these threats.


Risk Assessment: A risk assessment involves evaluating existing physical and environmental security and controls, and assessing their adequacy relative to the potential threats of the organization.


Business Impact Analysis: A business impact analysis involves identifying the critical business functions within the organization and determining the impact of not performing the business function beyond the maximum acceptable outage. Types of criteria that can be used to evaluate the impact include: customer service, internal operations, legal/statutory and financial.


Risks for a software product can be categorized into various types. Some of them are:

Product Size Risks:


The following risk item issues identify some generic risks associated with product size:

Estimated size of the product and confidence in estimated size?
Estimated size of product?
Size of database created or used by the product?
Number of users of the product?
Number of projected changes to the requirements for the product?


Risk will be high, when a large deviation occurs between expected values and the previous experience. All the expected information must be compared to previous experience for analysis of risk.

Business Impact Risks:


The following risk item issues identify some generic risks associated with business impact:

Affect of this product on company revenue?
Reasonableness of delivery deadline?
Number of customers who will use this product and the consistency of their needs relative to the product?
Number of other products/systems with which this product must be interoperable?
Amount and quality of product documentation that must be produced and delivered to the customer?
Costs associated with late delivery or a defective product?

Customer-Related Risks:


Different Customers have different needs. Customers have different personalities. Some customers accept what is delivered and some others complain about the quality of the product. In some other cases, customers may have very good association with the product and the producer and some other customers may not know. A bad customer represents a significant threat to the project plan and a substantial risk for the project manager.


The following risk item checklist identifies generic risks associated with different customers:

Have you worked with the customer in the past?
Does the customer have a solid idea of what is required?
Will the customer agree to spend time in formal requirements gathering meetings to identify project scope?
Is the customer willing to participate in reviews?
Is the customer technically sophisticated in the product area?
Does the customer understand the software engineering process?

Process Risks:


If the software engineering process is ill-defined or if analysis, design and testing are not conducted in a planned fashion, then risks are high for the product.

Has your organization developed a written description of the software process to be used on this project?
Are the team members following the software process as it is documented?
Are the third party coders following a specific software process and is there any procedure for tracking the performance of them?
Are formal technical reviews are done regularly at both development and testing teams?
Are the results of each formal technical review documented, including defects found and resources used?
Is configuration management used to maintain consistency among system/software requirements, design, code, and test cases?
Is a mechanism used for controlling changes to customer requirements that impact the software?

Technical Issues:

Are specific methods used for software analysis?
Are specific conventions for code documentation defined and used?
Are any specific methods used for test case design?
Are software tools used to support planning and tracking activities?
Are configuration management software tools used to control and track change activity throughout the software process?
Are tools used to create software prototypes?
Are software tools used to support the testing process?
Are software tools used to support the production and management of documentation?
Are quality metrics collected for all software projects?
Are productivity metrics collected for all software projects?

Technology Risk:
Is the technology to be built new to your organization?
Does the software interface with new hardware configurations?
Does the software to be built interface with a database system whose function and performance have not been proven in this application area?
Is a specialized user interface demanded by product requirements?
Do requirements demand the use of new analysis, design or testing methods?
Do requirements put excessive performance constraints on the product?

Development Environment Risks:

Is a software project and process management tool available?
Are tools for analysis and design available?
Do analysis and design tools deliver methods that are appropriate for the product to be built?
Are compilers or code generators available and appropriate for the product to be built?
Are testing tools available and appropriate for the product to be built?
Are software configuration management tools available?
Does the environment make use of a database or repository?
Are all software tools integrated with one another?
Have members of the project team received training in each of the tools?

Read more »

Testing tools

Let us consider an example scenario to explain the need for proper testing tools. A team of programmers develop a system for a client say client X. All the requirements which had been put forward by the client X are implemented by the programmers in this system. For a few inputs the system is tested and it gives the desired output. This system is delivered to the client X .The client X uses his own set of inputs and finds that the system is not working satisfactorily. Here it can be clearly seen that this system has failed because it has not been subjected to thorough testing methods.

For a system to work without producing any glitches, it is imperative that testing methods have been applied to it at every stage of its development. Any product, device, software should be tested before it reaches its destination which can be a customer, client or market. Testing can be done on a piece of hardware such as a PCB board and software such as a website or a user application .A combination of both hardware and software testing tools are available which find their application in large industrial automated plants. Testing tools help in detecting the loopholes within a system. Major failures can be averted if the defect is detected and resolved at the right time.

There are different kinds of testing tools for different purposes. Some of the most common tools are mentioned here.

•           QA testing tools

•           Load testing tools

•           Performance testing tools

•           Product testing tools

•           Unit testing tools

•           Software testing tools

In order to maintain the standards and specifications for a particular system, quality assurance tools are used .These tools help in providing customer satisfaction by meeting their requirements and ensuring that no defects are present in the system. Load testing tools help in understanding the response of the system under heavy loads. These tools create a virtual environment under which the parameters of the actual system are tested. Performance testing tools ensure that a system is running efficiently and smoothly. A faster response time and accuracy is always desirable. This tool helps in estimating the performance of the system in quite a precise manner. Product testing tools are used at every stage of development of a system. This not only enhances the quality of a product but also helps in keeping a check on the profit-loss margin. Software testing tools are used to test an application or even a small program to detect bugs and avoid failure.

Read more »

Internationalization and localization

The general ability of software to be internationalized and localized can be automatically tested without actual translation, by using pseudolocalization. It will verify that the application still works, even after it has been translated into a new language or adapted for a new culture (such as different currencies or time zones).
Actual translation to human languages must be tested, too. Possible localization failures include:

• Software is often localized by translating a list of strings out of context, and the translator may choose the wrong translation for an ambiguous source string.
• Technical terminology may become inconsistent if the project is translated by several people without proper coordination or if the translator is imprudent.
• Literal word-for-word translations may sound inappropriate, artificial or too technical in the target language.
• Untranslated messages in the original language may be left hard coded in the source code.
• Some messages may be created automatically at run time and the resulting string may be ungrammatical, functionally incorrect, misleading or confusing.
• Software may use a keyboard shortcut which has no function on the source language's keyboard layout, but is used for typing characters in the layout of the target language.
• Software may lack support for the character encoding of the target language.
• Fonts and font sizes which are appropriate in the source language, may be inappropriate in the target language; for example, CJK characters may become unreadable if the font is too small.
• A string in the target language may be longer than the software can handle. This may make the string partly invisible to the user or cause the software to crash or malfunction.
• Software may lack proper support for reading or writing bi-directional text.
• Software may display images with text that wasn't localized.
• Localized operating systems may have differently-named system configuration files and environment variables and different formats for date and currency.

To avoid these and other localization problems, a tester who knows the target language must run the program with all the possible use cases for translation to see if the messages are readable, translated correctly in context and don't cause failures.

Read more »

Security Testing

Security Testing is an indispensable part of Web application development life cycle due to increase in privacy breaches in businesses and organizations. Testree embraces the industry standard testing methodology and keeps track of new vulnerabilities. Testree has repository of reusable Security test cases and gained proficiency in using Security testing tools (open source and industrial standard). Offer

Testree helps to identify business risks that are caused by the security vulnerabilities on in-house developed applications, COTS products or third party applications. Testree offers the following solutions:
Web application penetration testing
Product security testing
Information Systems Risk Assessments / Security Audit
Security Policy and Process Design
Analyzing security vulnerabilities in the applications
Analyzing security quality of internally developed applications
Ensuring compliance with PCI standards, SOX, and HIPAA
Advice on fixing loopholes and future security vigilance plan Value Proposition

Testree has expertise in performing security / penetration testing on the web applications. It follows the industry standard guidelines by Open Web Application Security Project (OWASP) and Web Application Security Consortium.
Comprehensive security analysis
Potential security issue coverage Competency

More than one tool is required to accomplish security / penetration testing of web application. Tool evaluation is executed based on the nature of application and environment. Testree has expertise in using open and industry standard tools (IBM Rational AppScan, Web Inspect, Typhon III, WebGoat, and WebScarab).

Testree has expertise in testing web applications for OWASP Top 10 vulnerabilities, few of them are listed below:
Cross-Site Scripting (XSS) – (Session hijack, Track user activities, Browser exploitation)
Injection Flaws – (SQL injection, XPath injection, LDAP injection, SSI injection)
Malicious File execution
Insecure Direct Object Reference
Cross Site Request Forgery (CSRF)
Denial of Service
Buffer overflow

Read more »

Usability Testing

For many of us, usability testing is a necessary evil. For others, it’s too much work, or it’s too disruptive to the development process. As you might expect, I have issues with all that. It’s unfortunate that some teams don’t see the value in observing people use their designs. Done well, it can be an amazing event in the life of a design. Even done very informally, it can still show up useful insights that can help a team make informed design decisions. But I probably don’t have to tell you that.

Usability testing can be enormously elevating for teams at all stages of UX maturity. In fact, there probably isn’t nearly enough of it being done. Even on enlightened teams that know about and do usability tests, they’re probably not doing it often enough. There seems to be a correlation between successful user experiences and how often and how much the designers and developers spend time observing users. (hat tip Jared Spool)

Observing people using early designs can be energizing as designers and developers get a chance to see reactions to ideas. I’ve seen teams walk away with insights from observing people use their designs that they couldn’t have got any other way – and then make better designs than they’ve ever made. Close to launch, it is exciting – yes, exciting – to see a design perform as useful, usable, and desirable.

Read more »

Stability Testing

With companies worldwide failing to get market access due to insufficient stability data the pressure to ensure that the regulatory requirements for stability testing are fulfilled, first time, has never been higher with the pharmaceutical industry as a whole focusing on cutting margins and avoiding recalls.

ICH has created guidelines to regulate the major markets, however the WHO is pushing to amending those guidelines in favor of global harmonization.

Coupled with the trend towards global supply chains and working in climatic zones there are many challenges when it comes to assessing humidity and temperature levels in different supply zones - are you fully prepared for these changes?

Being involved in stability testing, you are well aware that high standards have to be maintained whilst dealing with significant budget and time constraints. Establishing an effective and efficient stability testing programme is key to ensuring regulations are upheld whilst staying within your margins.

Read more »

Java Test Tools

Java Test Tools

Arquillian - Arquillian from JBoss enables you to test Java business logic in a remote or embedded container. Alternatively, it can deploy an archive to the container so the test can interact as a remote client. No longer does writing a test involve system administration tasks, custom scripts, copy-paste Maven configuration, full builds, test classpath mayhem, looking up resources manually or reliance on coarse-grained, black-box testing. The goal is to provide a simple test harness that abstracts away all container lifecycle and deployment from the test logic so developers can easily produce a broad range of integration tests for their enterprise Java applications. Arquillian can either execute a test case inside the container, in which case the test class is deployed by Arquillian along with the code under test, or hold back the test class so it can act as a remote client to the deployed code. All the developer has to do is write the test logic.

RTI - Java performance analysis tool from OC Systems Inc. helps quickly find and resolve complex performance problems. RTI delivers lightweight deep diagnostics for distributed Java applications in production and test environments. Monitor transaction performance end-to-end from client across network, web and application tiers – deep dive to method level. RTI integrates with LoadRunner, JMeter and soapUI. Targeted to JBoss, Tomcat, Linux, Windows.

Yourkit Java Profiler - Java profiling tool from Yourkit; CPU and memory hot spots; memory leak detection; memory distribution reports;on-demand profiling; utilizes all of the advanced Java 5/6/7 profiling features; runs on most platforms.

VisualVM - A free visual tool, originally from Sun, to monitor and troubleshoot Java applications. Runs on Sun JDK 6, but is able to monitor applications running on JDK 1.4 and higher. Utilizes various available technologies like jvmstat, JMX, the Serviceability Agent (SA), and the Attach API to get data and uses minimal overhead on monitored applications. Capabilities include: automatically detects and lists locally and remotely running Java applications; monitor application performance and memory consumption; profile application performance or analyze memory allocation; is able to save application configuration and runtime environment together with all taken thread dumps, heap dumps and profiler snaphots into a single application snapshot which can be later processed offline.

LAPSE - Lightweight Analysis for Program Security in Eclipse. Designed to help with the task of auditing Java J2EE applications for common types of security vulnerabilities found in Web apps. Targets parameter manipulation, header manipulation, cookie poisoning, command-line parameters, SQL injections, cross-site scripting, HTTP splitting, path traversal vulnerabilities Developed by Benjamin Livshits as part of the Griffin Software Security Project. Not intended as a comprehensive solution for Web application security, but rather as an aid in the code review process.

Checkmarx - Code security analysis tool from Checkmarx; capabilitites include hundreds of out of the box security checks, checks for architectural flaws, coding practice enforcement. Runs on Windows against Java or any other programming language code.

Window Licker - An open source framework for the test-driven development of Java systems through the GUI. Provides a high-level API for controlling and making assertions about graphical user interfaces: for Swing and Dynamic HTML (aka "AJAX") including GWT. Deals with the asynchronous nature of GUI and AJAX programming so the tests don't have to; Controls the GUI by sending native mouse and keyboard events; Handles different keyboard layouts; Produces high quality error messages to help you easily diagnose test failures; Easily extensible to cope with new user interface components

Cobertura - Free Java tool to identify which parts of a Java program are lacking test coverage and calculate % coverage; based on jcoverage. Instruments already-compiled Java bytecode; execute from ant or from the command line; generate reports in HTML or XML; shows % of lines and branches covered for each class, each package, and for the overall project. Shows McCabe cyclomatic code complexity of each class, and average cyclomatic code complexity for each package, and for the overall product. Can sort HTML results by class name, percent of lines covered, percent of branches covered, etc. and sort in ascending or descending order.

JProfiler - Java profiling tool from ej-Technologies GmbH. Check for performance bottlenecks, memory leaks and threading issues.

Parallel-junit - Open source small library extensions for JUnit. Extends the junit.framework.TestSuite class by running tests in parallel, allowing more efficient test execution. Because TestResult and TestListener aren't designed to run tests in parallel, this implementation coordinates the worker threads and reorder event callbacks so that the TestResult object receives them in an orderly manner. In addition, output to System.out and System.err are also serialized to avoid screen clutter.

EMMA - Open-source toolkit, written in pure Java, for measuring and reporting Java code coverage. Targets support for large-scale enterprise software development while keeping individual developer's work fast and iterative. Can instrument classes for coverage either offline or on the fly (using an instrumenting application classloader); supported coverage types: class, method, line, basic block; can detect when a single source code line is covered only partially; coverage stats are aggregated at method, class, package, and "all classes" levels. Reports support drill-down, to user-controlled detail depth; HTML reports support source code linking. Does not require access to the source code; can instrument individual .class files or entire .jars (in place, if desired). Runtime overhead of added instrumentation is small (5-20%); memory overhead is a few hundred bytes per Java class.

PMD - Open source static analyzer scans java source for problems. Capabilities include scanning for: Empty try/catch/finally/switch statements; Dead code - unused local variables, parameters and private methods; Suboptimal code - wasteful string/stringBuffer usage; Overcomplicated expressions - unnecessary if statements, for loops that could be while loops; Duplicate code - copied/pasted code - could indicate copied/pasted bugs.

Hammurapi - Code review tool for Java (and other languages with latest version). Utilizes a rules engine to infer violations in source code. Doesn't fail on source files with errors, or if some inspectors throw exceptions. Parts of tool can be independently extended or replaced. Can review sources in multiple programming languages, perform cross-language inspections, and generate a consolidated report. Eclipse plugin.

TestNG - A testing framework inspired from JUnit and NUnit and developed by Cedric Beust; supports JDK 5 Annotations, data-driven testing (with @DataProvider), parameters, distribution of tests on slave machines, plug-ins (Eclipse, IDEA, Maven, etc); embeds BeanShell for further flexibility; default JDK functions for runtime and logging (no dependencies); can run tests in arbitrarily big thread pools with various policies available.

Concordian - An open source testing framework for Java developed by David Peterson. Utilizes requirements in plain English using paragraphs, tables and proper punctuation in HTML. Developers instrument the concrete examples in each specification with commands (e.g. "set", "execute", "assertEquals") that allow test scenarios to be checked against the system to be tested. The instrumentation is invisible to a browser, but is processed by a Java fixture class that accompanies the specification. The fixture is also a JUnit test case. Results are exported with the usual green and red indicating successes and failures. Site includes info re similarities and diffs from Fitnesse.

DBUnit - Open source JUnit extension (also usable with Ant) targeted for database-driven projects that, among other things, puts a database into a known state between test runs. Enables avoidance of problems that can occur when one test case corrupts the database and causes subsequent tests to fail or exacerbate the damage. Has the ability to export and import database data to and from XML datasets. Can work with very large datasets when used in streaming mode, and can help verify that database data matches expected sets of values.

StrutsTestCase - Open source Unit extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework, including validation methods. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing of Struts code with or without a running servlet engine. Uses the ActionServlet controller to test code, enabling testing of the implementation of Action objects, as well as mappings, form beans, and forwards declarations.

DDSteps - A JUnit extension for building data driven test cases. Enables user to parameterize test cases, and run them more than once using different data. Uses external test data in Excel which is injected into test cases using standard JavaBeans properties. Test cases run once for each row of data, so adding new tests is just a matter of adding a row of data in Excel.

StrutsTestCase for JUnit - Open source extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing Struts code with or without a running servlet engine. Because it uses the ActionServlet controller to test code, can test not only the implementation of Action objects, but also mappings, form beans, and forwards declarations. Since it already provides validation methods, it's quick and easy to write unit test cases.

JavaNCSS - A free Source Measurement Suite for Java by Clemens Lee. A simple command line utility which collects various source code metrics for Java. The metrics are collected globally, for each class and/or for each function.

Open Source Profilers for Java - Listing of about 25 open source code profilers for Java from 2006 from the Manageability.org web site.

SofCheck Inspector - Tool from SofCheck Inc. for analysis of Java for logic flaws and vulnerabilities. Explores all possible paths in byte code and detects flaws and vulnerabilities in areas such as: array index out of bounds, buffer overflows, race conditions, null pointer dereference, dead code, etc. Provides 100% path coverage and can report on values required for 100% unit test coverage. Patented precondition, postcondition and presumption reporting can help detect Malware code insertion.

CodePro - Suite of Java tools from Instantiations Inc. CodePro AnalytixX is an Eclipse-based Java software testing tool and includes features like code audit, metrics, automated unit tests, and more. CodePro Profiler, an Eclipse-based Java profiling tool enables inspection of a running application for performance bottlenecks, detect memory leaks and solve thread concurrency problems. EclipsePro Test automatically generates JUnit tests and includes an editor and analysis tool, provides test cases/results in tabular layout; mouse over failing case and Editor shows the failure message. WindowTester Pro for Swing or SWT UI's enables recording of GUI tests; watches actions and generates test cases automatically; customize the generated Java tests as needed. Provides a rich GUI Test Library, hiding complexities and threading issues of GUI test execution; test cases are based on the JUnit standard

Squish for Java - Automated Java GUI testing tool for Java Swing, AWT, SWT and RCP/Eclipse applications. Record or create/modify scripts using Tcl, Python, JavaScript. Automatic identification of GUI objects of the AUT; inspect AUT's objects, properties and methods on run-time using the Squish Spy. Can be run via a GUI front-end or via command line tools. Can execute tests in a debugger allowing setting breakpoints and stepping through test scripts.

Klocwork - Static analysis technology for Java, C, C++, C#; analyzes defects & security vulnerabilities, architecture & header file anomalies, metrics. Developers can run Klocwork in Eclipse or various other IDE's. Users can select scope of reporting as needed by selecting software component, defect type, and defect state/status.

Coverity Prevent - Tool from Coverity Inc. for analysis of Java source code for security issues. Explores all possible paths in source code and detects security vulnerabilities and defects in multiple areas: memory leaks, memory corruption, and illegal pointer accesses, buffer overruns, format string errors and SQL injections vulnerabilities, multi-threaded programming concurrency errors, etc.

GUIDancer - Eclipse-based tool from Bredex GmbH for automated testing of Java/Swing GUI's, Tests are specified, not programmed - no code or script is produced. Test specification is initially separate from the AUT, allowing test creation before the software is fully functional or available. Specification occurs interactively; components and actions are selected from menus, or by working with the AUT in an advanced "observation mode". Test results and errors viewable in a results view, can be saved as html or xml file.

CMTJava - Complexity measurement tool from Verifysoft GmbH. Includes McCabe cyclomatic complexity, lines-of-code metrics, Halstead metrics, maintainability index.

JavaCov - A J2SE/J2EE Coverage testing tool from Alvicom; specializes in testing to MC/DC (Modified Condition/Decision Coverage) depth. Capabilities include: Eclipse plugin; report generation into HTML and XML; Apache Ant integration and support for test automation.

Jameleon - Open source automated testing harness for acceptance-level and integration testing, written in Java. Separates applications into features and allows those features to be tied together independently, in XML, creating self-documenting automated test cases. These test-cases can then be data-driven and executed against different environments. Easily extensible via plug-ins; includes support for web applications and database testing.

Agitator - Automated java unit testing tool from Agitar Software. Creates instances of classes being exercised, calling each method with selected, dynamically created sets of input data, and analyzing results. Stores all information in XML files; works with Eclipse and a variety of IDEs. Also available are: automated JUnit generation, code-rule enforcement, and more.

PMD - Open source tool scans Java code for potential bugs, dead code, duplicate code, etc. - works with a variety of configurable and modifiable rulesets. Integrates with a wide variety of IDE's.

JLint - Open source static analysis tool will check Java code and find bugs, inconsistencies and synchronization problems by doing data flow analysis and building the lock graph.

Lint4j - A static Java source and byte code analyzer that detects locking and threading issues, performance and scalability problems, and checks complex contracts such as Java serialization by performing type, data flow, and lock graph analysis. Eclipse, Ant and Maven plugins available.

FindBugs - Open source static analysis tool to inspect Java bytecode for occurrences of bug patterns, such as difficult language features, misunderstood API methods, misunderstood invariants when code is modified during maintenance, garden variety mistakes such as typos, use of the wrong boolean, etc. Can report false warnings, generally less than 50%.

CheckStyle - Open source tool for checking code layout issues, class design problems, duplicate code, bug patterns, and much more.

Java Development Tools - Java coverage, metrics, profiler, and clone detection tools from Semantic Designs.

AppPerfect Test Studio - Suite of testing, tuning, and monitoring products for java development from AppPerfect Corp. Includes: Unit Tester, Code Analyzer, Java/J2EE Profiler and other modules.

Cactus - A simple open-source test framework for unit testing server-side java code (Servlets, EJBs, Tag Libs, Filters, etc.). Intent is to allow fine-grained continuous testing of all files making up an application: source code but also meta-data files (such as deployment descriptors, etc) through an in-container approach. It uses JUnit and extends it. Typically use within your IDE, or from the command line, using Ant. From Apache Software Foundation.

JUnitPerf - Allows performance testing to be dynamically added to existing JUnit tests. Enables quick composition of a performance test suite, which can then be run automatically and independent of other JUnit tests. Intended for use where there are performance/scalability requirements that need re-checking while refactoring code. By Mike Clark/Clarkware Consulting, licensed under the BSD License.

Abbot Java GUI Test Framework - Testing framework by Timothy Wall provides automated event generation and validation of Java GUI components, improving upon the very basic functions provided by the java.awt.Robot class. (Abbot = "A Better 'Bot'). The framework may be invoked directly from Java code or accessed without programming through the use of scripts via 'Costello', a script editor/recorder. Suitable for use both by developers for unit tests and QA for functional testing. Free - available under the GNU Lesser General Public License

JUnit - Framework to write repeatable java unit tests - a regression testing framework written by Erich Gamma and Kent Beck. For use by developers implementing unit tests in Java. Free Open Source Software released under the IBM Public License and hosted on SourceForge. Site includes a large collection of extensions and documentation.

jfcUnit - Framework for developing automated testing of Java Swing-based applications at the UI layer (as opposed to testing at lower layers, for which JUnit may be sufficient). Provides recording and playback capabilities. Also available as plugins for JBuilder and Eclipse. Free Open Source Software from SourceForge site.

JBench - Freeware Java benchmarking framework to compare algorithms, virtual machines, etc. for speed. Available as binary distribution (including documentation), source distribution, or jar file.

Clover - Code coverage tool for Java from Atlassian. Fully integrated plugin for Eclipse, IntelliJ IDEA and projects using Apache ANT and Maven. View coverage data in XML, HTML, PDF, or via a Swing GUI. Tracks cyclomatic complexity. TestOptimization automatically prioritises just the tests needed to cover the particular changes made.

Structure101 - Java source code visualization tool from Headway Software. Lets user understand, measure, and control architecture, design, composition, and dependencies of code base. Analyzes byte code and shows all dependencies, at all levels and between all levels; method, class, package, application. Measures code complexity using a measurement framework called XS. For Windows, Linux and Mac OS X.

Java Tool Suite from Man Machine Systems - Includes JStyle, a Java source analyzer to generate code comments and metrics such as inheritance depth, Cyclomatic Number, Halstead Measures, etc; JPretty reformats Java code according to specified options; JCover test coverage analyzer; JVerify Java class/API testing tool uses an invasive testing model allowing access to internals of Java objects from within a test script and utilizes a proprietary OO scripting language; JMSAssert, a tool and technique for writing reliable software; JEvolve, an intelligent Java code evolution analyzer that automatically analyzes multiple versions of a Java program and shows how various classes have evolved across versions; can 'reason' about selective need for regression testing Java classes; JBrowser class browser; JSynTest, a syntax testing tool that automatically builds a Java-based test data generator.

JProbe Suite - Collection of Java debugging tools from Quest Software; includes JProbe Profiler and JProbe Memory Debugger for finding performance bottlenecks and memory leaks, LProbe Coverage code coverage tool, and JProbe Threadalyzer for finding deadlocks, stalls, and race conditions. JProfiler freeware version available.

Krakatau Professional for Java - Software metrics tool from Power Software includes more than 70 OO, procedural, complexity, and size metrics related to reusability, maintainability, testability, and clarity. Includes Cyclomatic Complexity, Enhanced Cyclomatic Complexity, Halstead Software Science metrics, LOC metrics and MOOD metrics. Has online advisor for quality improvement.

Jtest - ParaSoft's Jtest is an integrated, automatic unit testing and standards compliance tool for Java. It automatically generates and executes JUnit tests and checks whether code follows 400 coding standards and can automatically correct for many.

DevPartner Java Edition - Debugging/productivity tool from Microfocus (formerly from Compuware, formerly from NuMega) to detect and diagnose Java bugs and memory and performance problems; thread and event analysis, coverage analysis. Integrates with several Java IDE's.

TCAT for Java - Part of Software Research's TestWorks suite of test tools; code coverage analyzer and code analysis for Java; written in Java.

Open Source code analyzers listing - A listing of open source Java code analysis tools written in Java.

Open Source code coverage tools listing - A listing of open source Java code coverage tools written in Java.

Open Source Java test tools listing - A listing of open source tools and frameworks for Java testing, written in Java.

Open Source web test tools listing - A listing of open source web test tools and frameworks written in Java.

Read more »

Load and Performance Test Tools

Load and Performance Test Tools

Tsung - Free open-source multi-protocol distributed load testing tool supported by Process-One. Can be used to stress HTTP, WebDAV, SOAP, PostgreSQL, MySQL, LDAP and Jabber/XMPP servers. SSL is also supported. OS monitoring (CPU, memory and network traffic) using SNMP, Munin or Erlang agents on remote servers. XML configuration system; several sessions can be used to simulate different type of users. Dynamic sessions can be described in XML. User think-times and the arrival rate can be randomized using a probability distribution. HTML reports can be generated during the load to view response times measurement, server CPU, etc. Developed in Erlang.

PerformanceXpert - Performance and load-testing solution available as a service over the Internet. Includes unlimited hardware and software already pre-configured and ready to run on-demand. Can realistically simulate thousands of virtual users and execute various load scenarios from multiple geographic locations (North America, Europe, and Asia). Includes web-based test management, archiving, repository, cloud-based monitoring, rich scripting language, and supports HTTP, HTTPS, web-services, XML, TCP, SQL, Login and more. Utilizes JMeter and Selenium. Lite and Pro versions.

LoadUI - Free open source cross-platform load testing tool from Eviwear/SmartBear Software. Using the soapUI Runner component (also from Eviwear/SmartBear), can leverage pre-existing functional soapUI TestCases and run them in loadUI. This integration enables support for HTTP(S), HTML, SOAP/WSDL and REST to AMF, JDBC, JMS and POX. Using loadUI Agents, can distribute loadUI TestCases to any number of Agents locally and remotely. Comprehensive analystics/reporting capabilities.

AppViewWeb - Cloud-based performance testing service from AppNeta that provides visibility into the network performance of web applications. Especially useful for QA test engineers conducting application pre-deployment testing on WAN networks. When testing web-based applications for CRM, VoIP, Video, Citrix, VMware and database management applications it's helpful to understand the performance of these applications from the perspective out to remote site end users; AppViewWeb provides such insight.

Sandstorm - Load testing tool from Impetus Technologies Inc., supports Web, Mobile and Email protocols. Supports Flex and Ajax, protocols such as Http, Https, Web Services, POP3, SMTP, DNS, SIP, WAP, Applets, and Java serialized objects. Rich and extensible framework for runtime modification of test case using JavaScript and core Java APIs. Integrated resource monitoring for most of the popular web, app and db servers.

Multi-Mechanize - Multi-Mechanize is an open source framework by Corey Goldberg for web performance and load testing. It allows you to run simultaneous python scripts to generate load (synthetic transactions) against a web site or web service. Results can be saved in CSV format along with an HTML report containing stats and graphs. Proficiency with Python, HTTP, and performance/load testing is recommended to use Multi-Mechanize successfully.

Load2Test - Performance/load/stress/high availability testing tool from Enteros Inc. Can capture real production workload for playback simulating thousands of users for top down testing or back-end testing at the component layer. Also available in the Amazon EC2, RackSpace or PlatForm Labs cloud environments. Integrated performance management and root cause analysis system automatically collects performance metrics across load test infrastructure and applies cross-component correlation to identify performance degradations and bottlenecks. Targeted platforms/OS's: Load2Test Controller: Windows, Linux; Load2Test Test Nodes: Windows, Linux. Load2Test performance monitors and root cause analysis data collectors - OS: Windows, Linux, HP/UX, SUN Solaris, IBM AIX Databases: Oracle, DB2, SQL Server, Sybase, MySQL, PostgreSQL; App: .NET, JBoss, WebLogic, WebSphere, Oracle Application Server (OAS), GlassFish, Tomcat, JettyStorage: NetApp Filers, IBM DS8300

Xceptance LoadTest - Load testing and regression tool from Xceptance Software Technologies, Inc for web and Java and other app load testing. Includes recording capabilities. XLT Cloud Service available. Tests implemented as JUnit 4 test cases. For web-based tests, the framework provides a (headless) browser that can emulate Internet Explorer or Firefox behaviour. Can execute client-side JavaScript in the emulated web browsers and that way it simplifies the creation of test cases for Web 2.0 applications. Platform independent due to tool being implemented in Java; test scripting in Java or Ruby. Free for up to five virtual users.

SiteBlaster - Web site load and stress testing tool; shareware. Can be used to rapidly submit requests to a site, or can pause a random amount of time between submissions, approximating user behavior. During testing the pages being tested will be displayed. Reports created on test completion. Designed to be very easy to use; intended for software developers and architects who want some early indication about performance characteristics of the web sites they create. Simulates MS IE web browsing functionality; a web page that is well behaved in IE should be well behaved in SiteBlaster. Best used to test those sites that use URL query strings to pass data to its web page(s). PDF user guide available. For Windows.

Load-Intelligence - Affordable load-testing “Software as a Service” from Cloud-Intelligence. Software and unlimited hardware all included. JMeter users can execute their test-scripts in an unlimited, pre-configure, distributed environment. Neither setup nor installation are required. Immediate access to JMeter logs, reports, test script, CSV files and more.

LoadStorm - A web-based load testing tool/service as a distributed application that leverages the power of Amazon Web Services to scale on demand with processing power and bandwidth as needed. As the test loads increase to hundreds or thousands of virtual users, LoadStorm automatically adds machines from Amazon's server farm to handle the processing. Tests can be built using the tool in such a way as to simulate a large number of different users with unique logins and different tasks.

BrowserMob - On-demand, self-service, low-cost, pay-as-you-go service from Neustar enables simulation of large volumes of real browsers hitting a website. Utilizes Amazon Web Services, Selenium. Uses real browsers for each virtual user so that traffic is realistic, AJAX & Flash support is automatic. Browser screen shots of errors included in reports.

Load Impact - Online load testing service from Gatorhole/loadimpact.com for load- and stress- testing of your website over the Internet; access to our distributed network of load generator nodes - server clusters with very fast connections to enable simulation of tens of thousands of users accessing your website concurrently. Free low level load tests for 1-50 simulated users; higher levels have monthly fees.

Pylot - Open source tool by Corey Goldberg for generating concurrent http loads. Define test cases in an XML file - specify requests - url, method, body/payload, etc - and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support. Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, ramp-up time, test duration. Real-time stats and error reporting are displayed.

AppLoader - Load testing app from NRG Global for web and other applications accessible from a Windows desktop; generates load from the end user's perspective. Protocol independent and supports a wide variety of enterprise class applications. Integrates with their Chroniker monitoring suite so results of load testing can be correlated with system behavior as load is increased. Runs from Win platforms.

fwptt - Open source tool by Bogdan Damian for load testing web applications. Capabilities include handling of Ajax. Generates tests in C#. For Windows platforms

JCrawler - An open-source stress-testing tool for web apps; includes crawling/exploratory features. User can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load on the web application. Load parameters (hits/sec) are configurable via central XML file; fires up as many threads as needed to keep load constant; includes self-testing unit tests. Handles http redirects and cookies; platform independent.

vPerformer - Performance and load testing tool from Verisium Inc. to assess the performance and scalability of web apps. Use recorded scripts or customized scripts using Javascript. Targeted platforms: Windows

Curl-Loader - Open-source tool written in 'C', simulating application load and behavior of tens of thousand HTTP/HTTPS and FTP/FTPS clients, each with its own source IP-address. In contrast to other tools curl-loader is using real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of openssl. Activities of each virtual client are logged and collected statistics include information about: resolving, connection establishment, sending of requests, receiving responses, headers and data received/sent, errors from network, TLS/SSL and application (HTTP, FTP) level events and errors.

Gomez Web Load Testing - An on-demand load testing service from Gomez.com/Compuware. Utilizes Gomez’s Active Network providing on-demand active monitoring from 150+ enterprise-grade servers running in world-class, high-bandwidth data centers around the globe, and Gomez’s Active Last Mile which provides on-demand active monitoring from 150,000+ real, consumer-grade desktops in 168+ countries.

StressTester - Enterprise load and performance testing tool for web applications from Reflective Solutions Ltd. Advanced user journey modeling, scalable load, system resources monitors and results analysis. No scripting required. Suitable for any Web, JMS, IP or SQL Application. OS independent.

The Grinder - A Java-based load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.

Proxy Sniffer - Web load and stress testing tool from from Ingenieurbüro David Fischer GmbH Capabilities include: HTTP/S Web Session Recorder that can be used with any web browser; recordings can then be used to automatically create optimized Java-based load test programs; automatic protection from "false positive" results by examining actual web page content; detailed Error Analysis using saved error snapshots; real-time statistics.

Testing Master - Load test tool from Novosoft, capabilities include IP spoofing, multiple simultaneous test cases and website testing features for sites with dynamic content and secure HTTPS pages.

JKool Online - Performance measurement and monitoring service from Nastel Inc. for web-based J2EE and SOA applications; start and stop live data monitoring whenever needed; drills down to JMS, JDBC, method calls, servlets and sessions with simple one-click option to view live session details; built-in support for jBoss, WebLogic and IBM WebSphere Application Server.

Funkload - Free web load testing, stress testing, and functional testing tool by Benoit Delbosc written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF.

Avalanche - Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Simulates as many as two million concurrently-connected users with unique IP addresses, emulates multiple Web browsers, supports Web Services testing Supports HTTP 1.0/1.1, SSL, FTP, RTSP/ RTP, MS Win Media, SMTP, POP3, DNS, Telnet, and Video on Demand over Multicast protocols.

Loadea - Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.

LoadManager - Load, Stress, Stability and Performance testing tool from Alvicom. Runs on all platforms supported by Eclipse and Java.

QEngine Performance Tester - Automated testing tool from Zoho Corp. for performance testing (load and stress testing) of web applications and web services; J2EE, .NET, AJAX, PHP, Ruby on Rails, SOAP Web Services etc. Supports multiple browsers on Linux and Windows.

NeoLoad - Load testing tool from Neotys for web and mobile apps, with clear and intuitive graphical interface, no scripting/fast learning curve (Javascript for most advanced cases). Can design complex scenarios to handle real world applications. Clear and comprehensive reports and test results. Supports latest technologies, including AJAX, FLEX, GWT, Java Serialization, and more. Load test from the private lab and from the cloud. Compatibility with mobile applications. Multi-platform.

LoadComplete - Automated web load test tool from SmartBear Software.

QTest - Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.

Test Perspective Load Test - Do-it-yourself load testing service from Keynote Systems for Web applications. Utilizes Keynote's load-generating infrastructure on the Internet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web application infrastructure.

SiteTester1 - Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests, HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.

httperf - Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code.

WAPT - Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.

http://www.microsoft.com/visualstudio/en-us/products/2010-editions/test-professional/overview Visual Studio Test Professional 2010 - A suite of testing tools for Web applications and services that are integrated into the Microsoft Visual Studio environment. These enable testers to author, execute, and manage tests and related work items all from within Visual Studio. Includes Lab Management capabilities.

OpenLoad - Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum hardware.

Apache JMeter - Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.

TestMaker - Free open source utility maintained by PushToTest.com and Frank Cohen, for performance, scalability, and functional testing of Web application. Features test authoring of Web applications, Rich Internet Applications (RIA) using Ajax, Service Oriented Architecture, and Business Process Management environments. Integrates Selenium, soapUI, TestGen4Web, and HTMLUnit to make test development faster/easier. Repurposes tests from these tools into load and performance tests, functional tests, and business service monitors with no coding. Repurposes unit tests written in Java, Jython, JRuby, Groovy, and other dynamic scripting languages. Runs on any platform.

Siege - Open source stress/regression test and benchmark utility; supports basic authentication, cookies, HTTP and HTTPS protocols. Enables testing a web server with a configurable number of concurrent simulated users. Stress a single URL with a specified number of simulated users or stress multiple URL's simultaneously. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein's torture.pl, but allows stressing many URLs simultaneously. Distributed under terms of the GPL; written in C; for UNIX and related platforms.

JBlitz - Load, performance and functional test tool from Clan Productions. Runs multiple concurrent virtual users.to simulate heavy load. Validates each response using plain text or regular expression searches, or by calling out to your own custom code. Full Java API. For testing and 'bullet-proofing' server side software - ASPs, JSPs, servlets, EJBs, Perl / PHP / C / C++ / CGI scripts etc.

WebServer Stress Tool - Web stress test tool from Paessler AG handles proxies, passwords, user agents, cookies, AAL.

Web Polygraph - Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C++ source available; binaries avail for Windows.

OpenSTA - 'Open System Testing Architecture' is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.

PureLoad - Java-based multi-platform performance testing and analysis tool from PureLoad Software. Includes 'Comparer' and 'Recorder' capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.

ApacheBench - Perl API for Apache benchmarking and regression testing. Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.

Torture - Bare-bones Perl script by Lincoln Stein for testing web server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.

WebSpray - Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. For Windows.

eValid LoadTest - Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.

Load Testing by Web Performance - Load test tool emphasizing ease-of-use, from Web Performance Inc. Supports all browsers and web servers; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. For Windows, Linux, Solaris, most UNIX variants.

Optima Quality Studio - A collection of load testing, capture/playback, and related tools from Technovations for performance testing of web sites. Modules include WebCorder, Load Director, Report Generator, Batch, Manager, and others. WebSizer load testing module supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. For Windows.

FORECAST - Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C++ scripting; windows browser or network recording/playback. Supports binary encoded data such as Adobe Flex/AMF, Serialised Java objects etc.SSL; supports NTLM, kerberos, proxies, authentication, redirects, certificates, cookies, caching, bandwidth limitation and page validation. Virtual user data can be parameterized. Works with a wide variety of platforms.

http-Load - Free load test application from ACME Labs to generate web server loads, from ACME Software. Handles HTTP and HTTPS; for Unix.

QALoad - Tool from Microfocus (formerly from Compuware) for load/stress testing of web, database, and character-based systems. Supports HTTP, SSL, SOAP, XML, Streaming Media. Works with a variety of databases, middleware, ERP.

IBM Rational Performance Tester - Performance testing tool from IBM/Rational; has optional extensions to Seibel applications and SAP Solutions. Supports Windows, Linux and z/OS as distributed controller agents; provides high-level and detailed views of tests.

SilkPerformer - Enterprise-class load-testing tool from Microfocus (formerly from Borland, formerly from Segue). Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity.

Radview's WebLoad - Load testing tool from Radview Software. Capabilities include over 75 Performance Metrics; can view global or detailed account of transaction successes/failures on individual Virtual Client level, assisting in capturing intermittent errors; allows comparing of running test vs. past test metrics. Test scripting via visual tool or Javascript. Wizard for automating non-GUI-based services testing; DoS security testing.

Loadrunner - HP's (formerly Mercury's) load/stress testing tool for web and other applications; supports a wide variety of application environments, platforms, and databases. Large suite of network/app/server monitors to enable performance measurement of each tier/server/component and tracing of bottlenecks.

Read more »

Difference between Retesting and Regression Testing

When I think about regression testing, I think about any testing that involves the reuse of tests (manual or automated) or test ideas (regression charters for example -- a regression test does not necessarily need to be the exact same test) to manage the risks of change. This could include testing for bug fixes, testing to make sure a bug fix didn't break something else, or testing to make the introduction of new or changed features didn't break something else.

If you think about regression testing in that broad sense, then retesting is most likely a subset of that definition. It's a test that you're repeating for some reason or another; so I would call it a regression test. The only place I can't think of where I would use the two terms interchangeably is when talking about a specific defect. I probably wouldn't say "I'm regression testing defect X." I'd say "I'm retesting defect X." But it's semantics more then anything. That retesting is still regression testing.

Read more »

How to start the recording using QTP?

• Record or click the Record button. When the Record and Run Settings dialog box opens to do this ;
•   In the Web tab, select Open the following browser when a record or run session begins.
•   In the Windows Applications tab, confirm that Record and run on these applications (opened on session start) is selected, and that there are no applications listed.

Read more »

Software Testing Wiki: My view

Software Testing Wiki: My view: Clean Code has more to say to the programmer. Clean Coder also resonates to testers from my point of view.

Read more »

My view

Clean Code has more to say to the programmer. Clean Coder also resonates to testers from my point of view.

Read more »

Test Harness

In software testing, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It has two main parts: the Test execution engine and the Test script repository.

Test harnesses allow for the automation of tests. They can call functions with supplied parameters and print out and compare the results to the desired value. The test harness is a hook to the developed code, which can be tested using an automation framework.

A test harness should allow specific tests to run (this helps in optimising), orchestrate a runtime environment, and provide a capability to analyse results.

The typical objectives of a test harness are to:

• Automate the testing process.
• Execute test suites of test cases.
• Generate associated test reports.

A test harness may provide some of the following benefits:

• Increased productivity due to automation of the testing process.
• Increased probability that regression testing will occur.
• Increased quality of software components and application.
• Ensure that subsequent test runs are exact duplicates of previous ones.
• Testing can occur at times that the office is not staffed (ie. at night)
• A test script may include conditions and/or uses that are otherwise difficult to simulate (load, for example)

An alternative definition of a test harness is software constructed to facilitate integration testing. Where test stubs are typically components of the application under development and are replaced by working component as the application is developed (top-down design), test harnesses are external to the application being tested and simulate services or functionality not available in a test environment. For example, if you're building an application that needs to interface with an application on a mainframe computer but none is available during development, a test harness maybe built to use as a substitute. A test harness maybe part of a project deliverable. It’s kept outside of the application source code and maybe reused on multiple projects. Because a test harness simulates application functionality - it has no knowledge of test suites, test cases or test reports.

Read more »

How to insert a Checkpoint in QTP

Inserting Checkpoints

Check point is a verification point, it takes expected result from the user and compares with actual results during execution and provides test results.

There are 11 Checkpoints available in QTP:

1.      Standard check point

2.      Text check point

3.      Text area check point

4.      Bit map check point

5.      Data base check point

6.      Accessibility check point

7.      XML Check point (from Application)

8.      XML Check point (from Resource)

9.      Page check point

10. Image checkpoint

11. Table checkpoint

Note 1: From 6 to 11 checkpoints are only for Web.

Note 2: From 9 to 11 checkpoints are hidden checkpoints, we can insert these checkpoints through standard checkpoint.

1.    Standard Checkpoint:

It checks object property values. We can use this checkpoint for checking any property value.

Navigation for Inserting standard checkpoint

Keep tool under recording mode > place cursor in desired location > Insert > check point > Standard checkpoint > Show the object  > click okay > select property and enter expected results & click Ok and stop Recording.

Navigation for Editing standard checkpoint

Select Checkpoint statement and right click > choose checkpoint properties option > modify the value > click Ok.

Navigation for Deleting standard checkpoint:

Select Checkpoint statements and right click > choose delete option.

Inserting Standard check points through active screen:

View > Active Screen >place cursor in desired location >Place mouse pointer on active screen & right click> choose insert standard checkpoint option > click ok > enter expected result > click ok

Note: Inserting Standard check points through keyword view same as in expert view.

Note: We can insert standard checkpoints in 3 ways.

a.      Through expert view,

b.      Through keyword view,

c.      Through Active screen.

2) Text Checkpoint:

 It Checks object’s text property value in different ways.

Navigation:

Keep tool under Recording mode >Insert menu > checkpoint > Text checkpoint > Show the object > click ok > Select options

(Match case; ignore spaces, exact match, text not displayed.)

We can select one or more options > click ok & stop Recording.

3)  Text Area Checkpoint:

It checks the text area present in the application.

Navigation:

Keep tool under Recording mode > Insert menu> Checkpoint > Text area checkpoint > Mark the area of text > select one or more options

(Match case, ignore spaces, exact match, text not displayed.)

Click ok and stop recording.

4.      Bitmap checkpoint:

It compares bitmaps; we can compare complete bitmaps as well as part of the bitmaps.

Navigation:

Keep tool under Recording mode > Insert menu > Checkpoint > Bitmap checkpoint > show the Bitmap >click ok >select “check only selected area” option if we want to compare part of the bitmap > click ok >stop recording.

5.      Database checkpoint:

It checks Content of the back end Database.

Navigation:

Insert > checkpoint > Database checkpoint >choose “specify SQL statement manually” option >click next > click create > select machine data source > Select DSN (QT_flight32) > click ok > enter SQL statement (select * from orders) > finish > click ok.

Note: here we do not need to put tool under Recording mode and we do not need AUT since data is from backend.

6.      Accessibility check point:

It checks whether the webpage in our web application is developed according to W3C (World Wide Web consortium) Rules and Regulations or not.

It is a configurable checkpoint, according to our requirements, we can customize.

Configuring accessibility checkpoint:

Tools menu> options >web > advanced > check/uncheck items > click apply > click ok

Invigilation:

Keep tool under recording mode with web environment >insert>checkpoint>accessibility checkpoint>show the webpage>click ok>click ok>stop recording.

Result Criteria:

a)     If item is available but not according to W3C rules then fail.

b)     If an item is available, according to W3C rules then Pass.

c)     If an item is not available then result would be pass

7. XML Check point (from Application)

It checks content of the XML file.

Navigation:

Keep tool under Recording mode in web environment > insert menu > checkpoint (from application)> show the xml pages >click ok > stop Recording.

8. XML Check point (from Resource)

It checks content of the XML file.

Navigation: Insert menu > checkpoint >xml checkpoint (from resource) > browse path of the XML File > click ok > click ok.

Note: 1. If XML file is an individual and path available, and then we can go for inserting xml checkpoint from resource.

Note: 2. If XML file is part of web application, separate path is not available then we can choose inserting XML checkpoints from application.

9. Page checkpoint:

It checks number of Links, Images and Loading time in a web page.

It is a hidden checkpoint; we can insert this through standard checkpoint.

Navigation:

Keep tool under Recording mode with web environment > Insert menu > checkpoint > Standard checkpoint >show the web page > click ok > click ok > stop recording.

10. Image checkpoint:

It checks Image property values.

Navigation:

 Keep tool under Recording mode with web environment > Insert menu > checkpoint >standard checkpoint > show the image > select image > click ok > click ok >stop recording.

11. Table checkpoint:

It checks content of the web tables.

Navigation: Keep tool under Recording mode under web environment > Insert menu > checkpoint > standard checkpoint >show the web table > click ok >stop recording.

Read more »