Software Testing Tools

There are various software testing tools available for QAs to increase efficiency, visibility, accuracy, and ability of their work. The tools available are fast evolving and vary in terms of form, type, purpose, size, price, and so on. Here I provide a bunch of examples categorized according to their domain.

Lifecycle Management (overall solution)

Azure, a.k.a TFS, a cloud computing service created by Microsoft for building, testing, deploying, and managing applications and services through Microsoft-managed data centers.

Test Planning

Testlink. TestLink is a web-based test management system that facilitates software quality assurance. The platform offers support for test cases, test suites, test plans, test projects and user management, as well as various reports and statistics.

Test Automation

Selenium(framework).Selenium is a portable framework for testing web applications. Selenium provides a playback tool for authoring functional tests without the need to learn a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages, including C#, Groovy, Java, Perl, PHP, Python, Ruby and Scala. The tests can then run against most modern web browsers. Selenium runs on Windows, Linux, and macOS. It is open-source software.
Robot(framework).Robot Framework is open and extensible and can be integrated with virtually any other tool to create powerful and flexible automation solutions. Being open source also means that Robot Framework is free to use without licensing costs. Robot Framework has easy syntax, utilizing human-readable keywords. Its capabilities can be extended by libraries implemented with Python or Java. The framework has a rich ecosystem around it, consisting of libraries and tools that are developed as separate projects.
Katalon(IDE).Katalon Studio is an automation testing solution developed by Katalon LLC. The software is built on top of the open-source automation frameworks Selenium, Appium with a specialized IDE interface for web, API, mobile and desktop application testing.

API Testing

SoupUI. SoapUI allows you to test REST and SOAP APIs with ease – as it has been built specifically for API testing.
Postman. Seems to be a bit more popular, more up to date look and more features than the previous.

Security Testing

OWASP. The Open Web Application Security Project (OWASP) is an online community that produces freely-available articles, methodologies, documentation, tools, and technologies in the field of web application security.
Kali Linux. Kali Linux is a Debian-derived Linux distribution designed for digital forensics and penetration testing.

Load Testing

JMeter. Apache JMeter is an Apache project that can be used as a load testing tool for analyzing and measuring the performance of a variety of services, with a focus on web applications.

Cypress Testing

Prerequisite for using Cypress.io

Node.js and NPM. Node.js is an open-source, cross-platform, JavaScript runtime environment that executes JavaScript code outside a web browser. NPM is a package manager for the JavaScript programming language. It is the default package manager for the JavaScript runtime environment Node.js.

Advantages of Cypress.io

Open source, free, automatic, scalable end to end testing, easier than selenium.
Tests run inside real browsers instead of a headless browser.
Visibility is considered during the test.
The scripts are readable.

Limitation

Cannot test multiple tabs at the same time.
Cannot test multiple superdomains in the same test.

Where to write your first test?

E.g. cypress > integration > first_test.js

Basic skills

• textbox input: get the element, test span, input
• selecting elements by: HTML tags or indexes or CSS classes or data-cy attribute (recommended)
• aliasing element: basically referring a long element selection as a short referral
• command results: assert on the element selected with cy.get
• beforeEach: easy repetition in beginning of each test
• base URL: specified in a JSON file
• interactions: double click, check(checkbox), uncheck, select(dropdown), hover over
• assertions: length, existence, contain, ...
• retry: some actions will be automatically retied to fight against flakiness

Postman API Testing in One Page

Why do we need to test API?

1. Verify functionality. 

2. Open to the public. So there is a risk for unwanted data access or system modification. 

What can we use Postman for during API testing?

1. Make API calls (manual/automated)

2. Assert responses using Javascript

3. Organize tests

4. Integrate tests with integration tools

Advantage comparing to UI automation

1. Faster

2. More stable

Postman Skills

1. Collections and folders. They are collections of API requests. It is a good way to manage tests and share with co-workers. 
2. GET requests. Retrieving data. 
3. POST requests. Publishing data. 
4. PUT requests. Updating data. 
5. DELETE requests. Removing data.  
6. Negative responses. To trigger negative responses, consider scenarios such as wrong url or missing parameter or illegal character, etc.. 
7. Common error codes:
   400 bad request; 401 unauthorized; 403 fobidden; 404 not found; 500 internal server error; 503 service unavailable; 504 timeout...
8. Environment variables. Like variables in programming languages, they are used to avoid repetition and increase efficiency. They may be defined manually or extracted and set automatically. 
9. Status code assertion. E.g. to.have.status() 
10. Body assertion. E.g. text().to.include("")
11. Header assertion. E.g. to.have.header("") 
12. Response time assertion. E.g. response.responseTime.to.be.below()
13. Debugging with the console. 
14. JSON assertion. E.g. response body JSON value check
15. Nested JSON assertion. Just include the path of the element you are asserting. 
16. JSON structure. An object is a collection of name-value pairs; an array is a group of objects. 
17. Running test collections as a group. 
18. Automating the test collections with NodeJS and Jenkins

Search Precision and Sort by Test

Searching bars are very common in most web applications. In order to test the search function, a few perspectives are available for consideration. For one example, we can calculate the search precision. In the context of information retrieval, precision is the fraction of retrieved entries that are relevant to the query.
Search Precision = Relevant Search Results / Total Search Results
As for the definition of relevance, it is totally up to the tester. It can be defined as the presence of search keyword in the result or even similar results if you are testing a fuzzy search. Here, I would like to provide an example of search precision calculation based on the keyword.

    //accumulate the number of search results 
    i = i + driver.FindElements(By.XPath(/*you need to specify*/)).Count; 
    //accumulate the number of search results that contains the keyword 
    j = j + driver.FindElements(By.XPath(/*you need to specify*/)).Count; 

    //calculate the search precision rate 
    r =  j / i;

Demystify DevOps

DevOps is short for (software) development and operations.
DevOps is a concept, which means it is like a large basket and contains many things including DevOps philosophies, culture, practices and tools. The main goal of DevOps is to accelerate the process from integration, testing, releasing to deployment and infrastructure management. Bass, Weber, and Zhu defined it by:
"DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality. "[1]
With this concept, development team and operation team are not siloed any more. Instead, they are merged and unified on a toolchain level across the following stages shown in the diagram:

Such toolchain involves all kinds of tools that you may already be familiar with:

1. Code — Git, Jira, Eclipse
2. Build — Maven, Gradle, Ant
3. Test — Selenium, JMeter, NUnit, Specflow
4. Release — Jenkins
5. Deploy — Chef, Ansible
6. Operate — Amazon Web Service, Azure 
7. Monitor — Nagios, Sensu

The advantages of DevOps are obvious: speed, reliability, scalability, security and collaboration.

[1] Bass, Len; Weber, Ingo; Zhu, Liming. DevOps: A Software Architect's Perspective.

A Picture that Demystifies Smoke Testing, Sanity Testing and Regression Testing

Click the image to view in larger size.

This picture explains what are Smoke Testing and Sanity Testing, and also illustrates the relationship among those plus Regression Testing.

*This original image is on Guru99.com

Common Interview Questions for Test Analysts

1. What is the difference between a use case and test case?

They are both instances of using the product. But they differ in terms of perspective and scope. A use case is describing a user achieving a general goal, and a test case is listing the particular steps to test a function which could be part of the general goal. For example, users to sign up at Facebook could be a use case, and the corresponding test cases could be testing signup button, testing personal information text boxes...



2. Which phase of SDLC does the tester begin to write test cases?

Ideally, the early the better which makes it the planning phase. This is mainly because of early tests prevents errors in the later stage.

3. What is the difference between functional documents and business documents?

Functional documents include the technical details of the software developed and usually maintained by the developers' team. Whereas business documents are kept by the product owners to keep track of the business requirements.


4. Who are the different stakeholders involved in different phases of SDLC?

Requirements– Business Analyst
Design – System Architect
Coding – Software Developer
Testing – Tester
Implementation – Operational Team
Maintenance – Production Support Team


5. Why do we need a separate environment for the developers and testers?

Because end users may not use the same environment as the developers, so testers have to make sure the product works properly in the users' environment, not the developers'.


6. What is the role of a Business Analyst in different phases of SDLC?

Requirements– Requirements gathering and analysis
Design – Document changes to the requirements
Coding – Keep in touch with the developers
Testing – Judging for user acceptance testing
Implementation – Forward beta version to users


7. What is the role of a Developer in different phases of SDLC?

Requirements– Offering technical suggestions
Design – Generating development strategy
Coding – Coding
Testing – Work with testers to fix bugs
Implementation – Fix bugs found in beta version


8. What is the role of a Test Analyst in different phases of SDLC?

Requirements– Offering technical suggestions
Design – Generating test strategy
Coding – Understanding codes
Testing – Testing
Implementation – Helping beta version


9. What is the role of a Customer in different phases of SDLC?

Requirements– Express their needs
Design –
Coding –
Testing – Helping with user acceptance testing
Implementation – Helping with beta version testing


10. What is the testing process followed in a company ideally?

Requirements analysis > Test planning > Test case development > Environment setup > Test execution > Test cycle closure


11. What are the activities performed in test design and test execution?

In test design, test conditions and test cases are specified according to the testing strategy.
During test execution, testers will run the test cases and report bugs or defects found. They will also follow up the bug fixing and retest. In this stage, testers have to work closely with developers and product owners.


12. Explain the difference between an SDLC and STLC?

SDLC is the whole picture of developing a software with multiple parties involved. STLC is actually part of SDLC that mainly focus on work related to testing.


13. What is test scenario, test condition and test case?

Test scenario and test condition are almost the same by describing the overall objective of a series of test cases. Test case is the detailed step by step instruction of verifying a particular situation.


14. What is the process to write a manual test case?

Understand the requirement > try it out manually > define the test condition > describe step by step execution > provide expected results and other criteria


15. What is a test cycle and why it is important to carry out testing in cycles?

It is a framework to carry out a structured series of testing including smoke testing, sanity testing, regression testing, system testing, integration testing, and end to end testing. The advantage of using such framework is to guarantee quality, uniformity, and efficiency.  

16. Explain Requirement Traceability Matrix (RTM) in simple terms.

It is a document that maps and traces user requirement with test cases.