QA by Justine

The QA Commandments

2021-01-07T16:08:32+00:00

Don’t test only what the story tells you to

User stories often have acceptance criteria added by a product owner or business analyst. These are usually helpful for QA to know when they can consider a story as “Passed QA”. However, it’s important not to just stick to the AC and test only the “Happy Path” for a feature. The AC usually tells us what we want to happen, but it’s our job as testers to think about what could happen. Remember to use your QA skills to think outside the box and run validation and negative tests to improve your test coverage on a story.

Don’t forget about user experience

Sometimes when we’re working to a deadline and have a lot of QA tasks we get so focused on just the functionality of a feature. If we can’t find any bugs and the tests pass then we move the ticket to the “Passed” column, and move on to next thing on the list. When this happens, we often miss user experience issues because we “can’t see the wood for the trees”. A feature might work as designed, and with no functional defects but it might be really painful and frustrating to use. Usability issues can result in loss of users just as much as a functional defect can. Remember to put yourself in the shoes of the end user when you’re testing, and if a behaviour would frustrate you, recommend changing it.

Don’t let developers bias you

Sometimes, with a complicated user story a developer may meet with the QA team to do a demo of the feature they’ve developed. This can aid understanding in the scope of what needs tested, and any risk areas. However, it can also result in the opposite effect where testers are biased by the developers into assuming something will work. A dev may demo a particular section of a feature and casually mention “I tested this really well and it all works great”. We might then gloss over thoroughly testing this section, and miss an important defect. The developer is already biased because they created the feature, and they’re confident everything works before they hand it over for testing. A testers motto should be “Everything is guilty until proven innocent.” Just because a developer is confident that something will work perfectly in production doesn’t mean we should be until we’ve seen it for ourselves in a variety of scenarios.

Don’t write bad bug reports

If a bug report is effective then the chances of having the issue fixed are higher, we’ve all had bugs rejected because the developer can’t reproduce it.

Ensure you have a clear, unambiguous, step by step guide to reproduce the issue. Putting these steps in a bullet point or numbered list makes it more difficult for a step to be missed. A screenshot of the issue is always useful, a screen recording of the steps is often even better. Some good tools for screen recording are: ShareX on Windows or LICEcap on macOS

Be specific with your report, don’t write an essay about what’s wrong, keep it to the point. Be careful also not to combine multiple problems even if they seem similar in nature. Compartmentalise your thinking and separate the issues into separate tickets so they can be addressed and retested separately.

Include as much relevant information as you can to make it quicker and easier for the developer to diagnose the problem. If the issue is only occurring in one browser – state that. If there are errors in the browser console log, copy and paste the full log into the bug report. If your application makes use of a logging tool like Splunk, check the logs for anything relevant and include these in your report.

XPath Selectors

2020-04-13T17:08:32+01:00

XPath is a query language that we can use to select elements on a webpage. In QA it’s commonly used in automation to select textboxes or buttons on the page that we can interact with.

Xpath is arguably the slowest way of finding elements on a page so you should use this as a last resort if your element does not have an id or name, and if CSS selectors aren’t an option. XPath is usually useful when you need more complicated logic than what CSS selectors allow for.

Testing XPath in your browser

In Chrome you can just open DevTools in the Elements tab, hit Ctrl+F or cmd+F and paste your xpath into the search.

In other browsers you can use the Console tab in DevTools to execute your XPath query eg.

$x("//div")

General XPath

Expression	Description
`nodename`	Selects all nodes with the name “nodename“
`/`	Selects from the root node
`//`	Selects nodes in the document from the current node that match the selection no matter where they are
`.`	Selects the current node
`..`	Selects the parent of the current node
`@`	Selects attributes

Wildcards

XPath wildcards can be used to select unknown XML nodes.

⚠️ Use these sparingly, wildcards increase the time it takes to find your element(s)

Wildcard	Description
`*`	Matches any element node
`@*`	Matches any attribute node
`node()`	Matches any node of any kind

Children and Descendants

Expression	Description
`//ul/li/a`	a single “/” finds a child element that is directly below the parent
`//ul//li//a`	two “//” finds any descendants of the parent at any level

Attributes

Expression	Description
`//input[@id="id"]`	id of element
`//input[@class="class"]`	class of an element, matches the class exactly, if your element has multiple classes then see Conditional Logic or contains() below
`//input[@type="submit"]`	matches an element of type “input” who has an attribute “type” with the value of “submit”
`//a[contains(@href, 'something')]`	matches an element of type a that has an href attribute that contains “something”
`//a[starts-with(@href, 'something')]`	matches an element of type a that has an href attribute that starts with “something”
`//a[ends-with(@href, 'something')]`	matches an element of type a that has an href attribute that ends with “something”

Attribute Operators

Expression	Description
`//a[@id = "xyz"]`	attribute equals a value
`//a[@id != "xyz"]`	attribute does not equal a value
`//a[@price > 25]`	attribute value is more than
`//a[@price = 9.80 or price = 9.70]`	attribute value equals a value OR another value
`//a[@price > 9.00 and price < 9.90]`	attribute value matches one condition AND another condition

Order Selectors

If you want to select the first, last or nth element in a list you can use [n] in XPath

ℹ️ XPath lists start at 1 not 0

Expression	Description
`//ul/li[1]`	returns the first “li” child of all “ul” elements
`//ul/li[2]`	returns the second “li” child of all “ul” elements
`//ul/li[last()]`	returns the last “li” child of all “ul” elements

Conditional Logic

You can use OR logic to have multiple selectors for one element, this can be useful for example if your DOM changes when the user is on a mobile device. Instead of having to use different elements for desktop vs. mobile you can use one element and have an OR selector that handles both cases

Expression	Description
`//a[contains(@class, 'enabled') and contains(@class, 'visible')]`	AND – if you need to match an element that has this attribute and value, and another attribute and value
`//a[contains(@class, 'enabled')][contains(@class, 'visible')]`	AND – this is another way of writing the previous expression
`//a[contains(@class, 'enabled') or contains(@class, 'visible')]`	OR – if you need to match an element that has either this attribute value or a different attribute value
`//a[contains(@class, 'enabled')] \| //a[contains(@class, 'visible')]`	OR – this is another way of writing the previous expression

Other Useful Expressions

Expression	Description
`//button[text()="Submit"]`	matches the text of an element exactly
`//button[contains(text(),"Go")]`	matches an element that contains text “Go”
`//product[@price > 2.50]`	you can use maths to match on an attribute that has a value higher or lower than specified

Types of Testing

2020-03-28T16:08:32+00:00

Testing is usually divided into two main categories – functional and non-functional.

Functional Testing

Functional testing is a type of testing which verifies that each function of the software application matches its requirements.

Every functionality of the system is tested by providing appropriate input, verifying the output and comparing the actual results with the expected results. The testing can be done either manually or using automation

Functional Testing types include:

Unit Testing
Integration Testing
System Testing
Sanity Testing
Smoke Testing
Interface Testing
Regression Testing
Beta/Acceptance Testing

Non-functional Testing

Non-functional testing is a type of testing to check non-functional aspects (performance, usability, reliability, etc.) of a software application. It’s basically everything that isn’t covered by functional testing.

A good example of non-functional test would be to check how many people can simultaneously login into a software.

Functional tests are usually easy to write expected results for eg.

1 + 1 should be equal to 2

With non-functional tests it can be more difficult to define expected results. How long should a page take to load? How many concurrent users should you expect to support at any one time?

Functional testing is based on customer’s requirements whereas Non Functional testing is based on customer’s expectations.

Non-functional testing is equally important as functional testing and affects client satisfaction.

Non-functional Testing types include:

Performance Testing
Load Testing
Stress Testing
Volume Testing
Security Testing
Compatibility Testing
Install Testing
Recovery Testing
Reliability Testing
Usability Testing
Compliance Testing
Localization Testing

What is QA?

2020-03-20T16:08:32+00:00

Quality assurance (QA) is a way of preventing mistakes and defects in products and avoiding problems when delivering products or services to customers. This defect prevention in quality assurance is often referred to as “shift left” since it focuses on quality earlier in the process.

Quality assurance is made up of activities implemented in a quality system so that requirements and goals for a product, service or activity will be fulfilled. It is the systematic measurement, comparison with a standard, monitoring of processes and an associated feedback loop that confers error prevention.

Quality assurance includes two principles: “Fit for purpose” (the product should be suitable for the intended purpose); and “right first time” (mistakes should be eliminated). QA includes management of the quality of raw materials, assemblies, products and components, services related to production, and management, production and inspection processes. The two principles also manifest before the background of developing (engineering) a novel technical product: The task of engineering is to make it work once, while the task of quality assurance is to make it work all the time.

Don’t Forget your QA Soft Skills

2019-05-09T17:08:32+01:00

Communication and teamwork are skills that are vital to effectively meet project goals. When it comes to QA, don’t spend so much time working on your technical skills that you neglect your soft skills.

Remember to be Tactful

As testers, we are often the bearers of bad news. A lot of the time we have to tell the developers that their baby is ugly. When we’re reporting bugs we have to be mindful that a dev may have spent weeks or months working on the feature we’re testing. They’re normally very proud of what they’ve created, and can get understandably defensive about any sort of criticism. It is possible to deliver bad news tactfully, and diplomatically in order to cushion the blow. Objective language can be used when reporting the bug to make the issue more palatable - “This is what I see happening” as opposed to “This is what you’ve done wrong, and this is what I think you should do differently.” People are usually more receptive to this type of language because it feels like less of a personal attack.

Don’t play the Blame Game

It can be stressful when you’re coming up to an important deadline and you come across a high impact bug during your testing. Sometimes it can seem as though a developer has performed absolutely no testing of their own before turning a feature over to QA. It’s important to remember that developers are often under pressure to get features finished as quickly as possible and they don’t always have time to test things as thoroughly as they would like. Blaming the developer for causing the bug is a waste of time and doesn’t help the team to meet project goals.

Similarly when a bug makes it to production – nothing is gained by spending time trying to figure out if it was the fault of the BA with missed requirements, the designer with bad mockups, the developer with buggy code, or another tester who missed it. When a failure happens in production the entire team is responsible for resolving the problem quickly and efficiently, and figuring out how their processes can be improved to reduce the likelihood of a similar failure happening again.

Celebrate the Wins

A big part of our jobs as QA Engineers is to constantly find the flaws in everything. It’s our responsibility to go through things with a fine-tooth comb and report any defect or inconsistency. It’s important to also remember to celebrate the things that are done right. Don’t forget to thank or celebrate a member of your team for a particularly well written requirements document, a really helpful design document, a great quality feature in the application. This attitude can be especially helpful in a retrospective meeting - remember to include items in the “what went right this sprint?” column instead of filling up the “what went wrong this sprint?” column.

How Not to do QA

2019-04-02T17:08:32+01:00

QA is not involved in each stage

QA is often referred to as simply “Testing”. In reality, testing is only one of the things that a good QA engineer brings to a team. QA stands for Quality Assurance - a QA engineer aims to ensure the quality of the development process and its results.

A good SDLC process has QA involved at each stage, right from planning through to support and maintenance. Testers often bridge the gaps between technical developers, product owners or stakeholders, and the end user. Testers are uniquely placed to have the most overall knowledge of how a product works from a user perspective, and also understand the technical details of what’s going on behind the scenes to make it work that way.

QA engineers then have a clear overview of the entire SDLC process and can point out inefficiencies and gaps. It’s their job to not only improve the quality of the end product, but also the process that’s used to develop it.

Focusing on Cost rather than Value

The earlier an issue is found the less money it costs. Consider a high priority issue reported to the customer support team that affects a large amount of users. The time taken to triage, investigate, debug, resolve, test, redeploy, and communicate the fix to stakeholders vs. if the problem had been discovered in a QA environment. The turnaround time for a fix would be a lot smaller and involve less people than dealing with a Production issue.

The severity of the issue also needs to be taken into account, companies have gone out of business due to manufacturing a poor quality product. This could be because they faced lawsuits, or simply because their reputation was irreversibly damaged with their target consumer.

An extreme example of this is the Therac-25, a radiation therapy machine that caused accidental radiation overdoses killing at least six patients. This was caused by poor software developed with no QA process.

Sacrificing Quality for Speed

There is a common misconception that QA slows down development. Many project managers/scrum masters see QA as a blocker to getting tasks into the Done column. However, if you factor the QA team into the process at an earlier stage it can actually save time in the long run.

One of the most important stages in the SDLC is Requirements Gathering. Mistakes made during this stage echo throughout the entire lifecycle and end up having an impact on the product that makes it into the live environment. QA engineers often know your application better than anyone else on the team, so having them involved in requirements gathering and scope setting enables them to point out which solutions will and won’t work, and to remind the team of areas in the application they might not have considered.

Introducing non functional QA like performance and load testing to your process can have a tangible impact on your server and hosting costs. Less hardware is required to run your application if performance is optimised, and less storage space is needed if we avoid saving data we don’t actually use.

We can also improve staffing costs by investing in QA. If we prioritise QA resources and catch issues early we don’t need to invest as many resources into a Support team because they won’t have as much to do!