Testing scope

The scope of a test defines what areas of a customer's product are supposed to get tested, what functionalities to focus on, what bug types the customer is interested in, and what areas or features should not be tested by any means.

If something is in scope, please test it; if something is out of scope, it should not be tested. Understanding the scope of a test is crucial to be a successful tester on our platform.

What information makes up the scope of a test?

Test environment

First things first – The URL given in section Access at the top of the test overview page defines what website or app should be tested. Any other websites or apps are not under test (unless stated differently in the test description). For more information, see Testing Environment.

You can only see access information once the test started (mind the countdown at the top)

Features

Each test contains at least one of what we call features. A feature can describe an area of a product, e.g. a landing page, product overview page or product detail page of a webshop, it can describe a process like the checkout process of a website or an app, or it can even be a particular functionality that should be tested. For example:

When a customer sets up a new test, so-called standard features will be added by default. Customers can use these standard features and modify them if they want or add completely new features.

You can only submit bugs if a corresponding feature exists in the test. For example, if none of the given features covers the checkout process, you cannot submit bugs related to it.

Bug types and severities

Customers are usually only interested in certain bug types and/or severities, not in all of them at once (Functional, Content, Visual, Usability). You can see which bug types and severities are in scope by checking out the payout table in the right sidebar on the test overview page:

If a bug type/severity is not listed there, you won't be able to select it in the bug form, and it is thus out of scope. E.g. if the payout table does not list the bug type Visual, you may not report visual bugs.

Test instructions

Test instructions are displayed below the Access section and they consist of three sections:

Goal of this test: This section usually covers a general purpose of the test. It often names what to focus on, e.g. new feature that is about to be released.

Out of scope: What is not supposed to get tested is specifically stated here. On live websites, you may oftentimes not complete orders at the end of a checkout process, or some areas of a website or an app might not be ready yet. Make sure to stick to this information – disregarding them can have big consequences.

Additional requirements: If there is anything else to point out,  information will be given in this section. This might be information concerning the whole test, credentials for user accounts, dummy payment information, etc.

Attachments

In rare cases, one or more attachments might be provided. Those are usually Microsoft Excel spreadsheets or PDF files given by the customer directly.

Requested devices

You will usually be invited for a specific device of yours. This device is displayed in the right sidebar on the test overview page. For an ideal device coverage, our distribution algorithm invites testers for different devices. You may only use the requested devices that were assigned to you.
If the seats for the requested device are taken, you can join the test with a different device. On the 'Submit a Bug' page you can check with which device you are in the test.

Chat information

The team leader or Customer Success Manager might have provided additional information about the scope or posted important reminders. Please always check the chat if you see that there are new messages.

Out of scope

Out of scope means that something is not supposed to be tested. Note that the following points are always out of scope (unless stated differently in the test description):

Legal problems. We are not legal advisors and the severity of a bug is not determined by legal provisions, frameworks, or standards.

Problems related to smart filters or virus scanners, e.g. blocking the execution of apps.