Testing terminology: a pragmatic glossary

A test case is a test instruction. One instruction out of many that needs to be performed as part of the testing process.

That instruction could be just a prompt or idea for something to look at, or it could be more elaborate with formal structure.

When formally structured, test cases typically include:

• title
• details about the environment
• any pre-conditions that need establishing
• the specific steps to execute
• a list of one or more expected outcomes
• and other metadata, such as priority, user roles, affected components, dependencies etc

Whether to use shorter or longer test cases depends on the context the test team is working in. In a process-heavy environment, where teams are using traditional Test Case Management tools, they will probably be required to structure each test case with many of the above elements. In a more agile environment, especially when the testing is more exploratory in nature, test cases can be just brief prompts of aspects of the product to remember to take a look at.

If you're new to testing, and/or don't have to use a Test Case Management tool, then we'd strongly steer you toward keeping your test cases (prompts) as short and simple as possible. They're much faster to write and much easier to maintain.

A Test Script usually means a collection of test instructions.

These might be instructions for a human to follow as part of manual testing, or might be the code used to define automated tests.

Test Scripts can sometimes be confused with Test Suites, but the distinction doesn't really matter. A Test Suite is usually only used for a collection of Test Cases as part of Test Case Management. Whereas a Test Script is more likely to mean either a piece of the code in an automated test environment or a list of test instructions for a human to follow.

In Testpad, a Script is used to mean the list of instructions for a human to follow – formatted as a checklist with outline structure.

A Test Plan is an ambiguous term: it can sometimes refer to the high-level document setting out the approach that a team will be using for the whole testing project, and other times it can just mean a list of test instructions for a team to work through.

As the high-level document, think of it as the strategy document, setting out the bigger picture and answering the "what," "why," and "how" of the testing effort, aligning stakeholders and setting expectations.

As just a list of test instructions, it's essentially the same as a Test Script.

Exploratory Testing is a type of (manual) testing where the tester is free to explore the product, inventing tests as they go, hunting for issues and inconsistencies, and crucially, reacting to how the product is behaving.

Exploratory testing can be contrasted with fully-scripted testing where every action to perform is defined by the test instruction (usually a detailed Test Case as part of Test Case Management).

Exploratory testing is easily the best kind of manual testing. It takes advantage of the tester's tenacity and intuition, and their experience of what the product did last time vs is doing this time. Exploratory testing allows for hundreds more test ideas to be carried out in the same time as there's no documentation overhead to write (and maintain) a Test Case per test idea.

A common formalization of Exploratory Testing is Session Based Test Management, characterized by directing test sessions with charters (a list of topics to investigate, along with time allocations for each) and testers documenting their test ideas and the results as free-form notes as they proceed.

See also Testpad's pragmatic approach to exploratory testing that adds a bit more control to the process (and therefore easier to plan, track and report on) without losing the exploratory nature of the testing itself.

Testpad's take on making exploratory testing more manageable by adding just enough structure through detailed (but not too detailed!) test prompts. Rather than "test the login system" as you might find in a high-level charter, prompts break testing down into specific aspects like "check password complexity rules" or "verify account lockout behavior."

Test prompts are therefore more specific ideas of what to investigate but leave the exact approach up to the tester. They avoid the rigid step-by-step instructions of scripted test cases, preserving the tester's freedom to explore while ensuring important functionality isn't overlooked.

Results can be collected as simple pass/fails against each prompt, creating a clear visual overview of what's working and what isn't, while additional comments and bug links capture any specific problems. This structure makes it easier to plan testing scope, track progress, generate clear reports, and reuse test ideas in future releases - all without losing the key benefits of exploratory testing's human-driven discovery process.

A quick check that the most basic, core functionality of your software works before investing time in deeper testing. If users can't log in, or the main page won't load, there's no point proceeding with more detailed testing of advanced features or edge cases.

While "sanity testing" sounds formal, it's really just common sense - check the obvious things first. Some teams treat it as a distinct phase with its own test plan, but that's usually overkill. Simply having experienced team members quickly verify basic functionality before starting full testing is often enough. The goal is to save time by catching major issues early, not to create extra processes.

Testing your complete software product as a whole, rather than testing individual components in isolation. System testing looks at how all the pieces work together - from user interfaces through to databases and external integrations. It's about verifying the entire system works correctly in a realistic environment.

Most testing is system testing, even if teams don't call it that. Unless you're specifically testing isolated components (unit testing) or how your system connects with others (integration testing), you're probably doing system testing. Don't get hung up on the term - it's just testing your actual product the way it will be used, which is exactly what most teams need to focus on.

Testing to make sure problems that you've fixed previously stay fixed. Think of it like a growing checklist - every time you find and fix a significant bug, you add a test to catch it if it reoccurs. No product is bug-free, but having the same bugs reappear is embarrassing and suggests a team that doesn't learn from its mistakes.

While often associated with automation, regression testing doesn't have to be automated to be valuable. Some tests are difficult or impractical to automate, and manual regression testing is better than none at all. The key is having these checks in place, whether they're automated or done by hand.

A visual way to organize ideas by creating branching diagrams that expand out from central concepts. In testing, mind maps are particularly valuable for breaking down complex products into testable areas - starting with core features at the center and branching out to explore different aspects that need testing and ensure comprehensive coverage.

A mind map starts with your product's core features or capabilities at the center, then branches out to explore related areas, test scenarios, and potential risks. It's a natural way to develop test plans - you can keep expanding branches as you think of new things to test, and the visual structure helps identify gaps in your testing strategy.

While mind maps are great for brainstorming test ideas, you don't need special diagramming tools to create test plans this way. The same structure works as a simple indented list, with main features as top-level items and related tests nested underneath. As you think of new test ideas, you can add them at any level - just like drawing new branches on a mind map. This makes it practical to turn your test ideas into executable plans.

A single run-through of a test plan or script, collecting results, bugs, notes and perhaps other evidence (e.g. screenshots) along the way.

If you have a list of tests, think of a test run as one column next to your tests, recording what happened when someone worked through the tests - what passed, what failed, and any observations made along the way.

During the testing of a product, as part of one release cycle, you might make multiple test runs through your tests, covering different:

• environments
• browsers or devices
• testers
• builds

Some teams (or indeed tools) build lots of processes around a test run, with effort spent preparing them and getting sign-off when completed etc. Depending on your process requirements and context, this can easily be overkill. What matters is having a clear record of what was tested, when, by whom and with what result.

Testing to verify software meets its intended purpose, typically near the end of development. In a client project, this means the client checking the delivered product matches what they ordered. For end-user products, it means verifying the software works effectively for its target users - either through testing with actual users or by having the test team work through realistic user scenarios.

Neither approach needs complex processes or special tools - what matters is getting meaningful feedback about whether the software truly works for its intended audience before release. This could be as simple as observing users interact with the software or having clients verify their requirements have been met.

Usually identical in meaning to User Acceptance Testing, however, can sometimes refer to the more general process of agreeing (through verification checks) that the requirements of a product have been met, whoever is performing those checks.

Testing that verifies your software does what it's supposed to do - the basic "does it work?" testing that most people think of when they hear "testing." It focuses on checking features against their requirements or expected behavior.

While it sounds formal, functional testing is just checking if your software's features work correctly. This could be verifying a login form accepts valid credentials and rejects invalid ones, checking that a shopping cart calculates totals properly, or ensuring a document editor can actually save files.

Most testing is functional testing, even if teams don't call it that. If you're verifying features work as intended, you're doing functional testing.

This differs from non-functional testing like performance testing (is it fast enough?), security testing (is it secure?), or usability testing (is it easy to use?).

(Also called Bug Tracking Tools)

Tools for recording and managing software issues found during development and testing. Issues are often bugs (defects), but can also be features, roadmap ideas, enhancements and other items of work relating to the development of the software.

They help teams track issues from discovery through to resolution, storing details like severity, steps to reproduce, screenshots, and discussions about the software changes required.

Traditional issue trackers are standalone tools focused solely on issue (defect) management. Modern development platforms often include integrated issue tracking as part of broader project management features - think Jira, GitHub Issues, or GitLab - allowing teams to manage issues alongside code, features and other work items.

Key features typically include assignment to owners, status tracking, and reports showing trends over time. The best tools let teams quickly capture issues without excessive form-filling while still maintaining enough detail for developers to understand and get to work on the issue.

Issue trackers shouldn't be confused with test management. Test management tracks what's being tested in the current release, while issue tracking manages issues throughout their lifecycle - often spanning multiple releases until they're fixed.

See Issue Tracking Tools

Tools that help teams organize and track their testing process. They typically store test plans, record results, and generate reports to show testing progress and product quality.

Traditional test management tools are built around formal "test cases" - requiring detailed documentation of steps, expected outcomes, and other metadata for each test. While this structured approach suits some contexts (especially highly regulated industries), it can be unnecessarily rigid and time-consuming for many teams.

Modern alternatives take a more flexible approach, letting teams create simple checklists of what needs testing, add detail where valuable, and generate clear reports showing what works and what doesn't. This lighter approach particularly suits exploratory testing, where testers benefit from the freedom to investigate and react to what they discover rather than following rigid scripts.

The best tool choice depends on your needs. Spreadsheets work well for simple testing. Traditional test case management tools suit process-heavy environments. Lighter tools like Testpad offer middle ground - more capable than spreadsheets but without the complexity of traditional platforms.

Testing performed by humans rather than automated scripts or programs. Testers directly interact with the software by clicking, typing and observing behavior. Manual testing covers a spectrum from fully-scripted test case execution (following precise steps) through to exploratory testing (reacting and adapting to what's found).

The exploratory form of manual testing is particularly valuable as it leverages human intuition and adaptability to discover unexpected issues that would be difficult to find through scripted or automated approaches.

While automation can be more efficient for repetitive checks, manual testing often remains more practical. The effort to automate some tests may outweigh their value, especially for features that change frequently or are difficult to verify programmatically.

Testing scenarios that are unusual, extreme, or at the boundaries of normal usage. Edge cases are situations that might be rare but could still occur, like entering the maximum possible value in a field, testing with unusually large files, or trying operations with poor network connectivity.

While it's important to verify edge cases work correctly, don't get carried away trying to test every conceivable unusual scenario. Focus on edge cases that are reasonably likely to occur and would have significant impact if they failed. For example, testing how your e-commerce site handles a shopping cart with 1000 items might be less valuable than testing what happens when a payment transaction times out.

A structured approach to Exploratory Testing where testing is organized into time-boxed sessions guided by a charter. The charter defines what aspects of the product should be explored in each session and how long to spend on each aspect.

During testing, testers maintain a detailed journal of their activities, observations, and any issues discovered. These session notes form the primary output and documentation of the testing performed.

SBTM helps make exploratory testing more manageable by providing a framework for planning sessions and tracking progress. However, the reporting quality depends heavily on testers' note-taking skills, and the session notes don't easily provide quick summaries of testing status or product readiness.

A collection of related test cases or test scripts, typically grouped by feature, component, or testing objective. While some tools and processes make formal distinctions between suites, scripts, and plans, these terms often mean similar things in practice - they're just ways to organize collections of tests.

Data used during testing to verify how software behaves with different inputs. This could be sample customer records, transaction histories, or any other information needed to exercise the software's functionality.

Creating good test data is often overlooked but important - you need enough variety to test different scenarios while keeping it manageable. Rather than creating massive datasets, focus on having representative examples that cover key use cases and edge conditions. For example, an e-commerce site might need just a few orders with different payment methods and shipping options rather than thousands of transactions.

Most teams benefit from maintaining a core set of reusable test data, supplemented as needed for specific test scenarios, and iterated on over time as experience grows from release to release.

Automated testing at the code level, where individual components (functions, classes, modules) are tested in isolation to verify they work as intended. Unit tests are often written by developers alongside their code and can be run automatically whenever code changes, typically as part of continuous integration.

While valuable for catching issues early in development, unit testing only verifies individual pieces work correctly, not how they work together or how the complete product behaves. Additionally, since unit tests are usually written by the same developers writing the code, they may embed the same incorrect assumptions about how components should behave.

Unit tests complement rather than replace system-level testing of the whole product.

Testing how different parts of your software work together, rather than testing them in isolation. While it sounds formal, Integration Testing is really just checking that when you connect different pieces of your system - whether that's different services, APIs, or third-party tools - everything still works as expected.

Where System Testing looks at the complete product working as a whole, Integration Testing focuses more narrowly on the connections between specific components. However, in practice, the distinction often blurs - if you're testing your product as users would actually use it, you're already testing both how components integrate and how the whole system works.

The distinction becomes more relevant when you're specifically focusing on testing components separately (Unit Testing) or when you have complex systems with many external connections that need special attention. For most teams, especially those with straightforward products, you don't need to treat Integration Testing as a separate process. Just make sure your normal testing includes checking any connections between different parts of your system, particularly after making changes that affect how these parts interact.

A quick check that basic functions work before proceeding with more thorough testing. The term comes from hardware testing - if you turn on a circuit board and it starts smoking, there's no point testing anything else yet.

In software development, smoke testing is essentially the same as Sanity Testing: verifying nothing obviously broken has made it into the build. It's about making sure there aren't any major showstoppers before investing time in detailed testing. Rather than being a formal process, it's about efficiently catching major problems early - if the software is clearly unstable, you can fix those issues before spending time on more nuanced testing.

The term can sometimes be used to mean "soak testing" - running systems for prolonged periods to find stability issues that only emerge over time - but the "sanity testing" interpretation is the more common one.

Testing your software's performance characteristics - for example: speed, responsiveness, stability and scalability. This helps identify how your software behaves under different conditions, from normal loads to peak loads and beyond, to better understand its limits.

In web apps, performance testing might look at factors like how quickly pages load, how many users can be supported simultaneously, and whether the software remains stable during extended use. It helps find issues like slow response times, resource constraints and breaking points before they impact real users.

While dedicated performance testing tools exist for simulating heavy loads and gathering detailed metrics, basic performance issues can often be uncovered through manual testing.

Load testing, often used interchangeably with stress testing, is seeing how your software performs under heavy usage. This includes testing expected peak loads (what's typically meant by Load Testing) and testing system limits by deliberately increasing usage until performance degrades or breaks (what's typically meant by Stress Testing).

While specialized tools exist to simulate many users or process large amounts of data, even basic testing with multiple team members can reveal important performance issues. The key is understanding your usage patterns and testing within - and beyond - those boundaries.

Stress testing, often used interchangeably with load testing, is seeing how your software performs under heavy usage. This includes testing system limits by deliberately increasing usage until performance degrades or breaks (what's typically meant by Stress Testing) and testing expected peak loads (what's typically meant by Load Testing).

While specialized tools exist to simulate many users or process large amounts of data, even basic testing with multiple team members can reveal important performance issues. The key is understanding your usage patterns and testing within - and beyond - those boundaries.

Testing how easy and intuitive your software is to use, typically by observing users attempting specific tasks. Can be performed at any stage - early in development using prototypes (even paper mockups) to validate design choices before implementation, or later with beta/released versions to identify improvements needed.

The scope can vary from informal observations by coworkers to formal studies with recruited participants representing your target users, complete with video recording and eye tracking. Many companies now outsource this to specialist usability testing services that handle participant recruitment and provide detailed analysis of user behavior.

The key is watching users interact with your software without helping them, noting where they get confused or stuck. These insights help improve the user experience before and after release.

Testing to find vulnerabilities in your software that could be exploited by malicious users. This ranges from basic checks like ensuring passwords are properly encrypted and users can only access their own data, to sophisticated penetration testing that actively tries to break into the system.

While specialized security firms offer comprehensive testing services using advanced tools and techniques, development teams should still perform fundamental security checks throughout development. Essential areas include input validation, authentication, access control, and data protection - especially for sensitive information.

The goal is identifying and fixing security holes before attackers find them. This applies not just to obvious entry points like login screens, but to the entire system including APIs, databases, and infrastructure.

Alpha testing isn't a well-defined concept for many teams. Where a formal definition is attempted, it typically refers to the final phase of testing a product internally, before releasing it for real users to try in a "beta testing" phase. However, some teams use "alpha testing" to simply mean "very early testing" within the development lifecycle, possibly even conducted during ongoing development.

It's not a critical concept or type of testing to worry about per se. What matters more is that you have comprehensive internal testing comprising unit tests, integration/system tests, and exploratory testing. Ideally, the bulk of this testing should happen on stable release candidates just before releasing (either as a general release or during a beta testing phase).

Rather than getting caught up in terminology, focus on ensuring your product works as intended through whatever testing methods make sense for your team and project. The label "alpha testing" is less important than having effective testing processes in place at the appropriate stages of your development cycle.

Beta testing is when you release your product to a limited group of real users before making it generally available. It's essentially a controlled first exposure to actual users in their real environments, allowing you to gather feedback and catch issues that internal testing might have missed.

While many companies make beta testing sound formal with elaborate "beta programs," at its core, it's simply getting your software into the hands of people who aren't part of your development team. These users will inevitably use your product in ways you didn't anticipate and on configurations you hadn't considered.

Beta testing is most valuable for products with large or diverse user bases where internal testing can't realistically cover all usage patterns. For simpler products or those with a small, well-understood user base, an extensive beta phase might be unnecessary. Some teams use "beta" labels primarily as risk management - signaling to users that they should expect some issues.

Black box testing is testing a product without knowledge of its internal code or structure. You interact with the software purely as a user would, focusing on inputs and outputs rather than how the system processes information. You're essentially treating the software as a "black box" where you can't see inside.

This approach is particularly valuable for finding usability issues, verifying that the software meets requirements from the user's perspective, and catching problems that might occur in real-world usage. Most manual testing is naturally black box testing - testers work through features, trying different inputs and checking if the outputs match expectations.

Black box testing is especially effective when performed by people who weren't involved in development, as they don't have preconceptions about how things should work. This often leads to discovering unexpected bugs that developers might miss because they test according to how they built the system rather than how users might actually use it.

While understanding the code can sometimes help testers be more thorough (see White Box Testing), black box testing remains essential because it mirrors how your actual users will experience the product.

White box testing is testing with full knowledge of the code and internal structure of the software. Instead of just looking at inputs and outputs, testers examine how the software processes information, often with access to the source code, data structures, and algorithms used.

This approach is valuable for ensuring all code paths are tested, finding issues with specific functions or modules, and verifying that internal operations work as intended. Unit testing is a common form of white box testing, where developers test individual components in isolation.

White box testing is typically performed by developers or technical testers who understand the codebase. It's particularly effective at finding issues like memory leaks, security vulnerabilities, and edge cases that might be difficult to trigger through regular usage.

Both white box and black box testing have their place in a comprehensive testing strategy. White box testing helps ensure technical correctness, while black box testing confirms the software actually meets user needs.

Gray box testing is a hybrid approach that sits between black box and white box testing. Testers have partial knowledge of the internal workings of the software, but don't need complete access to or understanding of the source code. They understand the general architecture, data flow, and algorithms without necessarily examining the code line-by-line.

This approach is particularly useful when testers need enough structural knowledge to create more effective tests but don't require (or have access to) the full codebase. For example, understanding the database schema can help testers craft better tests for data integrity, even if they can't see the application code that interacts with the database.

Gray box testing often occurs naturally in smaller teams where testers work closely with developers and gain familiarity with the system architecture over time. It's also common in security testing, where testers need to understand system components to identify potential vulnerabilities, but approach the system from an external perspective.

The benefit of gray box testing is that it combines the user-focused perspective of black box testing with enough technical understanding to target likely problem areas more efficiently. This makes it particularly valuable for complex systems where purely external testing might miss important scenarios.

A flaw in the software: it does something wrong, or fails to do something it should. "Defect" and "bug" mean the same thing - defect is the formal term you'll find in textbooks and process standards, bug is what everyone actually says.

Some methodologies draw fine distinctions between errors (the human mistake), defects (the resulting flaw in the code) and failures (the flaw showing itself when the software runs). It's a tidy chain of causation, and knowing it might help you pass an ISTQB exam, but no working team talks this way.

What matters more than the terminology is a good bug report: what you did, what happened, what you expected to happen. A clear reproduction beats any amount of classification.

The sequence of states a bug moves through from discovery to closure. A typical version: new, confirmed, assigned to a developer, fixed, verified by a tester, closed - with detours available for "won't fix," "can't reproduce" and "duplicate." Every issue tracker implements some flavor of this as a workflow.

The useful part is the reminder that "fixed" isn't the end. Someone should verify the fix actually works before the bug is closed, and skipping that step is how bugs come back.

Beyond that, don't over-engineer the workflow. Small teams do fine with three states (open, fixed, closed), and elaborate ten-state life cycles with strict transition rules mostly generate admin. Use whatever your issue tracker does by default and spend the saved effort on testing.

Digging past a bug's symptom to find the underlying cause, and, done properly, asking why the process let that cause through. The classic technique is the "five whys": keep asking why until you hit something fundamental. The report timed out; why? The query was slow; why? The table had no index; why? Nobody reviews query performance before shipping; and so on.

For most bugs, full RCA is overkill: find it, fix it, add a regression test, move on. Save the deeper analysis for the bugs that matter - the outage, the data corruption, the same class of bug appearing for the third time. Those are the ones where fixing only the symptom guarantees a repeat.

When you do it, keep it blameless. The interesting answer is rarely that someone made a mistake; it's whatever made the mistake easy to make and hard to catch.

The number of defects found per unit of software size - classically bugs per thousand lines of code, sometimes per module or feature. The idea is to compare components or releases: a module with unusually high defect density might need a rewrite, extra review, or extra testing.

As a rough pointer to where the risky code lives, it has some value. As a quality metric, it falls apart quickly: it counts bugs found (which measures the testing as much as the code), every bug counts equally (a typo and a data-loss bug are each "one"), and lines of code is a poor measure of size to begin with.

For most small teams this is metrics theater. You already know which module everyone's scared to touch; you don't need a ratio to tell you. Unless you're in an organization that requires reporting it, you lose nothing by ignoring it.

Testing without any plan or documentation - someone sits down and pokes at the software, following their nose. No test cases, no charters, no record of what was tried.

It's often described dismissively, as the unprofessional cousin of exploratory testing, but the two are close relatives. The difference is that exploratory testing works from at least a lightweight plan and leaves a record of what was covered; ad-hoc testing does neither.

Don't let it be the whole strategy forever: with nothing written down, you can't say what was tested. But doing ad-hoc testing is infinitely better than doing zero testing, and it's a fine starting point - the journey of iterating on your test process usually begins here. Write down what you tried as you go and you're already most of the way to a reusable checklist.

Testing done the agile way: continuously through development, inside the team, rather than as a separate phase at the end handled by a separate department. Whoever's testing - dedicated testers, developers, or a mix - works alongside the code being written, testing each slice of functionality as it's built.

The word carries an ambiguity worth knowing about (our agile testing guide unpacks it): capital-A Agile is the formal methodology world of ceremonies, sprints and frameworks; lowercase agile is the broader working style - short cycles, feedback, working software over comprehensive documentation. Agile testing splits the same way: there's a formal school with its own quadrants and vocabulary, and there's the everyday sense of testing continuously with lightweight plans that change as the product does.

Since almost everyone now develops in something they call agile, for most teams agile testing is simply testing. The useful core is the Manifesto's trade applied to testing: time spent learning about the product beats time spent maintaining paperwork about the product.

The practice of merging code changes into a shared repository frequently - usually many times a day - with an automated build and test suite run on every change. The point is to catch integration problems within minutes of the code being written, instead of discovering weeks of accumulated divergence when everyone finally merges.

CI is a development practice more than a testing one, but it changes the testing picture: the automated checks (mostly unit tests, perhaps some integration tests) act as a constant safety net, so obviously-broken builds rarely reach human testers.

What CI doesn't do is replace testing the product. The suite only checks what someone thought to automate. New features, usability, and the "does this feel right?" questions still need humans - ideally testing release candidates that CI has already confirmed are stable enough to be worth the time.

The idea of testing continuously throughout development, rather than saving it all for a phase at the end. In practice the term usually means running automated tests at every stage of a delivery pipeline - on every commit, every merge, every deployment - so feedback about quality arrives constantly rather than in one late batch.

It's a sensible idea wrapped in an unnecessary label. Testing early and often has been good advice for decades; "continuous testing" mostly restates it in DevOps vocabulary, sometimes in aid of selling tooling.

Take the substance and skip the ceremony: automate the checks that are worth automating, run them often, and keep some human testing in the cycle for each release. Whether you call the result continuous testing doesn't matter.

A measure of how much of something your tests exercise. For automated tests it usually means code coverage: the percentage of lines or branches executed when the suite runs. For manual testing it's fuzzier - coverage of features, requirements, or risks.

The number is seductive and mostly misleading. "85% coverage" sounds like a fact about quality, but it only says those lines were executed, not that anything meaningful was checked - a test can run code and assert nothing. And coverage of what? Lines of code, features, requirements, browsers, user types? Each gives a different number, and none of them is "how tested is this product."

Coverage earns its keep as a gap-finder: a module at 0%, a feature nobody has looked at. Treat it as a way of spotting what you've missed rather than a score to maximize. Chasing a coverage target produces tests that satisfy the metric, which is not the same as tests that find bugs.

A report at the end of a test cycle summarizing what was tested, what was found, and whether the product looks ready to release. Formal versions (there's an IEEE template) include scope, environments, results by area, outstanding defects, deviations from the plan, and a recommendation.

A good test report answers three questions fast: what's been tested, what's broken, and what happens next. A page of narrative and a results table covers that - and if your test tool can share a live report of results, that beats a document someone compiles by hand a week after the decision was made. (More in our post on simple test reports.)

If nobody ever asks a question about your test summary, it's too long, aimed at the wrong audience, or both. Short and current beats comprehensive and late.

The average time a system or component runs before it fails. It's a reliability-engineering measure at heart: if you make hardware, MTTF (and its repairable-system cousin MTBF, mean time between failures) is how you predict how long units will survive in the field, plan warranty periods, and decide how long a soak test needs to run.

Hardware teams meet these numbers for real: burn-in and soak testing exist precisely to catch the early failures that drag the average down, and results recorded per unit per revision are the raw material the figures get computed from.

Software teams mostly meet MTTF secondhand in SLAs. Software doesn't wear out the way components do - it fails when it hits a state or input nobody tested - so the average time between failures says little about where the next one will come from. Knowing what the term means is usually enough.

The average time it takes to restore a system after a failure - from things-are-broken to things-work-again. Like MTTF and MTBF it comes from reliability engineering, and it shows up in SLAs and DevOps metrics dashboards (it's one of the DORA metrics, as "time to restore service").

Of the mean-time family, this one has the most practical use, because it measures something you can improve: how fast you notice failures, diagnose them, and ship a fix. A team that can recover in minutes can afford to move faster than a team that takes days.

For a small team, though, formally tracking the number is usually metrics theater. You already know whether recovering from problems feels fast or agonizing. Put the effort into the things that improve it - monitoring, deployment speed, the ability to roll back - not into the measuring.

The setup your tests run against: hardware, operating systems, browsers, databases, configuration, test data, and any connected services. "Works on my machine" is the classic symptom of a test environment not matching the real one.

The pragmatic goal is an environment that's realistic where it matters and convenient everywhere else. A staging setup resembling production (same versions, sensible data) covers most needs. Perfect production parity is expensive and rarely necessary; what you want to avoid is testing against a setup so unlike production that the results don't transfer.

Two things are worth doing. Record the environment alongside the results, because "passed" means little without knowing where it passed. And keep the environment somewhere near reality as the product evolves - environment drift is a slow leak, where each release the staging setup gets a little less like production until bugs start slipping through.

A short test instruction describing what to test without spelling out how - "password reset with an expired link" rather than ten numbered steps. It is a test case stripped back to the idea itself.

"Prompt" is Testpad's word for the lines in a script: each line is a prompt, and prompts nest in an outline so related ideas sit together. But the concept travels beyond any one tool. Whether you call them prompts, checks, or one-line test cases, terse instructions that trust the tester to fill in the details are faster to write, easier to maintain, and leave room for the exploratory instincts that find real bugs.

Getting the level of detail right takes a little practice. "Test login" is too vague to direct anyone; full numbered steps are too rigid. A good prompt names one specific aspect and implies its expected outcome - specific enough that a competent tester knows what to look at, open enough that they choose how.

Severity is how bad a bug is; priority is how soon it should be fixed. They often align but don't have to. A crash in a feature nobody uses is high severity, low priority; a typo in your pricing headline is low severity, high priority.

The distinction is worth keeping in your head because it dissolves a common argument: testers rate impact (severity), whoever owns the product decides order (priority). A tester saying "this is severe" and a product owner saying "not this release" can both be right.

Don't build a bureaucracy around it. Five-point scales for each, matrices mapping one onto the other, mandatory fields on every bug - most teams need nothing more than a plain note of how bad it is and someone deciding what gets fixed first.

Divide the possible inputs into groups the software should treat the same (valid ages, ages too low, ages too high) and test one value from each group instead of hundreds.

Of the formal test design vocabulary, this describes something experienced testers do by instinct: don't test fifty middling values that all take the same path through the code. You don't need to document which technique each test used; knowing the name mostly helps with exams and with reading other people's test plans.

Its natural partner is Boundary Value Analysis - once you have the groups, the interesting values sit at their edges.

Test at the edges of your input groups (see Equivalence Partitioning): for an 18-and-over age field, try 17, 18, and the extremes - because bugs cluster at boundaries.

If the terminology reminds you to try zero, negative one, the maximum, the maximum plus one, and the empty input, it has done its job.

The framework that runs automated tests and collects their results - the scaffolding around the tests rather than the tests themselves.

If you're not writing the automation yourself, the useful summary: automated tests run inside a harness against partly simulated surroundings, which makes them fast and repeatable - and is also why a passing automated suite doesn't prove the fully assembled product works. Testing the real thing, connected to real things, is still a job.

The configured setup automated tests run on - hardware, software versions, data. In practice it means much the same as Test Environment (see that entry), and in automated contexts it overlaps with the Test Harness.

You'll hear all three used loosely; the distinction rarely matters.