EDITORIALS

What is software testing?

Man at computer with extra screens

What is software testing?

Software testing isn’t about formalized test cases or fancy burndown charts. It’s not even a quality assurance mechanism. So, what is it? Why does it matter? And how hard can it be?

Testpad

By Testpad

December 18, 2024

Linkedin Logo Twitter Logo Facebook Logo
t

he tech industry has a tendency to overcomplicate testing. There’s formal terminology. There’s a rigid process. There’s a constant push for automation.None of this is wrong, per se. It just makes testing seem more intimidating and difficult than it really is.

But what, exactly, is software testing, and how hard can it be?

Keep reading to learn our take on software testing, why it’s a critical part of the software development life cycle, and how to get started with testing yourself.

What is software testing?

Software testing is one of those terms that you’ll get a different definition for depending on who you ask (we do have a lot of that kind of vocabulary in tech). To IBM, for instance, software testing is:

The process of evaluating and verifying that a product or application does what it’s supposed to do.

ChatGPT has a slightly different (and a bit more convoluted) answer:

The process of systematically evaluating an application or system to identify and address defects, ensure it meets specified requirements, and verify its functionality, performance, security, and usability under various conditions.

What all this jargon really indicates, at least to us, is that software testing is about discovery — learning about your product’s current state.

It’s important to note here that testing and quality assurance (QA) are not one and the same. After all, learning about your product’s current state doesn’t produce quality in its own right. But testing is a key part of the QA process. Because, by exploring the product, you reveal where quality needs attention.

Why is software testing important?

You might immediately jump to “testing surfaces bugs” or “testing uncovers potential user experience enhancements.” Do a cursory Reddit search, and it’s easy to see why:

HB_propmaster: Found a bug in a timesheet/payroll system where the employee was able to get themselves paid double. Bug had been live for at least two years. No employee had ever found it.

Hello____World_____: Huge sections of the database were going missing overnight in our test environment (not production). It turned out an upcoming feature had a schedule overnight task that was actually deleting tons of valid data. If that new feature made it to production, it would have been a disaster.

nutrecht: I worked for a large bank and replaced a developer who created a bug that allowed clients to see the account balances of other people. Basically a mistake of keeping state in a (supposedly) stateless microservice. It even managed to go through code review by security. Fortunately, they found out before the press got wind of it.

While bug reports and lists of feature suggestions might result from testing, the actual value in knowing how your app is performing is that it helps stakeholders decide what happens next. More specifically, testing can answer questions like:

  • Is abc feature ready to ship next week?
  • If not, what needs fixing now?
  • Can the more trivial issues wait until our next release?

At the root of it, testing should answer: where should everyone spend their time?

5 types of software testing

Like most things, there are many ways to approach software testing. Below, we cover a few top ways to go about it and when you might employ each strategy.

1. Sanity testing

Sanity testing is what it sounds like — a quick, manual check to make sure the basics are working. It’s a lightweight form of testing that verifies critical functions work as expected and that it’s worth doing a deeper dive.

Benefits:

  • Limited scope. If an update was made to a particular feature, testers can concentrate on that and make sure it’s working fine before testing the modification in relation to the rest of the app.
  • Saves resources. For small teams or teams on tight deadlines, sanity testing is a good way to make sure obvious issues are caught early before investing in more intensive testing.

When to use it : Before you launch a thorough round — we’re talking days or weeks-long — of scripted or exploratory testing (coming up next).

2. Exploratory testing

Exploratory testing is a hands-on, unscripted approach that encourages testers to think critically and creatively. Instead of following predefined test cases, testers actively explore your software, trying to find hidden bugs and unexpected behaviors — as they see fit. At Testpad, we like to say that exploratory testing is “making it up as you go along - with your brain engaged.”

Benefits:

  • Easy to get started. Testers just need a login and a place to write down their results, observations, and new test ideas.
  • Catches the “unknown unknowns.” Letting testers interact with the product and improvise tests on the fly helps them discover bugs that could’ve slipped through scripted or automated testing.

When to use it: To find edge cases after scripted or automated testing. Ideally, an exploratory tester is someone who did not run scripted tests.

3. Session-based testing

If exploratory testing feels a little too unstructured for your team, session-based testing (SBTM) might be a good happy medium. Session-based testers still have the flexibility to test as they please, but they are confined to testing a specific area of the product for a specific amount of time.

Benefits:

  • Focus. Exploratory testing can be a little too “out there” for some people’s tastes. SBTM gives you the best of both worlds — the freedom to test however you want and the ability to share what you’ve tested, for how long, and what you have left to test.
  • Time management. Your testers might have a lot on their plates. Time-boxing their testing sessions helps them stay on track.

When to use it: When you see the value in exploratory testing but want more control over what you test and/or to give more concrete testing progress updates to higher-ups.

4. Automated testing

So far, we’ve only discussed manual testing strategies. But there is a way to automate your testing: by feeding test scripts into testing software. The main benefit of these tools is speed. They can run millions of tests simultaneously, helping you test your product way faster than a human could.

That said, there are some worrisome limitations to automated testing:

  • Automated tests are only as good as the test scripts you write.
  • They can’t test edge cases and dynamic scenarios that require human intuition.
  • Automated testing platforms can be expensive and time-consuming to set up.

We saw some of these dangers firsthand with the Crowdstrike outage earlier this year.

Reddit Screenshot

Credit: BananaLegitimate6528

So, while automation can increase your efficiency, it’s not a panacea. To deliver reliable, high-quality software, you need to balance automation with human-centered testing.

When to use it: For repetitive tests that need to happen on a regular basis. This frees up your team to focus on manual testing.

5. Regression testing

Regression testing confirms that new changes haven’t broken existing functionality. Technically, you can run automated regression tests. But, again, you run the risk of not testing something you haven’t thought of.

Benefits:

Uncovers unforeseen problems. Whenever something new gets added to your codebase, there’s a possibility of unexpected side effects, especially in complex systems. Regression testing helps you avoid them.

When to use it: After you’ve pushed up new code but before a production release.

How to get started with software testing

The thing about software testing is that it’s easy to go overboard. Testing every minute detail of your app isn’t necessarily the best way to expose the kind of data your stakeholders need. Then again, you don’t want to be so broad with your testing that you accidentally skip over things that end users complain about later.

Since testing exists within real-world constraints, teams must balance available time, resources, and expertise to maximize what they can learn about their product's state within those limitations.

The best way to do that? Keep things simple. Here’s a four-step process to get you going:

1. Assemble a team

The size of your team will depend on your budget and the scope of your product. If it’s a huge platform with tons and tons of features, you may need multiple teams — one for each area of the product. For more straightforward apps, you may only need a testing team of one or two.

Pro tip: If you need to bulk up your team but don’t have the resources to up your staff, don’t be afraid to ask trusted customers or folks on other internal teams to help you. New perspectives can help you identify new areas to test (and thereby uncover new insights). Some tools, like Testpad, have built-in features specifically for guest testing, permitting users to test your software (as prompted by a test plan) and log their results.

2. Choose your testing approach

We’ve already shared five ways to go about software testing. Review those again, with your team constraints in mind, and choose an approach (or set of approaches) that gives you the most bang for your buck.

3. Optimize your strategy for efficiency

Don’t spread your team too thin by testing the whole app at once. Focus your efforts where problems are most likely, and take advantage of practical trade-offs that move the process along.

For example, maybe you spend more time testing a feature on browsers you know for sure end users tend to use versus less common ones if you’re really pressed for time.

4. Pick the right software testing tool

In truth, you could just jot down your software testing notes in a journal, but that’s not super sharable (especially if your handwriting is a mess). Excel is a good start because it organizes your notes into a legible system.

But Testpad is perhaps an even better solution, allowing you to add new tests on the fly in just a few clicks, invite guest testers to specific projects, and run reports to keep stakeholders updated on your team’s test results.

Plus, it’s super intuitive — you can drag and drop rows to rearrange them, group rows together and create hierarchies by indenting them, and comment out non-test rows using shortcuts. And, because it’s easy to use, your testers will be more likely to use it, and your stakeholders will have the most accurate, up-to-date information at their fingertips.

Whatever you do, don’t overcomplicate

Remember, software testing doesn’t have to be as elaborate and sophisticated as you might think. Start small, see how it goes, then learn and grow from there.

That advice applies to your testing software, too. If Testpad piqued your interest, sign up for a free Testpad account. You can use it for 30 days to decide whether it’s the right software testing tool for you and your team.

Or, if you’re still working up the courage (or your boss’s approval) to start software testing in earnest, check out our blog. We’ve got plenty of articles to get you inspired and up to speed on best practices.

Green square with white check

If you liked this article, consider sharing

Linkedin Logo Twitter Logo Facebook Logo

Subscribe to receive pragmatic strategies and starter templates straight to your inbox

no spams. unsubscribe anytime.