EDITORIALS

Don’t overlook sanity testing in your development process

Image of a foot testing the temperature of pool water

Don’t overlook sanity testing in your development process

Sanity testing doesn't have to be overwhelming— a quick check to make sure the basics are working before diving into more complex testing can give you confidence in the state of your product. It can also save you time and money when done right.

Testpad

By Testpad

September 24, 2024

Linkedin Logo Twitter Logo Facebook Logo
s

oftware testing doesn't have to be complicated. It's a quick check to make sure the obvious stuff works before you dive into the deep end of testing.

Think about it. Your team's just finished a new build. Before you unleash the full might of your testing process, wouldn't it make sense to spend a few minutes making sure the basics still work? That's all sanity testing is. It's a simple, pragmatic way to catch glaring issues before you waste time on more detailed tests.

What is sanity testing?

Sanity testing really is as simple as it sounds—a quick check to see if your software's basic functions are working as expected. It's not about running all your tests or diving into extensive regression testing. Instead, it's about efficiently assessing whether it's worth committing to a full test run.

Think of it as a quick temperature check for your software. Before you call in the full medical team (your comprehensive testing process), you want to make sure there's actually a fever to treat. This approach is all about being time-efficient, catching obvious issues with minimal effort before you invest more resources.

Consider this scenario: Your development team has been working on a new feature for your web app. Before you unleash your QA team for a week-long testing marathon, wouldn't it make sense to spend 15 minutes checking if the basics still work? Can users log in? Does the main page load? Does the new feature do anything at all? That's sanity testing in action.

The goal is to go wide before you go deep. You're looking for obvious issues that might derail more extensive testing efforts. It's far better to discover a major problem in a 15-minute sanity test than five days into a comprehensive test run.

Remember, sanity testing helps by being pragmatic. It's not about perfection — it's about quickly gathering the information you need to make smart choices about your testing process. If you're not doing some sanity testing already, and your full test run takes non-trivial effort, then it'd be a good idea to start with some simple quick checks.

Sanity testing vs. smoke testing

Sanity testing and smoke testing are often used interchangeably, and in many contexts, they mean the same thing: a quick check of basic functionality. However, some teams do make a distinction:

Sanity testing sometimes refers to a narrower focus, zeroing in on specific areas that have been changed or updated. For instance, if you've just fixed a bug in the user registration process, a sanity test might focus solely on that feature.

Smoke testing, on the other hand, might cast a slightly wider net, covering a broader range of core functionalities even if they weren't directly changed.

That said, these distinctions aren't universal. Many teams use the terms synonymously, and that's perfectly fine. The key isn't the terminology, but the core concept: perform quick, efficient checks to ensure basic functionality before diving into more comprehensive testing.

Sanity testing vs. regression testing

Sanity testing should not be confused with regression testing – the two are quite different.

As we've mentioned, sanity testing is a quick check that all the obvious stuff is working before committing to a longer, more expensive full test. It's about quickly assessing the current state of the software, particularly after recent changes or bug fixes.

Regression testing, on the other hand, is the rigorous testing to ensure that everything that has been broken and fixed in the past isn't broken again this time. It's about ensuring that existing functionality hasn't been compromised by new changes.

The automated part of your regression tests will probably have already been run before you're considering a sanity test, as they're typically quick and easy to run once written. But the manual part of your regression testing (and yes, regression testing can be manual) will be part of the very "full test run" that you want to protect with a quick sanity check first.

In some cases, sanity testing might overlap with a subset of regression testing. For instance, if automated regression tests flag several issues, a quick sanity test might be used to verify these problems and assess their impact before diving into more detailed testing.

Remember, the goal isn't to rigidly categorize every test you perform. The key is to use these different testing approaches pragmatically, choosing the right tool for the job at hand.

Whether you're performing sanity testing or running a full regression suite, the ultimate aim is the same: to gather valuable information about your software's current state and make informed decisions about your development process.

The importance of sanity testing

Sanity testing is all about efficiency in the software testing process. It recognizes that testing happens within the constraints of limited resources, and we're always trying to find as many issues as possible with the least effort.

The importance of sanity testing lies in its ability to catch obvious problems quickly before investing time in more thorough testing phases. It's a smart way to allocate your resources and potentially save significant time and effort down the line.

However, the scope of sanity testing should be proportional to your overall testing process:

  • If your full test run only takes an hour, you might not need a separate sanity check.

  • If your full test run takes several hours, a quick 20-minute sanity test might make sense.

  • For test cycles that take days or weeks, a more extensive "quick" test lasting a day or two could be beneficial.

The key is to keep things in perspective and tailor your approach to your specific needs. Sanity testing is about making sure it's worth proceeding with more comprehensive testing, not adding unnecessary complexity to your process.

If a sanity test fails, it often indicates a fundamental issue that needs addressing before further testing. This early detection can save your team valuable time and resources, ensuring you're not investing effort in testing a build that's fundamentally flawed.

Ideas for applying sanity testing

When it comes to applying sanity testing, remember this: IT DEPENDS. Your specific project, team structure, and development process will all influence how you implement your sanity testing.

You perform sanity testing any time you're about to embark on a longer test cycle.

This could be after receiving a new build, following significant code changes, or when you need quick feedback on the software's state. The key is to use sanity testing as a quick check before investing time and resources into more comprehensive testing.

Practical steps:

  • Keep it simple: Don't overcomplicate it, just work out a small collection of simple checks that test if the main features are working.

  • Use checklists: Checklists are an obvious and simple way to document your sanity tests. They help ensure you don't forget anything, and give you consistency from release to release.

  • Focus on what's new or changed: Yes, include quick checks for all the main features of your software, but it can often be worth a few extra quick checks on what's new or changed since last time (as that's the most likely areas for new and obvious bugs)

  • Consider a subset of regression tests: If you have an established set of regression tests, consider using a small subset for your sanity testing.

  • Incorporate exploratory testing: Don't be afraid to go off-script. Sometimes, a few minutes of freeform exploration can uncover issues a scripted test might miss.

Sanity testing really doesn't have to be complicated. Your checklist could look as simple as:

  • Can customers log in?
  • Does the product catalog load?
  • Can a product be added to the cart?
  • Does the checkout process work?
  • Is the new "Recommended Products" feature visible?

The exact contents of your sanity test will depend on your specific application. Focus on the core functionalities and any recent changes or additions.

By applying sanity testing, you quickly assess whether your software is ready for more thorough testing. This simple approach can save your team valuable time and resources, catching major issues before you invest in more comprehensive testing.

Sanity testing tools

Choosing tools for sanity testing isn't hard. In fact, you don't need complicated tools at all.

A pen and paper or whiteboard can work, but they're easy to lose or accidentally erase. Many teams find starting with a list in a spreadsheet is the best approach. It's simple, shareable, and gets the job done.

When spreadsheets start becoming unwieldy due to formatting issues or growing complexity, you might consider upgrading to a tool like Testpad. It's perfect for managing tests as checklists, offering more power than a spreadsheet without the complexity of traditional test management tools.

Remember, sanity testing is about quick, efficient checks. Avoid over-engineering your process with complex systems. The key is to keep it simple and choose a tool that aligns with your team's needs and workflow.

As for automation tools like Selenium, Cypress, or LoadRunner —these are primarily for regression testing, not sanity testing. While they're valuable in their own right, they're typically overkill for the quick, manual checks that characterize sanity testing.

The bottom line? Start simple. Use what works for your team. And only add complexity when it truly adds value to your sanity testing process.

Conclusion

Sanity testing is simply about doing quick checks before diving into a longer, more expensive full test run. It's an obvious concept, but one that's sometimes overlooked in the rush to complete testing.

The beauty of sanity testing lies in its simplicity. Whether you're using a basic checklist or a more structured approach, the goal is the same: to quickly assess if your software is ready for more thorough testing.

In the end, sanity testing is just common sense applied to software testing. It's a straightforward way to catch glaring issues early, potentially saving your team from wasted effort down the line. So keep it simple, keep it focused, and let sanity testing do what it does best—provide a quick reality check for your software.

And… if you want to see how Testpad can support your sanity testing, you can sign up for a 30-day free trial.

Green square with white check

If you liked this article, consider sharing

Linkedin Logo Twitter Logo Facebook Logo

Subscribe to receive pragmatic strategies and starter templates straight to your inbox

no spams. unsubscribe anytime.