How to Fail Fast with Software QA (and why that’s good)
Many leaders try in vain to help keep their team from screwing up. Instead, embrace failure early and often!
News flash: Everyone writes buggy software. Even your most seasoned engineers will introduce countless bugs and regression issues.
“Thanks, Captain Obvious!” You’re probably thinking.
But here’s the point: It’s not a matter of IF your code is going to have bugs in it. The question is: WHEN will you KNOW that your code has bugs in it? Compressing the duration of your code-fail-fix loop can mean huge wins for your company’s bottom line, product quality, efficiency, time-to-market, culture, developer sanity, and customer happiness.
What is this “fail fast” business?
In the business world there is a movement called Lean Startup, of which I am a huge fan. One of the key principles is this “fail fast” mantra, which basically means the sooner you know your idea sucks the better.
Example time! Let’s say I’m inspired to go into the chocolate-covered pickle business. I am thoroughly convinced that these cocoa infused vinegary cucumbers will surely be the next big thing!
So I go out and I rent some space in a commercial kitchen. I find a retail storefront, which I decorate craftily with a hipster kind of vibe. Finally, I hire some staff and start a Facebook marketing campaign! We throw a grand opening extravaganza and …. <dramatic pause> …. nobody shows up.
Wouldn’t it be nice if we would have found out earlier that everyone but me was disgusted by the thought of my twelve inspired flavors — including balsamic peanut butter cup — of sweet + sour confectionary goodness??!!
It would have been much better for me to trial balloon the product first. “Run it up the Flagpole,” if you will. I could have made up a few batches in my home kitchen and set up one weekend at a farmer’s market. That way the marketplace would have rejected my idea long before I put out a huge outlay of time and money.
That is what failing fast is all about.
How do we apply this to software development?
A lot of ways actually. We could take this same intro and then talk about things like A-B testing or creating MVPs. But, no, today we are here to focus on quality assurance in software engineering.
The purpose of our engineering lifecycle should be to find issues as soon as possible. Ideally, of course, we catch them during the planning stages when there is little cost to change. Sub-optimally, we can find it while the designers are mocking it up. It may require another conversation and some rework on the wireframes, but we can deal with that.
Next best is finding issues as we’re still in the midst of developing it. Slick modern IDEs with realtime feedback are great about doing this, as are strictly typed languages or transpiling tools. But… stay on topic, Jason!
One area with huge potential gains is catching bugs right after we think we’re done, but before it’s been merged with the wider code base. You may already have unit tests, but there is more you can do. And making it automated is essential because if we expect a developer to always and continuously remember to run a test they have to kick off manually, then forget about it.
Part of the just-after-code automation should be unit tests, linters, and other code quality checks. But also thinking about adding another layer of testing that falls somewhere between that and full blown E2E, UI, or System testing. It must be a tool that can test a wide swath of the actual application without taking a long time and slowing down development. Some may call this “smoke” or “integration” testing.
There are a number of strategies to trigger these tests to kick off. One of my favorites is to make it part of the “git commit” process.
Many people don’t know this, but you can actually hook into the git commands and automatically run scripts immediately before or after a commit or push (as well as other commands). This can be really useful for many reasons, but (staying on task) for this kind of early-as-possible test automation we can use it to prevent a commit from happening.
Here is a great article about implementing these git hooks:
So what should we aim for with these git hook triggered tests?
- Run in just seconds, not minutes
- Prevent code from being committed, pushed, or merged that doesn’t pass basic tests
- Tests basic function of the code (like unit tests)
- Tests code quality (like linters, code sniffers, and other code quality tools)
- Dead or unnecessary code detectors
- Automatic documentation generation
- Test coverage analyzers
- Integration/Smoke and API end point tests to asynchronously hit large potions of the application and check for expected results. Not individual functions, bu
We have open sourced a new project created here at FloSports that might be helpful for HTML integration and REST API endpoint testing. Or at least maybe it can give you some ideas.
