Monday, September 29, 2014

Test Early, Test Often



Of late, I have been enamored of testing techniques that come earlier and earlier in the development cycle. It can be called static analysis, design auditing, prospective testing, shift -left or the like, but the research is in: testing before you get something bears fruit in most organizations.  Here I present a few examples from my own experience.

At the start of a sprint, we leave Sprint Planning with the requirements.  The next interaction with developers is when we review their Developer Design Overview document that spells out the development approach and helps QA scope their testing effort.  This developer had chosen to put an error message into a file usually reserved for configuration.  QA saw the DDO and raised concerns immediately.  Why was a message being added to this file when they were usually reserved for the language DB?  With this one question, before QA saw the code, we changed the trajectory of development.  The fix was in before we got our first build, and the story closed with the Sprint instead of carrying over with the do-over.

An even earlier example came when we looked to implement secure communications between two servers.  While I couldn't code my own implementation, I was able to provide recommendations at design-time by staying educated and confirming my understanding with developers who had dealt with crypto.  By starting early, we were on surer footing when troubleshooting and confirming the implementation was sound.

As the examples above illustrate, QA often saves time for developers by defending standards and consistent implementation early in the cycle, but that is not the only savings that comes from shifting left.  Often, test environment issues can also be aided by an early understanding of requirements.  In one case, as story had carried over from a previous sprint which meant we were already behind.  The roadblock was a production issue pulling the developer away from the story.  Instead of sitting on our laurels, QA worked with the configuration manager to make sure our test environments were ship shape before the code was completed.  When the developer's changes passed build verification, we were off and running almost instantly.  Not only did our preparation help us get to the work of testing faster, but it also helped us close more stories as environments were made ready before they could become an obstacle. Not only was I able to test early, but it lead to me testing more and in greater depth.

Most modern test engineers have their own war stories from early testing.  For every story where requirements changed and early notes became meaningless, there are ten stories where early questions lead to greater clarity, fewer bugs, and more time for digging in.  I consider projects that foster this early access for QA to be among the most fruitful and least volatile.