CONTACT US 224 265 0400 or Email

The Agile Incubator Blog

Rethinking How to System Test Your BI Project, Part 5: The Case for Thorough System Testing

My last couple blogs set the stage for building test data sets of the right size and content. It’s time to take a breather to understand why system testing is so critical to our BI solutions.

Imagine if you will a crystal ball on the desk in front of you.

Inside the orb, you can view your project performance.  You see a project that gathers all the business rules correctly and understands and mitigates data quality issues. You see developers implement complex rules and properly integrate and test their code. At completion, a satisfied business user community praises the BI team.

Unlike the crystal ball, where all the business rules are known and data quality is predictable and resolved flawlessly, reality can be opaque and imperfect.  System testing can help us address both the substantial and subtle obstacles.

System testing highlights shortcomings in our business rules and brings unforeseen data quality issues to light.

The validity and scope of the business rules defined for a BI effort may not be apparent until system testing starts. This maxim doesn’t prevent our diligent business analysts and development teams to offer up their best attempts to fill out incomplete, vague, or poorly written business rules.   Challenges with data quality can appear anywhere. Unit testing rarely exposes such problems. Neither does cursory system testing. No business analyst or user can anticipate all possible data conditions, so the rules defined to address data quality will come up incomplete at best, or even incorrect.

System testing authenticates our designs.

Complexity motivates our designers to break solutions apart. Similar logic is consolidated rather than distributed for the sake of consistency and maintenance.  Performance considerations mandate certain expensive logic be performed first. The design solution was parsed out amongst the development team. Our competent developers completed and unit tested their code components, but assembly concealed unexpected ‘system’ behaviors. Unit testing does not address these anomalies.  The interplay of independently developed code sets within a designed solution can cause our work to miss the mark. The business goals can become anything but transparent in the designed solution.

Forget the idea that system testing is just a formal validation of development. It’s not. Done right, system testing will validate your business rules, elicit acceptable data quality, and prove out the solution design. All those steps are essential if you want to deliver a trustworthy solution to your end users.

What about that crystal ball sitting on your desk?

Use it to magnify, not mask, project issues.

My next blog will explain how to automate your system data sets.

– Jim Van de Water contributed to this blog.