August 20, 2010 | By Rey Villar
Every Project Manager (PM) worth his/her salt (and who is acting in accordance with The Project Management Institute’s guidelines outlined in the “Practice Standard Project Risk Management” manual) has a list of all the things that can go wrong during a project, as captured in the Project Risk Log. The good PMs work with the project leadership (the business sponsor, the IT executive, team leads, analysts) in a Risk / Mitigation session at project commencement—anticipating any potential chinks in the project’s armor. And they hold a weekly meeting to review and update the risks for the project’s duration.
So, having seen how risks become issues and issues become scope change / budget increase / missed project deadlines, I had one savvy client ask as part of their data warehouse request for proposal (RFP) process that each service provider bidding compile a list of their top five risk factors and mitigation steps in order to achieve a greater degree of confidence that the project would be delivered within the budget. The list that emerged consisted of the following:
- Consensus on and completeness of reference data and hierarchies with an ability to reconcile across business units and functions
- Scope creep and SME expectation
- Availability of SMEs, system experts and other pertinent resources, timeliness of decision-making
- Excessive complexity of reporting environment and / or data transformation
- Lack of data cleanliness
Before I go any further, I’ll concede that this list is not comprehensive. And I’ll also acknowledge that risks need to be assessed for each project. But I stand by my assertion that this set of risks creates huge headaches if they are not addressed and tackled head-on as early as possible. Let’s explore further.
Risk #1: Reconciliation of Reference Data and Hierarchies
Project Status: Stalled. Evidenced usually in Requirements and Analysis stages. Reason: We can’t get the Finance SME and Marketing SME to agree on the Product Hierarchy or perhaps align on the definition of Net Revenue. If the data warehouse is the central “single version of the truth,” how do we keep moving forward if the LOBs have problems agreeing on that single version?
- Clearly defined point of authority
- Early start of this task
- Emphasis on financial reconciliation
- Use of commercial tools (COTS) accompanied by short-term consultation to leverage industry solutions
- Careful attention to managing the schedule for early identification and impact of prolonged data investigation and data resolution
- Weekly monitoring of budget using predictive tools
Risk #2: Scope Creep
Project Status: Behind schedule. Evidenced usually in Design and Development stages. Reason: Requirements keep changing. As we moved from the Analysis Phase into Design Phase and your designers had questions about the data or the “meaning” of a requirement, your Business Analyst team would go to the LOB for answers and come back with more requirements.
- Careful documentation
- Review of requirements by the business, as well as intermittent audits and evaluations by PMO Quality Assurance
- Cross-reference requirements to project plan
- Manage expectation-setting of SMEs and business units
- Careful impact analysis and change control
- Personal accountability of project members
- Open, honest and ego-free environment
Risk #3: Availability of SMEs
Project Status: Behind schedule. Evidenced usually in Design stage. Reason: Design and development team asked for business rules on how to handle exceptions, or create a metric, etc., but the subject matter expert was on vacation or wouldn’t respond. The design team forges ahead with a “best guess.”
- Early notification of SME participation, including expectations of amount of time over duration of project that will be needed and committed to
- Consideration of SME time – avoid wasting their time through the appropriate skill-set and preparedness of the BA team and facilitators
- Involvement of the “right resources” rather than a less-informed substitute
- Effective scribes during interviews and meetings: Note-taking, action list and follow-through
Risk #4: Complexity of the Reporting Environment and Transforms
Project Status: Behind schedule. Evidenced usually in Development stage. Reason: Underestimation of the number of reports, complexity of the reports, metrics and measurements needed for the reports, performance of the reports.
- Early evaluation of complexity of reports and queries
- Validation and clarification of business requirements, as well as early and careful translation of the data requirements
- Validation of the number and accessibility of source data fields (get a sense of this before project kick-off)
- Validation of the number of reports (get a sense of the number of existing reports that will be replaced as well as how many new reports will be created)
- Project tracking
- Addressing data problems as they occur
- Incorporation of checkpoints and team structures. E.g.:
- Project Management Office
- Staffing Team
- BI-COE or BICC
- Training Office
Risk #5: Data Cleanliness
Project Status: Behind schedule (or worse – on schedule and “discovered” during Testing/QA). Evidenced usually in Development or Testing/QA stages. We’re always promised at the beginning of the project that the quality of the data is good / acceptable and we’re always so shocked when the development team says they are rejecting a large percent of the data (unless they just pass through everything. Yikes!) Or – WORSE – the Testing/QA team finds it out. Or – WORST CASE SCENARIO – in production the information consumers start screaming that the data warehouse is unusable.
- Formal data sampling and profiling early in the project (get that development environment set up right away, during requirements gathering, and start getting samples from the source systems)
- Early attention to the definition of data quality criteria (need those SMEs again) — and refer to Jill Dyche, who has continually sounded the clarion call for DQ
- Early evaluation of complexity (your Architect is key)
- Validation and clarification of requirements, as well as early and careful translation of the data requirements
- Project tracking
- Addressing data problems as they occur
- Use of commercial tools (COTS) accompanied by short-term consultation
- Point of authority for quality issues and prioritization
Implications of Risk Mitigators on Project Delivery
As noted, early and ongoing evaluations of these risks should be conducted. For those risks which are determined to be of a greater degree than an anticipated “acceptable level of risk,” (e.g, excessive degree of report complexity or greater number of data quality issues than anticipated), the joint team of Business and IT project leadership must work to achieve a balance for inclusion / exclusion into the effort while still meeting the project’s objectives.
What are Your Top Project Management Risks?
Perhaps I’ve been “preaching to the choir,” and all you saavy PMs out there already observe these best practices. On the other hand, we’ve all lived through nightmare projects at some point in our careers and carry with us bitter lessons learned. What are the top 2-3 project risks that keep you awake at night? Would you like to share your horror stories and lessons learned? (Hmmm…no names, please. Use the “I have a friend with a problem…”)