Agile Business Intelligence

An iterative methodology for fast, flexible and cost-effective Business Intelligence

Can Agile Business Intelligence finally deliver analytics and insights to the people that need it? Or is it potentially a distraction? Agile philosophy has been around for more than a decade, and it looks like BI is catching up to it, with increasing talk of Agile BI. While Agile development emphasizes technology and business teams working side by side, Agile BI puts forth the notion that business end-users should be free of the technologists altogether – with more self-service tools and ability to design interfaces and analytics to explore information without having to first go through the IT department.

Traditional approaches to BI cannot deliver solutions and reporting fast enough in this era of hyper-competitiveness. Companies spend a lot of time modeling data, and that’s precisely what IT team does very well. They collect requirements and transform those requirements into data models. But the problem is it takes too long. By the time the development is done, the requirements have changed. If the IT team didn’t foresee some of the requirements, and didn’t model them, well then, the organization really cannot analyze the condition. So we definitely need environments that run on the data itself, not from data models. Given the difficulty that many organizations have faced in delivering the BI applications their managers and executives need to understand performance and make critical business decisions, it’s not surprising that an alternative development approach is being embraced. Indeed, there is a broad and growing consensus that Agile BI’s time has come.

 

Some of the key traditional challenges that the Agile BI addresses are as follows

Growing Demand:

Demand for information about business performance has risen dramatically. (The Information Age could just as well be called the BI Age.) BI delivery teams have a large backlog of projects from business users looking for more information to support their decisions. But it’s not just more information users want; it’s more information faster. Agile BI helps IT meet the imperatives for quantity and speed in unlocking the full value of data assets.

Flexibility:

The agile methodology is designed to adjust to changing requirements – and BI requirements change more frequently and profoundly than those for nearly all other types of software projects. In fact, in a 2011 survey of 200 businesses and IT executives conducted by Forrester, 67% of respondents said that BI requirements change at least monthly. A full 20% of respondents said their BI requirements change on a daily basis. Such changes wreak havoc on the traditional waterfall delivery cycle, yet they are inevitable during the lifetime of any BI project.

User Engagement:

The great strength of the agile methodology is that it fosters collaboration between IT and the business. While traditional approaches have struggled to place user needs at the core of the process. Agile BI is all about giving users faster access to functionality and more opportunities to provide feedback. Ultimately, user engagement equates to higher user satisfaction and adoption rates.

Manageable Scope:

Budget overruns and blown schedules can damage IT’s credibility, besides costing the company real money. Because Agile BI focuses on the delivery of smaller sets of functionality in shorter time periods, projects are driven by business defined scope and value. Project timelines and budgets can be tracked in smaller units, and users pay for the value defined. Avoiding scope creep is good news, but it’s better news that these budgets are significantly smaller and the project timelines much shorter.

Lower Costs, Higher Value:

Agile methods in BI have a strong track record in reducing project costs and shortening timelines. Further, because project budgets are aligned to high-priority deliverables and outcomes – that is, high-powered, easy-to-consume applications that users like and that meet real and urgent business needs – overall technology ROI also increases.  Conventional SDLC approaches are poorly suited for BI. Traditional waterfall methodology for SDLC calls for collecting user requirements, documenting them, transforming them into specifications, and then turning specifications over to developers, who then go through the design, build, test, implement cycle. While this approach is often successful for traditional enterprise application implementations, it is almost guaranteed not to work for the majority of BI requirements. The “build it, and they will come” mentality is directly applicable — and recommended — for BI, since only once an end user sees something in front of him, something he can touch and feel and “play with,” will the real requirement materialize.

Clearly a different approach is needed to make BI applications more flexible and able to react much faster to ever-changing business and regulatory requirements. Agile BI is first and foremost a different approach to designing and building BI applications.

The purpose of Agile BI is to: 1) get the development done faster, and 2) react more quickly to changing business requirements. Mostly Agile BI is no different than any agile development methodology that calls for incrementally delivering products versus a big-band approach; for rapid prototypes versus specifications; for reacting versus planning; and for personal interactions with business users versus documentation. The Agile BI methodology differs from other agile approaches in that it requires new and different technologies and architectures for support.

The key to driving an Agile BI project is minimizing project management overhead, reusing existing assets, and automating inefficient manual tasks. Aligning your project budgets to deliverables and outcomes generates more value – that is, high-powered, easy-to-understand, easy-to-consume business intelligence solutions that users like and that meet real business needs.

How to achieve agile development

Agile BI projects should focus on people over process. This does not mean that Agile BI is inherently opposed to thorough and careful development processes, but is guided by a minimalistic business user focused approach, to streamline development cycles. There are three key agile processes that should be adhered to ensure an Agile BI rollout:

Iterative ‘Sprint’ development cycles

1. Systematize ongoing BI processes
2. Implement Barely Sufficient Processes

Iterative ‘Sprint’ development cycles

In Agile development methodology, teams work in ‘sprints’ to produce bite-sized deliverables in an iterative manner. Sprints can be one to four weeks long depending on the size and complexity of the project. At the end of each sprint, the business has a working deliverable, such as a new report or dashboard, delivered to them in a production setting.

By contrast, the waterfall development cycle used in traditional BI rollouts, is cemented in a regimented, sequential progression. It is inflexible to changing reporting needs and costly to make changes to. Agile BI, as a process, is about delivering functioning software regularly in short weekly or monthly timeframes – the shorter and more frequent the better (working software is the measure of BI success). Agile BI is about responding to the immediate needs of the BI user, rather than working to establish and deliver ALL potential reporting needs upfront. To establish an approach that facilitates Agile BI, reporting objectives should be readjusted regularly based on available resources and intermediate business goals to help focus attention on what is really important. Working in this manner ensures the relevancy of reporting to business goals and enables a faster, more flexible approach to changing reporting needs.

Systematize ongoing BI Processes

Agile BI development teams must automate any repetitive tasks/processes to allow more time and focus to be spent on developing and delivering end-user features. For example, whilst critical, testing the BI system manually takes up unacceptable resources each and every sprint cycle. Automated testing conducted by the users actually involved in the development of new reports, means that new changes can be quickly tested within the sprint, rather than waiting to be processed by a separate testing team. This way, accountability resides within the team.

Implement Barely Sufficient Processes

Minimizing the amount of ‘ceremony’ associated with BI development reduces the length of development cycles and allows development teams to concentrate on the work that matters. This minimalistic approach does not suggest that careful planning is unnecessary during the development process, but that formal planning and documentation should be aimed at satisfying the practical needs of the project. For example, a concept document for each sprint should focus on business user requirements and nothing more. Additional verbiage simply adds no value. Agile BI is just as much about maximizing the amount of work not done.

Summary

Successful Agile BI deployments enhance organizational flexibility and responsiveness. Increasingly, businesses and their personnel are exploiting the benefits of Agile BI, allowing them to respond with immediacy to business demands. To survive and prosper in a competitive marketplace, businesses from all industries and sectors have to be able to scan their external environment, review their internal processes and make appropriate proactive and reactive changes. Modern companies are striving to spread fact-based decision-making throughout their organizations. Agile BI solutions, with their end-user centric approach, enable organizations’ to anticipate and adapt to shifting market conditions.

References

• Information Management Blog : Agile BI
• Yellowfin : Agile Business Intelligence
• Forrester report on Agile BI Out Of The Box
• Balanced Insight: Enabling Agile Business Intelligence with Balanced Insight

 Don’t forget to leave your comments below.