Skip to main content

Author: Richard Larson

With over a decade of experience in working for GoPromotional UK, Richard Larson has been an integral part of marketing GoPromo’s award-winning brand. Richard’s skillset varies from a deep understanding of product strategy to his consistent ability to design, analyze, and execute team-based marketing projects. Richard’s team leadership at GoPromo is complemented by a love for data-driven results and his ability to connect with audiences."

Why Agile isn’t Enough Part 2: Lean Startup Build Phase and BA Techniques that Enhance it

In part 1 of this article, we covered an overview of Lean Startups and its Build-Measure-Learn (B-M-L) methodology.

To recap, in the B-M-L cycle we build a product increment, measure how it is adopted and used, and learn from the measures about what worked and didn’t and use it for the next cycle.
In this part we’ll examine the Build part of the cycle in more depth and see which Business Analysis techniques are most helpful to propel the Lean Startup to providing innovation and value. Part 3 will cover the remaining two parts, Measure and Learn.


To review, the Build-Measure-Learn methodology within Lean Startup is the central engine driving the process. Lean Startups rely on repetitive cycles of B-M-L to produce various product increments as recapped in Figure 1. An important milestone while building a product is to reach a Minimum Viable Product or MVP such as in “P2” below.


The concept of a Minimum Viable Product was popularized by Eric Ries in The Lean Startup1 and has been genericized quite a bit in our industry. In Lean Startup an MVP is the product produced with the minimum effort and the quickest path or paths through the B-M-L loop. Product P2 in Figure 1 shows an MVP produced after two times through the loop.

BA July 9 1

Figure 1: Lean Startup Framework

More than a prototype or proof of concept, the goal of an MVP is to test fundamental business hypotheses about a product. For example, the hypothesis of whether customers would pay to have DVDs shipped to them and return them by mail was the basis of Netflix.

Another way to think of an MVP is “a first attempt at building a solution to a problem worth solving.” The learning starts with testing the fundamental hypothesis and continues with each new feature added. We’ll return to learning in part 3.

Take the video communication app called Marco Polo. This app allows family and friends to communicate by directly sending videos right from the app. That’s basically all the app does, but I can tell you our family is totally addicted to it! It is valuable to us since we live in cities across the US and can hear and see what family members are doing. It’s incredibly valuable to us since our kids and grandkids keep us updated much more often with it than by phone or email.

Marco Polo is now a bit beyond an MVP as of this writing, but not much. The initial MVP just let users record and play videos and was enough for their startup to learn from. They added the ability to Fast Forward and Rewind based on user feedback. They also added a few other features, but it is still fairly basic as of this writing. Without a Lean Startup mentality, though, adding additional potential features would have delayed their launch and may not be widely used.

MVP: “A first attempt at building a solution to a problem worth solving.” Eric Ries

Leap of Faith

All startups are based on assumptions and get started as a “leap of faith.” It’s the basic hypothesis of a new product or company. There is a paradox here, though: we aim to ultimately build things for the “masses”(see Figure 2) but need to start with the early adopters or our product likely won’t get off the ground.

BA July 9 2

Figure 2: Product Adoption Over Time

It’s important to know and note the assumptions that form the “leap of faith.” Business Analysis skills can help by ferreting out and documenting those assumptions. As we do that, it’s important to avoid false analogies that obscure the “true leap of faith.” Eric Ries in The Lean Startup suggests we shun borrowed analogies like “this worked for Apple so it will work for us.”

Speaking of Apple, their Apple Watch was a leap of faith whether people were willing to read texts or listen to music on their watches, which many did want and still do.


Build – 13 techniques

Figure 3 lists the BA techniques for the Build phase. Looking through the list, which techniques have you found to be the most helpful in developing new products? If I was on a desert island and could only use 5 techniques, I’d have to say Benchmarking, Brainstorming, Lean Canvas, Observation, and Prototyping would be my choices. The details and applications of the techniques are beyond the scope of this article but are intended to be a reference.

BA July 9 3

Figure 3: Techniques Useful in the Build Stage

The Business Model Canvas is becoming more and more popular and for good reason. But it is meant more for established business and doesn’t help as much as a newer technique called the “Lean Canvas.” The Lean Canvas tool helps focus on customers and their solutions based on need. See Figure 4 for an example. Like the Business Model Canvas the Lean Canvas has nine categories to help spur thinking and help create a product of value.

I particularly like the first two categories, “Problem” and “Solution,” since they focus on two of the most critical aspects of delivering a valuable solution. I also like including “Key Metrics” in the middle and we’ll discuss metrics in Part 3. The upper right-hand section labeled “Customer Segments” is also a key factor since it focuses on the customer and possible segments or “cohorts” of users.

BA July 9 4

Figure 4: Lean Canvas Example

Customer Segments is also valuable since we have found as entrepreneurs it is critical to first find customers in need and then build or deliver solutions that will appeal to them. It is much riskier and potentially wasteful to build products and then find customers we think will need them. Take the example of Teforia, who built a “tea-infusion system” that sold for up to $1,300 per machine. As it turned out, people weren’t willing to pay that much for tea so the product was a failure (and the startup folded).


In summary, the Build portion of the Lean Startup methodology leads off the cycle. It relies on vision and ideas for solving a problem at first. It then uses learning from product measures to adjust the product with new or changed features in future cycles. The final part of this article will explore the latter two portions, Measure and Learn, in more detail.

Why Agile isn’t Enough Part 3: Lean Startup Measure Phase and BA Techniques that Enhance it

In previous parts of this article, we covered an overview of Lean Startups and its Build-Measure-Learn (BML) methodology.

We also detailed the Build portion of the cycle, including the Minimum Viable Product and its importance to the Lean Startup approach.

To summarize Lean Startup, the method was created by Eric Ries and detailed in his groundbreaking book The Lean Startup[1] back in 2011. Lean Startup can guide product development, shorten development cycles, and help us deliver the products and features customers need, not just what they tell us they need.

In this part we’ll examine the next portion of the B-M-L cycle, Measure and why it is crucial to success.


After building a product or features of a product, the second part of the Lean Startup method is measuring how that product is adopted and used. Tools such as focus groups and surveys are useful measuring tools. But beware of over-relying on them since focus groups only reveal how people feel and surveys only tell us what they think.

Beware also of so-called “Vanity Metrics” since they tend to make us feel good and don’t lead to the kind of learning we need. These metrics tend to be easy to capture, such as number of users or site visits, and may show false progress. For example:

  • The number of visitors to a web site is easy to measure and seeing an increase after a product launch feels good.
  • Let’s say your team uses a strategy of offering a free version of a product to attract potential paying customers. A large spike in new product users is a vanity metric since it masks how people like the product or how likely they are to become paying members.

Instead, we need to find what I call “Metrics that Matter.” Those are usually harder to obtain, but typically more meaningful. For the above two examples:

  • Measuring the number of site visitors who return to the website and how quickly they return is harder to measure but gives a better sense of meaningful customer interest.
  • Tracking the free members who convert to paying customers is harder than just measuring the new members. But those conversions are a much more meaningful measurement – just ask any digital marketing person about the importance of conversions.


To focus on metrics that matter, Eric Ries suggests we use AAA metrics1, which are:

  1. Actionable – these must prove a clear cause and effect otherwise they are vanity metrics. They should also be significant enough to base a decision on. Example: discovering a new feature that is not used much and then dropping it or deciding not to enhance it.
  2. Accessible – the product team needs to be able to access the measures to gather, analyze, and learn from them. For example, Google Analytics is useful for web site measurements and is highly accessible. But, that tool can provide plenty of vanity metrics such as number of site visits per month. More significant would be to measure the number of people returning in subsequent months and then doing “cohort analysis” on groups of returning customers using Google Analytics.

Find AAA metrics:

·    Actionable

·    Accessible

·    Auditable

  1. Auditable – ensure the data is credible and can be verified. Often this means spot-checking the data with real customers. Auditable also means to keep reporting processes simple, preferably with data directly from operational data (vs. manual manipulation). What this means is that the process by which measures are obtained is simple and can be independently verified. Example: surveying customers directly to verify their purchase decisions.

Measure9 techniques

Figure 1 below shows a list of nine standard Business Analysis techniques that will assist in the Measurement part of the B-M-L cycle. In addition, below are two that are not yet in the IIBA or PMI standards, but are still important in the Measuring phase

Cohort Analysis – important for a startup to track how groups of related customers or customer segments are acquired and retained. Cohorts can help us determine which features will be valued by different segments. Google Analytics has a new cohort analysis tool that can help. Cohort analysis is like the use of personas which group users by various characteristics. But the two differ in that cohorts are established by their current and past behavior, not by projected traits, and we can measure cohort behavior.

A/B Split testing – is a simple and effective way to run an experiment to learn which features are used, which marketing messages appeal better, even which kinds of web pages are more effective at promoting a product.

Two test groups are given different product versions to see which group adopts and uses which version to determine the one that is most effective. The differences are usually controlled to see which feature(s) made the biggest difference. Often the versions are tested in a live, production environment to not slow down progress.

It is not an official technique in any BA standard but is referred to as part of experiments.

BA July 1 1

Figure 1: Techniques for Measuring

In summary, the Measure stage of the Lean Startup methodology lays the groundwork for us to learn what worked and what didn’t with what we built. The measures and the learning from those measures are what separates an ordinary Agile effort from a Lean Startup. I also feel the “M-L” work we do contributes to greater – and faster – success with products we release than with typical delivery cycles.

Some important facets to remember about measuring are to avoid “vanity metrics,” which are easy-to-capture measures that are inclined to make the team feel better about making progress. It is far better to discover “metrics that matter” using AAA metrics –Actionable, Accessible, and Auditable. These are often harder to obtain but will enable the greatest learning.

Speaking of learning, the final part of this series covers the Learn portion of the B-M-L methodology of Lean Startup.


[1] Eric Ries, The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses, New York: Crown Business Books, 2011

Why Agile isn’t Enough Part 1: How Lean Startups fulfill the Promise of Agile

Let me ask you a dumb question: Does your organization produce or provide products or services that customers use? Of course.

But maybe a better question would be: how many of you would like to produce them faster or with more innovation?

If you produce new products or services, you potentially could use a Lean Startup approach and create them faster and with more value. It doesn’t matter if you work for a small company, at a major multi-national company like GE (who uses lean startups), or for government or a non-profit. Lean startups are useful and needed in all industries and organizations.

The Lean Startup method was created by Eric Ries and described in his groundbreaking book The Lean Startup back in 2011. I first came across his work a few years after that and it struck me as intuitively sensible. Part of the reason it resonated with me was a few years earlier our company conducted a lean startup type of effort. We built our Online Study Exams product based on customer needs and Ries’ concepts really hit home. They also made we wish I had known then what I know now. So here is a series of 3 articles to help others based on lots of mistakes and learning! It takes the concepts of the Lean Startup process and shows how business analysis techniques support this process.

What we’ll explore in this 3-part series is:

  • Part 1 (this article): What are the main principles of the Lean Startup framework so you can see their importance for developing innovative and winning products quickly. I’ll touch on why Agile is a necessary but not sufficient condition to building the right products.
  • Part 2: The Lean Startup methodology. This part includes a list of 12 key Business Analysis techniques that are valuable throughout the Lean Startup process and provide needed “fuel” for them.
  • Part 3: Explores the “Measure” and “Learn” parts of the methodology, keys to a successful lean startup. There are an additional eight Business Analysis techniques that help these two portions work successfully.

Part 1: What is Lean Startup and why is it needed?

Lean Startup is a “methodology for developing businesses and products, which aims to shorten product development cycles by adopting a combination of business-hypothesis-driven experimentation, iterative product releases, and validated learning.” Wikipedia
The reason we all need to care reflects the state of business today :

  • Pressure to innovate or lose market share. In some cases, companies who don’t innovate end up stagnating like Blackberry or worst case going out of business-like Blockbuster.
  • Innovation is not a black box. You can’t just buy it or demand it.
  • Jumping to solutions is as rampant as ever. Executives feel the need to “get going and build something!”
  • The need to deliver value and deliver it faster than the competition. For non-profits, this is expressed by the need to deliver value or face reduced funding.
  • “The need to focus on what customers want, not just on what we think they want or what they tell us they want.” Ries
  • The innovator’s dilemma: adding incremental product improvements is always safer and easier than new breakthrough products. Even Apple has been playing it safe recently by adding small incremental features rather than new, innovative products.

But which solutions does an organization pursue? The key is to define the problem(s) first and use them to figure out what our customers need. Then we can work on solutions. It seems obvious, but that approach is not always followed for a multitude of reasons.

Kathy Fish, the CTO of Proctor and Gamble and a big proponent of Lean Startups, puts it well: “When you fall in love with the problem and not the solution it opens your mind up in a really different way. ” Fish goes on to say by following a different funding approach to new products, they are moving faster and spending 25-50 percent less money to boot.

Lean Startups are based on five principles , most of which are beyond the scope of this article. We will focus on the Build-Measure-Learn methodology.
1. Entrepreneurs are Everywhere
2. Entrepreneurship is Management
3. Validated Learning
4. Innovation Accounting
5. Build-Measure-Learn


Lean Startup Methodology

Lean Startup shares its heritage with lean manufacturing. It relies on the knowledge and creativity of individual workers, shrinking of batch sizes, Just-in-Time (JIT) inventory, and faster cycle times. You might recognize the underpinnings of Agile here, too, since they are also based on these principles. Lean Startup borrows from other management and product development ideas as well, including design thinking and customer experience/development.

All these disciplines share a focus on value, which from a lean perspective is anything that provides benefit to the customer (and not just what we think they want or might want). Plus, they all have an aversion to non-value-added features.

But, wait, if they are similar, what’s wrong with just using Agile? Well, Agile alone is not enough. It helps you build a product the right way; it doesn’t mean you will build the right product. We need to figure out what the right product is, then use Agile to build it right. Lean Startups help us determine the right products and features.

In The Lean Startup1 Eric Ries describes the methodology for building the “right product.” The basic kernel of what makes Lean Startup work is cycling multiple times through what is called the Build-Measure-Learn (B-M-L) cycle. See Figure 1.


Here’s a typical flow: After building an initial product increment, we measure how it is adopted and used. From those measures we learn what worked and didn’t and use it for the next cycle. I think it literally flips around the normal cycle of learning what customers need and then building what we think will need. It also promotes innovation since we can incrementally discover what customers truly value and deliver it to them as we learn what is valuable to them.

Figure 2 shows the cycle from a flowchart perspective:


• Lean Startups begin with ideas that are used as inputs for what to build. Those ideas are guided by the vision for what the product should achieve.
• A product increment is output from that and the use or adoption of the product and its features are what is being measured.
• Measurements produce data, which is what we use to learn from.
• The output from learning are new ideas for what to build for the next increment.

The methodology assumes you have identified a problem and have a vision and ideas on how to solve it. From a process standpoint those are “pre-conditions.” It also relies on having an overall strategy for the product’s direction to support it. Without those, we’d be meandering around a lot instead of focusing on solving particular customer problems. Each time through the B-M-L loop we produce a new product increment. See Figure 3 for a graphical view of this “framework.”


To conclude, Lean Startups give us a simple, repeatable, and rigorous way to build products that customers want and need, not just what they say they want. The core of the method is the Build-Measure-Learn cycle, which produces product increments that we can measure and use data to learn whether the features are valuable or not. It is more complex, of course, but my aim with this article is to give a practical overview of the methodology.

In Parts 2 and 3, I’ll continue by exploring the B-M-L cycle further and showing the business analysis techniques that can propel Lean Startups to build creative and effective solutions.



[1] Eric Ries, The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses, New York: Crown Business Books, 2011


[1] Adapted from Clayton M. Christensen, The Innovator’s Dilemma, New York: Harper Collins, 1997-2011

[1] Downloaded from, Downloaded Sept. 4, 2018

[1] The Lean Startup,

How Data Modeling Has Helped me in Business. No, really!

I’ve been working with and doing data modeling training for many years, more than I want to recount.

I feel like the essential principles of data modeling are now baked into my thinking. So much so that I find myself using the concepts and terminology in my day-to-day business life as well.

I know, call me weird, but it really has helped me in business and to communicate more clearly. Let me explain with a few points.

  1. Entities. Entities define business objects, and often have several synonyms. Getting clear definitions can clarify ambiguous roles and responsibilities. For example, we have various types of customers: A few key ones include private onsite customers who arrange training for groups, individual public open-enrollment students needing training for themselves, and licensees/distributors who license courseware from us. 
    They are all customers to us yet have different needs and we work with each type differently, even having different people assigned to help each type. Minimally we need unique names for each and if we extended that to a data model we’d have a customer supertype and various subtypes. Each customer has things in common with each other type (part of the supertype) and some data and processes unique to their type (the subtypes). 
    The clarity this brings is to remind us that subtypes might seem separate and distinct, especially with their various names. But there are many functions and data points that unite them and can be leveraged to take advantage of. Examples include marketing, accounting, or customer care to name a few. 
  2. More Subtypes and Supertypes. Speaking of subtypes, we sponsor and exhibit at several events a year. They include everything from local events in one city to regional conferences that attract a wider audience to national ones that pull in international attendees. 
    These events are more a repeatable process than a project, and we tend to do the exhibit materials planning similarly for each of the three types. To simplify the planning, we organize the event materials and logistics according to the size. For instance, we only ship our large booth to the national events. We bring smaller batches of literature to the smaller shows since we usually only have a table for us to exhibit them. 
    What occurred to me long ago was that an event is a supertype, with some commonalities between them all. The subtypes are the three types of small to large conferences I mentioned earlier. This has simplified planning for us, especially reducing the amount of decision-making for each one and making them more repeatable. 
  3. Relationships. I read in a sales and marketing book a while back that sales is a 1-1 activity while marketing is 1-Many. Is that not data modeling language? The distinction helps us at Watermark decide which tactics should be sales activities and which would be better done as marketing.
  4. Relationships redux. It’s helpful to remind anyone doing data modeling that relationships between entities are business rules. For one company’s example, say a Service Technician is assigned to a repair. He or she can only work on one repair at a time. But once we factor in time, a technician might be assigned to 5 or 10 in a day. 
    A given repair is usually handled by a single tech in most instances. Is it always? When we assign a trainee to shadow an experienced technician, then there are multiples. What starts out as a seemingly simple relationship quite often ends up being a more complex, Many-to-Many one needing an associative entity to resolve them. When setting company policies and business rules, I find it helpful to think about a rule over time and the M-M type of relationship helps guide me. 
  5. Flexibility. Associative entities remind me that every business needs flexibility. Example: today we have one duration and price combination for our self-paced ATLs, but would like the flexibility to offer multiple combinations. Some ATLs might have only one but others may have, 2, 3, or more. I won’t bore you here with how to solve this in a data model. But, the associative entity concept provides guidance that every business needs the flexibility that they provide to account for changes and future expansion.


Now, I don’t want to leave you with the impression I go around thinking like a data modeler most of the time. I don’t! But, when a problem arises that has any sort of categorization or complexity of objects, data modeling certainly helps. Post your thoughts about this – I’m curious if anyone else has also found these benefits.

The Ultimate Guide to CBAP Certification FAQs, Part 2: Exam and Application

Watermark Learning holds monthly IIBA Certification chat sessions, and we get great questions each time.

I compiled many of the questions received over several months and answered them below. Most are about CBAP certification and we’ll publish a separate blog post on ECBA questions.

In part 1 of this blog, I addressed parts 1-3 (general, study tips, and study materials) of our extensive guide on CBAP certification FAQs. Part 2 will cover the last two sections on the CBAP exam and application.

For space reasons, we list only a few questions in this post. For all of the FAQs, including those from part 1 and more, download a pdf of The Ultimate Guide to CBAP Certification FAQs.


Q. Will my CBAP score be shown immediately after exam?
A. Yes, you will know immediately if you passed or not. Plus, you will get feedback on which Knowledge Areas you were above average, average, and below average compared to other examinees. This feedback is helpful in case you need to re-take your exam.
Q. Do we have questions on tools & guidelines, perspective, core concept, elements, management theories (theory X)?
Q. How important is “Perspectives,” which is added newly in v3?
Q. What about the Agile extension?
A. You will have questions on all of these except none on Perspectives and none on the Agile Extension. The management theory you mention is an example of underlying competencies and you will be tested on a few of these.
Q. Some techniques may require some calculations, does the exam include that type of questions?
Q. How many formulas will I need to know, and which?
A. We have reports from successful CBAP candidates that they have had to perform calculations on their exams. Our recommendation is to memorize some of the basic financial formulas like ROI, but not the complicated ones like Net Present Value. We also suggest memorizing the formula for PERT estimates, since it is straight-forward and widely known. Our Online Study Exam has some calculations like the ones I just described.
Q. What are the important Techniques I should focus on?
A. The techniques are challenging, since you will receive many questions on them. There are 50 techniques and we can’t imagine you will be tested on all of them. I always recommend focusing on the ones most widely referred to in the BABOK. Appendix B in that Guide has a handy cross-reference table called “Techniques to Task Mapping” to help.
Q. What is the breakdown weight of each knowledge area on the exam?
A. We list the exam blueprints for all IIBA exams (CBAP, CCBA, and ECBA) on our CBAP FAQs page.


CBAP Application

Q. Do I need to complete training before I fill out the Education Hours of the application?
Q. Are there any restrictions on the professional development training requirement?
A. Yes, your 35 BA education (professional development or PD) hours need to be complete before you can finish your application. Bear in mind the training must be BA-related, and “close calls” such as Six Sigma or CSPO Certification won’t count. You also cannot include PDU/CDU activities such as webinars, conferences, articles, etc. You will need CDUs for re-certification, but not for initial qualification for the CBAP (or CCBA or ECBA for that matter).
Q. When filling out the application and listing projects, are we supposed to post projects from most recent to the oldest worked on?
Q. When completing the application, does the “Your Role” field always have to be Business Analyst or can it be Business SME or Tester?
A. Yes, IIBA requests that you list projects in reverse order, newest to oldest. I would recommend you list your role as “BA” and only include hours spent doing BA tasks. If your role was as an SME or tester, you should not include those projects. If you had multiple roles, only list the hours spent doing BA work.
Q. I’ve heard IIBA’s application process for CBAP is pretty strict. Do applications happen to get denied or asked for more info?
Q. Is there a sample tracking sheet available where the six knowledge areas are provided – something that would facilitate tracking the 7500 hours prior to registering for the test?
A. IIBA has made the application process simpler and we think easier with version 3. That said they are strict with the requirements of 7500 BA hours, 35 hours of PD, two references, etc.
We have a tracking worksheet available that you can download and use to list your hours. It has many handy calculations, including a way to see if you meet the 900-hour requirement and the overall 7500-hour requirement.
Q. Regarding the application process, I paid for my application and instantly got an email stating my application is APPROVED. (No wait or lead time.) As for the next step, I believe I need to pay the exam fee. Does that have any sort of processing time or do I expect an instant “Go ahead and schedule” email once I pay the exam fee?
A. You should be able to schedule your exam instantly once your application is approved. If anyone knows differently, please leave a comment!