Covers: theory of Product Development
Estimated time needed to finish: 10 minutes
Questions this item addresses:
  • What are the basic frameworks to follow for product development?
How to use this item?

Product development is all about iteration over and refinement of ideas and eventually turning them into profiltable businesses.

Product Development Landscape

A lot of us start machine learning projects by coding, and that is not the right place to start. Some of us, start from an idea that we think is brilliant, that’s definitely better, but not a good place to start either. The reality, however, is that most successful teams start with many ideas and go through a systematic process of eliminating bad ones, until they’re left with better ones.

Usually better ideas are the ones that involved solving a specific problem for a specific user, therefore getting that user’s feedback is extremely important.

The next step is to grab the better ideas, create mockups for them, and see how the users react to those. This will weed out another subset of ideas.

Then, out of the remaining ones, you want to create prototypes to answer a few questions like “how would the user react to a more realistic looking version of your idea”, “would the technical parts that are most risky work”, or importantly “would someone buy this” where you try to sell the most basic version of your idea and see if anyone buys it.

Again, after invalidating more of your bad ideas, you start coding the more promising ones, and that’s the real validation of your idea being technically feasible. Once you have the most basic version of your end to end software ready, you’re ready to start validating other product aspects, like how can you bring users to it, can they use it, would they continue using it, etc.

Finally you get to the point that the real big user/solution fit question comes up, do they want it enough to pay for it enough to justify the cost? In an ideal world, they even want it enough that you can have a profit and grow over years to come.

In order to ensure that you can move along your journey as fast possible, and with as little risk as possible, you have to make sure you spend just enough time on each stage. For example, you should be able to come up with lots of ideas in span of hours, and you should be able to invalidate some of them with a mockup in a day or two. And putting the alpha version of your product in the hands of user shouldn’t really take more than a month. This is extremely important because this journey is not as linear as I said, it’s very likely that you hit roadblocks in every stage that forces you to go back all the way to the drawing board where you have to come up with new ideas and restart the process all over again.

Maybe one of the the most famous examples of this is Slack, where they went all the way to the business stage, and then had to abandon everything and go back to coming with new ideas to pivot to.

To put some labels on all these stages, and be able to refer to them later, the first few stages are discovery parts, then we go through short term validation, and finally through long term validation.

In the case that you can actually do all this for your core product feature, and make it to the end, you’ll have to repeat it for all other product features you will be adding

(in)Validation Framework

As you can see, a lot of what we are doing in product development is various types of validation, so let’s establish a framework for it, after all the science in data science should mean something! One point of caution that i would like to make is that, what we are trying to do here is in fact “invalidation” of bad ideas. It is very dangerous if you think your task is to prove that your idea is good, because we’re all human and prone to confirmation bias. We see what we want to see. Your task is in fact to identify ideas that won’t work as early as possible to avoid any long term investment that leads into disappointment

The framework that I’m about to lay out is fairly rigorous and can be used for all stages, discovery, short term validation, and long term validation

The first and foremost is writing down what you want to do. It’s amazing how many times I have met people who are well into their projects but can’t still articulate what it is that they are doing

Next, you need to know what assumptions you are making. For that you can do some exercise that I will lay out in more detail shortly. Those exercises will help you articulate what it is you are doing, why to do it all, and why to do it now, and eventually how you think it will happen. This will easily tease out a lot of assumptions that you are making about what you are doing

Next you need to come up with a way to validate these. This usually involved doing some research, either googling or talking to your potential users. Hopefully, you’ll learn a lot about the domain you are working in through these, and invalidate some of your assumptions. In all likelihood, you end up repeating this a few times, every time equipped by learnings that will refine what you want to do, why, and how.

For the validation plan, often you need to design experiments and create artifacts for them. Artifacts can be things like low finesse mockups, to the basic version of your application, all the way to an engineered version of your application at different stages. And you use these artifacts to obtain, qualitative, quantitative, and large scale data about how and why users interact with your app.

This framework can be applied to validate your massive transformative purpose all the way down to what color a particular button has to be

"so what" Exercise (abstraction ladder)

The “so what” framework, although it might sound dismissive at first sight, but it’s one of the most illuminating frameworks that i frequently use. It can really be used everywhere, but most importantly I use it when I want to articulate what it is I’m doing and why. When I want to identify the assumptions I’m making about a problem or a solution. And when I want to understand fundamentally why I want to do something to see why that’s a priority.

The way it works is the following: 1. You write down what you want to do 2. You read that and ask yourself “so what”, and write down the answer 3. Then you read that answer, and ask yourself “so what” again, and then answer 4. And you repeat 3-5 times before you get to the most fundamental reason

Now a few things might happen, you might go through this with no problem, which either means that you are doing it wrong, or you really know the domain of your problem. More likely, you might really struggle to do this beyond the second or third iteration, and that’s a good sign: it means that you have identified things that you don’t know which you have to think about more, or research more Remember one of our principles, if you are stuck and can’t move forward, you have to research more, and mostly likely, you need to talk to your users for that. In all likelihood, once you get to the 5th iteration, where you started might not sound the most reasonable solution any more. So, you might have to go back and modify that and repeat the exercise until everything flows through nice and smoothly.

"why now" Exercise

A very similar framework to “so what” is “why now”. Here your attempt is to map out your assumptions about the urgency of you work with the same pattern as we did previously. Very similar to what we did in "so what" exercise, we repeatedly get to the bottom of why we want to do something right now.

But let me make a few points:

  • All you are doing here is writing down what you want to do and why to document your assumptions. You still need to go through the validation framework to verify them
  • One of my favorite additions to this to really tease out all the hidden assumptions is to also write down how I envision the transition between each of the statements I wrote would happen. This should move you upward through the ladders by going from the more abstract ideas to the more concrete ones.
  • Final point, try to write these from the view point of your user as much as possible. At the end of the day, you want to articulate why they want what you’re building and why they want it now (a point that I clearly failed at in the above example)
Fail to play? Open the link directly:
Author(s) / creator(s) / reference(s)
Amir Feizpour
0 comment

AI Product Development

Total time needed: ~20 minutes
This RECIPE walks you through the most important steps of AI Product Development. By the end of this you should be able to know, on a high level, what information you need to gather to build products that users actually want and leverage AI to achieve that goal
Potential Use Cases
Identifying problems worth solving with AI
Who is This For ?
INTERMEDIATEData Scientists, Machine Learning Engineers, Product Managers, Software Developers
Click on each of the following annotated items to see details.
Resource Asset5/15
VIDEO 1. Product Development Principles
  • What are the principles of product development?
10 minutes
VIDEO 2. Product Development Frameworks
  • What are the basic frameworks to follow for product development?
10 minutes
RECIPE 3. Discovering the Right Problem to Solve
10 minutes
RECIPE 4. Rapid Validation of the Product Idea
10 minutes
RECIPE 5. Preparing your Product for Long-term Operations and Use
10 minutes
VIDEO 6. Building (AI?) Products; Step by Step Guide
10 minutes
VIDEO 7. Product Ideation - Art of Finding the Right Problem to Work on!
10 minutes
VIDEO 8. Product Ideation: From a Hunch to a Concrete Idea
10 minutes
VIDEO 9. How to build products people actually want to use
10 minutes
VIDEO 10. Building your Product Strategy - A Guide
10 minutes
VIDEO 11. The Importance of Strategy in AI Product Management
10 minutes
VIDEO 12. Operationalizing the AI Canvas for AI Product Success (and profit)
10 minutes
VIDEO 13. Defining your AI Value Model for Product Success (and Profit)
10 minutes
VIDEO 14. Identifying Big ML product opportunities inside Big organizations
10 minutes
ARTICLE 15. Data Science Project Management
10 minutes

Concepts Covered

0 comment