Case study: Continuous journey of improvement

Presented by: Dilla Anindita

cookpadteam.com

Hello! I’m Dilla!

Lead Product Manager - Cookpad Global (Bristol, UK)

Current Role:

Past Roles:

Product, Strategy, and Growth in SEA market@dilleuh

linkedin.com/in/anindilla

cookpadteam.com

Our mission is to make everyday cooking fun!

100M+ users in 70+ countries globally with 4M+ recipes published by home cooks worldwide

cookpadteam.com

Why we need to constantly improve

Why should I improve my product? Let’s reminisce the usage of our messaging apps

SD SMP SMA Kuliah & Kerja

Kerja

“Change is inevitable. Growth is optional.”John C. Maxwell

As we seek for growth — craving for improvements while building our products, sometimes we are faced with two common options

Create?Do I need to introduce new

features? Shall we build something new from scratch?

Iterate?Do I need to continue iterating on

this feature? What else can I improve?

In this talk, I want to focus on the iterate piece.

Create?Do I need to introduce new

features? Shall we build something new from scratch?

Iterate?Do I need to continue iterating on

this feature? What else can I improve?

“We’ve iterated on this product every release cycle. I wonder why the metrics aren’t moving?”

AmirThe “Blindfolded Sprinter”

Common challenges faced by product managers: Introducing Amir

The basic thinking or framework we can start using is the classic lean startup cycle you’ve probably seen 99 times

Source: Mind Tools, adapting from Lean Startup

Case Study: Cooking Log

Cookpad introduced a concept called Cooking log around last year

A space where users can document their whole cooking process from start to finish: planning the cooking, during cooking, and result after cooking.

Other users can also chime in to the space and have conversations around the recipe.

We had to evaluate how it works before we get to iterate on it

Source: Decision Diamond (Joe Leech, 2020)

Parameter Example Metric

Business Driver “Make everyday cooking fun”

User Research Value: “I will create cooking log every time I cook!”

Data Frequency: N times / week? Day? etc.

Gut Feel Are we confident that this will work?

It didn’t work as planned! But how did we act upon this insight to improve the product?

Parameter Result

Business Driver 👍

User Research 👎👎👎

Data 👎👎👎

Gut Feel 👎👎

Setting a better desired outcomes

My team had to reset our understanding of the product and redefine the goals.

Make sure to clearly understand the role of each metric.

● Primary: main measure of success● Key drivers: what drives the

success of the primary metric● Health: what should be monitored to

ensure that it won’t break other product (or the superset of the product)

Map the desired outcome to opportunities for improvements

Break down to smaller chunk of potential improvement items or experiments

Example: HMW improve the visibility of cooking log?

We did 1-2 week size of experiment

We conducted rapid testing by interviewing core users to quickly close the loop of build, measure, learn — without too much cost of launching it

Set clear goal for the tests: Value or usability?

Always have clear hypothesis!

Test multiple variants, 1-2x a week

...And first major iteration result was made, and shipped to production. It did improve the experience, but we still haven’t move the needle!

Parameter Result

Business Driver 👍

User Research 👍

Data 👎

Gut Feel 👎

We noticed that a bigger investment is needed to solve this problem, hence we tried to conduct in-depth interviews...

Encourage first-hand experience for the whole team

...And try doing design sprints to quickly make the research findings actionable

Sprints help your team to focus and be on the same page.

Initially we came up with an extreme version for riskiest assumption tests

Launch an “extreme” version in BETA environment: 40 core users

Qual & quant evaluation every week

We conducted A/B testing to see if our new version will work better while continuously iterating on the new version...

Source: Mixpanel

Never test something you built by launching immediately to 100% of all users

Parameter Result

Business Driver 👍👍

User Research 👍👍👍

Data 👍👍👍

Gut Feel 👍👍

...And it did significantly improve the experience for users!

The change was pretty major, the old concept of “Cooking log” introduced before is now history, retaining mainly only the user problem we wanted to solve

Know when to stop doing small iterations (tweaking) when a product reaches its local maximum and invest on bigger bets instead

Source: Crisp Spotify Case Study (Henrik Kniberg)

Tips for Amir: The Blindfolded Sprinter

Some tips about continuous iteration

● Healthy pace: Min. one new learnings per week

● When there’s no previous benchmark (most likely early-stage products), take the market standard to indicate success: 10% WoW growth

● Take your time — real impact takes time, tydacc instan seperti bikin indomie

● There’s technically no “golden recipe” that applies to all companies & industries to build “the perfect product”, you have figure it out by trying!

Thank You!

Presented by: Dilla Anindita

cookpadteam.com

All GIFs credits to: Giphy