sds podcast episode 353: how to practice human … · thinking about your customers throughout the...
TRANSCRIPT
SDS PODCAST
EPISODE 353:
HOW TO PRACTICE
HUMAN-CENTRIC
DATA SCIENCE
Kirill Eremenko: This is Episode number 353 with Founder of
Designing for Analytics, Brian T. O'Neill.
Kirill Eremenko: Welcome to the SuperDataScience podcast. My name
is Kirill Eremenko, Data Science Coach and Lifestyle
Entrepreneur and each week we bring you inspiring
people and ideas to help you build your successful
career in data science. Thanks for being here today
and now, let's make the complex simple.
Kirill Eremenko: This episode is brought to you by my very own book,
Confident Data Skills. This is not your average data
science book. This is a holistic view of data science
with lots of practical applications. The whole five steps
of the data science process are covered from asking
the question to data preparation, to analysis, to
visualizaton and presentation. Plus you get career tips
ranging from how to approach interviewers, get
mentors, and master soft skills in the workplace.
Kirill Eremenko: This book contains over 18 case studies of real world
applications of data science. It comes off algorithms
such as random forest, k-nearest neighbors, naive
bayes, logistic regression, k-means clustering,
Thompson sampling, and more. However, the best part
is yet to come. The best part is that this book has
absolutely zero code. So how can a data science book
have zero code? Well, easy. We focus on the intuition
behind the data science algorithms so you actually
understand them, so you feel them through. And the
practical applications, you get plenty of case studies,
plenty of examples of them being applied. The code is
something that you can pick up very easily once you
understand how these things work. The benefit of that
is that you don't have to sit in front of a computer to
read this book. You can read this book on a train, on a
plane, on a park bench, in your bed before going to
sleep. It's that simple, even though it covers very
interesting and sometimes advanced topics at the
same time.
Kirill Eremenko: Check this out, I'm very proud to announce that with
dozens of five-star reviews on Amazon and Goodreads,
this book is even used at UCSD, University of
California San Diego, to teach one of their data science
courses. So if you pick up Confident Data Skills, you'll
be in good company.
Kirill Eremenko: To sum up, if you're looking for an exciting and
thought provoking book on data science, you can get
your copy of Confident Data Skills today on Amazon.
It's a purple book, it's hard to miss, and once you get
your copy on Amazon, make sure to head on over to
www.confidentdataskills.com where you can redeem
some additional bonuses and goodies just for buying
the book. Make sure not to [inaudible 00:02:47] that
step. It's absolutely free. It's included with your
purchase of the book but you do need to let us know
that you bought it. So once again, the book is called
Confident Data Skills and the website is
confidentdataskills.com. Thanks for checking it out
and I'm sure you'll enjoy.
Kirill Eremenko: Welcome back to the SuperDataScience podcast,
everybody. Super excited to have you back here on the
show. Today, we've going to very cool and interesting.
All of our episodes are very cool, but today's a very
interesting episode because approaching data science
from a different perspective. To provide some context,
let's try to answer this question. What are the
outcomes that we're going for in this specific data
science project? So you might be working on
something and how often do you ask yourself, what
are the outcomes we're actually going for? Or for
instance, this question. How are we going to measure
success in this data science piece of work or project or
analytics tool, decision support system that you're
building, model, insights, how are you going to
measure success?
Kirill Eremenko: So the thing is that very often, we get caught up in lots
of different things that comprise data science, from
thinking about AI and data science strategy to juggling
around different components of IOT systems, to
working with data preparation or building different
models, gathering insights, creating business decision
support systems, and so on. Visualizing our data,
presenting on it. We get caught up in all these different
things. But what we might get in the end is in the
words of Brian T. O'Neill, a technically right result or
insight, but effectively wrong. What does that mean?
Well ... and or actually, why does that happen? Well,
that can happen because along the way, we hadn't
been thinking about the end user, about their
experience, about putting them in the middle of
everything. That's where human-centered design
thinking actually comes in.
Kirill Eremenko: Brian T. O'Neill is an expert in the space of human-
centered design, specifically for enabling decision
making to be precise, human decision making, in data
science. So he's bringing in the field of human-
centered design which exists in other areas of the
world as well, he's bringing it into, or he has been
bringing it for many years now into the space of data
science and enabling decision making. So basically
thinking about your customers throughout the way.
Kirill Eremenko: That is a very powerful tool. It's a specific soft skill but
it's not just about presenting the insights. It's about
thinking about your user throughout the whole
journey. In this podcast you'll get a lot of tips on how
to do that. So Brian T. O'Neill's a consultant in that
space, he's been doing it for many years. In this
podcast, we will learn how to ask the right questions to
understand the business needs and what actually is
desired from a certain piece of work that you're doing,
the seven steps that he performs when he goes into
companies to do his consulting work, understanding
outputs versus inputs and the consequences or the
outcomes that come out of your outputs.
Kirill Eremenko: So lots of interesting questions will be raised in this
episode and I have a feeling if you apply the things
that you learn here in your next data science project,
you'll see a different attitude from the people that
you're going to be presenting and delivering it to.
They'll have a much better experience and results from
your project.
Kirill Eremenko: So there we go, that's what this episode is all about. As
usual, all of the links to connect with Brian T. O'Neill
will be mentioned at the end of this episode, but I want
to mention one already now. In case you don't get to
the end of the episode, but you want to learn more,
Brian has set up a special page for us, thank you so
much Brian. It's
designingforanalytics.com/superdatascience. You can
learn more about his work there if you'd like.
Kirill Eremenko: On that note, let's dive straight into it and let's learn
about human-centered design thinking in data
science. Here we go. Without further ado, I bring to
you Brian T. O'Neill, founder of
designingforanalytics.com.
Kirill Eremenko: Welcome back to the SuperDataScience podcast,
everybody. Super excited to have you back here on the
show and today's guest is Brian T. O'Neill calling in
from Boston. Brian, how are you going today?
Brian T. O’Neill: I'm doing great. How's it going?
Kirill Eremenko: Very, very good. I'm super excited about today's show
because we're going to be talking about some really
cool things, but first of all, you are a man of many
activities and things that you do. So in addition to
data science, you play the drums, right?
Percussionist?
Brian T. O’Neill: That's correct. I do do that.
Kirill Eremenko: That's fantastic. Where can our listeners hear some of
your work, because ... we've chatted a bit about it
before the podcast, but you play jazz mostly, is that
right?
Brian T. O’Neill: Jazz, chamber music, orchestral music, Broadway
shows, that type of thing. Little less on the rock pop
music. Occasionally, some stuff like that, but yes, a lot
of jazz, classical, world music.
Kirill Eremenko: It's not just like a hobby because you know, I play
around, or I enjoy dabbling on the piano but I know
two pieces. You actually play professionally. You have
two lives effectively. There and here. How do you
combine that?
Brian T. O’Neill: Oh yeah. Well that was my training. You know, my
formal training was in music. So I have a degree in
percussion studies. I work as a freelance musician
around Boston doing as I was saying, a lot of classical
work. I play with a lot of the Broadway theater shows
that travel through town in the pit orchestras for
those. Occasionally some star attraction work, video
game orchestras will come to town and they pick up
musicians. Then I run a group called Mr. Ho’s
Orchestrotica, which sounds like it's spelled, it spells
like it sounds, just orchestrotica.com if people are
interested in that, that's more of what I call my
startup. It's like running your own a little business
and promoting original music in the kind of chamber,
jazz, and global jazz kind of space.
Brian T. O’Neill: So yeah, so I do that and then I've been designing for
the webs for since 1996, if you blur that out. But yeah,
about 25 years doing design, started out as a web
designer, gradually moved into the kind of the Boston
startup dot com scene and then got into more
enterprise stuff, fidelity investments and JP Morgan
and working in some banking and larger enterprise
kind of contexts. Then kind of just got into some very
nerdy IT related software products. There's actually a
lot of enterprise B2B companies here in the Boston
area. I had clients when I started, when I went
freelance in like 2006, I started working for myself so I
could kind of balance my two careers. I just had
clients that kept kind of bringing me along to their
next projects and they tended to be at very technical
companies so products for other IT people, technical
data products, this kind of thing.
Brian T. O’Neill: Analytics kind of was simmering behind the scenes
and all of these and so that just became a kind of a
focal point for me. About four years ago, I decided to
kind of specialize my consulting work in data
products. My goal is really to help companies design
innovative and engaging data products powered by
data science and analytics and really to focus on that
last mile, which is where humans interact with the
stuff that all of this great smart people that are doing
things with data and math. At the end of the day if
there's a human in the loop in your system and you're
not developing a fully automated solution, then they
are a factor in the success of the work that gets done.
Doing that well is a different skillset than the modeling
piece and the training sets and getting the data
cleaned up and all those other things. You can get it
technically right and effectively wrong, so I want to
make sure that you know my clients and the people
that I train in my seminars in this are focused on that
human last mile piece. Really understanding the
problem space, understanding how humans are going
to perceive the work that's being done, how they're
going to understand the data.
Brian T. O’Neill: The visualization is part of this. We typically jump to
that when we talk about design, we think of data
visualization. I tend to think that there's a layer,
there's a perspective that's a little bit higher elevation,
which is in the design world we call this the user
experience kind of layer which sits above the interface.
Because you can actually, you can technically get the
data visualization piece right, but if you don't have the
right data to begin with and you don't understand the
context of use, then it doesn't matter if the
visualization is the best way possible to do it, whatever
the heck that means, it doesn't matter. Right? Because
if no one's logging in to use the service and to make
decisions, you know ...
Brian T. O’Neill: That's really what a lot of our work is about, right? It's
really about decision support. So if we don't create
decision support with these models and analytics
services, then we're really not having an impact with it.
We have to look beyond the ink and the data ink that's
on the page and think about workflows and how do
people do their jobs and what are they concerned
about with this technology, do they understand it?
What's the change management that may be required
there? That how I see it and one of the concerns I have
... and jump in if I'm just like babbling here, but it's
the-
Kirill Eremenko: [inaudible 00:12:25] very interesting.
Brian T. O’Neill: Yeah, the reason that ... one of the reasons that I know
this is failing, I keep hearing this repeated over and
over, is that what's sometimes called the
operationalization of models in the non-software
companies, they tend to talk about AI and predictive
analytics in this context of improving an internal
business as opposed to creating a software product
that has data science and analytics behind it. Those
are kind of two branches and they use words like
operationalization and change management.
Brian T. O’Neill: From a design perspective, from a designer's lens on it,
I don't like that perspective because I feel like what
that means is this team goes and does the technical
part and they're going to spit out a spreadsheet, a
visualization, a Tableau thing, a field and the CRM.
There's some output and then it's some other team's
job to go in and make the business use that stuff. This
is where I think things can break down, right?
Because you've got two teams trying to do the right
thing but I think the perspective that's more important
is did we design the solution, the model, the thing, the
software application, whatever the output medium is
with the engagement model in mind from the start and
look at that as part of the success of the overall data
science work?
Brian T. O’Neill: It's not a second thing. It's not something that you
pass off to another group. It's integral to the work.
Think of it as integral, not a deliverable that you pass
to someone else to go shove down people's throats.
That's not how you build stuff people want to use.
Instead, you get them involved from the start, right? If
it's the sales team or it's the CMO or the marketing
department or whatever, they should be involved from
the beginning of the project so there's no big giant
reveal. Like all of a sudden, the CEO is like, "Today,
we're making a big change. We're releasing a new
model to do X," and jaws are hitting the floor and
people are like, "F that. I'm not using that. I'm going
to, I'm still calling the same ... I've got my sales
prospects, right? I'm not calling this list of sales
prospects you came up with in the dark with some
magic AI stuff. I don't know what that's about, but I
know who's going to buy this week and I'm going to
call those people. That's my job as a salesperson."
Brian T. O’Neill: Well right there, there's your six months of data
science work down the toilet, right? Because this
salesperson does not want to, they don't know how
you came up with this list of strange customers that
they've never talked to, but you're saying, "Oh they're
going to close next week. They'll sign on the dotted line
next week. And oh, by the way, here's what you should
charge them. Here's the price quote that you should
use," and the salesperson is like, "How did you come
up with this number? Where does this from, I have no
idea what this is about." Because they weren't involved
with the solutioning and the problem discovery and
they weren't ... There was no research done. That's
where things can totally break down.
Brian T. O’Neill: So the designer lens is know these people are integral
and we have to factor them in from the start and we're
all going to have a better time doing this work together
because I'm sure ... [inaudible 00:15:33] I'm sure
you've had this experience, it's more fun to work on
stuff people want to use, right? Not stuff like, it just
like-
Kirill Eremenko: For sure.
Brian T. O’Neill: You're hoping this reveal, it's like where's the smiles?
And instead it's like, kind of quiet in the room and
people are like, "What does that mean? What do I do
with that number?" We don't want to have those kinds
of experiences. We want to deliver, like, "Yes, yes.
When can I get more of that? Oh, could you also show
me this? Does the model factor in this thing? Oh it
does? Oh that's awesome. I hate doing that work in
spreadsheet." That's the kind of stuff we want to hear
at the end.
Kirill Eremenko: Yeah, totally agree. It's interesting because I was
interviewing Stratos, one of our students, just
yesterday on the podcast as well. He said that when he
was applying for data science job last year, at the
interview for the job that he actually ended up taking,
one of the questions was related to exactly this about
soft skills. How he would present the data and data
science project, how he would talk to executives, how
he would go about helping people understand what
insights he's communicating. So it's exciting to see
that companies are not only realizing this after the fact
now, once the data science projects are starting to fail,
they're doing it preemptively. They're hiring people who
know what they're doing in terms of this, what you call
it, operationalization and soft skills that change
management's in.
Kirill Eremenko: What I wanted to ask you, so walk us through the
process. You've outlined how important it is, it's totally
a critical part. What's the point of doing a project if
nobody's going to end up using it? But once you go
into a company and they need your help with this
operationalization or change management or soft skills
and data science, what are your typical steps? How do
you identify the problems? Are they at the start, in the
middle, at the end of the data science project pipeline
or work life cycle? Then once you've identified the
problems, what do you do about them?
Brian T. O’Neill: Well, the most popular response to this question by
consultant ever is, "It depends." There is a broad
design process that I typically use, but that process is
more like, it's more like a shelf of ingredients. I may or
may not use this ingredient with this particular pie
this week with this client or I may use a ton of it. Or I
may start with the flour and then add the water later
and the next time, the flour doesn't come in until way
later in the process.
Brian T. O’Neill: One of the things that clients need to understand
when they're doing this type of work, when you're
doing creative work, when you're doing discovery work
to get into people's heads and understand the problem
space and all of this, is that it's not highly analytical
work, ironically. You're going to have to ping pong
back and forth to understand what's needed. So
sometimes you actually, you need to get into the
design itself in order to figure out what needs to be
designed.
Brian T. O’Neill: Research is a big thing that's often missing in this
place and that can sound like this really expensive
long thing that takes forever to do. Nope. A lot of times
what I'm talking about is having one-on-one
conversations with the actual consumer of whatever it
is that's going to ... whoever's going to use this. You
know, if we're talking about predictive analytics,
whoever's going to use this predictive score, we need to
figure out what is going to make them want or not
want to use this from the beginning.
Brian T. O'Neill: You may even need to start with, well, we don't even
know who's going to use this, and so right there we
haven't even figured out who is our team, who are the
stakeholders in this project, what are their interests in
this, and what is going to make or break this? You
may need to have a conversation with your senior level
stakeholders, because sometimes what can happen is
you can't even get the time you're trying to... Say
you're helping out the marketing department and
they're like, "We don't have time to sit in your ideation
sessions to go through this." Well, senior management
needs to hear that, and the response from... If the data
science experts are the ones that are leading the
process here, what they need to hear is, look, we can
build a model for anything if we have the right data for
it.
Brian T. O’Neill: If you want us to build a decision support mechanism
for the marketing department, the marketing people
need to be involved in the process of helping us
understand the pain and the need, and how they're
going to use this. If they're not, what you're going to do
is pay my department $5 million over the next three
months, and you're going to have a high risk output at
the end of that. So, do you want to take the chance
that all the work we're going to do is going to hit the
floor and it's never going to be used, or do you want to
make sure that the marketing people are saying,
"Wow, we know how to stop advertising in the wrong
spot. We know who to send our mailers to for the next
campaign that's coming out. This is really helpful
information."
Brian T. O’Neill: And if you want the ladder, you've got to have those
people involved at the right time. So, there needs to be
a clear understanding of who our users are, our end
users, who our stakeholders are, how we're going to
understand what it means for this output. Again, our
visualization, our predictive model, whatever it's going
to be, how will we measure the success of that at the
end of the project before we get into building anything,
right? And what we may find is that you don't need a
machine learning model for this product. Maybe the
first version of something is like, you know what?
Right now you're taking a wild ass guess every time
you decide which cohort of people are we going to send
this campaign to, right?
Brian T. O’Neill: Well what if we could just simply tell you how many
people open the mail that we sent last year, and we
can compare it to these other metrics. And it is, is just
historical data, but we can at least get that going.
Would that be an improvement? In one month, would
that help you start making better decisions about
where to spend marketing dollars? And if they said,
"Yeah, that would actually be really great," well now we
have a way to start small, and we're really focused on
that business outcome, right? Instead of focusing on
building a model, we're focused on help the marketing
department know who should we send mailers to and
who should we not send mailers to for the Spring
20/21, whatever, shoe campaign, right?
Brian T. O’Neill: So,
Kirill Eremenko: Yeah.
Brian T. O’Neill: So, that's part of it. There are several different steps.
In my seminar, I have these, approximately seven
different steps. You have your team building, we have
a stage of research and problem finding or problem
definition, which is where we get really crystal clear
with our team about what problem we're trying to
solve, not what data science problem we're trying to
solve, but what people problem are we trying to solve,
and what business outcomes we're going for.
Brian T. O’Neill: We then move into starting a design brief, and this is
where the question of ethics starts to come in. So, we
may be looking at what are the second order
consequences of the work that we're doing here?
Where might we need to put checks in place with the
work that we're doing so that the solutions we're
building are ethical, and useful, and usable, and
actually consciously thinking about this, and not just
waiting for a story to hit the news that we don't want
to hear about. So, that's a factor in the process. From
there, I'm a big fan of using a couple of tools from
design, which are called journey maps or service
blueprints. So, maybe you've heard about these before,
but this is a visual way of talking through the
customer's journey, where they are today, and how
they do their work today, and plotting that out visually
over time so we can understand what's it like to be
this marketer that needs to send out things.
Brian T. O’Neill: How do you decide how to do that today? What
process do you go through? Well, we collect these
analytics from this tool, and then we go into Tableau
and then we look at, whatever, the CRM, and then I
kind of take a guess based on what I think the
market's doing. Anyhow, we map this thing out, and
by understanding this customer journey and all the
departments that may be involved, we can start to
have a bigger picture about how does our little, or
maybe major data science initiative fit into that
workflow, and where might it hit the ground, right?
Where is there a gap that may not be a data gap, but it
may be an engagement gap, like trust.
Brian T. O’Neill: Maybe we find out the salespeople are on the road all
the time. They're not going to open up a PDF, they're
not going to open up Tableau in some desktop thing or
whatever. They're only going to respond to text
messages, whatever's on the screen in their little app
that they use. We need to provide them with really
good recommendations on which door should I go
knock on next if I'm selling widgets door-to-door, or
something like that. We need to understand what it's
like to be that person and to do that job, so that we
have that context the entire time that we're doing our
work. So, the journey maps and service blueprints can
help with that. The difference there is really whether
you're talking about external customers' experience, or
you're talking about internally how a business process
works. So, the service blueprint is really for if you're
building a model or something to improve operations
inside a business, it's... They talk about the front stage
and the backstage. So that's what that service
blueprint version is.
Brian T. O’Neill: But they're very similar. They look very similar. And
then from there-
Kirill Eremenko: So, the service blueprint is internal?
Brian T. O’Neill: Yes, it covers... think, again, think of it as the
backstage, like the behind the scenes. When you go to
the Apple store, it's like thinking about all the process
of how they onboard you when you come in and have a
customer service on your iPhone, right? And then, all
of a sudden they walk away with your cracked iPhone
screen. Well, behind the scenes there's a whole bunch
of stuff happening there. They're probably like, is it
under warranty? No. Okay. If that's the process, then
we do a quick five minute check of the screen, and see
if we can do a hot swap. Nope. Okay. There's a whole
process that they go through there, but the customer
doesn't see that, right? So, that's more of that service
blueprint version.
Brian T. O’Neill: So that may be applicable for your audience that's
working as an employee inside a business, trying to
improve the internals of that business from there,
yeah, and another part of this is what I call the
honeymooning and the onboarding. And this is what,
again, we sometimes use this term operationalization,
but I also like to think about what I call the
honeymoon period, which is the period between when
we make the announcement or we "launch," launch or
put into production, whatever the output of our
analytics work is, there's a period of time here where
it's new and it's different, and I call this the
honeymoon, and your design and the way this works,
you may need to consciously put intentional effort into
how we help with that transition, instead of relying
heavily on training, which it's hard to get people to
show up for this stuff. It may be something where we
actually need to design into the software.
Brian T. O’Neill: For example, how do we transition someone from the
old way? We know that you used to use a spreadsheet
here. Well you can actually upload your spreadsheet
here, and then we'll map this into our predictions, and
we'll help you save some steps of maybe they need to
key in a bunch of data in order to get back some
recommendations on something, and by
understanding what these blockers are, the friction
points, we can actually smooth this transition out by
really... And I'm sure you and your listeners have
experienced clunky onboarding when you download a
new app for your phone, and a lot of times they force
you through a tour, and you're like skip, skip, skip,
skip, skip. Just get me into the product, right?
Kirill Eremenko: Yeah.
Brian T. O’Neill: You kind of want the product to just be intuitive. You
don't want to read a bunch of screens about all this
stuff it's going to do, because you probably
downloaded it because you have one thing you want to
do, and now they want to tell you about 20 things that
you need to do through a video or whatever. All that
stuff is just in the way. But if you really understand
how someone wants to use the service, and what their
job is, or what they need to do, you can design that
experience to gradually bring them into the new way of
doing whatever that may be. So, from there you get
into actually doing the sketching, algorithm design
planning, getting visual with workflows. If there are
visualizations that need to be presented here, then I
like to work low-fidelity.
Brian T. O’Neill: So, I kind of teach this idea of working lo-fi with a
small team with your power team that we talked about
in the first module there, but working at a whiteboard
together, and trying to get visual, and to prototype or
simulate what might our outputs look like in low-
fidelity before we ever do any data work whatsoever.
And again, partly what we're doing here is we're taking
away the giant reveal. It shouldn't be the... There's a
black cloak. You walk into this dark room and then,
bang, the lights go on, and here's the data science
model. That is not how you want to release stuff.
Kirill Eremenko: It's kind of the difference between Waterfall and Agile, I
guess.
Brian T. O’Neill: Yeah, exactly. So, we want people nodding their heads
as we move through this process because they know
what we're doing, they know why we've done it the way
we're doing it, and they've been involved throughout
the process. So-
Kirill Eremenko: And they feel like owners as well.
Brian T. O’Neill: Yeah.
Kirill Eremenko: In the end the product is you're presenting it to their
bosses, they'll be on your side helping you present it
rather than on the opposite side.
Brian T. O’Neill: Exactly. Exactly. So yeah, so there's this visual
process there, and we're doing this work and iteration
with our team. And then, kind of the last two, really
the last formal phase here. And again, remember by
this point, you may realize, wow, we don't even know
what we're trying to do here. We haven't clarified the
problem yet. We're, we're sketching stuff, but we're
realizing by getting visual here, our chief marketing
officer has been on our team here participating, and
they still don't really understand how they're going to
use this number, that our model is going to come up
with, to do their work.
Brian T. O’Neill: We might need to go back to the drawing board, do
some other research or talk about the problem space
more before we go any further, before we start doing a
ton of work, collecting data and building pipelines and
all this stuff. You may ping pong back and forth
between these different stages before you move
forward. But the last kind of formal... If we were to do
this in the perfect theoretical way where it was step
one through six, perfectly, the last step here would be
doing validation of the results here.
Brian T. O’Neill: So, what does that mean in the context of a predictive
model or something like this? Well, the easiest way to
boil this down would be, let's say you're going to
present a score from zero to 100. You have a
probability that's what your model spits out. And the
CMOs going to... Every day they log into this
dashboard and your model's going to produce a 67. On
Tuesday, let's say the score is 67. Well, what are you
going to do with that 67? And 67 as compared to
what? And asking this person, well, what would you
do with this 67, and letting them talk about how are
you going to react to this score?
Brian T. O’Neill: So, by presenting them a visual and having a
conversation with them, we sometimes call this
usability testing or design validation, we can start to
tease out what might need to go into the engineering
and the modeling. So, what you might hear is, "Well,
67 doesn't feel very certain to me, but if I understood
why it was 67, then I might know who to send my
mailers to, right? But right now you just say it's 67,
and I don't really know how you guys came up with
that." So, ding-ding, light goes on, right? We might
need model interpretability here, right? We may need a
way to show which features contributed most to that.
And if they didn't say that, and someone said, "You
know what? Anything above an 80 I'm cool with that. I
don't really give a crap how you guys came up with it
because it doesn't matter. Anything above 80 is
awesome. Anything below 50 I'm just going to totally
ignore. I don't care."
Brian T. O’Neill: Well, at that point you may say, "Well you know what?
We can come up with a much better algorithm here
that's 92% accurate, if you don't really need to know
how we came up with this, and there's no compliance
issues," or whatever, that can start to guide the
technical decisions that are made in terms of how the
actual data science part works. But guess what? You
don't need data to test this out. You may find out that,
oh with the 67 we need this kind of model
interpretability, and in fact maybe we need a scoring
system, qualitative ranges like, anything between 67
and 75 is a buy. Anything that's below 67 and 52 is a
hold. Red, green, yellow. Sometimes we talk about the
traffic lights.
Brian T. O’Neill: The point here is you're talking about putting a
qualitative measure onto this quantitative score that
came out, and the only way you're going to know what
those qualitative ranges should be is by having a
conversation with your customers, with your users, to
understand how they're going to perceive this
information and how they're going to act on it. And
you don't actually have to build the entire model to
know that. You can also start to tease out how
accurate does it need to be, and this is something I
hear about, a fair amount is data scientists, especially
young ones, they want to do cool data science work.
Some of the more academic ones want to publish
papers about how accurate their models are, et cetera,
and they're focused heavily on the accuracy of the
model, and not so much on the, "did someone use my
model to make decisions?"
Brian T. O’Neill: In a business context, that's what they care about.
And what you may find out, I have a podcast episode
about this, the title was something along the lines of,
"When does the 60% accurate model beat an 85%
accurate model?" And the joke is, well it's the one that
actually gets used to make decisions. That's what
matters. And the reality was is this person was David
Stevenson, he was talking about this was a big light
bulb moment for him. It was when he learned from his
client, I forget it was an employee, or if he was
consulting there, but the client, he was spending all
this time trying to get the accuracy up from 80 to 82%
or 85%, and his business sponsor was like, "What the
hell are you doing? This is so great. If you can tell me
that this is 65% accurate, let's go onto the next thing. I
don't care. I've made my decision. It's a yes/no
decision."
Brian T. O’Neill: I forget, I'm kind of paraphrasing it. This could be
wrong, but the point here was that was more than
enough accurate for some really great business value
to be created. And spending twice as long to get a 5%
increase in the quality of the prediction was not a good
business decision whatsoever, because now you're
spending all your time doing this work that the
sponsor or the user doesn't care about. It won't make
any difference on how that person does their job,
whether it's 80% or 85% accurate, it's completely
meaningless. But if you never have those
conversations with your stakeholders and your users,
you're never going to know that. You're not going to
know what the pain and gain looks like from their
perspective. This empathy is what's required for us to
understand how to design really effective solutions. We
have to put ourselves in their perspective and take off
our technical hats, and look at things from the
perspective of the person that's consuming them.
Brian T. O’Neill: That's what design is really about. It's really about
empathy and being able to put ourselves in their seat
and their role, and relate to what that person's job is
like, and how they make decisions about things, and
how do we slide our technology in there too to help
with that.
Brian T. O’Neill: I know that was a long winded explanation of the
process, but it's mushy, it's gray, it's mushy, it's not
perfect, it's not clean.It can be, we can put structure
on it. That's partly what we talk about in my seminar
is, yes, it's supposed to be a little bit messy. We may
need to ping pong back and forth. That's the
innovation space. But we're also trying to fail fast here.
We're trying to learn quickly what's working and
what's not without spending a ton of time building the
wrong stuff. So, when you get that question, what is
our machine learning strategy? Right there, your
alarm should be going off. This is a bad question. This
question needs to be unpacked. And the reality is, is
your business sponsor, if you're not at a software
company, your business sponsor probably doesn't
understand what's possible.
Brian T. O’Neill: So you may need to have a separate discussion about,
well, what is AI? What is possible with these
technologies? And realize together we need to have a
better conversation about what business outcome we
want. Yes, we will try to use machine learning if that's
the best thing possible for us. And if you really just
want machine learning no matter what, then let's talk
about this in laboratory, what I call, lab mode. Let's
have a project where really what we're doing here is
we're going to rehearsal and we're practicing. We're
having a scrimmage.
Brian T. O’Neill: And if that's the point, let's take a really tiny project.
There's no expectation of business value. We're just
here to exercise our abilities to see if, can we collect
data? Can we put the training data together? Can we
test it? Can we deploy it? If that's really just to exercise
our skillsets and perhaps to see where do we need
more talent? Is it visualization? Is it data engineering?
Fine. But the point is you have a clear plan and a clear
conversation to set expectations that the goal of this
project is a lab mode to work on these skills, to see if
in the future we actually have the skills to put AI to
good use in a business context and produce some
value.
Brian T. O’Neill: I don't think that's what's happening. I think usually
it's like, go give us some cool (beep), excuse my
French, go build some amazing thing with... hire some
PhDs, and they're going to come up with this magic
sauce, and they're going to give it to us at the end. And
then, there's this big disappointment, or the data
scientists are saying, "Well, what is the problem you
want me to work on?"
Brian T. O’Neill: And the business person is saying, "Well, what's
possible?" And they're like, "Well, I don't know, you're
the product manager. What would you like us to help
you with?" And you can see there's like a tennis game
going back and forth. And my feeling is, and the people
that I talk to in my show, it's time for the data people
to step up here and start to have a better
understanding of the people that are going to use
these solutions.
Brian T. O’Neill: And it's not to say that business people don't also have
a responsibility to become more data literate, but I
tend to think that the last straw in this game are the
people that are writing the code and pushing this stuff
out. It's the data people. And so they are the linchpin
in this.
Brian T. O’Neill: And I think that skill set needs to be developed, at
least in part, by the data people. They need to learn
how to ask good probing questions. They need to learn
how to extract these needs from a stakeholder who
may not understand what's possible yet with these
techniques, and to try to really guide that person to
express their need more clearly so that your team, if
you're the data people, you can be assured that my
work is not going to fall on the floor in six months.
People are not going to be wondering what is the value
of paying... And this is expensive, right? People are
paying top dollar right now for this talent. But guess
what? It's going to change.
Brian T. O’Neill: The salaries are going to come down. Everyone's
jumping into this space and at some point there's
going to be a lot of people with "data science" in their
title or claiming this, and who's going to be left
standing are the ones that can actually turn data
science into value and outcomes. And that requires a
different skillset. It's not Python, it's not R, it's not
Kubernetes, it's not all that technical stuff. That's part
of it. But there's another part of it here if you really
want to connect it to the people.
Brian T. O’Neill: So, anyhow. I'm blabbering. You asked me some
questions, but I'm hoping this is helpful to your
listeners.
Kirill Eremenko: Very helpful. I'm listening, soaking it all in. Very
interesting insights.
Kirill Eremenko: What I think would be very helpful for our audience is
the concepts you identified fantastic. From team-
building to design brief, journey maps, service
blueprint, honeymooning, sketching, algorithm design,
validation, usability testing, very useful tips.
Kirill Eremenko: However, it sounds like that is something more of, it's
good to be aware of for anybody, but also that's a
framework for a consultant who goes in to analyze a
business or maybe for even a business leader or
manager to work with their stakeholders around data
products.
Kirill Eremenko: The question that I'd love to get your opinion on is,
what can an individual contributor, an IC data
scientist, somebody who's there building the code,
who's not the only data scientist in the whole company
where he would obviously have the mandate to apply
this framework of certain steps, but he's a part of a
bigger team. Maybe there's a hundred people in this
team, maybe there's five or 20 people in his team. He's
not the manager, he's not the leader or he or she. He
or she, they are part of this bigger team so they don't
have this control or say over what's going to happen,
what's not going to happen. They're just doing their
job. How can they be better at design thinking?
Brian T. O’Neill: Sure. It's a fair question. And ultimately these are
strategic questions. They do come down to that. I think
the way to think about this is by being objective and
asking good questions. For example, how are we going
to know that we did a good job with this project? If
you're having questions about the work you're doing
and you feel like, "God, this project is going off the
rails," well maybe it's time to get your team together
and just have an informal conversation and say, "It
would really help me, could we come up with just five
bullets that are going to dictate the success of this
project?"
Brian T. O’Neill: How would we know, not technically, not anything to
do with the data, but at some point we're going to
present something to somebody, right? They're going
to consume this and they're going to make a decision
about whether it's good, bad, okay, excellent,
whatever. How was that going to happen?
Brian T. O’Neill: And if there's silence in the room or it's really mushy,
then you can express that. Should we spend some
time clarifying this so that we can make sure that we
really hit a home run here? To use a baseball analogy,
are we hitting a home run or a base hit, here? Well, if
no one can tell us the difference between what a base
hit and a home run is, then what do you think the
chance of us hitting a home run is? It's probably pretty
low.
Brian T. O’Neill: It sounds really simple here, but if you have responses
like, "We will impact the business, we will use AI in the
CRM. Well, we could probably find an office shelf
thing, create a field in the CRM and shove a data point
into it and say, "We created AI in the CRM." Whatever,
right?
Brian T. O’Neill: That's not clear enough for us to actually be actionable
here and produce value. And I think if you really ask
these questions with your team, and it's not meant to
challenge anybody, it's in pursuit of clarity for the
team so that we don't create a data output that falls on
the floor.
Brian T. O’Neill: And part of this is how you ask the questions. I have
articles on my site about how you do this kind of
research, but a lot of this really comes down to asking
really good open-ended questions, listening and trying
to kind of facilitate the conversation here so that
everyone sees why we're asking these questions. But I
think that's one way to do it, is simply to have a
question about what are the outcomes we're going for,
here, and how will we measure that these were
effective? It's very rare, I hate to say it, it's rare that I
see that because most of the time employees are
usually compensated for inputs and effort. You trade
your time, we pay you a salary every month to come in
and use your data science skills and you're really
paying for time.
Brian T. O’Neill: This model focuses more, even though you're still
going to get paid for your time, this model comes from
a different one, which is, "What if our compensation
was based on the results that we created, the
outcomes and the value that we produce?" And I think
part of the reason we don't have this question as much
is partially because of the way most companies
compensate their employees. And I'm not suggesting
we change that or anything, but that's, I think, part of
the reason these questions don't always come up. And
instead we just kind of wait for a boss to tell us "This
is the next thing we're doing. Here's the project, here's
where the data are, build some connectors. We're
going to need to clean up this X, Y, and Z, dah, dah,
dah, dah." And you kind of just go in and do the work.
Brian T. O’Neill: At some point in your career, if you're a junior and
you're probably going to need to go kind of down the
expert/contributor path, or you're going to go into
management, but either one of those two things, the
more you can start to realize that I'm not really being
paid here for our programming, that's not really what
they want. The reason why someone is funding my
team is they want the value that our programming
theoretically can produce for the business. And if you
don't know what that value is, it's going to be a lot
harder for you to be seen as a great contributor.
Brian T. O’Neill: And so trying to align your work with that bigger
picture, that's one way to really start to connect the
dots, here. Is to say, "You know what? Yes, I know how
to do the model. Yes, we have the data here. But my
concern is when we talked to so-and-so, the CMO,
they said, we're not going to use this. If we can't
understand how you came with this prediction, we're
not going to be able to use it. We can't take the risk,
and we're going to keep the status quo here."
Brian T. O’Neill: So when your boss is saying, "Look, we can get 95%
accuracy by using this deep learning model, blah,
blah, blah," then you can say, "Well that's fine. I'm
with you if that's what you want to do. But didn't we
hear so-and-so said they're not going to use this? Do
we want to maybe try a first version here, maybe little
bit less accurate, but we know this person's going to
use it because we can prove to them how the model
came up with these recommendations. Maybe that's
where we should go first and then we can see if we
should make it more accurate." You can try to have
these conversations and I know that's tough.
Brian T. O’Neill: Sometimes it's hard to have these conversations when
you're early in your career, but I would challenge your
listeners, you're not really there to write code. What
really a line of business wants is they want the value
that your code produces, but it's not really the code
and the modeling and all the stuff that you learned in
school, that's not really what they want. It's the
output. It's the outcome from your outputs.
Brian T. O’Neill: So if you always have that kind of lens in your mind
and you can connect it to the people who are going to
consume those outputs, you're probably going to be
more successful in your career in general.
Kirill Eremenko: Wow. That's golden. I think if people follow that advice,
they'll be twice as successful already.
Brian T. O’Neill: Yeah.
Kirill Eremenko: Indeed.
Brian T. O’Neill: Sure.
Kirill Eremenko: It's the output. As you said, the outcome of the output.
Brian T. O’Neill: Yeah.
Kirill Eremenko: That matters.
Brian T. O’Neill: Can I give you an example of this? Like real quick.
Kirill Eremenko: Sure.
Brian T. O’Neill: This is my learning moment for your listeners. When I
was working, I worked at Lycos. If you remember,
Yahoo was big. 20 years ago, Yahoo was the big search
engine before Google and Lycos and AltaVista were
competitors. And I worked at Lycos and I was a
designer and we each belonged to different verticals
and I focused a lot on the financial services products
and some online trading platforms and things like this.
Brian T. O’Neill: I remember one day when I was designing one of the
stock research pages for the Lycos Finance or
whatever it was, and I was talking to the product
manager about the ad placements here. And for years,
and most designers, they hate the ads. If you're
working in a media company where advertising is the
model, you have to find these slots to put banner ads
on the page and you hate it and all this kind of stuff.
Brian T. O’Neill: And it didn't click until this moment, this conversation
with him, that, wait a second, these ads are what fund
my salary here to be a designer and to do the work
that I love to do. So what if I change my perspective to,
look, no one really likes looking at ads. We know the
customers, they hate that, but at some point this
funds the business. So what if I can use my creative
energy to figure out how to fit adds into the experience
in a way that's not so annoying, maybe it's a little bit
smoother or maybe there's an add at a surprising
place where it actually has some interesting context for
the customer. Something like that.
Brian T. O’Neill: I looked at it more like I actually want to help my team
produce more advertising revenue because that funds
my salary, but that's really what they're asking for.
And it was just this light that kind of went on for me.
And so I stopped fighting it and I realized it's never
going to go away. This is a media company. At the time
they were looking at subscription businesses and
other models, but at the time it was a media company,
which meant advertising.
Brian T. O’Neill: And so I actually started coming up with some other
advertising products that we could actually go out and
sell, that kind of stayed out of the way of the UX,
which was my job, is to make really great user
experiences with these interfaces. But also, I've started
to think about what would be some other ways we
could sell creative advertising, because that's really
what it was about.
Brian T. O’Neill: But I was fighting it constantly. And part of that, you
want that ying and yang, right? You kind of want some
of that in the business, which is you have kind of our
purist designers. Designers can relate to this, they're
always going to go for simplicity and usability. And
sometimes you may want to encourage people to opt
into a form, right? To provide some more data. Well,
maybe there's a creative way to collect that data that is
both transparent and ethical, but perhaps it's fun.
Maybe you turn it into a game instead of asking
someone to fill out a survey. We put that into a game
context, but we realize that we actually do need to
collect this information. How can we do that in the
interest of the business and the customer?
Brian T. O’Neill: It's not just about the user experience piece. It's about
the business value that we're creating, too. That was
kind of the moment when the light went on for me in
the advertising context. But I'm sure your listeners can
probably find a way that they can start to see, "Wow,
we're going to help salespeople know who to call
instead of just opening the CRM and smile and dial."
Right? What if we could tell them, "Here are the next
20 people, based on all the data we have, we think
these 20 people are most likely to sign on the dotted
line within the next two months.
Brian T. O’Neill: Put yourself in their shoes. What is it like to be a
salesperson? And when you start to realize that's
really what the business hired you and your team for,
is to help these sales people know who to call so they
spend less time calling the wrong people. That's what
you're there for. Not Python. Not R.
Kirill Eremenko: Gotcha. Yeah. Wow, okay. So in a nutshell, keep in
mind what you're there for, in terms of you're there for
the outputs and the outcomes that come from your
outputs, not for your inputs. And also, empathy. I love
that you mentioned that, really understanding and
sitting down. This really helped me many times. Sitting
down with the people that I'm creating a model for, or
creating some analytics, or doing analytics for. Sitting
down with them, even just living with them through
their whole working day. Understanding what they
experienced, what they feel throughout the day, really
helps inform what I need to do.
Brian T. O’Neill: One other comment on this is, I'm going to totally cast
a generalization, there's lots of different people out
there, but I'm going to say generally speaking, people
with STEM backgrounds tend to be a little bit more
introverted. They may find some of these kinds of
research discussions a little bit uncomfortable. And
here's the great thing about doing good research: your
primary job is to listen, it's not to talk.
Brian T. O’Neill: So if you're not comfortable doing this, really your job
is to come up with some good questions, we call them
open ended questions, which means questions that
generally don't start with the word "do," because we
don't want questions to end with "yes" or "no." We
want to ask, "Tell me about X. Tell me about how you
decide who to call when you're on the hook for your
sales numbers. How do you decide who should get
what offer?"
Brian T. O’Neill: Ask open-ended questions, here, and just listen. And
this is a way to kind of get get comfortable with this
process where you don't feel the need to talk like I am
right now. I'm babbling, but it's really about listening,
is really what it's about.
Kirill Eremenko: Gotcha. Totally agreed. Brian, you have quite a few
things that you're doing at the same time. Of course in
addition to your music, but at the same time you do
consulting for companies, you run seminars, you are
about to launch a course, which is very exciting.
You've got a podcast of your own. First of all, I want
everybody to know that Brian's podcast sounds
amazing. It's called Experiencing Data With Brian T.
O'Neill. Check it out on iTunes. Congrats, Brian.
You've done what, like a year now? Of podcast.
Brian T. O’Neill: Yeah, right. Episode 34, I think, comes out tomorrow
actually, based on what we're recording this now. So
it's been good. Yeah. Every two weeks we drop.
Kirill Eremenko: Fantastic, really cool. So the episode you were talking
about earlier is Episode 24, How Empathy Can Reveal
a 60% Accurate Data Science Solution. So everybody,
you're on this podcast, you're listening to this, this
means you love podcasts already. Check out
Experiencing Data With Brian T. O'Neill. I think you're
going to love it. And Brian, I wanted to ask you, what
do you teach? You've already shared quite a lot of
things on the podcast today. What is it that you teach,
any additional insights you can provide from the
seminars that you run? What are the discussions
around there? I'm just curious what other themes
exist in this space of using human-centric design and
data science?
Brian T. O’Neill: Are you asking what's in a seminar or a course? Is
that what you're asking?
Kirill Eremenko: Yeah, typically. What's the news online [crosstalk
00:00:54:30]?
Brian T. O’Neill: Sure, sure. I'll give you an idea of the self-guided video
course, which I just put up. The curriculum, like that
process we talked about, those six or seven steps that
were there, that's a video course. So what it is, is for
each module of those six or seven, there's I think
seven, there's a short video where I kind of talk about
the key concepts in there. And then there's a written
module that goes with that. And it's really focused on
doing the work. It's not a read a book and you kind of
digest 10% of it and 90% goes out the door.
Brian T. O’Neill: Most adults learn by doing. And so what I really tried
to do with this course is provide actionable steps for
each module. What do I literally go out and do if I'm
doing this work in my own organization? What do I go
and do to put this into action?
Brian T. O’Neill: When you talk about coming up with your team, well
what does that mean? Literally? So that's what I have.
The video is kind of an overview for the module and
then there's kind of step-by-step activities there. And
then I link to examples when it's relevant. I try to
provide some examples there. And one of the ways it's
different is that the course is called Designing Human-
Centric Data Products. It's loosely based on, you've
probably heard of the term "design thinking" before,
but what I felt was missing was this lens on data
products.
Brian T. O’Neill: So each module specifically talks about what is
different in the context of data products. When I'm
working with AI or probabilistic types of software
applications, what are some of the considerations that
are different? This is still a very new space. But
generally speaking, I would say right now, a good 70%
of the process is the same and 70% of the meat-and-
potatoes of doing good design work is the same.
Brian T. O’Neill: Whether or not it's a machine-learning model or just
descriptive analytics or some other technique, most of
it's the same, but there are other considerations we
need to add on when we're talking about probabilistic
models. And so each module has a specific call out
about what are the considerations here if I'm building
a predictive model or something like that. So that's the
course.
Brian T. O’Neill: And then there's an instructor-led online seminar
version, which is the same modules. The only
difference is I release two modules per week, and then
we have a call together with a cohort of people that are
in Slack. So this is the doing it with other people,
which partially helps keep you on track and makes
sure that you actually go and do the work.
Brian T. O’Neill: And some people like to work alone, other people want
to kind of have a cohort of people to go through it with
and hopefully they can learn from each other, and so
we have live Q&As. On Mondays, we release the new
modules and have a discussion about those, and then
Fridays we do a check-in and it's actually spread out
over four weeks. It's not four weeks of 40 hours a
week. It's very much designed on it's you get in what
you put ...
Brian T. O’Neill: What you put into it is what you get out of it, and the
goal here is to give you time to actually go and do
some of this work, and some of it takes time. It takes
time to set up, like I want to go do a ride along
interview with my salesperson. I don't know how they
do they work, but I understand that I need to go
understand a day in the life of the salesperson before I
do this. Well, it could take a few days to set that up,
and so I intentionally spread that seminar out over
four weeks, so that there's time for people to put this
into play and then get feedback from me on it, so
that's the seminar and that's how the training works.
Kirill Eremenko: That's very cool. Can you give us an example of, like
you said, you provide hands on exercises for people to
do on their jobs already? Maybe can you give us an
example of a simple hands on exercise somebody can
go and do at their work already tomorrow, to actually
experience that feeling?
Brian T. O’Neill: Well, in terms of actually literally spelling out how to
do it, I don't know if I could shortly do that on the
podcast, but if you're talking about what are some of
the types of activities, like this journey mapping and
service blueprinting we talked about, that's one of the
things we talked about, so who's involved? Who do I
need to bring to a session to do that? How do I set it
up? In this case I actually provide a graph, a visual
template that you can use here to get going with this,
but we talk about literally setting up the room and
who needs to be involved here. What is the goal of
doing a journey map and what do you do with the
thing? Okay, I have the map. Now what?
Brian T. O’Neill: That's what we go through in the course, is literally
what is this for? What is the value of it? What do I do
with it in order to move forward, and when do I do
this? When is it important for me to use this particular
tool? Because again, you may not need this tool or it
may be too late for this at the stage of the product that
you're in, so that's another aspect of this, is you don't
have to do everything and every single module all the
time on every project. What I want to do is give you
seven kind of core areas, and this is not all of design.
This is it's just like algorithms, or different modeling
methods.
Brian T. O’Neill: You don't necessarily use all of them all the time on
every project, but I wanted to give people seven kind of
core areas that they can go deep on. Six months later
you may say, "Oh, wow! I remember we did something
with usability studies and we have a lot of screens to
show people. I'm going to go dig out that module on
testing that Brian had, and then I can use that on this
project, because we're doing lots of visuals," or
something like that, so ...
Kirill Eremenko: Okay. Got you. What I love about online teaching is
success stories, where people have applied what I
wanted to convey, and they've gotten a job or a
promotion or some kind of success. Any cool success
story you can share about somebody who wasn't using
design thinking, and then through listening to you on
the podcast, or taking your course, or somehow
interacting with your work they decided, "I'm going to
try design thinking," and that completely changed
their career? Anything inspiring like that, that you can
share?
Brian T. O’Neill: I can't. For the seminar and the course I can't,
because the course is just about to come out, and the
seminar just is actually really new and we just, I'm
actually hoping to put some testimonials up soon. I'm
in the process of actually gathering some feedback
from my first cohort of students that went through
this, so hopefully you'll be able to see some of those,
the results from the seminar and the course on my
website in the near future.
Brian T. O’Neill: So I can't specifically say there's that, but one of the
things I got from one of the people in the course, and
so this person was actually, she's at an AI consulting
firm and she's technically an account and project
manager, but what she realized was a lot of the
challenges they were having, they were working with a
pharma company, was in this kind of user experience
space, because the client kept saying, "Well, just build
some stuff and we'll figure out whether it's good later,"
and she could smell that, "Well this could be really
risky for us, because they're happy with the work as
long as we're doing stuff, as long as we're building
pipelines and showing data," but they couldn't give a
clear expression of how is this information going to be
used to make decisions?
Brian T. O’Neill: And she wasn't sure. She could see the team was
struggling with this and they had had some difficult
conversations with their client, and so by taking the
course she said, "I feel a lot more armed now about
what tool do I use in this toolbox based on what the
current client situation is that we're having? And now I
feel a lot more armed. When it's time to test this, I
actually know how to test the results of our
visualization or the application that we built. I know
how to go do that now with them, so that they can see
whether or not the work we did was useful or not, and
then we can learn from that," and before it was very
much is the client say they're happy or not?
Brian T. O’Neill: Well, the client may look at something and say, "I'm
happy. That looks really nice," but if you didn't
actually talk to them about, well, does it work well? If
no one can measure how it's supposed to work, then
you might be just happy with the way it looks on the
surface, but underneath the covers it's not actually
producing that value. And so now she knows how to go
and have that conversation with the client and use
that toolkit, so that was one of the big things for me,
was that she had gotten that out of it.
Brian T. O’Neill: Another student was actually managing director at a
big supply chain finance kind of related company, and
he's trying to figure out, "How do I bring some of our IP
and our analysts' work that we do on every single
project? We have some IP here. We want to productize
this into an application, so that we're not spending as
much time doing the same types of manual tooling
work here," and he just didn't have a framework for
how do you get from ad hoc presentations of work to a
software application that will express this information
on a routine way? He was really struggling with that,
and so he feels that now he has a much better idea of
what that process looks like to get to that UI, that will
help him spend more time doing higher value work for
his clients.
Kirill Eremenko: Amazing. That's really, really cool examples of this
stuff in action, and from quite some senior people as
well, so you should give out certificates for your
trainings, because this would be so valuable. It is
already so valuable, but imagine somebody, I can just
see somebody coming to an interview and saying,
"Okay, this is a data science interview. What else do
you know?"
Kirill Eremenko: Python R, and normally people are like, "I know SQL, I
know this. I know Tableau. I know Kubernetes. I know
TensorFlow 2.0," whatever else, and that's all great,
but imagine saying two, three, four, five, whatever
tools, technical tools, and then in addition saying,
"Plus I did a whole training on design centered
thinking in the space of analytics, data science and AI,
and this is what I know. This is the framework I apply.
This is my awareness of the situation. This is how
confident I am about dealing with internal and
external stakeholders," and boom!
Kirill Eremenko: You just blow them away. Nobody ever says that at
interviews, if you're looking for a job or if you're
looking for getting better at your company, getting
promotion or growing your existing business. It's your
annual review, whatever it is, or talking to your
manager at your next one to one you. You start
mentioning these things, discussing these things.
You'll be the first person in the whole AI team talking
about design centered thinking.
Kirill Eremenko: I think this is a great addition to anybody's career,
very exciting. I'm glad, very, very cool that you decided
to go from music into this, and also, in addition, you're
finding time to teach this and spread this knowledge. I
think that's very, very, very cool. And also, plus this
podcast, some interesting guests that you're
interviewing. Hats off to you for the amazing
contributions you're making to this-
Brian T. O’Neill: Great. I appreciate the work. As for the certificate
thing, someone asked me about this and I told her,
"Look, this is the real world. It's not school, and
school's about getting a grade," and if you look at why
is school the way it is? School was designed as, from
what I understand, listening to a podcast on this,
school was designed to optimize factory workers
basically, right? You process them, you train them. If
they don't pass, you send them back, until they've
learned the skill, and you move them up the ladder
and you use quantitative testing to figure out whether
or not they pass or not.
Brian T. O’Neill: That is not the world of business. That's not what
we're there for. It's not to say I know more Python
functions by memory than you do. That's easy stuff to
measure, right? The biggest reward I think you could
get out of taking my course or seminar and the thing
that would be the biggest thanks for me is when your
profile, when your résumé starts to talk about the
results of the work that you have created with your
team, right? When it says I helped the business save
$2 million a month by building a model that did X,
that is going to make you stand out at your next job.
When you say, "I know R Python, Kubernetes. I've
certified Microsoft Cloud, whatever, blah-blah-blah."
Brian T. O’Neill: Well, guess what? You look like all the other people
that are now coming into this field, except your
number, I have seven years of this instead of six. Well,
if you want to command a higher salary, when you can
say, "Look, yeah, I don't know Python as well as this
other guy that you do, but did they help create a $2
million savings with their data science work? Do they
know how to go talk to a stakeholder who says, 'Give
me AI,' and they have no idea what they want? I
actually can go in there and help you figure out what
do they mean when they say they want AI, what
should we really spend our precious data science
dollars on?"
Brian T. O’Neill: When you could have that conversation, and I think
my course will help people learn how to do that, that's
a good thank you to me and that's going to speak way
louder than any certificate with my logo on it. I
appreciate the gesture there and I know where you're
going with it, but that's really what's going to make a
bigger difference for you as a data science practitioner,
is to be able to really show the results and the
outcomes that you have helped produce, and it's hard
to do this. Sometimes it's hard to measure it and really
track it back to exactly your work. I get it, but at least
have that in mind with the work you're doing and
you'll probably see you're going to have a great career
ahead of you I think.
Kirill Eremenko: Fantastic, Brian. Well, that's a great note to end this
episode on. I think everybody's gotten what they
needed out of this and much, much more. Before I let
you go, what's the best place to find you? Where can
our listeners contact you or get in touch, or just follow
your career and follow the things that you share?
Brian T. O’Neill: Sure. Yeah. I'd say if you go to my website,
designingforanalytics.com, that's probably the best
place. I have a mailing list, so if you want to just keep
track of what I'm doing, I do send out little insight
articles every week and updates on the podcast, so
each time we release an episode that gets sent out to
the list. I'm also pretty active on LinkedIn. Twitter, my
handle is rhythm spice, R-H-Y-T-H-M spice.
Brian T. O’Neill: I'm not super active there, so I'd say hop on the list if
you're interested in following my work, and usually we
have little, I offer little deals sometimes, especially
when I'm putting out a new offering, a new training
thing or something like that. I usually, in the spirit of
doing MVPs, right? I may be contacting you to do a
little research and like, "Is this a useful service?" And
then offering coupons and discounts and things like
that to my subscribers, so yeah, that's probably the
best place.
Kirill Eremenko: Fantastic. Thank you, so once again, the website is
designingforanalytics, all one word, .com. Definitely
check it out, and the podcast is called Experiencing
Data with Brian T. O'Neill. Brian, is LinkedIn a good
place to, for people to connect?
Brian T. O’Neill: Yes, that's a great place to connect as well.
Kirill Eremenko: Awesome. Great, and connect, make sure to connect
with Brian on LinkedIn. Okay. Great, and one more
question I have for you today is what's a book you can
recommend to our listeners? I'm sure you have
something special prepared.
Brian T. O’Neill: Yes. Well, there's two books. I would say they're both a
little bit more on the business side, but are you
familiar with Karim Lakhani, at-
Kirill Eremenko: No.
Brian T. O’Neill: He's at Harvard Business School and he just wrote a
text called Competing in the Age of AI, so I actually
went to the little, the book release. I live close to
Harvard here and I'm really just about 20% of the way
through that, but if you want to look at, if you want to
start to understand how is AI really changing the
business landscape? And maybe you can start to feel
like, "Oh, I can see how I fit into this," I think that's a
good text to start to look at a high level, how your
business stakeholders are looking at it. It's not a
technical book whatsoever.
Brian T. O’Neill: The other text that I'm in the middle of reading that
I'm enjoying is called Infonomics. I don't know if you
know Doug Laney, but this is a Gartner book and it
really talks about how to monetize, manage and
measure information, so it's looking at data as an
asset instead of this kind of like sawdust, right? It's
not sawdust. That dust has a lot of value, and so what
do we do with it though, right? How do we create
products with it? How do we improve products and
services using data? And so Doug's got a book there
and I'm really interested in finishing that, so those are
the two things that I'm reading right now.
Kirill Eremenko: Fantastic. Thanks. Thanks for your recommendation,
so Competing in the Age of AI and Infonomics.
Brian T. O’Neill: Yes.
Kirill Eremenko: On that note, Brian, thanks so much for coming on
the show. Really enjoyed our chat and I learned a ton
from here, and I'm sure our listeners will pick up great
things from here as well. Thank you so much.
Brian T. O’Neill: Awesome. Yeah, it's been a pleasure to chat with you.
Kirill Eremenko: So there you have it, everybody. That was Brian T.
O'Neill and human-centered design thinking for
enabling decision making in data science. How exciting
was that? Lots of valuable insights, which you can
already apply in your career already. Now, what was
my favorite part?
Kirill Eremenko: My favorite part was the concept of thinking about
your end user throughout the whole process, so
something that I've talked about before quite a lot is
that the most in demand data scientists are the ones
that can connect insights to end users, so that's the
last stage of the data science project lifecycle, the
presentation, the visualization, the presentation,
communication of insight, but Brian takes it a step
further.
Kirill Eremenko: He says that you need to be thinking about your end
user not in the very end of your project, which is
important, which indeed already sets your part, but
he's saying think about your user throughout your
whole project. From the moment you ask the
questions, to then preparing your data, to then
building your model, to then visualizing and
presenting it, the whole five steps you need to be
thinking about your user. That's what human-
centered design thinking is all about in data science. It
doesn't matter if you're creating a data science
product, or you're building a model, or just delivering
an insight, or a decision support application, whatever
it is, think about the end user.
Kirill Eremenko: I think it's a skill. It's a skill, it's an art, something
that needs to be learned and practiced, and hopefully
now after this podcast everybody will be a little bit
more inspired to practice it. And as mentioned
throughout the podcast, you can find Brian at
designingforanalytics.com, so if you're a business and
you want to engage Brian to look at your analytics
products, then head on over to
designingforanalytics.com/superdatascience. That's a
way to get in touch with him, and we don't have any
affiliate arrangement with him. That's just a nice link
that he set up for our listeners.
Kirill Eremenko: On the other hand, if you are an individual contributor
in the space of data science, if you're a user or if you're
a data scientist basically, the best way, the best things
that will benefit you from his suite of products or
things that you can find on his website are his
podcast, which is called Experiencing Data with Brian
T. O'Neill, and since you're listening to this podcast
you already like podcasts, check that out. It's
Experiencing Data with Brian T. O'Neill. Then his
seminar, which is an online seminar, and his course,
which he just published recently or is publishing in
the coming days. Check that out as well, so maybe you
want to learn more about design thinking in data
science, so there we go. That's where you can find
Brian T. O'Neill.
Kirill Eremenko: Of course, you can connect with him on LinkedIn as
well, and all of these links plus all the materials that
we mentioned throughout this podcast will be
available as usual at superdatascience.com/353.
That's superdatascience.com/353, and another
exciting piece of news is Brian T. O'Neill's coming to
DataScienceGO, so if you haven't booked your tickets
yet, this is the DataScienceGO US version, United
States, in October 2020.
Kirill Eremenko: Brian T. O'Neill will be presenting there. We're getting
him to come all the way from Boston, so if you haven't
gotten your tickets yet head on over to
datasciencego.com, get your tickets today, lock them
in and we will see you there. You'll see Brian T. O'Neill
and lots of other exciting speakers, so there we go.
That's the end of today's podcast. Thank you so much
for being here today and I look forward to seeing you
back here next time, and until then, happy analyzing.