AVINASH KAUSHIK: I’ve a short 30 minute presentation– I think 10 slides And what I want to do is cover this very, very difficult topic of creating a data driven culture, because all the data and technology coolness is great But this is very, very hard to pull off It’s really hard to fundamentally change the culture in your organization, especially organizations that are big It actually gets much harder, the bigger they are In fact, I find that it’s not unusual for me to go and give a talk and show many of the testing experiments and for the most part lots and lots of tests and experiments are actually being done by small companies Actually the bigger the company, the dumber it is in terms of it getting to a cutting edge It really is I think it’s something about the culture, and something about the pace at which they can move in the bureaucracy and the agility that gets lost when you get big So it’s sort of a interesting little problem This topic is also in the book So you can read more, because I will run through this quickly So you all have a book, which is great It’s very heavy It took me a long time to write My publisher Wiley– they just called me in September or October and said– after six months, I had started blogging– and they said we should make this into a book And I said who the hell would want to pay for a thing that’s already free? It turns out a lot But the core reason that I decided to write the book is that all of the proceeds that I make from the book actually go to two charities My wife and I decided that everything we make from the book will go two charities, and the first one is Doctors Without Borders It’s a Swiss charity It does some really amazing work around the world They’re a charity that’s known to be first in every sort of troubled spot in the world, and they won the Nobel Peace Prize in 2003 It’s just an amazing charity We love it And the other half of my proceeds go to the Smile Train, which is a charity that does cleft palate surgery in the third world It’s a pretty terrible condition, emotionally, not physically, emotionally, for a child to go through And it only costs $50 to actually do the surgery And so we support the Smile Train So thanks to Brett and Google, you all get a book My unbiased opinion is this a awesome book But it also raises money for charity, which is something that’s great about a book Lots of things you’ve heard today– hopefully you get lots of ideas, and you’re excited to go back and do things And often people want to– because they are really easy I’m just going to go, and I’m just going to do it And a bunch of them are And lots of things that I especially talked about this morning, with the GA talk is how easy it is to do these things But often, we have to have an inspirational call there But often, things are actually not quite straightforward Life is tough And I’m a huge fan of Guy Kawasaki You’ll see me thanking him in the preface of the book And this is a quote that I take from Guy Kawasaki It’s true If you want a roast duck, you have to work for it Nothing is easy in life And I think this quote captures it very well So I’m going to go though five things that I have come to learn during my experience that help create a data-driven culture Because it’s hard to get it right, but you can And the very first lesson that I have learned is always focus on the outcomes, especially at the outset And I like to say this story that when I first got to my former employer, they were using Webtrends And they were publishing 300 Webtrends reports So I walked around I did not know anything about web analytics It’s embarrassing Just a few years ago, I knew nothing And so, I met everybody for two weeks, started to figure out what this web thing is, and this was just four years ago It’s more embarrassing And then at the end of my second week, I simply turned off Webtrends I simply switched it off

And no one in this $2,000,000,000 company actually called me, not a single human being called me And it was not that Webtrends is a bad tool Webtrends is actually a pretty good tool It’s that all of this massive data puking did not focus on outcomes Because at the end of the day, all of us are motivated by outcomes– either outcomes for yourself, or outcomes for the company It’s really, really important that if you want to get start or end up with creating a culture that thrives on data, you have to connect to people at a very, very deep level And there is nothing deeper than focusing on figuring out how I, as the person, director of analytics, can help you make more money by getting your bigger bonus People respond to you helping them improve their salaries No amount of visitor trends help John improve his salary The amount of orders I get from jelly beans improves his salary And my job, as the director of data, is to figure out how I can help him sell more jelly beans So at the end of the year, he will get a higher bonus It seems kind of odd, but tons of reports that sit out there don’t focus on outcomes The very first thing we did is we said we’re going to hack together a data warehouse, quick data warehouse, and all it will report is on one number How much revenue did we make? And the very first time we put this number in front of the CEO, we shocked him and he hired a VP And the reason he hired a VP, because he was surprised at how much money we were making on the web And so he hired a VP for the web Because the web was that important to the company And remember, they had 300 Webtrends report available all of this time And yet they had no idea what the outcomes were So at the start, create a simple dumb graph that plots revenue Each color in this stacked bar represents a product I’m not going to tell you what, obviously But it helps them understand two simple things How much money do we make? What are we selling? Everybody responds to that Because it’s the bottom line So as you focus, as you go back, if you want to prioritize the actions you want to take, the first filter that you should put on is am I reporting on bottom line? Because people care about bottom line– theirs and the company’s Then you’ll be ready to go back and say– after a few months of doing this, they’re like oh! Why are we making so much money? Where are people coming from? They’re ready for this And then at some point they’ll say, ah, I know Google is great or whatever this is saying And then they’re ready to say what is happening on my website that is causing me to make money? It’s a gradual transition What happens in reality in many companies, is it’s easy to throw a tag on the website and you start going the reverse way It increases complexity, and there’s less skin in the game for people you’re working with Solve for outcomes, and you will create a culture that will demand data I’ll give you this– another great example The first time we did a test at Intuit, we built a platform and ran a test– actually for Quicken John’s from Quicken And we created a test for Quicken and it was a simple test. You know, three different [UNINTELLIGIBLE] images is as simple a test as you can get And what I said at that time is why don’t we bet on it? So we have this idea three, let’s everybody bet on which one is going to be the winner It’s only a dollar each Everybody had skin in the game The next day, in the earlier [UNINTELLIGIBLE] Tom, you get data The next day, everybody was checking in to see if they’re going to win the pool Because they had skin in the game And people would come up to me and say oh, what’s the definition of this conversion rate thing that you’re measuring? I’m not winning I think this is wrong Great, right? People got skin in the game And everybody was interested in knowing how multi-grade testing worked They all wanted to know what the winner is going to be And by the way, they all lost The techie guy won, because the interesting thing is when we said in order to win the pool, you have to predict which recipe would win and by how much And so they all predicted– it’s sort of easy to say which recipe might win, but then everybody said improvement in conversion rate will be 45%, 15%, 19%, 28% The improvement in conversion was 1% Important lesson learned, right? You do small things, you win small

But all of this happened because we focused on outcomes This is another very, very important thing, and the reason that I think a lot of companies fail– the smartest people in each company are probably sitting in this conference room, sadly But you are You’re the smartest people in the company, because you know data And you want to play with data, and you want to use data to make decisions In some sense, you already get it But yet I find that especially as you approach the HiPPO level, people have a deep distrust for data Here is the other facet of this problem You know every nuance of how the data is collected, processed, and presented You are the smart ass And when you present data up, remember the person on the other end of the table or the phone is not as smart as you are about data They obviously got there because they have other attributes They not that smart about data So what happens is a lot of times they’ll say oh, you’re saying this thing is bad about this recipe, or this [UNINTELLIGIBLE] is not working They think it’s your opinion It’s very, very, very important that if you want to create a culture where people use data, you have to bepersonalize decisio-making It cannot be your opinion about something, not even remotely If people think it is your opinion, and you’re not a very nice person, you’re screwed There are lots of simple ways to depersonalize decision making Every single report you put out, you should pass through this filter Does this report meet the criteria that this is not my opinion? This is data speaking So a simple way to do it is something that I had shown you many screenshots of this morning Give context We got a million people this month, and it is bad Oh, it is bad, because last month we had three And we have 3,000,000 versus 1,000,000 So do a simple graph that says oh, the blue is how we did last year, and the red is how we’re doing this year Or the red is the goal, and the green is how we’re doing This is not your opinion anymore Whether the performance is good or bad is not your opinion anymore Depersonalize it Or you can compare it to other benchmarks and indexes Like, for example, you cannot go to your CEO and say we suck at 1% conversion We can tell him shop.com just published a study that says that an average commerce retailer– top 300– converts at 2.3%, and our conversion is 1% The message that you suck is included But it is not your opinion Right? A lot of analysts make this mistake– a– because they are smart, they can do math, and they portray data as their own opinion and it’s not Another one is– I am a huge fan of doing surveying And so this is the American Customer Satisfaction Index You can go to the ACSI.org and you can actually get satisfaction scores for the last 10 odd years for any company you want So if you’re running a customer satisfaction survey, you can say to your CEO, our score is 53, and Marriott is 79 Or Marriott is only 79 We’re 96 You get a bonus right away obviously Because it’s not your opinion anymore And that’s the same reason that I like testing experimentation, because it’s not your opinion You will always lose against the HiPPO You have no chance of winning Or very little But it’s easy to say let’s put it up there– a, b, and c, and see which one wins In this case, that’s the control, that’s the one I like, it’s clean and it has a heart Angels are missing It’s clean, right? The interesting thing is the woman cooing soft nothing into the guy’s ear did not work This did not work either It’s the woman and what appears to be an androgynous person winning And if that was your idea, your HiPPO would have never accepted it But now it’s not your opinion It’s the customers voting In all examples, whether you create index against yourself, you compare against others, or you’re in testing and experimentation, all you’re doing is depersonalizing decision-making You have to take the people around the table out of the equation It’s not our opinion And then, it turns out people are more willing to listen People listen to customers Most reasonable HiPPOs listen to customers and then listen to the bottom line

They listened to comparisons to others in the industry But they don’t want to listen to you Not as much as you’d like And the other one, obviously, is if you want to actually create a data driven culture, you have to figure out how to empower your analysts I mentioned this morning that there is a huge difference between doing reporting and analysis Reporting means everybody gets the data and nobody takes any action But everybody gets data If you actually want to create a culture where you want to have a lot of data being used to drive decisions, you have to figure out how to empower analysts in your team You have to make sure that they have 20% time for reporting, 40%, 70% time for doing unstructured analysis If you don’t allow people to go explore and find insights, how are you actually going to create a culture that thrives on data? Reports are not going to tell you what to do by themselves And the other important thing is if you do have analysts, encourage a culture of risk taking I’m astounded that people hire an analyst for $100,000 a year, and then make him into a reporting monkey They will get somebody from Harvard and Yale, and then rather than letting this extremely brilliant person with their Ph.D. to do the kinds of analysis that he has learned to do, they will say take a, multiply it by b, and then divide it by c, and that is what you’ll do day and night And that’s not how you create a culture that thrives on data You take risks, and you do big things And the reason is this wonderful little graphic It is actually amazingly true When you touch data, you have absolutely no idea how big your problems are If you don’t take risks, if you don’t spend time in doing unstructured analysis, you’re actually not going to be able to figure out how big your problems are And the next one is this philosophy that I have. It’s called the Trinity Strategy It’s covered in the book in some detail, but it was developed after a little while of me thinking through all of the problems around the web, especially the web And at the center of the Trinity Strategy is what I had said this morning, is the desire to find actionable insights and metrics that drive change, not data And one facet of the Trinity Strategy is doing to click-stream analysis– what you would do with Google analytics, lots of quick-stream analysis– is understanding the patterns of people And that’s good But it’s important to realize that there are a lot of weird people in the world And they all have ADD And they are all using your website It’s astonishingly sad, but it is true So as you analyze trends and patterns in your click-stream data, it is not unusual for you to struggle to find a semblance of insight, because people do weird crap on your site So when you attack click-stream analysis, the important thing to remember is you’re inferring intent And hence the second element of the Trinity is really important to do outcomes analysis If you have a support website, figure out how to measure problem resolution rates If you have an e-commerce website, figure how to measure revenue If you have a lead generation website, figure out how to measure conversion But the second element of the Trinity is to analyze outcomes But the thing that I have learned that creates data driven culture is this number three, which is experience analysis It’s doing testing, experimentation, lab usability studies, surveys– all of these wonderful things that help you getting into the head of all this collective people who had ADD It’s me trying to get into your head as you’re clicking around on our site to figure out what the hell are you thinking? There are many simple and complex ways of doing it You could simply run a survey Make a survey and say hey, what do you think? You could do tests That’s a way of getting into a customer’s head and saying what is going to work for you? But another way is doing remote There’s this company in San Francisco called Ethnio– E-T-H-N-I-O. It’s a great company And the wonderful thing about Ethnio is you can buy their services, and then show a DHTML window on your site as people come, and you say would you like to make $50? Everyone does Who doesn’t want $50? And if you say yes, they fill out a quick short survey– say what’s your age group? What products do you use? How long have you used our site? Do you use our competitors’ site? Literally anything you want As soon as you meet the criteria–

you say fill out your information, say yes, it comes over to this live database, and I have your phone number to call you And I call you and I say hey, John, you wanted $50– no, I don’t say that– you wanted to participate in a study with us So you press this button, and boom I can see exactly your browser window I teach them to think aloud for two minutes, and say, hey, here’s how you think Whatever you’re doing, just tell us And then we sit back, relax, and let the person go Say do whatever you’re doing on our site And it will blow your mind when you see the person behaving on your site, telling you what they’re thinking, or what they’re looking for, and then struggling to actually solve their problem It gives you amazing insight into a person’s head as to what they’re thinking And often you want to cry, very often you want to cry, because the interesting phenomenon about human beings is that a lot of these people who participate in these experiments, often if they’re not able to complete their task– I want a prize, I want to order something, I want to give [UNINTELLIGIBLE] I want to know how much this product costs– they tend to blame themselves very quickly And they say oh I’m so sorry I’m such an idiot I can’t do this simple thing The reality is you are the idiot for making this human being feel bad You should have figured out how to get the price in front of them really quickly That’s doing experience analysis There is a lot that it takes to understand the experience of your customer, so that you can understand the behavior that they have on the site And the goal of the Trinity Strategy is to understand the experience so well using all this methodologies so they can influence their behavior which leads to the right outcomes That’s the goal As you go out and want to create a data driven culture in your companies, remember this is what you have to solve for And the reason is because at the end of the day, we’re not analyzing cookies and sessions and Javascript tags and other BS like that We’re actually trying to solve for people And it is very, very often that we all forget that we’re analyzing people Some of them are frightened by you– obviously, as this woman on the left Lot of people love you Lot of people are screwing with you And your goal, as you do all of the analysis, when you log in to Google Analytics tomorrow morning– which all of you will, yes?– you should try and figure out and remember that you’re solving for people And you’re trying to analyze people, and not cookies Because it’s so easy in our web world to just forget that you’re actually analyzing people The last one is a particular point of view of mine is that web analytics over the last few years has evolved tremendously Decision-making on the web has evolved tremendously We started with log files, and analog– which is one of the old log file parcels that are still around My personal point of view is that web analytics as a responsibility should sit with the business, and not the IT team It sounds crazy, but in a lot of companies, web analytics is still owned by IT teams. And that’s OK, you know? But the interesting thing I have found is that IT people have completely and utterly different requirements from data than do business people As the web has matured and become an integral part of any business, it’s no longer IT that is driving the website It is the business team and the market that are driving websites So my personal point of view is the person who should own web analytics in your company is the person whose neck is on the line if the site fails to deliver for the customer Sometimes it’s somebody in strategy and operations Often I find a lot of organizations meeting success by having web analytics owned by marketing, because marketing is responsible for creating the experience You can figure out what is the right outcome for your own company, but it is important to realize that it is probably an optimum outcome to have business own data and decision making from data Because of this one singular reason, you have to think, imagine, and move at the pace of business And the pace at which a business moves is radically different from a pace at which IT moves And it’s not that one is better than the other, but they both solving for different things Inside this complex graphic that you should not read– that I will do as a part of an hour presentation– but rather than having a lot of different silos on pieces of the puzzle from collecting data all the way to making decisions in your companies, because of all of the transformations with Javascript tags and ASB models and hosted vendors, a lot of this complexity is going out

of the equation now So you can stuff a lot of things to the vendor and create a responsibility in your company that is end-to-end and owned by the business It’s really, really important that if you want to create a data driven culture that you have a data team, an analytics team, that is owned by the business and reports to the business, because they will make sure that they kick them enough so that they’re doing the right thing That’s it

You Want To Have Your Favorite Car?

We have a big list of modern & classic cars in both used and new categories.