What we learned from 5 million books

Erez Lieberman Aiden: Everyone knows that a picture is worth a thousand words. But we at Harvard were wondering if this was really true. (Laughter) So we assembled a team of experts, spanning Harvard, MIT, The American Heritage Dictionary, The Encyclopedia Britannica and even our proud sponsors, the Google. And we cogitated about this for about four years. And we came to a startling conclusion. Ladies and gentlemen, a picture is not worth a thousand words. In fact, we found some pictures that are worth 500 billion words.
1:02
Jean-Baptiste Michel: So how did we get to this conclusion? So Erez and I were thinking about ways to get a big picture of human culture and human history: change over time. So many books actually have been written over the years. So we were thinking, well the best way to learn from them is to read all of these millions of books. Now of course, if there's a scale for how awesome that is, that has to rank extremely, extremely high. Now the problem is there's an X-axis for that, which is the practical axis. This is very, very low.
1:29
(Applause)
1:32
Now people tend to use an alternative approach, which is to take a few sources and read them very carefully. This is extremely practical, but not so awesome. What you really want to do is to get to the awesome yet practical part of this space. So it turns out there was a company across the river called Google who had started a digitization project a few years back that might just enable this approach. They have digitized millions of books. So what that means is, one could use computational methods to read all of the books in a click of a button. That's very practical and extremely awesome.
2:03
ELA: Let me tell you a little bit about where books come from. Since time immemorial, there have been authors. These authors have been striving to write books. And this became considerably easier with the development of the printing press some centuries ago. Since then, the authors have won on 129 million distinct occasions, publishing books. Now if those books are not lost to history, then they are somewhere in a library, and many of those books have been getting retrieved from the libraries and digitized by Google, which has scanned 15 million books to date.
2:33
Now when Google digitizes a book, they put it into a really nice format. Now we've got the data, plus we have metadata. We have information about things like where was it published, who was the author, when was it published. And what we do is go through all of those records and exclude everything that's not the highest quality data. What we're left with is a collection of five million books, 500 billion words, a string of characters a thousand times longer than the human genome -- a text which, when written out, would stretch from here to the Moon and back 10 times over -- a veritable shard of our cultural genome. Of course what we did when faced with such outrageous hyperbole ... (Laughter) was what any self-respecting researchers would have done. We took a page out of XKCD, and we said, "Stand back. We're going to try science."
3:32
(Laughter)
3:34
JM: Now of course, we were thinking, well let's just first put the data out there for people to do science to it. Now we're thinking, what data can we release? Well of course, you want to take the books and release the full text of these five million books. Now Google, and Jon Orwant in particular, told us a little equation that we should learn. So you have five million, that is, five million authors and five million plaintiffs is a massive lawsuit. So, although that would be really, really awesome, again, that's extremely, extremely impractical. (Laughter)
4:03
Now again, we kind of caved in, and we did the very practical approach, which was a bit less awesome. We said, well instead of releasing the full text, we're going to release statistics about the books. So take for instance "A gleam of happiness." It's four words; we call that a four-gram. We're going to tell you how many times a particular four-gram appeared in books in 1801, 1802, 1803, all the way up to 2008. That gives us a time series of how frequently this particular sentence was used over time. We do that for all the words and phrases that appear in those books, and that gives us a big table of two billion lines that tell us about the way culture has been changing.
4:34
ELA: So those two billion lines, we call them two billion n-grams. What do they tell us? Well the individual n-grams measure cultural trends. Let me give you an example. Let's suppose that I am thriving, then tomorrow I want to tell you about how well I did. And so I might say, "Yesterday, I throve." Alternatively, I could say, "Yesterday, I thrived." Well which one should I use? How to know?
4:59
As of about six months ago, the state of the art in this field is that you would, for instance, go up to the following psychologist with fabulous hair, and you'd say, "Steve, you're an expert on the irregular verbs. What should I do?" And he'd tell you, "Well most people say thrived, but some people say throve." And you also knew, more or less, that if you were to go back in time 200 years and ask the following statesman with equally fabulous hair, (Laughter) "Tom, what should I say?" He'd say, "Well, in my day, most people throve, but some thrived." So now what I'm just going to show you is raw data. Two rows from this table of two billion entries. What you're seeing is year by year frequency of "thrived" and "throve" over time. Now this is just two out of two billion rows. So the entire data set is a billion times more awesome than this slide.
5:59
(Laughter)
6:01
(Applause)
6:05
JM: Now there are many other pictures that are worth 500 billion words. For instance, this one. If you just take influenza, you will see peaks at the time where you knew big flu epidemics were killing people around the globe.
6:16
ELA: If you were not yet convinced, sea levels are rising, so is atmospheric CO2 and global temperature.
6:24
JM: You might also want to have a look at this particular n-gram, and that's to tell Nietzsche that God is not dead, although you might agree that he might need a better publicist.
6:33
(Laughter)
6:35
ELA: You can get at some pretty abstract concepts with this sort of thing. For instance, let me tell you the history of the year 1950. Pretty much for the vast majority of history, no one gave a damn about 1950. In 1700, in 1800, in 1900, no one cared. Through the 30s and 40s, no one cared. Suddenly, in the mid-40s, there started to be a buzz. People realized that 1950 was going to happen, and it could be big. (Laughter) But nothing got people interested in 1950 like the year 1950. (Laughter) People were walking around obsessed. They couldn't stop talking about all the things they did in 1950, all the things they were planning to do in 1950, all the dreams of what they wanted to accomplish in 1950. In fact, 1950 was so fascinating that for years thereafter, people just kept talking about all the amazing things that happened, in '51, '52, '53. Finally in 1954, someone woke up and realized that 1950 had gotten somewhat passé. (Laughter) And just like that, the bubble burst.
7:52
(Laughter)
7:54
And the story of 1950 is the story of every year that we have on record, with a little twist, because now we've got these nice charts. And because we have these nice charts, we can measure things. We can say, "Well how fast does the bubble burst?" And it turns out that we can measure that very precisely. Equations were derived, graphs were produced, and the net result is that we find that the bubble bursts faster and faster with each passing year. We are losing interest in the past more rapidly.
8:24
JM: Now a little piece of career advice. So for those of you who seek to be famous, we can learn from the 25 most famous political figures, authors, actors and so on. So if you want to become famous early on, you should be an actor, because then fame starts rising by the end of your 20s -- you're still young, it's really great. Now if you can wait a little bit, you should be an author, because then you rise to very great heights, like Mark Twain, for instance: extremely famous. But if you want to reach the very top, you should delay gratification and, of course, become a politician. So here you will become famous by the end of your 50s, and become very, very famous afterward. So scientists also tend to get famous when they're much older. Like for instance, biologists and physics tend to be almost as famous as actors. One mistake you should not do is become a mathematician. (Laughter) If you do that, you might think, "Oh great. I'm going to do my best work when I'm in my 20s." But guess what, nobody will really care.
9:14
(Laughter)
9:17
ELA: There are more sobering notes among the n-grams. For instance, here's the trajectory of Marc Chagall, an artist born in 1887. And this looks like the normal trajectory of a famous person. He gets more and more and more famous, except if you look in German. If you look in German, you see something completely bizarre, something you pretty much never see, which is he becomes extremely famous and then all of a sudden plummets, going through a nadir between 1933 and 1945, before rebounding afterward. And of course, what we're seeing is the fact Marc Chagall was a Jewish artist in Nazi Germany.
9:55
Now these signals are actually so strong that we don't need to know that someone was censored. We can actually figure it out using really basic signal processing. Here's a simple way to do it. Well, a reasonable expectation is that somebody's fame in a given period of time should be roughly the average of their fame before and their fame after. So that's sort of what we expect. And we compare that to the fame that we observe. And we just divide one by the other to produce something we call a suppression index. If the suppression index is very, very, very small, then you very well might be being suppressed. If it's very large, maybe you're benefiting from propaganda.
10:34
JM: Now you can actually look at the distribution of suppression indexes over whole populations. So for instance, here -- this suppression index is for 5,000 people picked in English books where there's no known suppression -- it would be like this, basically tightly centered on one. What you expect is basically what you observe. This is distribution as seen in Germany -- very different, it's shifted to the left. People talked about it twice less as it should have been. But much more importantly, the distribution is much wider. There are many people who end up on the far left on this distribution who are talked about 10 times fewer than they should have been. But then also many people on the far right who seem to benefit from propaganda. This picture is the hallmark of censorship in the book record.
11:11
ELA: So culturomics is what we call this method. It's kind of like genomics. Except genomics is a lens on biology through the window of the sequence of bases in the human genome. Culturomics is similar. It's the application of massive-scale data collection analysis to the study of human culture. Here, instead of through the lens of a genome, through the lens of digitized pieces of the historical record. The great thing about culturomics is that everyone can do it. Why can everyone do it? Everyone can do it because three guys, Jon Orwant, Matt Gray and Will Brockman over at Google, saw the prototype of the Ngram Viewer, and they said, "This is so fun. We have to make this available for people." So in two weeks flat -- the two weeks before our paper came out -- they coded up a version of the Ngram Viewer for the general public. And so you too can type in any word or phrase that you're interested in and see its n-gram immediately -- also browse examples of all the various books in which your n-gram appears.
12:06
JM: Now this was used over a million times on the first day, and this is really the best of all the queries. So people want to be their best, put their best foot forward. But it turns out in the 18th century, people didn't really care about that at all. They didn't want to be their best, they wanted to be their beft. So what happened is, of course, this is just a mistake. It's not that strove for mediocrity, it's just that the S used to be written differently, kind of like an F. Now of course, Google didn't pick this up at the time, so we reported this in the science article that we wrote. But it turns out this is just a reminder that, although this is a lot of fun, when you interpret these graphs, you have to be very careful, and you have to adopt the base standards in the sciences.
12:42
ELA: People have been using this for all kinds of fun purposes. (Laughter) Actually, we're not going to have to talk, we're just going to show you all the slides and remain silent. This person was interested in the history of frustration. There's various types of frustration. If you stub your toe, that's a one A "argh." If the planet Earth is annihilated by the Vogons to make room for an interstellar bypass, that's an eight A "aaaaaaaargh." This person studies all the "arghs," from one through eight A's. And it turns out that the less-frequent "arghs" are, of course, the ones that correspond to things that are more frustrating -- except, oddly, in the early 80s. We think that might have something to do with Reagan.
13:28
(Laughter)
13:30
JM: There are many usages of this data, but the bottom line is that the historical record is being digitized. Google has started to digitize 15 million books. That's 12 percent of all the books that have ever been published. It's a sizable chunk of human culture. There's much more in culture: there's manuscripts, there newspapers, there's things that are not text, like art and paintings. These all happen to be on our computers, on computers across the world. And when that happens, that will transform the way we have to understand our past, our present and human culture.
13:57
Thank you very much.
13:59
(Applause)

Ambition. Discipline. Purpose. The Journey of Being an Entrepreneur with Gurbaksh Chahal

0:01this program is presented by University of California television
0:04like what you learn visit our website or follow us on Facebook and Twitter
0:09to keep up with the latest you see TV programs
0:20the new the normal
0:31little
0:36cool
0:39cool
0:40cool
0:44good evening we've got an
0:46and exciting a guest speaker this evening someone who's
0:51set some world records in and being an entrepreneur and some other things he's
0:55done
0:55my name's Ron lesniak I'm a lecture in the CT 191
0:59entrepreneurs in class and I was asked to say a few words
1:03at the beginning in this talk by device in Sholl
1:06out his family came the United States from India
1:09when he was four years old a at 25 cents to the
1:13twenty-five dollars to the name and by the age 18 he had sold his first company
1:18click agents and Internet advertising company he created as a teenager
1:23I believe you started when he was 16 for forty
1:26million dollars since then he has acted as the youngest executive
1:31a multi billion dollar Corp corporate Nasdaq corporation
1:34and started and so the second company blue vitzium
1:38to Yahoo for a mere 300
1:41million dollars so this is a man who is
1:44a unbelievable are entrepreneur
1:47a believer in passion with action and and and a mover and a shaker
1:53unparalleled in young entrepreneurship he has been the cup
1:56discussed at the on the cusp every internet advertising trend
2:00for the past 10 years this is a man who pioneered
2:03the Internet and in in the advertising world um
2:07he starred in fox's secret millennium he has appeared on The Oprah Winfrey Show
2:12I'm he's with been interview by Bonnie Hunt New York Photo
2:16and profile the New York Times Entrepreneur Magazine
2:19in the San Francisco Chronicle his currently developing several other
2:23television shows
2:24and recently launched his third company G wallet
2:28in he lives in San Francisco I like you to introduce to
2:31under Varsha Jul let's give him a nice warm welcome
2:35thank you for the
2:43cilantro were I'll let me just put this in presentation
2:47good alright
2:55so how many you guys wanna be on for are
3:00good good so I guess this mostly this will apply to Cal University everything
3:05I guess is the some background on Conway amor talk about today
3:09yeah is a very different than what you're probably
3:12used to hearing in a classroom a most everything I
3:16I talk about whether it's in my book or even today I
3:19it really comes from the heart and it comes from experience that comes from
3:23very unorthodox way of doing things and
3:27it also comes from reality so everything that I've realizes that
3:31in the world have business a too many times we get absorbed
3:35especially in the classroom or on textbook and weak sometimes forget that
3:39businesses are are started with relationships
3:43and the desire to go and succeed comes from passion
3:46and a lot of things I was talking about is exactly those
3:50few things so before you go them most the time when people
3:54here my story they are purple disclaimer stay in school
3:58so I did drop out %uh high school when I was 16
4:02I to start my first company by that does not tell
4:06by anybody bat that's the way to go
4:09because in essence I think the perfect thing that school teaches us
4:12is it teaches us how to learn and all of us have a very different wavering
4:17and you can say that I kind of learned very differently
4:21a at sixteen reality I've
4:24the way to start a business and from myself and in the business world but
4:28that doesn't mean that you guys should be about the same I think that the
4:31platform you guys have
4:32is probably a gift that I didn't have in terms of this education
4:36as well as this this opportunity so stay in school
4:39couple other things entrepreneurship is in general
4:44is like a roller coaster it has its ups and downs but it's your choice to
4:47screamer enjoy the ride
4:49so another way of looking at that is that it takes a lot I got
4:53and it takes a lot of emotion to succeed in the business world
4:56I and you gonna have so many crazy days
5:00and you're gonna have probably one or two really good and then you have to
5:04basically realize that okay to the bad days that you have
5:07it's okay because you know what you gonna go ahead I'll learn
5:10and your adapt and your stomach yourself into surviving the next day
5:14and sometimes that's the worst thing because that is
5:17most the time when people give up and that's what you're not supposed to
5:21because if you're passionate about something you believe in something you
5:23want to do it
5:24I you have every opportunity to
5:27have more those good days to take me on the next annex ride
5:31so what are the things I also wanna share before we start is a
5:35go to reach you the regime on mantra actually my third companies call radium
5:39one
5:39was formally called you all that but this is actually something that we have
5:43hanging in our lobby
5:44and this is what I i spire everybody to actually
5:48think this way because I don't think that I hire employees even though we
5:51have to an employee's
5:52across different offices I do not think your arms employees I think them as
5:56entrepreneurs
5:57so I was read this to you as and I hope you guys get
6:01what I think a people can get out this we believe anything is possible
6:06we see opportunity one other see impossible we take risks we're focused
6:11we hustle we know that nothing's unrealistic
6:14we feel overwhelming love we embrace are childlike wonder and curiosity
6:19we take flying leap into the unknown we contribute to something bigger than
6:24ourselves
6:25we create we learn we grow we do
6:28we believe it's never too late to start living a dream
6:31we are entrepreneurs so
6:35couple other things I guess to start off just because it's very unconventional
6:38and I wanna go ahead and
6:39and let you know what you're not going first
6:42so the next big doing our idea
6:46right I get I get asked many times I just need an idea just hook me up just
6:50give me one great idea
6:51folks me on a and I'm gonna become that next Mark Zuckerberg right
6:55and wanton most people don't realize is that Mark Zuckerberg did increase of not
6:59working right
7:00I it was already there he made it better he made it
7:03in a way that actually could grow into a fifty billion dollar company today
7:06and that's what you have to remember that the next best thing
7:09is actually something that's already there but the better iteration of it
7:13the quickest way to sell a company for a lot to me
7:17so a lot of times people are in it for the wrong reasons
7:20I'll if you look at my resume you can quickly say that okay well you obviously
7:24know how to
7:25start a company in for a bit it's not so easy right
7:28think that's not the way the business world works is because you cannot create
7:31just a business for the sole purpose to sell it that's just not the way it is
7:35mean sometimes it works but let's call him the lottery
7:39but that's not what we want to do we actually want to work for it and
7:42the lottery is not something that we can predict what we can predict
7:46is when we can create value and that's what I want to talk about today
7:50how your investors for you next big idea right most the times people think that I
7:55have
7:55an amazing idea and all I need is money for it
7:59and there is another thing that just doesn't work because if you think the
8:02all the greatest companies for the last ten years
8:05we actually already started a business Andrew traction
8:08and then they got investors right a mere idea does not get you money
8:13it's the aspiration to actually have traction that shows investors as well as
8:17yourself they actually on to something
8:20to be successful all you need is to think of something different again going
8:24back to my first point
8:26uge you should not I'm a big proponent
8:29I've something that most PC's probably will disagree with most PC's
8:33always say what's so different about you what's actually gonna go ahead and make
8:36me want invest in you
8:38and what you doing different than someone else right
8:41and when you think about things that way what ends up happening is
8:45you try to create something that is unproven
8:48and when the odds are against you want things are unproven what are you really
8:51gonna do
8:52you're probably not gonna go ahead and be successful
8:55you know in that regard so it's very very key to go ahead and
8:59figure out one thing that's already out there that's proven so if you can look
9:02at my motto
9:03in my career in advertising advertising's a $35 billion dollar
9:06industry its
9:07it's been around people make money through ads you know all the stuff that
9:11we've done is to make
9:12advertising more intelligent right and it's already been proven
9:15you know there's different variations avi as as the internet moved on but
9:19it's something that was already established in out there
9:23how you will change the world with your business this is another common dilemma
9:26that ends up happening
9:28I think arch winners take far too much credit a
9:31one successes hit the Academy ovett and
9:35say okay started his business to change the world that's not the way it works
9:38ride I could tell you right now that some the biggest companies that today
9:42that have changed the world
9:43change the world because they grew into what they became
9:47they did not change the world because that's cool flatter p
9:50and philanthropy in the business world doesn't go hand in hand ago only happens
9:54after the fact so if you have an idea that want to change the world
9:58I E think about it as a later stage
10:01in the process not the beginning aspect other
10:06how to write a business plan I this is something I'm also very very
10:10negative on because I we ventured
10:13the economies of scale where things move so quickly
10:17and the business world has changed so much for a
10:20innovations going so fast so if you're gonna spend
10:23months and months on to write a business plan which is
10:27all about phiri I you're actually we think I'm
10:30so I know this is probably not the textbook way
10:33love it because most textbooks teach you what a business plan is to give you
10:37practice
10:37on how to actually I formally things
10:41but the idea that is is that the number one thing that's and I keep you away
10:45from being successful is time and that is something that we do not have
10:48in our favor in this was a role especially as it becomes more
10:52competitive
10:53so move away from the business plan and focus on traction
10:58and how you live a stress leave life as an entrepreneur that is one thing
11:02that life just doesn't give you especially if you choose the path to
11:05being an entrepreneur
11:06so here's the three things that I hope you do learn today
11:10the definition of ambition disappoint
11:13and purpose as it is defined by you as an individual
11:18that is by the most important because if you can figure those three things out
11:21you will become a very successful entrepreneur
11:25fears your worst enemy risk is your best friend
11:30that's what you have to basically realize is that the only thing that I
11:33stop you
11:34is the fear a failure the only thing that's actually gonna help you is when
11:38you actually decide to take that risk
11:40they go hand in hand and the last thing
11:43so from a little tired it's because a number three insomnia will become your
11:47best friend
11:48so keep that in mind
11:51so it's a let's talk about this as well I shared this
11:55slide with a during my last sale: summit add cream 1
11:59and I said this to my sales force I said that
12:03the hierarchy of success whether your salesperson or their product person
12:07whether an engineer
12:08whatever you're in the company's by defined by these three
12:11very basic principles so if you have the stupid 3
12:15you will become successful the first one is obviously ambition
12:19and this is just the mere desire up wanting to win
12:23right if you have it you will go ahead and make that your fuel
12:27the second thing is disappoint too many people sometimes
12:31want to be successful but don't want to put in the work
12:34right sets the stage to if you have that ambition
12:37the second phase of the is this a fun because you gotta realize
12:40the sleepless nights there's things that you have to do that you never dreamed of
12:44but you have to do them in order to go and put in that works sweat equity in
12:47place
12:48and the third thing is making a difference and making a dent
12:52and that's come down to purpose what drives you what motivates you
12:56what actually wants to make you want to put the pieces together
12:59because if those pieces are together in the way that you envision it
13:03I and you have the right basis of cream that you can be successful
13:07otherwise if you're in it for the wrong reasons saying okay I wanna be the next
13:10billionaire
13:11I wanna be on Forbes magazine I want to do this on
13:14those are all the wrong things does all those happen
13:17by the by product 56 s nothing happened to begin
13:21and I know that some this may be too apparent to you but I'm just trying to
13:24go ahead and
13:25make it very clear because sometimes people are in it for the wrong reasons
13:27and
13:28and and you want to make sure whatever your delivering whatever you're making
13:32whatever you're treating
13:33has the right foundation morally ethically motivational a
13:37because all that composes your best outcome for actually wanting to one
13:43so and here's a quick diagram above what an arch winners brain looks like
13:48so you know obviously work ethic from the beginning
13:51you probably trust not that many people but in the middle you have this
13:55imagination you actually have the ability to create your own reality
13:59and you have very little patience right because that's one thing that time
14:03doesn't give you
14:05but your creative you have the charisma you have the backward selling accum and
14:10and you have the self-esteem to inspire only people
14:13so these are all the different characteristics you need
14:16in order to connect these dots so
14:20a couple other things just think about from business that you can learn from
14:23like what's happened in the last ten years
14:25right and one of my favorite quotes in business actually doesn't come from
14:29a business book actually comes with science right daren
14:32is not the strongest the species that survive nor the most intelligent that
14:37survives
14:37is the ones that are most adaptable to change
14:42right if you can just look at the last fifteen years
14:45love the Internet how much has it evolved and
14:48probably the last man standing that still be here bungalows Google
14:52but just think about the era we had the netscape we had
14:56up the Vista we had Yahoo then we have Google
15:00I'm sure we had a lot of stuff in between but the that's the evolution of
15:04it and this is the span of 15 years
15:06right even look at social networking right I mean look at how much has
15:09changed from Friendster
15:10myspace to Facebook too whatever else is out there so
15:14that is the difference right I mean you have to adapt to change
15:19so here's four examples of companies that we can all kinda look back at last
15:2215 years
15:23and say that where the I was of the ball
15:27first one
15:30since it's all about innovate or die here's for example zav
15:34where you don't wanna die Kodak
15:38the cease to be a very famous thing great this is a Kodak moment
15:41right you take a family picture above a block but
15:44at its peak it at 260,000 employees
15:49right 160 thousand it a lot and
15:52in 1997 at a market capitalization of $28 billion dollars
15:56so obviously at that was the peak love it if file for bankruptcy it's no longer
16:00out there
16:01right simply because very innovate par the digital revolution
16:05and here's something that has become like the vocabulary
16:08I've are a you know that defines a verb for us
16:12right is no longer even in business because the sore fact I've
16:15it in adapter its reality in change
16:20blockbuster 30,000 employees in its peak
16:23five billion dollar market cap less than 10 years ago right
16:26we actually should drive to watch movies and bring home
16:30baby obviously to innovate fast now for the digital
16:33revolution and now this acquired its assets for
16:37very little money but that's just another example
16:41blackberry anybody have lacked a reason the audience shame on you
16:45so
16:48you won't be soon enough but
16:51in 2008 June 2008 blackberry had a market capitalization
16:55$83 billion dollars like it owned a smartphone market
16:59right it was like you were the cool kid if your BlackBerry if you raise your
17:03hand not so cool right now
17:04by but the the drift love it is is that
17:08they thought they owned it right they thought that simply by having a keyboard
17:12attached to a so-and-so screen that was enough for them
17:15Apple came along ball in less than four years
17:19since this peak I Apple went from
17:22I guess 50 or 100 doing our bit to all the way a $600 billion dollar company
17:26right like a multiple the $83 billion dollars because they created something
17:31that didn't exist and they made it better and evolve
17:35and they pushed the market that way right and that's something that happened
17:39in a matter for years
17:40by that that's that's that's phenomenal and that is a speed
17:44love above the business world today one thing that I've even seen last 14 years
17:48my life is that
17:49I innovation is kinda like it grew and I think the Internet allowed it to go
17:54ahead make a lot of fast decisions and companies
17:56I drew fly by night and now it's actually
18:00become so intense even for the big boys to keep up
18:03because our star upstairs there's diff people that can be nimble enough
18:07and go after a problem and just own it in a matter
18:10short months for years myspace
18:14who had a myspace account back in the day yeah
18:18why heard the relaunching but the my wanna say that
18:22one they got acquired for $580 million dollars in July 2005
18:26they thought we all thought I was crazy and then just a couple years later
18:30I ya who attempted to buy it in a in a merger for $12 billion
18:35thank God that happen right but this guy tells you right there is that what
18:39people thought five inner ear million dollars for the largest social
18:42networking site in 2005 was a joke
18:44I it could have maintained right look at what Facebook is there a fifty billion
18:49dollar company
18:50right so this is just another great example in the last examples by the
18:54greatest lost in shareholder value in history a well
18:57I in they had this clever idea and December 99 they were gonna go ahead and
19:03create
19:03all media new media together comes out to under twenty two billion dollars a
19:07market cap
19:08that was in december ninety nine and what ended up happening Vienna for
19:11racing
19:12$220 billion dollars in a span of 13 years
19:16since it didn't work so what's the rule from here you better in a bit
19:20so what are the biggest things to learn here is
19:24the idea verses them right the idea versus action
19:28ideas or 1 percent of the journey right the
19:31actually just tell you what to do the action is actually what composes a
19:37above the result and most but most people again
19:40absorb themselves on I have the next big is cool is greatest amazing idea
19:45and that's where the absorbed 99 percent of the energy were
19:48what they really should be doing center if that's what I think I'm gonna do
19:52I wanna go ahead and actually plan out the roadmap to
19:55to create something out of it but then I'm gonna be open and I see where it
19:59takes me
19:59Google is a great example face because ik is a Greek
20:03example love multi-billion dollar companies again that started off as
20:06something that existed before
20:08even my company's i sorry click asians we were not
20:11be first performance based advertising network I had to have him
20:16we were not the first a behavioral advertising network
20:20and I even you all what I originally launched you all it I have the wrong
20:25idea
20:25than you think that third time entrenar with my success blah blah blah
20:30were actually know what I'm doing right and this is a clear example love me
20:34basically realizing that
20:36I don't know what I was doing by was quick to react to it I was quick enough
20:39to realize that this is the wrong
20:41area of focus on what to do and we pay for the entire company
20:44to redeem one which is based on social advertising in today
20:48it's bigger than you my prior to companies but that's the kinda DNA that
20:51you have to have
20:52I and realize that you can launch a product get the parity
20:56innovate and then start separating from the pack so
21:01the second lesson here is that you wanna be open to change take the blinders off
21:06look around don't be afraid to go on all sorts of unusual directions
21:10because that's where you might find a most unusual and surprising
21:13opportunities
21:15so a lotta people for stuff to work
21:18they get obsessed they think that it's passion that drives them but it's
21:21actually obsession
21:22there's two different things you can get obsessed with the product you can get
21:25obsessed with an idea
21:27because you get obsessed Europe or something to succeed
21:30verses if you actually just say okay this is one direction
21:34but I'm I keep myself entirely open to realizing where else I can go
21:38thats were the best result ends up happening
21:42valuation is built by scarcity valuation is based on what your value today
21:48and it is completely rebuilt around scarce it's not built around
21:52necessary how much profit you maker how much revenue make
21:56bet that a component but the real valuation of what drives you
22:00in the marketplace is a scarcity I you have the biggest deals
22:04are when companies are not sold one thereby right
22:07I companies that are sold in only being taken over companies that are being
22:11bought
22:12are actually bob because a motion and if you can actually ever saw your company
22:17by a motion
22:18it's based on a scarcity value that he built up based on someone that actually
22:22need something from me
22:23verses buying something right that's there's a very big do any Asian
22:27and every entrepreneur should start strive to build scarcity value from day
22:32one
22:34higher only rock stars this is actually a quote from my book that I think
22:38actually summarizes a a great way to look at
22:42leadership and and teamwork higher the smartest people you can find
22:46smart people make beautiful music together lots a smart people
22:50working Eunice I'm can have the power and you have a beethoven symphony
22:55you want a pic quality over quantity too many times people want to discard people
22:59to do
23:00jobs and I and I don't look at it that way I can look at every single person
23:05that
23:05I actually and the being part of my team they look at this as a career
23:10they don't look at this as a job you stamp out of 5 o'clock
23:13you not part of the team right doesn't mean that you have to devote
23:1724 hours every day to actually go ahead and work for a company but you gotta
23:21have that passion where you're not looking at is a
23:23as a in and out time clock it's one passion is driving you did make
23:27decisions and make results because your part is greater team
23:31and your party that connection that drives you to be successful
23:34the other thing I learned is again I had to learn this very very differently
23:38because I started 16
23:40I is not to be frugal Leidy can you imagine being a 16-year-old CEO and
23:44saying oh my god I want to pay someone that much
23:47right I've never even thought about that before and that's the kinda way I would
23:50I would think but I would actually say
23:52that person to make more money to me that's not fair I mean that's the kinda
23:55stuff obviously you you think about when you're
23:57adolescent now I don't look at it that way if somebody's worth five hundred
24:01thousand dollars I will pay them five thousand dollars because that means that
24:04there's greater value attributed their so all in all the net-net have that is
24:08that you ought to be frugal
24:10you wanna pay people you wanna give them the equity
24:13or give them the cash in for whatever they deserve because that's the end
24:17result
24:18you get what you pay for you get what you actually connect with
24:21and you want to be able to have the right people with you
24:27never key piece of vulnerable never make someone have control
24:31view that lets you not control your own destiny
24:35so when I was 16 I E
24:38had a business partner that I met on the internet
24:41surprise that actually was poor the person that
24:44I created the technology the beginning at serving technology for
24:49for quick asians and you know I'm indiana my technical so I i
24:53I had to have someone create this for me
24:57I and what I realize is that I just you know outsource
25:00this to someone and forgot about technology
25:04did my deals called calls and
25:07within six months at a business I was going two hundred thousand dollars a
25:10month
25:11and I feel great and then I realize that this person basically told me that
25:15if I did not give much 30 the company they would be if we shut me down
25:20and again I like I'm 16 II ended up saying
25:23try go for it let's see what happens and I was shut down
25:27and what I ended up learning there is a I actually
25:30could have lost everything I would not be here today if I did not make the
25:33right decisions from that point on
25:35and the decisions I have to make is basically realize that I can never
25:39let someone have full control and visibility
25:42have everything if I don't know how to program that's okay I just need to
25:45surround myself from people that actually do
25:47and her in her for the right reasons incentivize them that way
25:50hand up what ended up happening I
25:54found some people I found better people people that I could trust
25:58I got back up and running and that twelve months from that point ended up
26:02selling my country for 49
26:03so even what I thought to have been the biggest disaster had I not make the
26:07right decisions
26:09backer just completely killed my career right there but the most important
26:12lesson I learned there is Boehner bility
26:14and you never want to be vulnerable in that environment
26:18asking for help I don't expect
26:22help from others unless you're willing to do something or give something
26:25in return so I used to think that
26:28because I was Indian I would always look up to other Indian entrepreneurs and say
26:32oh well you know i mean the in their Indian they can help me out
26:34doesn't work that way and what I realize is that
26:38you have to change that dynamic I mean this is probably something that
26:42is as cold hard as it may sound but the reality that is if you want something
26:47I you know they're not gonna give it to you right you're only I get something
26:50out of it
26:51if you present something that as is actually value to
26:55right and I some that basic point I had to learn myself
26:58so I can change the conversation around to the conversation was known about how
27:02you can help me
27:03is about how was I help them but they were actually get me to do something for
27:07them
27:07or they're actually do something for me they're actually add value
27:10you have to change that dynamic otherwise you're never going to get
27:14people to actually do something for you
27:15its basic human nature but applied in the business world
27:19a whole lotta sense I no one help me a quick asians
27:22people only offer to help me a blue with him because
27:25my track record a quick Asian so there is that a
27:29there is a growing pain you have to do the first time around
27:33right as a entrenar to be the toughest thing that that you have to go to have
27:37almost
27:37on your stripes and you own your stripes by going through the painful process
27:41a doing a lot of these things on your own until you prove yourself that you
27:45can add value for others
27:48need versus necessity understand the difference between universes necessity
27:52the dot-com world was a prime example something
27:56a in that nature the subprime I mortgage fallout was another one
28:00and what you gotta realize greed is in good and you have to spend
28:04every dollar like the last right and that's a basic form lesson that
28:08we sometimes forget I in every different business cycle here on
28:13and the the net never is that people only to be a
28:16impressed by one thing that your performance nothing else sometimes
28:19people think they have to put on an image
28:22or for our show or put on this a thing that will drive them
28:26to succeed with a client or so investor or some degree
28:30but all that stuff is irrelevant what's really gonna drive them is on real merit
28:33a real performance exterior stereotypes
28:38what matters most is how you see yourself the number one characteristic
28:42in entrepreneurship
28:43what I realized is confidence
28:46and that is a difference between landing a customer
28:49landing an employee a and I you know actually growing your business
28:54and in in creating that or a that actually creates a dynamic around it and
28:59I don't have that early on my life I
29:02if you would have asked me at sixteen to stand up here in from class
29:05I'd be running out the door and I was the biggest introvert on the planet
29:09but it takes time takes practice you can overcome it
29:12but this is one of the biggest things that actually drives you
29:15to create their culture and create a company so confidence is key
29:19and the business world is a stereotype
29:23you can't let it get to you when I was 16
29:26I had two options when people ask me what I did before quickly asians
29:30I could tell the truth her which was like middle school or
29:33I coulda I could be so Impala ring
29:37engaged with what they were enthused about
29:40in about the business that they could they could care less
29:43right the fact that that the question above me being 16 would mean the poor
29:47the problem he would be the fact that while you're actually a million dollars
29:49a month
29:50and you doing at this when your profit warn you this year that because the
29:53entire conversations built around a
29:55it's not based are upon I you know what I did in the past for if I have a Ivy
30:00League education or this and that
30:02its all about actual real numbers
30:07relationships a real successful business is not run by algorithms
30:11its my relationships a relationship are one of the key things that drive you
30:16I through one phase real-life for the next and
30:20you ever want to burn a bridge I i've been insulted told I was in the field
30:24one one VC actually told me a year into
30:27him for me I was a miserably fail that I should save my years my youth
30:32and I should go into a completely different reaction right and I bit my
30:36tongue
30:37and I do not retaliate I said thank you very much for your advice
30:40and a you know life goes on right that's part of the thick
30:44that's par that fixed in you have to have but also the realization that you
30:48never want to burn a bridge in everyone react in a way
30:50because you don't know when that person you're going to be back in your life and
30:53we're
30:54you're actually in a go-ahead and have a better dialogue with them and words
30:57actually gonna make you do something that
30:58could be powerful because you prove them wrong
31:03the hunger too many people in this world one a handout
31:07surround yourself from people that wanna see successful
31:10and our are hungry and if you want a surly on
31:13successful repeat itself for db2 examples that
31:16when I the second time around a buffet and
31:20I knew it another ad serving technology engineering team
31:23so forth and iPhone a company in Belarus evolve with
31:27and IFU down in Belarus met these guys
31:31bad like a they're they're apart their apartments
31:35were rundown apartments but bout thats was their office to
31:39and they had some great technology very hungry very scrappy people
31:43very motivating and a when it was time to strike a deal
31:48I asked the guy with what do you want for this cuz I need this technology can
31:51you help me
31:52put it together and you know be part of this greater company
31:56and he said if you give me a half a million dollars all were like slaves
32:00you have the technology and it will be all all yours
32:04I actually said hey I'll give you a better deal actually let you be part of
32:08this company or the
32:09equity in it and three years there that became $50 million dollars for them
32:13so and and the desire is that
32:16that's not wrong I didn't do anything wrong I actually did right because
32:19that X for 14 and half million dollars they deserve right and that's the kinda
32:23motivation you want it is because
32:25have to spread the pie because that's the only way the pie actually gets
32:28bigger
32:29another example %uh hunger a the and hunger repeating itself
32:34another founding employees have mine was actually are
32:38try to be a doctor and you know you're such a bad influence he dropped out
32:43and I I actually hired him as one of my eye
32:46imp Bonnie employs trouble with him he worked his butt off
32:50you we all played an instrumental role a few and then after we saw the company
32:54I he actually started another one and after two years he sold it for fifty
32:58million dollars
33:00so that is kinda tells you hunger
33:03and the importance of hunger and the importance of actually thrown yourself
33:06from people
33:08that will go all out when you need to to actually connect the dots
33:15culture is everything the first 20 employees will make or break
33:19you hire the right be in a and that company culture
33:23will actually go so much farther for you and and this is key because
33:27cultures were those things that you're around these people every day
33:31you want to make sure you're around people that actually want to be there if
33:34they want to be there they're gonna hire the next twenty people were actually
33:37want to be there too
33:38and that's what creates this or a me if you look at Steve Jobs book
33:43right it was all about thinking differently sir on your cell phone you
33:46if people
33:47trying to let some people actually want to be there and that's the art of
33:50culture
33:51and cultures what drives you cultures what makes the difference between having
33:54a small company
33:55and a super large one because I cultures what actually amplifies
33:59the successful everybody
34:03rejection who likes rejection this room
34:07liar fire
34:10nobody likes rejection human nature right
34:13I but the art love the business world is you have to embrace rejection
34:18to the point you have to almost follow up with it and and and
34:22it is the hardest thing especially the 16-year-old growing up and so forth
34:25a I you know I was challenging matter
34:29in that arena growing up but I learn to accept that I actually let it few me
34:34I me that negative energy actually into positive energy
34:37and that's what you have to do because you everything happens for a reason
34:41and if it happens you have to learn to accept and embrace it
34:46because if if that happens that doesn't mean it's over
34:49that just means you have to try again anytime somebody actually tells me that
34:52all work for me I just end up saying
34:54not right now I'll get you some time or for clients is now
34:58we eventually got them on board not just because we've we we stock down but
35:02because we actually persisted on developing a relationship
35:06actually allow them to realize that that thats
35:10it's been a better for them and another problem examples are
35:15you know nobody wanted to invest in me even the second time around
35:18I had three term sheets from the season actually ended up saying
35:22nope not interested and then they eventually
35:25I found one that was in a better deal and I eventually got the right investors
35:28that believed in me
35:29to other in the is a few examples are JK Rowling probably one of the
35:33be most successful offers in history also a billionaire
35:37she got rejected six times six times from saying no Harry Potter not
35:41interested
35:42right a magellan's another example 1
35:46you know the GPS company had this crazy idea that people actually watch GPS is
35:49in their cars
35:50I 87 times they were told no
35:54that's a lotta rejection right for a passion because people actually ended up
35:58using GPS is in the car
36:00and these guys are one of the first people that want market for
36:05negotiation perception always negotiate
36:08from a position of strength if you need something from the other guy you've
36:12already lost
36:13people want what they can't have become the thing
36:16people want and perception is reality when he sees more important know what is
36:22so show them what they want to see and tell them what they want to hear
36:25you have to believe in yourself almost to the point of madness
36:29you have to be able to sell the dream build a dream yet have to be able to
36:32connect the dots
36:33in your heart in your mind with your team all over
36:36and and and when you do that things work out even if
36:40if if you don't early another example is Google
36:43one Google was just Amir employees
36:4620 employees the lani the Netscape account
36:49and that's cute back then was this big company right
36:53I and their servers crashed so
36:56imagine that you just signed the biggest to your life you're not prepared for it
37:01and the servers crash right so that's the pivotal point just like I had my
37:04people point when I was 16
37:06we're at it figure out what to do when I was normal
37:09they figured out what to do in those coming days I want to end up happening
37:13you know a company ten years later became twenty fifty billion dollars in
37:17in market cap right VAX those are some other decisions that you end up making
37:21that throw yourself in a situation and allow you to go ahead in creating a
37:25bigger
37:26I aspiration outta fix ken
37:30grow a fixed yet a very very
37:33thick skin people will question your ability to succeed
37:37and the loudest among them might make you doubt your own talents
37:41so you need to fix can to drown out the noise the silence will help you focus
37:45on your objective and you will prevail in my second company
37:49booth him in 10 my board meetings
37:52three years into it one my board members actually said hey you know what
37:55I think the company's run a lot faster than your
37:58so imagine that you're the founder
38:01supposedly you doing things right revenues profitability the whole nine
38:05yards
38:06everything's working the right direction why your board members just
38:09doesn't believe in you so I had two options I could react
38:12I start crying I don't do that
38:15I could go ahead and react negatively
38:18right and then I just basically instituting a fight which doesn't
38:22make sense or I could basically except
38:25the fact that that's out this one person feels
38:28and let that be the case but only prove them wrong
38:32their performance so far to fit the fix can help me
38:36keep on the objective and continuing and that's what you have to go
38:42so final thoughts take the advice the late great comedian Jimmy Toronto
38:47be nice to the people on the way up because you'll probably meet them on the
38:50way down
38:51and which means you gotta do the work keep your eye on the tiger
38:55fight like hell defy the odds it's worth it
38:59remember you never want to compromise you're more of
39:02Steve Jobs said it best stay hungry stay foolish
39:06and to mean what that means is never forget where you came from
39:10and never forget where you wanna go thank you
39:20sure a war I so
39:23I guess I i ke I was raised differently
39:26meaning my parents were very religious I were determined for the first eighteen
39:30years my life
39:31so I went through I a lotta love
39:35untypical situations at school and the
39:39that taught me a lot and I think that part of the problem is is that
39:43that usually when people feel that and the rebel leader they get that
39:47the end up becoming a singer to personality and I ended up realizing
39:51Diffley I never realize okay I'm different so what
39:54now let's let's move on acting what am I gonna do with my life
39:57and I think that kinda helps just I guess I was lucky with
40:01you know my grandmother raising me and so forth I in the way of instilling that
40:05but one thing that I did face that that that
40:08I think is to a lesser degree in the bay area now is
40:11is the racism with the stereotypes right I i think that
40:15ten years ago there was definite racism after type something
40:19ten years ago the idea that if you were the engineer are your the technical
40:22person
40:23you would get fired I after you got traction because
40:27they put in the Ivy League mature person to manage the business
40:31right that's all said and done so I think the opportunities for younger
40:34people minorities
40:35all that stuff a has been clear the road
40:39I so so I'm hope that summarizes our answer your question
40:45Jer sure so R you know I a
40:48I've been very a acted differently in flying to be
40:52and a I having gone through
40:55racism and having gone through I the issues
40:59love being the the the different kid
41:03and full-on facing racism I realize that a
41:06things probably got but I mean as I answer this question right here in the
41:09business world
41:10things have gotten better at least in Silicon Valley now
41:13I when the incident that happened a few weeks ago where
41:17you know basically about a supremist came and
41:20kill people and a go to our in Wisconsin
41:24that affected me because I realize that things should not be that way
41:28right things should not actually be effective that way people should
41:31understand that we're all different
41:33and we should be proud and we should be able to go ahead and
41:36and and transcend that message so a
41:39I created to be proud foundation were in the middle of shooting a media campaign
41:43to go on TV
41:45originally was gonna go on earlier but too much crap on politics
41:48for the next few weeks so we're actually launched in November
41:52and other social media component to it as well where you could I
41:55install the Facebook app change your profile picture with the be proud
41:59sticker on it
42:00social life feat of people joining the movement will have a Twitter integration
42:04with that as well but the whole concept if it is
42:06if you think about the coney movement a few weeks few months ago
42:09spy big a successful social cause ever on the Internet
42:13any cause this awareness and that's what inspired us to say that if we can
42:17inspire people
42:18to you social media as a way to
42:22recognize that they're different Mb private I hopefully that'll take one
42:25level away
42:26from problems that we faced a few weeks ago
42:33I think sometimes people learned through failure but
42:36if you don't have to learn it why do it so
42:40you know another way to think about it is is I mean
42:44I was driven to want to succeed no matter what and when you kind of go
42:48through those three things I talked about
42:49ambition discipline purpose and you have those three things
42:53underlying your motivation is clear
42:56there's a good chance or by not gonna fail so ya
42:59if this is again I'm not trying to get all warm and fuzzy here but
43:03I that is in the essence the art of business right it comes from within it
43:07comes from your heart
43:08and and if you can figure yourself out and figure out that
43:11what is it gonna take for me to react a certain way we're I will not accept
43:15failure
43:16that ends up happening no and failure on compu
43:26cool
43:31cool
43:36well
43:41on
43:44that
43:45of
43:48cool
43:54well
43:57cool
44:00of

The Lean Startup.

0:00
>>Thomas Sharon: Hi everyone. Thanks for coming. My name is Thomas Sharon. I'm a UX Researcher
0:01
here.
0:01
And without further ado, I would like to introduce you to Eric Ries, the founder of the Lean
0:10
Startup movement. And thank you for coming.
0:14
[applause]
0:16
>>Eric Ries: See, I don't know. It's hard to know when you have an audience you don't
0:25
know.
0:25
It's hard to know what to say.
0:27
So, some of you maybe know the blog, Startup Lessons Learned. Anybody? Quick show of hands.
0:32
OK, good. We have true new people. So, thank you. Thank you for coming to check it out.
0:36
So my name is Eric Ries. I don't really want your undivided attention. So keep your laptops
0:42
on. That's fine. And in fact, if everyone do me a favor and take your phones out of
0:46
your pockets.
0:46
I mean, I'm sorry. I know I'm not supposed to have this one, but if can turn them on.
0:50
No, I'm not joking around. Out of the pockets, please. 'Cause this is gonna be a talk. You
0:53
might get bored and you might wanna tweet amongst yourselves.
0:56
Or I don't know, whatever special internal tools you guys use. Whatever it is, please
1:00
feel free. All I ask is that you use the Lean Startup hash tag, at least if you're on Twitter.
1:03
Okay. Is that fair? I won't tell anybody. Don't worry.
1:09
I'm in New York because I'm writing a book. So one of things you do when you're writing
1:13
a book is you have to go to tell as many people as possible that this book is coming out.
1:18
It'll come out in the fall. I thank you in advance for preordering it. You can do so
1:21
at lean.st.
1:22
That's my life now is as a professional expert selling books, which is a far cry from how
1:27
I started, which is as a programmer writing code.
1:30
So I'm one of those people that grew up writing code. I used to write code for a living, which
1:33
is a job I knew really well and understood. I was pretty good at it.
1:36
And then, I started doing startups and I started to have to manage people who wrote code for
1:39
a living. I knew that slightly less well.
1:41
And then I was managing people who managed people who write code for a living.
1:44
And now, I am this professional expert. And so I advise people who manage people who manage
1:49
people who write code. So I've become very far removed from the actual work of product
1:54
development.
1:54
But that journey has taken me from just writing code to doing this because I actually think
1:59
the way that we organize new product development is basically wrong. And that most of the energy
2:06
that we are investing into what is called 'entrepreneurship', when it's two guys in
2:10
a garage, or 'disruptive innovation' or something else buzzwordy when it's done inside a big
2:15
company, is wasting a lot of people's time. And I think we can do something about it.
2:20
So that's what I wanted to talk to you guys about. Thanks for coming.
2:22
So anyway, and this book, of course, will be available in stores everywhere in the fall.
2:26
So, if you read the book, you will learn five principles of the Lean Startup. And we'll
2:33
go through them today.
2:34
And I wanna invite you to ask questions either on Twitter, if you're not feeling that courageous,
2:38
or interrupt me at any time or we'll have time at the end. All right?
2:41
So, entrepreneurs are everywhere. The first thing is, especially in an audience like this,
2:44
I wanna be clear, that entrepreneurship is not just about two guys in a garage eating
2:49
Ramen noodles. In fact, what makes you an entrepreneur is not what kind of noodles you
2:52
eat, but the context in which you operate.
2:54
And as I've been travelling around talking about Lean Startup, what I have learned is
2:58
that there are entrepreneurs in all kinds of places you wouldn't necessarily expect.
3:02
And we have a lot more in common than people realize, because entrepreneurship is management.
3:08
But not the kind of general management we're teaching MBAs and that we have studied for
3:12
the last hundred years, something fundamentally different. It is management of a kind of work
3:16
that is measured by validated learning, rather than just making stuff.
3:21
We accelerate that learning through something called the 'build-measure-learn feedback loop'.
3:24
And then we measure and hold entrepreneurs accountable using a new accounting system
3:28
called 'innovation accounting'.
3:31
Now, I apologize. You came to a very, I'm sure, you saw the signs up here like, "Ooh,
3:35
an exciting talk about entrepreneurship and startups and that's gonna be cool." And what
3:38
did you get?
3:39
You got management and accounting, which are perhaps the two most boring topics on Earth.
3:44
So if anybody wants to leave now, I won't be offended. It's OK.
3:48
Because the truth is, what do people know about entrepreneurship? I feel like -- who
3:54
saw 'The Social Network'? OK. Right. I feel like that's probably the best modern example
3:59
of the entrepreneurship story we're all used to.
4:02
And you see this in magazines and you see it in 'The Social Network'. I noticed last
4:06
night that the story, 'Ghostbusters' -- remember 'Ghostbusters', the movie? It's an entrepreneurship
4:11
story. They start a business. They, Dave, everything’s the same. It's like the same
4:14
plot structure as 'The Social Network', believe it or not, except it has a Stay Puft Marshmallow
4:17
Man, which is awesome.
4:18
In these entrepreneurship stories, what happens? It's a story in three parts.
4:24
Act One: The plucky protagonist, his character, his character flaws and how he came up with
4:30
his amazing idea.
4:32
Act Two: What I call the photo montage. It's usually about two minutes long. It goes from
4:37
"they finally get the thing to work". Then they're writing on whiteboards and drinking
4:41
some beer, pounding on some keyboards. And then they get their first customer. And then
4:46
that's pretty much it. No dialogue or anything in the photo montage.
4:48
And then Act Three: Now that we're on the cover of magazines, how do we divide up the
4:51
spoils? And who's in charge? And who's in Who's Who? And how do we deal with the EPA
4:55
and all that stuff? For fans of 'Ghostbusters'.
4:58
What I think is really interesting about these stories about entrepreneurship is that 95
5:03
percent of the time of the movie is spent in Acts One and Act Three, even though in
5:07
real life, all of the important work of entrepreneurship happens during the photo montage. But the
5:13
problem is, for a story-telling point of view, the photo montage, even though it has no dialogue
5:16
and only lasts two minutes, is it's unspeakably boring.
5:21
What do we do as entrepreneurs that actually makes a difference? We spend our time trying
5:25
to figure out which customers to listen to and who to ignore, how to product-prioritize
5:29
product features.
5:30
I mean you guys, how many product prioritization meetings do you go to? It's not exactly the
5:33
stuff of movies. It's unbelievably boring.
5:35
And how do we hold people accountable? How do we measure to figure out if we're actually
5:39
making progress or building something that nobody wants? See, watching somebody pretend
5:43
like they don't have anxiety that their vision is wrong, is not very good for movies. But
5:48
that's what most entrepreneurs do.
5:51
And so, we're gonna have to talk about stuff like management and accounting, 'cause it's
5:54
time to go inside the photo montage and try to figure out what can we do to make the actual
5:58
work of entrepreneurship more effective. So, entrepreneurs are everywhere.
6:03
My goal, my mission in doing this whole Lean Startup thing has been to try to put the practice
6:08
of entrepreneurship on a more rigorous footing. And so, I started out with a definition. Here's
6:13
mine: What is a startup? "A human institution designed to create something new under conditions
6:18
of extreme uncertainty."
6:20
So I think the most important part of this definition, and for our purposes today, a
6:23
very important part of our discussion is what it excludes. It doesn't say anything about
6:28
what the size of your company is. It could be five people, 5,000, or 50,000. It really
6:32
doesn't matter.
6:33
It doesn't matter what sector of the economy you work in. It really doesn't even matter
6:36
what industry you're in.
6:37
If you fundamentally are operating with extreme uncertainty about who is your customer, what
6:42
product do they actually want, and how do we build a sustainable business, then you
6:45
are an entrepreneur. And when I work with large companies, one of the things I have
6:49
been trying to do, is to get them to adopt entrepreneurship as a job title.
6:54
Entrepreneurship is a career. When you become an entrepreneur, you are no longer an engineer.
6:58
You are no longer a marketer. You are no longer a UX designer. Whatever it is you used to
7:03
do, all of a sudden now, you have a different job title and you've entered a new career
7:07
path.
7:08
But unfortunately, we don't get the memo that tells you that. So, it can be a little bit
7:12
confusing.
7:13
That's all a fancy way of saying a startup is an experiment. What I mean by "experiment"
7:19
is not just like let's ship it and see what happens, OK? That's not science. If you just
7:23
put some compounds in a beaker and heat it up, you might look like you're doing science.
7:27
But unless you have a hypothesis that you're trying to test, you have theory, it suggests
7:33
which experiments are gonna help you and then you make specific predictions, then, fundamentally,
7:38
you're not conducting an experiment.
7:39
And we mean, in a Lean Startup model, "experiment" in the scientific sense. We're trying to create
7:43
a science of entrepreneurship that will help us to stop waste people's time, because that's
7:50
what we're doing on an industrial scale.
7:52
And you guys know. Anyone's who's worked on new products knows that most of them are doomed
7:56
to failure. And when you get at the end of that product -- I mean as an engineer, I kept
8:00
having over and over again the experience of working on amazing technology that is today
8:04
sitting on a shelf or worse, that fundamentally nobody is using. And I kept looking for more
8:10
and more technical solutions to that problem. I thought, "If we could just get the right
8:14
development methodology, if we just had the right amount of unit tests or the right this
8:18
or the right that, then we could stop that happening."
8:21
But the biggest waste that product development faces today is not building things inefficiently,
8:26
but building things very efficiently that nobody wants. And I brought a demonstration.
8:31
We all know that most startups fail. Who remembers Web 2.0? Remember Web 2.0 when that was really
8:36
cool? At the height of Web 2.0, 2006, a graphic designer put together this graphic. Have you
8:41
seen this before? This was the like the logos of all the incredible Web 2.0 companies that
8:45
were gonna change the world.
8:47
And in just three years, in 2009, a different graphic designer was feeling a slightly different
8:53
set of emotions when they put together this graphic. Our three year report card in Web
9:00
2.0. I mean, the blood red Xs, these are all companies that are just dead. I think for
9:05
our purposes today though, a much more important part of that chart to look at are the green
9:10
circles. I won't point at any of them in particular, but some of those green circles are supposed
9:15
to be the success stories of Web 2.0. But for this chart, what that means is there are
9:19
companies that were acquired by a larger company, including this one.
9:22
And listen, I'm all for people making money.
9:24
So when a company gets acquired by another company, usually investors and the founders
9:27
make some money and that's all good. But my question is which of these companies are actually
9:31
success stories? Success stories by the higher definition, not of "did anybody make money?"
9:37
But rather, which of these companies succeeded in living up to the aspirations, dreams, time,
9:43
talent, and energy of the founders and their investors? And more importantly, their employees.
9:47
See, look, we all know that when big companies buy startups, at least half the time, they
9:54
die afterwards. So we buy something for hundreds of millions of dollars. And then we wind up
9:59
selling it three years later for tens of millions of dollars.
10:01
That's not supposed to happen. In general management, that doesn't happen. When you
10:05
buy an asset, it depreciates in a predictable way. But when big companies buy startups,
10:10
it doesn't happen exactly like it’s supposed to.
10:14
And I think the problems that corp-dev departments have when deciding how to buy startups and
10:20
which startups to buy and then how to integrate them into the parent company, are the exact
10:24
same problems that internal innovators who are trying to create brand new startups inside
10:28
big companies have. And they're the exact same problems that venture-backed entrepreneurs
10:32
have with their investors.
10:34
All of us lack a theory of entrepreneurship to guide our behavior and so we're falling
10:39
back on tools and methods that are not appropriate to entrepreneurship. That's my belief. So,
10:44
I don't think it has to be this way.
10:46
See, it's one thing if startups were failing because they were taking too much risk. If
10:50
one of these companies was working on teleportation and then it turned out to be too hard -- we
10:54
couldn't quite get the technology for quantum entanglement like we thought -- I accept that
10:58
kind of failure; that happens.
10:59
But I chose Web 2.0 for my demonstration, especially for you guys. You know. There's
11:05
not a single company on this chart where you would say, "Boy, I wonder if that can be built."
11:09
Right? "Geez, I wonder if that new social network -- is it possible to build it?" We
11:13
all know. Software companies, we can build anything we can imagine. Think about that
11:19
for a second.
11:20
The dominant question of our time is not can it be built, but should it be built. And the
11:24
issue is can we build a sustainable business around a particular product? So, the future
11:30
of our society, our economic growth in the future, the GDP growth of industrialized countries
11:35
in the future is going to be dependent on the quality and character of our collective
11:39
imaginations, which I think is a very strange place to be.
11:44
That is really different than the kinds of economic problems that general management
11:47
was designed to solve in the 20th Century. Now, most of my startups have failed. So I
11:54
know that's not how you're supposed start one of these talks, like, "Hi. I'm a professional
11:57
expert and I have had more failures than successes. So you can be just like me if you'll follow
12:01
my advice."
12:01
So, I'm sorry about that. But those of you who spend any time around entrepreneurship
12:05
know the truth that where there's startups, there's a lot of failure. And it has to be
12:10
somebody's fault in a talk like this and obviously it shouldn't be my fault 'cause I'm the expert.
12:15
And preferably, it should be the fault of someone who's dead so that they can't argue.
12:19
So I chose this guy.
12:20
[laughter]
12:22
This is Frederick Winslow Taylor. He died in 1915, which is very handy for the purpose
12:25
of this talk because it means he can't talk back. So, sorry, Fred.
12:30
Fred Taylor invented something called "scientific management" in the early 20th Century, which
12:34
today, we call "management". See, people don't really remember Fred Taylor.
12:39
And those who've studied scientific management in school probably remember him for some of
12:42
his outdated and really ridiculous ideas, like time and motion studies or the idea that
12:47
a worker is basically just an automaton and should be told what to do. The reason we don't
12:51
give Fred Taylor the credit he deserves is because he invented things that to us are
12:55
so obvious, we can't imagine them ever having been invented. It doesn't make sense.
13:00
Like, everybody knows that work should be done as efficiently as possible, right? And
13:05
that we should treat work like a system and that we should have managers organize that
13:08
system. That's just plain common sense.
13:11
And my favorite, Fred Taylor, invented something called "The Task and Bonus System", which
13:16
we just call "tasks". The idea was if you want to do a large project, the best thing
13:20
to do is decompose that project into a series of individual tasks, assign those tasks to
13:25
functional specialists. And everyone just does their part knowing that if everybody
13:28
does their part well and everybody else does their part, the whole will actually work out
13:32
like the manager said.
13:33
And here's the best part. If you do your task particularly well, better than expected, you
13:40
should be paid a bonus rather than being penalized. What could be more obvious? Except in the
13:46
19th Century, the way work was organized is that if you did your task better than expected,
13:51
you were penalized.
13:52
[pause]
13:52
Why? Because that showed a lack of integrity. You obviously could have done it better before.
13:57
But you didn't. So that proves that you're a liar.
14:00
It gets worse. Not only are you a liar, but what about all your compatriots, your coworkers,
14:04
who do the same task the old, slow way? They're liars, too. All of you have been lying this
14:09
whole time and you should all be penalized.
14:11
Imagine working in a factory where if you can come up with a better way to do your repetitive
14:16
job, not only would you be penalized, so would all your coworkers. Can you imagine the culture
14:21
that would grow up around such a thing? That everybody is working really hard to make sure
14:26
that nobody ever does anything in any way better because then we'll all be in trouble.
14:31
That phenomenon was so widespread in the 19th Century, they had a name for it. They called
14:34
it "soldiering". That all the workers were intentionally soldiering on, trying to do
14:38
work as slow as possible so that nobody would get in trouble. Now, we laugh when we think
14:44
about that kind of thing happening in a factory. Because to us, the way that we manage physical
14:50
products, and just all regular general management, is light years beyond what was possible in
14:55
Fred Taylor's day.
14:57
But they also believed something else that I think maybe you'll find a little bit familiar.
15:01
They had this idea that what basically was the great man theory of work. That fundamentally,
15:06
the job of managers was to select the best possible person. Of course, this is the early
15:11
20th Century.
15:12
So a great man of upstanding character with good integrity, the right attributes, put
15:16
them in the job and fundamentally leave them alone. Because if you just trust a great man
15:21
to figure out what needs to get done, they would figure it out.
15:24
Does that sound familiar? We still manage knowledge work and innovation work that exact
15:29
same way. We still believe in the great man theory of entrepreneurship and we believe
15:33
it especially about the great man software developers.
15:36
And yet, when we look back on this time, decades from now, I think we're gonna laugh at the
15:41
kinds of things that we do when we need to develop new products. It will seem as antique
15:45
to our future selves as Fred Taylor feels to us. Because I think that entrepreneurship
15:51
is management. It's just a different kind of management than the general management
15:55
that has been practiced since Fred Taylor's day.
15:58
So, we need to create a different paradigm for management that's not better than general
16:03
management. It's not worse than. It's simply a parallel discipline specifically for entrepreneurship.
16:09
And so, here's my attempt.
16:10
The first concept in the entrepreneurial management toolbox is this thing we call the "pivot".
16:15
Who's tired of hearing about pivots already? Anybody? OK, I apologize. In Silicon Valley,
16:21
everybody's hand is up, by the way. This word has become ridiculously overused. Believe
16:25
it or not, I saw this just the other day in the New Yorker magazine. Can you read this
16:30
caption? "I'm not leaving you. I’m pivoting to another man."
16:33
[laughter]
16:35
So this word started in Lean Startup and then it became this monster of an overused, overhyped
16:40
concept. Typical for Silicon Valley. We went from not knowing what the concept was to being
16:45
tired of hearing it and claiming that it's overhyped, without having passed through the
16:48
intervening stage of ever learning how to use it correctly. That's just – that's how
16:51
we operate with the hype-cycle in Silicon Valley. So, I'm sorry about that. I really
16:55
didn't intend.
16:57
But you can't understand anything about entrepreneurship unless you have a word for this concept, because
17:01
it is the universal constant of all successful startups. If you can get the real story of
17:06
what actually happened in the early stages of a company, you will find out that successful
17:11
startup founders do not, do not, have better ideas than the failed ones. Contrary to what
17:16
you see in the movies, most startup founders of successful companies had ludicrously bad
17:21
ideas at the beginning.
17:23
And what's amazing about the successful startup founders is not that they just persevered
17:26
indefinitely, but that they had this funny combination that when they run into difficulty,
17:31
it's not just that they gave up ship and went home. Neither did they persevere the plane
17:36
straight into the ground.
17:37
They do this thing we call the "pivot". By analogy to basketball, one foot firmly rooted
17:41
in what we've learned, changing one other thing about the business at a time. And the
17:45
premise of the Lean Startup is as follows: If we can reduce the time between pivots,
17:50
we can increase our odds of success before we run out of money.
17:53
See, the way you think about startup runway, is not how many months of burn do I have left?
17:58
It's fundamentally how many opportunities to pivot do I have left? And sure, we can
18:03
extend the runway by raising more money.
18:05
But we can also extend the runway by figuring out how to pivot sooner. And every day we
18:10
shave off that time is a magical extension of our runway by a proportional amount. Does
18:15
that make sense? OK, I'm getting at least a few nods. That's good.
18:19
So, to increase the odds of success, we need to figure out how can we pivot sooner. We
18:24
need to bring our focus to a validated learning. Anyone know this? This is the waterfall methodology
18:29
of software development. It's traditional in one of these talks when you're gonna beat
18:32
up on methodologies to call one the "waterfall methodology". So this is mine.
18:38
This is just Fred Taylor's idea as applied to software development. When I was trained
18:41
as an engineer in Silicon Valley, I was taught this as the manufacturing metaphor of software
18:46
development. You can imagine, incidentally, how pissed off I was when I found out that
18:49
they don't use this in manufacturing anymore. This is way obsolete.
18:52
So it's not clear to me what our excuse is in software development for copying an obsolete
18:56
model of manufacturing. But I understand why it’s so appealing.
18:59
The idea is that since software is so intangible, we like to imagine our work travelling on
19:03
an assembly line, a virtual assembly line, from department to department. And if everybody
19:07
does their part and trusts everybody else to do their part, everything works out fine.
19:12
It's entertaining to beat up on the waterfall methodology because it has such a bad track
19:15
record. But it’s important to understand that waterfall works as long as the solution
19:21
and the problem are both relatively well understood. So if we were building something that is fundamentally
19:26
similar as things we have built in the past, this works great.
19:29
If you're building a video game sequel, amen. If you're building the next iteration of a
19:32
product that zillions of people already use, that's fine. But entrepreneurship doesn't
19:37
deal with those circumstances. So it's basically a waste of time.
19:40
Now, when you do waterfall, you have this problem I call "achieving failure" where you
19:45
successfully build the wrong thing and you boast about how good you are at doing it.
19:52
My question is, if you go to a startup board meeting, or a milestone review meeting for
19:55
most new products, what do we talk about? Milestones, right. We are on track. We're
20:00
building features we said we were gonna build. We talk about our gross numbers like – hey,
20:04
we have this many customers just like we said.
20:06
I remember being in a company once that was looking, had a plan. They said we were gonna
20:08
have this nice hockey stick-shaped growth. And we're on the nice, long flat part, and
20:12
everything's going according to plan. Sounds a little familiar, maybe.
20:16
And you're going to court like, the plan is genius. Like, nobody is using our product
20:19
as expected, as expected. But next week, it should turn up.
20:22
And how do you know if you were on the long, flat part and you're gonna just, or you're
20:26
on the hockey stick and you're gonna just coast indefinitely? I believe we can actually
20:29
answer those questions quantitatively. We'll get to that.
20:32
So, to me, we are bragging about how we're building a product that nobody wants. But
20:36
we're doing it on time and on budget. As if, I have this image of a general manager driving
20:40
a car off a cliff and while they're driving their like, "But I'm getting great gas mileage,"
20:44
right off the cliff. That's what startup failures mostly look like.
20:49
And I think that's true in big companies, too. Not that I would presume to talk about
20:53
you. But in other companies I've seen, for sure. So, in manufacturing, they abandoned
20:57
that linear way of working. That's why we call it Lean Startup by analogy to lean manufacturing.
21:01
These are two other unfortunately deceased men who made this possible. Deming is famous
21:06
for having said, "The customer is the most important part of the production line." By
21:11
which he meant everything that we do should be gauged to be decided by whether the customer
21:16
cares that we do it.
21:17
So if the thing is of high quality in the customer's eyes, that matters. But if we're
21:21
doing a lot of internal transport, with the meetings that we have, the specifications,
21:25
the customer doesn't care about any of that internal stuff. So let's try to eliminate
21:28
it.
21:28
When these ideas were handed down to me as a Silicon Valley engineer, they looked like
21:32
this. There's our very own guru of extreme programming, Kent Beck. You guys know Agile
21:36
very well, so I won't bother getting into it.
21:40
Suffice to say that all Agile methodologies have their origins inside the IT departments
21:46
of big companies. Every single one. And there's a reason for that. They are designed for situations
21:51
where it is the problem that's known, but the solution is unknown. And so, by building
21:54
something that is well-understood iteratively, we can increase the odds that the project
21:59
will be successful.
21:59
So, the classic -- Chrysler Corporation needs a new payroll system. Agile to the rescue.
22:05
But this isn't the world that we live in as startups either. If the customer is the most
22:08
important part of the assembly line, what do we do if we don't know who the customer
22:12
is? That in whose eyes should we judge our work? In extreme programming, which customer
22:18
should we sit down next to the engineers to tell them what to do?
22:21
The assumption of Agile and all previous management approaches is that there is somebody who can
22:25
give us an authoritative, definitive answer to design questions. And in entrepreneurship,
22:31
that assumption breaks down.
22:33
We are working on products where nobody knows what the customer wants. At best, we have
22:39
a theory, a hypothesis, a plan, a hope. And so, this is what Lean Startup looks like.
22:44
Now, at Lean Startup we have our own guru, Steve Blank. He's still alive, but I put him
22:48
in sepia tones just to be consistent.
22:50
[laughter]
22:50
Steve invented something called "customer development", which is an iterative process
22:54
of trying to figure out who your customer is, which we can merge in parallel with Agile
22:58
development to this company-wide feedback loop of learning and discovery. This changes
23:03
the unit of progress from making stuff to validated learning.
23:07
Let me try to illustrate what I mean. I created a company called IMVU in 2004. We make a 3-D
23:13
avatar instant messaging technology. And at that time, we wanted to be the next AOL back
23:18
when that was still cool. And we wanted to take over the hot, new social technology of
23:25
IM. We really thought that was the wave of the future.
23:26
Whoops. And here was our plan. See, everybody knows that instant messaging is a network
23:30
effects business, right? So, therefore, if you wanna get someone to switch from their
23:35
IM network to yours, it's kind of a pain 'cause they'd have to bring all their friends with
23:38
them. So there's high switching costs.
23:40
And therefore, IM isn't an industry characterized by high barriers to entry. That's the MBA
23:45
analysis of the instant messaging market. And we spent a lot of time figuring that out
23:48
at the whiteboard.
23:48
And we said, "Ah, we need a strategy. A strategy for avoiding that problem and here it is.
23:53
We'll create an instant messaging add-on that interoperates with all of the existing networks
23:58
and can bring 3-D avatar technology to your IM client. So we take your boring 2-D IM and
24:03
make it 3-D. Wow."
24:04
This is before 'Avatar' and the current 3-D craze. So we thought we were on to a real
24:09
trend. And so, here's the reason we got so excited about that strategy. It would be inherently
24:14
viral. Because when you would decide you wanna go 3D, you would have to be IM’ing with
24:18
somebody and they would automatically get a text link inserted into the chat stream
24:22
that they could just one-click, pick-up boom, IMVU installs. Now you're both in 3D.
24:26
Doesn't this seem like a good idea? Well, we met with investors at that time. The strategy
24:31
part of it, they're like, "That does sound very promising." And when I tell the stories
24:35
to MBAs now, I get a lot of nods, like "That is good stuff. I don't know what the hell
24:38
this guy's been talking about until now, but this I understand. This is strategy."
24:42
And the strategy actually is very good, except for the tiniest, tiniest problem, which is
24:45
that every single thing I just said is false. Customers actually don't have high switching
24:49
costs for IM. Their network effects are way overblown and our customers refused to invite
24:54
their friends. It was a total deal breaker.
24:56
We'd have customers come in an in-person usability test. We're paying them to be there. And we'd
24:59
be like, "OK, download our instant messaging add-on." The customer would be like, “What
25:04
is that?" "An instant messaging add-on. It interoperates with all your IM." And you gotta
25:08
picture of a 16- year old like, "What? Is it an IM client?" And we're like, "No, no.
25:12
You won't have to run a whole other IM client." They're like, "Why not?" Like, "Oh, it would
25:16
be so complicated to download." They're like, "Dude, I have like seven IM clients. What's
25:20
the big deal?" And we're like, "There are seven IM clients?"
25:21
[laughter]
25:22
So that was problem number one.
25:23
We're like, "Listen, we are paying you to be here. So how about you download this thing?
25:27
OK?" "All right. Fine." "Download the thing." OK. "Customize your avatar." They love this
25:31
part. "Like, ooh, that's really cool, interactive like that." Great. "OK, now you customize
25:35
your avatar. Invite one of your friends." "No way." "Why not?" "I don't know if this
25:41
thing is cool, yet and I'm not gonna invite my friends to something that turns out to
25:44
be lame."
25:45
See, I know people who are selling like business software are used to the concept of "mission
25:48
critical". We didn't understand that in our business, mission critical, like the law of
25:52
commandments of mission critical software, one of them needs to be like, "Do not make
25:55
teenager look lame in front of their friends." Total deal breaker.
25:58
They wouldn't do it. We were like, "We're paying you to be in this usability test."
26:01
And it's just like, "I'm not doing it. You can keep your money. I will not. It's not
26:05
worth it to me." And they kept saying things like, "Let me use it with -- let me try it
26:10
out first. And if it's cool, then I'll invite one of my friends." And we were like, "Oh,
26:14
OK." We're from the video game industry.
26:15
So we knew what that meant. That meant single-player mode. So we built another version of the product,
26:21
single-player mode. Allowed you to -- we had another teenager come in for a usability test.
26:26
"Hey, download this instant messaging add-on." "I don't know. What the hell is that?" "Just
26:28
do it." "OK." "Customize your avatar." "Oh, that's cool." "OK, try it by yourself."
26:32
And they could check out the avatar, dress it up, do the moves and the whole thing. Learn
26:34
how to use the chat bubbles. And then we're like, "OK, now invite one of your friends."
26:39
Teenager, "No way." "Why not?" "This thing is lame." And we're like, "But we told you
26:44
it was gonna be so lame."
26:46
I mean, we're supposed to be listening to customers, but they don't know what the hell
26:48
they want. And they told us to build this thing and we're like, "It'll be so cool once
26:52
you invite some of your friends." And they're like, "Listen, old man. I'm not doing it."
26:56
[laughter]
26:57
And we were really devastated. Okay.
26:58
So anyway, long story short, this was a total deal breaker. This strategy is completely
27:01
flawed in every respect because it’s based on empirically incorrect facts.
27:08
Now, we wound up having to pivot the company and we created our own instant messaging network
27:11
and it all turned out fine. But I'd like you, if you would, just for a minute to sympathize
27:15
with me personally. OK? Because I was the engineer. I was the CTO of this company.
27:19
It was my job to write the software to do instant messaging interoperability. So I wrote,
27:24
I don't know, 25 thousand lines of code or something. I did it all Agile, refactored,
27:31
really elegantly structured if I do say so myself. Good unit test coverage, the whole
27:35
shebang. And all of my code got thrown out.
27:38
[pause]
27:39
The good code got thrown out and the bad code got thrown out. The well-factored code got
27:43
thrown out. The stuff I was proud to show my mom and the stuff that I wouldn't want
27:46
anyone to see at all was equally thrown out.
27:49
Because a [ ] of quality is if you don't know who the customer is then you don't know what
27:54
quality means. So failure is a great equalizer of quality. It all had to be thrown out.
27:58
And I was really depressed. Because you gotta understand, we had spent six months killing
28:02
ourselves to build this product. And we had spent I don't know how many hours of my life
28:06
that I can never get back arguing with each other about the following. Which bugs did
28:11
we absolutely, positively have to fix. And which ones could we live without? Sound familiar?
28:16
Which features just had to be in version one or which ones could we just maybe could maybe
28:21
postpone to a different release? That's what we spent all of our time doing.
28:25
And yet, we had this problem, which was that customers would not download our product.
28:28
Like, this product sucked. It was really buggy. It would crash your computer. I was really
28:33
embarrassed to have shipped it. And we almost didn't ship it, I was so embarrassed.
28:36
But then, I was actually relieved cause nobody found out how bad it was because nobody would
28:40
use it. And I was like, "Wait, something is not right here. Why am I relieved that nobody's
28:46
using the product? That doesn't seem right."
28:49
And long story short, my cofounders dragged me, kicking and screaming, to the realization
28:53
it was time to pivot. We had to throw that code away. And we created a standalone IM
28:56
network and we were much more successful, la di da.
28:58
But here's the thing. I had to make myself feel better somehow because I was like, "Gosh.
29:03
Would the company have been just as well off if I had spent the last six months on a beach
29:07
somewhere, having nice drinks and doing nothing?"
29:10
And I was, "Did I even need to be here given that all the work that I did was thrown away?"
29:15
Anyone feel like that's true? Anyone know what the excuse I used was to make myself
29:19
feel better?
29:19
You can shout it out. It's OK. I guess, yeah.
29:22
>>audience 1: You built a team.
29:24
>>Eric Ries: What's that?
29:24
>>audience 1: You built the team.
29:25
>>Eric Ries: We had the team at the beginning. What did they need me for? Why was it worth
29:29
having done this exercise in the first place? What's that?
29:31
[audience 2]: You learned something.
29:31
>>Eric Ries: Because I learned something. Thank you. The last excuse in the book. If
29:34
you've utterly failed to execute, you can always claim to have had a good learning experience.
29:38
At least you learned something. I mean, I don't tell you guys.
29:41
In general management, you claim to have learned something, you're likely to be fired. A general
29:45
manager who learned something -- one of two things is true. Either they didn't make a
29:47
very good plan, in which case, definitely should be fired. Or even worse, they made
29:51
a really good plan and failed to execute it. I'd definitely fire that guy.
29:55
So, I think it’s natural that we have a little bit of an aversion to wanna just say
29:59
that we learned anything because that is very dangerous. But in entrepreneurship, failure
30:03
is, not only is failure an option, it's practically the only option. It's what happens when reality
30:08
intervenes with our plans.
30:09
>>audience 2: So, what did you learn?
30:11
>>Eric Ries: So here's what we learned specifically. We learned the hard way, that customers did
30:16
not wanna use our product to connect with their existing friends. They wanted to use
30:21
it to make new friends.
30:23
That doesn't seem like a very big deal. I mean, it's all a very modest change in semantics.
30:27
But from a code and product point of view, that is a radically different product. It
30:31
required a very different experience. And we didn't throw out every line of code. But
30:35
we had to throw out a lot.
30:36
The pivot was quite dramatic. And I made myself feel better with this whole learning story
30:40
until I asked myself the following question. I mean, literally, I was up nights once I
30:44
had this question asked to me.
30:45
Which was, "Wait a minute. If my goal of the last six months was to learn this important
30:49
thing about customers, why did it take six months? How come the word 'learning' is only
30:54
coming up now after we failed and we need an excuse? We never used the word 'learning',
30:59
not one time during those six months. All we ever did was argue about features and bugs."
31:04
And then I was like, "But would we have had the same learning if we'd built a slightly
31:08
different first product?" Like, for example, did we have to support all seven IM networks?
31:12
What if we'd supported only three? Would the learning value have been the same? Sure. Customers
31:16
won't download, so who cares? What if we'd supported only one network? Learning values
31:20
the same. Now, that's a lot of code between seven networks and one, that's a lot less
31:23
code that needed to be written.
31:25
But this is the thought that literally made me sick to my stomach. I'd say, "Wait a minute.
31:29
What if we had just created a single web page and in three hours created a photo mockup
31:34
of what the product was going to look like and said, 'Hey, download this amazing 3D avatar
31:38
instant messaging add-on.' And had a big download button. Would we even have had to create the
31:42
second page where we admit that we didn't build the product, or would a 404 have been
31:47
adequate?" Come on, it's the 404, obviously. Because nobody would download the product.
31:53
It was a deal breaker. Nobody wanted it. That meant that we didn't even need page two.
31:59
And that was really upsetting to me, personally. Why? Because I look at my business card and
32:03
what did it say? It said a lot of things, but all I saw was "guy who writes code." My
32:08
job is to make features.
32:10
So if I went home at the end of the day and I write good code, I had a good day. And now,
32:14
but if my goal is to learn this thing about customers, and I can do it without code, is
32:19
that my job?
32:20
Is it possible that something I could do in three hours is just as meritorious as something
32:24
that requires 25,000 lines of code? It didn't seem right.
32:27
But I think that's actually true. Fundamentally, startups exist to learn how to build a sustainable
32:33
business. We call it "validated learning" 'cause we have to back up that learning quantitatively.
32:36
Any old idiot can tell a good story.
32:39
But we need a system for rigorously assessing, "Are we actually learning how to build a sustainable
32:44
business?" And everything else is a complete and total waste of time, including our precious
32:49
code.
32:51
Now, in the lean manufacturing revolution, the first question they had to teach people
32:55
to ask was "what is the difference between value and waste"?
32:58
And in a factory, this is actually relatively straightforward. Value is the stuff that we
33:01
make. The customers want. And waste is everything else. But if our unit of progress is gonna
33:05
be learning, then our unit of value has become intangible and now we have an issue. Which
33:10
is -- OK, we can eliminate all the stuff that we do that doesn't contribute to learning.
33:15
So, we have this concept in Lean Startup called "minimum viable product", which is, what really
33:20
needs to be in that first version? And now we have a good answer. Only what is necessary
33:24
to learn whether our plan is correct or not. Everything else is extraneous.
33:29
But that's still a little bit vague. And so the next step in lean manufacturing was to
33:34
focus on cycle time. And so what that looks like is this. Very simple heuristic. This
33:38
is the flux capacitor of Lean Startup.
33:42
All we are as a software startup is a catalyst that turns ideas into code. When customers
33:47
interact with that code, they create data which we can choose to measure quantitatively
33:50
and qualitatively. And then if we want, we can learn impacting our next set of ideas.
33:54
This, we can use to put the concept of the pivot on a more rigorous foundation.
33:58
A pivot is one major turn through this feedback loop. And the heuristic for any kind of startup
34:04
advice that anyone wants to give you is really simple. Does it minimize total time through
34:09
this loop?
34:09
So I don't know about you, I go to a lot of startup talks, I read a lot of startup blogs.
34:13
All the advice is like this. "You know, it's really important to have great design. Design
34:17
always wins. Except craigslist didn't have very good design and neither did EBay. So
34:21
sometimes it's fine to have no design. But make sure its very scalable, cause you don't
34:25
want to be the next Friendster. Except that Facebook wasn't very scalable and it was fine.
34:28
So make sure you have good design and design doesn't matter. It's scalable, but not too
34:32
scalable."
34:32
It's not very helpful advice and if you go down the list, it's like make sure you raise
34:35
plenty of money, but not too much money. Make sure you have the right kind of people, but
34:38
not those other kind of people, but actually, sometimes those other kind of people are fine.
34:42
And we focus on all this contradictory stuff 'cause for any particular piece of advice,
34:46
I can find you somebody who followed that advice and then made a lot of money. I can
34:49
also find you somebody who didn't follow that advice and made a lot of money. I can find
34:53
you people who followed that advice and made no money and people who didn't follow that
34:56
advice. I can find you all four quadrants of a logical possibilities chart.
35:00
So, how do we know what advice to take and what not? I think this is the heuristic you
35:04
wanna use. If it gets us through this feedback loop on a sustainable basis faster, it's a
35:08
good idea. And if not, not.
35:11
There's a lot more, of course, to Lean Startup. There's a zillion things on this graph. You
35:17
can read them all on my blog, Startup Lessons Learned. Of course, you can buy a certain
35:20
book. I've heard it's coming out in the fall. It's really good.
35:23
All of these techniques, like continuous deployment, where we put software into production, like
35:28
50 times a day on average. So, 20 minutes from the main trunk to production, no branches.
35:33
Things like net promoter score where we can evaluate in real time using a tracking survey,
35:37
what customers really think about our product. Everything you know about usability tests,
35:41
five whys, which is drawn from the Toyota production system.
35:44
Each of these techniques has -- they operate at one stage of the feedback loop, but they
35:48
have the net effect of minimizing total time. That's what the Lean Startup is about.
35:54
But I wanna mention one more really boring topic called "innovation accounting". See,
36:00
we've forgotten what accounting was designed for. I mean, we think of accounting as that
36:04
thing that the really boring people do to keep track of where the money goes, right?
36:08
That's pretty much what it is. It's just a ledger that says, "Where did all the money
36:11
go?"
36:12
But accounting was invented for a very different reason. It was invented to drive accountability
36:17
across departments. Because if you wanna have a large company with many different divisions,
36:22
you have to be able to hold the managers of those divisions accountable to some things
36:25
so that you know that they did a good job. General Motors, which invented most of our
36:30
modern management paraphernalia, had this concept.
36:32
When I first read this concept, I literally laughed out loud; I couldn't believe it. It
36:35
was called the Standard Volume. It was the ideal number of cars that General Motors should
36:41
sell, division by division, in an ideal year. And they actually had the math, and staff,
36:48
the macroeconomic staff to figure out, given all the macroeconomic data available, how
36:52
to translate the standard year into our coming actual year.
36:56
So they could go division by division and tell each manager, "Given that we're in a
36:59
recession, or the economy is booming, you should sell this number of Oldsmobiles. And
37:04
therefore, if you sell more than the standard number, you get a bonus. And if not, you failed."
37:09
And it's not fair if you didn't have that concept, then if it's a good year, all the
37:13
managers seem like they're doing well. And in a recession, everyone seems like they're
37:15
doing badly. You can't tell which manager actually made a difference.
37:19
Now when I read that concept, I laughed out loud because I was like, "Wait a minute. Are
37:22
you telling me there was a time when people could make forecasts about what was going
37:25
to happen in the future, and then it actually happened?"
37:29
I don't know about you. I have never in my whole life seen a forecast of anything that
37:33
turned out to be remotely accurate. No startup I have ever worked for has had a roadmap that
37:37
turned out to be remotely true. I have never seen a company say how many customers they
37:41
would have in the future and then deliver. Never seen it.
37:44
So to me, the idea of the standard volume is ridiculous. But I understand when you have
37:47
a sufficiently large company and you have a sufficiently long operating history, you
37:50
can do this. Maybe this sounds a little bit familiar.
37:53
So, if we're using accounting to drive accountability, but all of our accounting depends on having
37:58
a long operating history and a lot of customers, how do we drive accountability if there's
38:03
no customers yet?
38:04
If the CFO of a company, hypothetically speaking, gives a certain team a bunch of money and
38:09
sends them off to some remote location to do their work, like to Australia or something.
38:14
And then they hang out in Australia or whatever. And then a year later, they say, "What are
38:20
your results?" And the team says, "They're not very good, but we're on the brink of success."
38:25
How is the manager who gave them all that money supposed to know if A: They are in fact
38:28
on the brink of a success or they're just on the flat part of the hockey stick, or if
38:32
they've just been goofing off for a year? Or more likely, if they're just executing
38:36
a bad plan.
38:37
And at what kind of milestone should we hold them accountable to if we can't hold them
38:40
accountable to the gross numbers of customers? 'Cause that's fundamentally not fair. If we're
38:44
focusing on the gross numbers, incidentally, we might decide we're gonna do a lot of publicity
38:48
and PR and be like, "This thing is gonna be amazing," to drive customer awareness.
38:51
But we all know if you happen to find yourself on such a team, that that early awareness
38:55
is fundamentally lethal. But it's not fair to just say, "Well, just let them do whatever
38:59
and hope for the best." You guys know exactly how that turns out. So, what is the solution?
39:04
I think we can answer that question now with something I call "innovation accounting".
39:08
Here it is.
39:08
Instead of focusing on product milestones and gross numbers, we have three learning
39:12
milestones we can focus on. We have to take our attention away from the vanity metrics.
39:18
Vanity metrics are the numbers you put in a press release to make your competitors feel
39:21
bad. Like the total number of pages on the internet that you've indexed. I happen to
39:25
like that one a lot.
39:26
There was a time when we had a big index, number of in pages indexed battle. And it's
39:30
like, "We have four billion and you only have two billion." But like, what does that actually
39:33
tell you about the quality of somebody's business? Absolutely nothing.
39:35
They could be four billion really dumb pages. It could be one guy's website who's just really
39:40
excited about the number four billion. Or, it could be four billion people who each have
39:43
one crappy website.
39:44
If you read "TechCrunch", you're gonna see a zillion stories about "This company has
39:48
sent 400 messages through their platform." But is that 400 million people who are all
39:52
about to turn out, or one very excited customer? We don't know.
39:56
We used to have a competitor in IMVU that would report on the gross GDP of the value
40:00
of their whole user to user economy. And my CEO would sometimes come to me and be like,
40:04
"These guys have a four hundred million dollar GDP. What's our GDP?" I was like, "What does
40:08
that even mean in our context? If two users exchange some virtual currency, is that part
40:13
of the GDP? I don't know." It's a completely meaningless number, but it sure made us feel
40:17
bad.
40:17
I'm all for vanity metrics and press releases. Go to town. That's fine for making your competitors
40:21
feel bad.
40:21
But what happens when we use those numbers to guide our own business? Is that "when the
40:26
numbers go up, it's always because of what I was working on"? Everyone thinks I made
40:30
this feature last week and now the numbers went up. So obviously it's due to me. Of course,
40:34
the people in marketing feel like it's 'cause their new marketing campaign, etc. What happens
40:37
when the numbers go down?
40:39
Anyone ever been in that meeting? Oh, it's seasonal effects. Did anyone ever hear seasonal
40:43
effects used to describe numbers going up? Never in my career. It's always like, if it's
40:46
going up, it's features. If it's down, it's seasonal effects, or worse, those idiots in
40:50
marketing.
40:50
And over time, each us lives in our own private reality where the stuff I do makes numbers
40:55
go up and the stuff that those guys do make numbers go down. So is it any wonder that
40:58
we think each other are idiots?
40:59
Now, expand that organization larger and larger and larger as people are in ever more permanent
41:04
silos, speaking their own language, living in their own private reality. Is there any
41:07
wonder that they have trouble working together?
41:09
Maybe that sounds a little bit familiar. Okay. Just checking. So instead of that, we're gonna
41:15
use actionable metrics, which are about per customer behaviors, things that can be measured
41:19
at microscale.
41:20
And the first thing we're gonna do is establish the baseline. So now we can put the purpose
41:24
of the minimum viable product on a much more rigorous setting. Somewhere in our business
41:28
plan, there is a model that says, "Hey, if customers behave in this way, then we’ll
41:31
have zillions of them over time." And we can't get into all the details on how to build those
41:35
models. Of course, there's a great book coming out. You can learn all about it.
41:39
In the meantime, what we wanna do is just figure out what are the real numbers for each
41:42
of those inputs at microscale? That's what the minimum viable product is for. So, if
41:47
there's some number, some spreadsheet somewhere, that says, "Hey, ten percent of customers
41:50
who come to our website should register for our product." Then we should have a big banner
41:55
in our office somewhere that's like, "We must have ten percent conversion or we die." And
42:00
then, we each have a minimum viable product as soon as possible to find out what that
42:02
number is today.
42:03
And most likely, when you do that experiment, the baseline will be horrible. Like, it'll
42:08
only be one percent and it's supposed to be ten percent. And like, oh my God. In general
42:12
management, that provokes a crisis cause now we failed and uh-oh. There's this thing called
42:16
the "audacity of zero", which is how much easier it is to raise money and get people
42:19
excited when you have no results. Or having zero dollars of revenue in a startup is a
42:23
great time to raise money. Having one dollar of revenue is a disaster. 'Cause with zero,
42:28
it's like, "Well, why is it zero?" "'Cause we haven't launched." So, obviously it should
42:30
be zero. Everyone's like, "Oh, that makes sense."
42:32
God forbid you have one dollar of revenue, 'cause then they'd say, "Why is it only one
42:35
dollar? I thought this thing was gonna be an overnight success and now you're proving
42:39
to me that it isn't." So with zero you can always be an overnight success. With any other
42:42
number you're screwed.
42:44
But we need to change that. We need to say, "Finding out the truth of where we are right
42:47
now is progress. It's a milestone that we should celebrate." And then we do step two,
42:53
which is we tune the engine. We make product development changes that are not designed
42:56
to drive huge gross numbers, but to make those conversion numbers go from the horrible baseline
43:01
to the ideal in our business plan.
43:03
And whenever I've done this with teams, I've only ever seen two cases. Case one, it's supposed
43:08
to be one percent. It's one percent but it's gotta be ten percent. So, a few iterations
43:12
in, it's one percent, three percent, six percent, six and a half percent, seven and a half percent.
43:17
Now, it's not ten percent yet. So the model isn't exactly working, but you can say, "Are
43:21
we gonna get there?" Yeah, probably. Each thing that we do seems to drive the number
43:25
up a little bit. We seem to be heading in the right direction. We're split testing to
43:29
make sure that the changes we're making are in fact driving the change. It's all good.
43:33
Here's situation number two. It's one percent, three percent, three and a half percent, 3.75
43:37
percent, 3.8 percent, 3.81 percent. Now, the numbers are going up every time. So it's not
43:43
like the numbers are going down. It's not like it's zero. But you might ask yourself
43:45
a question four, five, six months into hitting that asymptote. Are we ever gonna beat ten
43:50
percent? I think it's safe to conclude the answer is no.
43:52
Of course, theoretically, it is possible. The next iteration will be that magic one
43:57
more feature that gets you to ten percent. But in reality, that's not the case.
44:01
When the team gets to the point where hitting that diminishing returns, everybody knows
44:06
you're not gonna make it and you enter the land of death march. So instead, I recommend
44:10
we do three. We schedule the meeting in advance. That three months from now, six, whatever
44:16
it is, we're gonna have a meeting to decide whether to pivot or persevere.
44:20
And by that meeting, we will have the data about whether our efforts to tune the engine
44:24
are working or hitting diminishing returns. And so, we have all these concepts in entrepreneurship,
44:29
like product market fit, that are very vague. This system allows us to put those concepts
44:33
on a much more quantitative basis. We can't turn whether to pivot into a formula.
44:38
I can't tell you what to do. I still rely on human judgment, just like science does.
44:43
But if we make specific predictions, if we use innovation accounting as our accountability
44:47
model, then we can be training our judgment to get better over time, just like in science.
44:53
So, don't do product development astrology. Do product development science. I left a bunch
44:58
of questions unanswered 'cause we only have a short time together. Like, how do you know
45:02
specifically when to pivot?
45:03
What's the relationship between our vision, our strategy and our product? What exactly
45:06
should we measure in each of the engines of growth? How is it that products grow? How
45:11
do we know if we're on that hockey stick, or on the long, flat part forever? How do
45:15
we test if we're creating value?
45:16
What specific features should be in the MVP? Can we go faster? The answers to these questions
45:21
and so many more are in the new book coming out in the fall, called "The Lean Startup".
45:24
You can, of course, preorder it at lean.st.
45:26
Thank you very, very much for doing so. I'll just give you my contact information. Please
45:30
be in touch if I can be helpful in any way. We have a brand new website, which is itself
45:34
a minimum viable product about theleanstartup.com. Please check it out. We would love your feedback.
45:38
And you are all officially invited to the Startup Lessons Learned conference, which
45:42
is gonna be May 23rd in San Francisco, but we also simulcast. Last year, we were in 50
45:46
cities.
45:46
So presumably we'll be in New York, too. I hope if you can make it, you will drop me
45:50
a line. And if this proves in any way helpful, I hope you'll email me and tell me about it.
45:55
Thank you all very much.
45:57
[applause]
45:58
So we have time for a few questions? OK, let her rip. If I stumped all of Google, I'm gonna
46:04
be pretty proud of myself. That's going right on Twitter.
46:07
[pause]
46:08
Sweet.
46:10
[pause]
46:11
>>Female Audience Member #1: So, I just wanted to touch on something that you mentioned is
46:23
reviewed too many times; the pivot.
46:25
>>Eric Ries: Uh-huh.
46:26
>>Female Audience Member #1: I have trouble understanding exactly how much work to put
46:30
in for the first pivot. I think the second and the third might be a little bit more,
46:36
OK, maybe not. [laughs] But just starting off, like how much work should you really
46:41
put in for that first pivot?
46:44
>>Eric Ries: There's no way to answer that question in general, honestly.
46:47
You have to put yourself into a position where the team will know if it's working or not
46:54
working. And the problem is that most teams have a plan, which is basically to ship it
46:58
and see what happens.
46:59
And the problem with ship it and see what happens, you can feel like you're being very
47:02
agile, but you're guaranteed to succeed at seeing what happens. And so, you'll therefore
47:09
always feel like it was worth doing and you'll feel like you're on the right track no matter
47:11
what.
47:12
The only way to get yourself into a position where you have to pivot is to make specific,
47:16
concrete predictions ahead of time that if they turn out to be wrong, will actually call
47:20
your theory into doubt.
47:22
And the issue is that we all know that most projections for new products are complete
47:25
BS. You have to tell the CFO, or whoever, that you're gonna have a zillion trillion
47:29
customers in year five. Otherwise you won't get the money to do your project.
47:33
But we know that we just made those projections up. So when they don't prove to be correct,
47:37
we're like, "Well, that doesn't prove that our vision is wrong. It just proves that it
47:39
took longer than we expected." So, yeah, the hockey stick is still gonna happen, but it's
47:43
taking longer.
47:44
If we do innovation accounting and we make very specific per customer behavior predictions
47:48
-- like one thing we'll often have people do is sell the magic version of their product
47:53
on a landing page somewhere.
47:54
And it's like, don't even say what the product, like how it works. Just say the benefit that
47:58
it gives you and see if you can get people to sign up. If people won't sign up for the
48:01
magic, they're certainly not going to sign up for your product, 'cause magic is always
48:05
better.
48:06
And if magic isn't even good enough, if the conversion rate on magic is too low, then
48:10
you already know that you have a problem. Not that that means that therefore give up,
48:13
go home. It just means there isn't already enough latent demand for what you're doing.
48:17
And so you're gonna be in a different kind of market then maybe you expected. Does that
48:21
help? So the minimum viable product truly is the minimum, the least amount required,
48:26
to get that first information. It's not, "Oh, if it doesn't turn out into an overnight success,
48:31
we give up."
48:32
And if you release that pressure to get it right the first time, like, I feel like a
48:36
lot of us feel like we have to do this circus act to make it seem like we could predict
48:40
the future. Like, one of those brilliant visionary, the next Steve Jobs. Not even Steve Jobs is
48:44
as good as Steve Jobs. That's a story that we've all been told.
48:47
And every company I've worked with internally, there are these genius heroes who always seem
48:51
to be able to get it right on the first try and everyone else is trying to emulate. But
48:54
when you meet the hero, you're like, "How do you do it?" If they're being honest with
48:58
you, they're like, "I actually don't know."
49:01
Or, "Actually that's not how it happened and it wasn't as easy as everybody thinks. Now
49:04
I feel incredible pressure to replicate that success again." And so, you can imagine the
49:10
negotiation that happens with the superstar over their next project. They don't ever want
49:13
to be in a position where they do something and it doesn't work, or they'll have to quit
49:16
and go to convince some other company they're a superstar.
49:16
I hear that happened recently. Anyway. Is that helpful?
49:16
>>Female Audience Member #1: Yeah.
49:16
>>Eric Ries: OK. Any other questions?
49:16
[pause]
49:16
>>Male Audience Member #4: So most of the rapid feedback you're talking about seems
49:18
to be very applicable to consumer applications where the cost of trying something in the
49:28
amount of time it takes to try something is pretty low. I will or will not sign up for
49:37
Twitter or Facebook or whatever the next thing is.
49:39
>>Eric Ries: Yeah.
49:42
>>Male Audience Member #4: How does it apply if it's a bigger thing? If it's an enterprise
49:50
sale or if it's a bigger commitment, do you just basically take the same process, but
49:55
the time scale is longer because the commitment process is longer for each step?
49:59
>>Eric Ries: Yes. I mean, that's the short answer. I mean, the long answer is my background's
50:05
in consumer internet. So I talk that way just naturally about large sample sizes and split
50:10
tests. Just the way that I, I can't help it. That's my whole career.
50:13
What's funny is that Steve Blank, who I mentioned earlier, has a great book called "The Four
50:17
Steps to the Epiphany", that I hope you all read. When he talks, his background is in
50:20
enterprise software. He gets this question too, all the time. Except when he gets the
50:23
question, people say, "Well, of course, this will work in enterprise software. But how
50:26
will it work on consumer internet?"
50:28
Because we just assume that the tactics that have worked in one industry don't work in
50:31
another. So the answer is truly, it's the principles that matter, not the tactics.
50:35
And so, the principle of the build-measure-learn feedback loop is cross-industry. So for example,
50:40
in consumer internet, because we're used to having large numbers of customers, we do all
50:44
this analytic stuff as a crutch. It's actually, it's actually -- it's because we have it worse
50:49
than the enterprise people.
50:50
When you have a lot of customers, you can't get to know any of them particularly well.
50:54
In fact, you just have, you have to rely on archetypes and summaries and assumptions.
50:58
When you have a small number of customers, it's a huge asset because you can know each
51:02
of them extremely well and the experiments that you do can have much higher fidelity.
51:05
So like, physicists don't get to ask electrons what they're thinking about doing. They're
51:08
not available for comment.
51:10
But when you're studying something that can actually interact with you, it's a whole different,
51:13
whole different ballgame. Question over here? Yeah.
51:18
>>Male Audience Member #5: What product should Google be pivoting on right now?
51:22
>>Eric Ries: What? What product should Google be pivoting on right now? Listen, I would
51:23
never presume to tell Google anything about that. I mean, look.
51:26
>>Male Audience Member #5: You were invited.
51:27
>>Eric Ries: What's that?
51:27
>>Male Audience Member #5: You were invited.
51:28
>>Eric Ries: I was invited. But look, but seriously. Like, the outside expert doesn't
51:32
know anything. You guys know your business way better than I do. And you know what products
51:37
need to be pivoted. You know. You don't need me to tell you.
51:40
The better question is, how on Earth are we gonna get to pivot them? Because we're stuck
51:45
in a management and accountability framework where pivoting is failure is a problem. So
51:49
for example, a certain Google product that I know especially well because it was competing
51:56
with something that I was working on at the time, I won't get into too much detail.
52:04
One of our cofounders at IMVU went to work on this product for Google. So, you're Google
52:08
people. You can talk about this. So they had a lot of inside information about what we
52:12
were doing available to them. And at first, we were really nervous because it meant that
52:16
we were gonna face competition from Google.
52:18
Here's what happened. Google spent two years working on that product. Spent, I can't imagine
52:24
how much money developing it. And when it finally came out, they launched it with a
52:27
big bang, put the Google brand right on it. And then when some things didn't go as expected
52:32
and it turned out to be an embarrassment, like, it got pulled and killed.
52:36
And the whole time, we were like, "What is going on over there?" 'Cause we were iterating
52:39
and changing constantly over that whole two year period. By the time they launched the
52:42
product, we felt like they'd launched a product that did what our product did two years ago.
52:47
And by giving it the big launch hype, putting the Google stamp on it, the inevitable problems,
52:52
the same problems we had had when we launched, but we were able to pivot because we had a
52:56
pathetically small number of customers and no one had ever heard of us.
52:57
That's a huge asset. It's actually a relief. 'Cause it means you can screw up. Nobody knows.
53:02
Nobody cares. Obscurity is a benefit. By putting the Google name on it, by claiming it was
53:06
the next big thing, Google put that team, I assume, in a position where there was no
53:10
way for them to pivot. It became an embarrassment to corporate, or somebody, and the thing got
53:14
pulled.
53:14
Now, [pause] that product could have pivoted. It was a really good product. It had a lot
53:19
of really good things about it. There was no physical reason why it couldn't pivot.
53:22
But two million dollars, two years and millions of dollars in, with all these expectations
53:27
and the Google name, I can't imagine the pressure they were under. I felt bad.
53:33
So, the right question is, what could Google have done to build that same product and put
53:37
it in a position where it could have pivoted? I actually think that leadership in today's
53:42
economy means creating platforms for experimentation for your teams.
53:47
I borrow that phrase from Scott Cooks, founder of Intuit, who's been a big doctor of Lean
53:50
Startup.
53:51
So if you want to be a general manager in a big company like Google and you want Google
53:55
to be more entrepreneurial, my point of view is it's on you, not on your team, on you to
54:00
give them a safe sandbox for experimentation.
54:02
So for a risky new product, do not, I would never put the Google brand name on it. For
54:07
shame.
54:08
And I would never talk to the press about it at all, if you can avoid it. And I would
54:13
try to have it have as small a number of customers as humanly possible while still doing that
54:17
innovation accounting.
54:18
Then, teams will pivot just fine. Nothing wrong with Googlers. It's a management problem.
54:22
In my humble opinion. Yes.
54:23
>>Male Audience Member #6: So let me follow up with that directly.
54:25
>>Eric Ries: OK.
54:26
>>Male Audience Member #6: Let's say you write a new mobile application and you post it to
54:34
the--
54:34
>>Eric Ries: Hypothetically speaking.
54:34
>>Male Audience Member #6: Yeah, yeah. The app store or the market. If you do that at
54:38
Google, with the Google name in a blog post, you'll get a hundred thousand downloads no
54:42
matter what it is.
54:42
>>Eric Ries: Yeah.
54:43
>>Male Audience Member #6: It can be almost anything. But it's true. And by virtue of
54:47
that, you are now, when people browse to categories, shopping, personal finance?
54:50
>>Eric Ries: You'll be number one.
54:51
>>Male Audience member #6: you will be up there. You will have that visibility and those
54:55
will lead to clicks and you will be featured.
54:56
And you will jump start in a way that you can't do if you're nobody and you put up an
55:00
app out there. You may very well just be in the noise and you'll never break out of that
55:02
no matter how great the product is.
55:06
>>Eric Ries: Right.
55:07
>>Male Audience Member #6: So, I mean having done this twice--
55:10
>>Eric Ries: Entrepreneurs never ask hypothetical questions, by the way. So I understand.
55:14
>>Male Audience Member #6: Yeah. Yeah. So I can tell you there's real value. I mean,
55:18
I know the successes that I've had are in large part because of having that name attached
55:20
to it.
55:20
And that initial jumpfrog, the initial leapfrog made up for, was market that I couldn't have
55:26
bought.
55:26
>>Eric Ries: Yeah, look, you get free marketing, but so what? Free marketing is worth so much
55:34
less than we think. Like, yes, that accelerated your process of success, but only because
55:37
you had the right product and it worked for your customers.
55:40
So putting rocket fuel into a rocket that has problems with it is not a good idea, right?
55:44
You accelerate too fast. The thing blows up. Now, sometimes you achieve liftoff because
55:49
you actually do have the right product.
55:52
But marketing, marketing dollars are not as hard to come by as they used to be. The really
55:56
great products today have an engine of growth that allows them to grow organically anyway.
56:01
So, yes. Google has huge advantages of putting their brand name on it. It can get you a lot
56:04
of downloads. But also, there's a huge liability about that because--
56:08
>>Male Audience Member #6: Well, that's a good question then. So why is it not OK for
56:13
Google to fail? Why not take risks and just fall on your face?
56:15
>>Eric Ries: Listen.
56:15
>>Male Audience Member #6: and admit it quickly and move on?
56:16
>>Eric Ries: If it was me, I would be celebrating failures and I would make those people heroes.
56:20
If it was up to me, but that's the culture that I live in now.
56:23
But see, failure is not what we want to celebrate. We wanna celebrate successful pivots away
56:27
from failure. And that's challenging because it's easy.
56:31
If you're charismatic, you can get resources for anything and you can ship stuff and get
56:36
people to use it and then you can engage in a ton of what I call "Success Theater", to
56:40
try to make it seem like you're a big success. And so, those people tend to get the celebrations,
56:45
whether they actually add any value or not. They're like a cancer on most organizations,
56:49
in my opinion.
56:51
But what do we do instead? We don't want to celebrate those people. We don't wanna just
56:54
celebrate people who fail because they accomplished nothing. So we have to have a system for evaluating
56:58
which failures were actually instructive and then we have to celebrate the learning and
57:02
what the learning turned into in the next try.
57:05
So yeah, if Google corporate was fine with people failing and having the Google name
57:09
be associated with nasty stuff, that'd be fine. But I just think that's unrealistic.
57:13
I mean, I wouldn't be that comfortable.
57:15
If I was Google, I would be launching those things under a different brand name altogether.
57:18
I wouldn't tell anybody that they're made by Google until they're actually proven to
57:22
be viable.
57:23
How do you think Apple does it. The stuff that we all consume and wait in line for,
57:27
we're the first people -- the first person who buys an iPad at the Apple store is the
57:30
first person to have used the iPad? Come on. That's my humble opinion. Helpful?
57:32
>>Male Audience Member #6: I didn't follow the last line.
57:36
>>Eric Ries: Then maybe I shouldn’t have said it.
57:40
[laughter]
57:41
Cancel that. I do think giving teams a platform for experimentation, having clear analytics
57:45
for testing whether they're making progress or not, and celebrating pivots is the answer.
57:51
Whatever Apple does, who knows?
57:52
>>Thomas Sharon: All right. Last question.
57:53
>>Eric Ries: Make it count.
57:54
>>Thomas Sharon: Or this was the last question.
57:59
>>Eric Ries: Too much pressure? It's a Google-branded official question. It's going on the internet.
58:06
[pause]
58:06
>>Thomas Sharon: All right then.
58:06
>>Eric Ries: Ok, thank you all very much. I appreciate it.
58:07
[applause]