A-B tests are not psychology experiments (and you don’t want them to be)

In July 1961, Yale University psychology professor Stanley Milgram sought to understand how far people would go against their conscience when following orders from an authority figure. Milgram was trying to understand the culpability of  Nazi soldiers and other accomplices to the Holocaust and ran an experiment where subjects (at the behest of a lab proctor) thought they were sending electric shocks to other subjects (they were not).

Apart from the amazing findings (that more than 60% were often willing to provide high levels of shock), the main takeaway was the physical and psychological toll taken on the test subjects who thought that they had inflicted harsh electrical shocks on innocent people. As a result of this and other questionable social science research projects, most (all?) educational institutions now require all research get approval of an institutional review board.

When I was in grad school and studying the efficacy of anti-drug advertising on teens, I had to jump through a number of hoops to ensure the safety and well-being of the participants. A key component of the whole process was one of informed consent, when dealing with minors, I had made things more complicated.

All of this background should be helpful in understanding why there is legitimate concern about Facebook performing a psychology experiment  with 689,000 of its users. The lack of informed consent and the apparent lack of institutional review would be problematic in dealing with 100 people. When you’re talking about six thousand times that many people being affected, it becomes a whole other story. Studying user behavior makes a lot of sense, but manipulating the news feed to purposefully provide more negative content leads it open to a lot of questions about the impact.

I’m sure that $FB shareholders would hope that their was some business objective from this study, but at this point that seems entirely unclear. In fact, seems more like a skunkworks psychology experiment because valuable business findings are rarely published in peer review journals. This clearly separates it from an A-B test because those have tangible business objectives.

But now that the argument has been raised that A-B tests are psychology experiments, new questions arise.
Should all A-B tests be subject to an institutional review?
What is the moral and ethical responsibility of computer engineers and web developers?
If their impact can be as great on a person’s well-being as a doctor or lawyer, should there be a code of conduct that needs to be followed?

My guess is that rather than having this debate, it would be easier to admit that A-B tests are not psychology experiments.

 

Hacker News commenter syndicate and other AngelList syndicates I’d like to see

Hunter’s post about the types of syndicates that he’d like to see got me thinking. And maybe it’s the Friday afternoon beer blogging, but if we really wanted to shake things up, here’s five syndicates that might do the trick.

Hacker News commenter syndicate: Look, anyone who reads the comments on HN knows these folks are the cream of the crop. They’ve never made a mistake, what they’re working on is much better than your business and they won’t be afraid to tell you if your marketing strategy is shit or if your site’s font is too small.

Google/Facebook/Twitter pre-IPO chef syndicate: They have more money than you and maybe they’d be willing to throw in some of that special sauce to help you get to IPO.

Guys named Kevin syndicate: Don’t you know a Kevin? Isn’t he a great guy and didn’t he have some good ideas for your last startup. Bring all those dudes together.

Tech recruiter syndicate: Don’t these folks have their finger on the pulse of what’s hot? They are always proactive telling you about the can’t-miss, sure-thing, Pre-IPO startup with an amazing technology and engineering-friendly workplace.

Non-sports fan who always wins the March Madness bracket syndicate: We all know these people. Don’t you want them investing in your company? Especially if you’re idea is as ridiculous as Villanova winning it all.

4 reasons your startup can’t get press coverage

Photo by Flickr user Tech Cocktail

If you’re like a lot of startup founders,  you started your startup with visions of getting in TechCrunch and then instant success. Chances are that it didn’t work out. You may have gotten coverage in one of the startup lists or maybe even the local paper, but still you don’t think that you’re getting the press coverage you deserve. There’s generally three reasons that you are not getting the press coverage you want.

Lack of crafted pitch

A well-crafted pitch is ideal to getting press coverage. I’ve read hundreds of pitches and you’d be surprised how many startups miss the important components of a well-crafted pitch. Here’s what a well-crafted pitch should include:

  • What type of news is this? Funding announcement? Product release? User milestone?

These are the three types of announcements that you can get the tech press to cover, probably in reverse order of interest. Funding announcements are bread and butter. Plan simple and if you raise enough, people will want to write about it. Product release is better than funding announcement because it has the additional value of actually providing useful news to the reader. A user milestone is even better, because it shows that people find value in your product ad actually USE it.

  • Will my readers find interest in this? Can you answer the “so what?” question?

“We raised $2.5 million for our photo-sharing app?” So what? “We raised $2.5 million to topple Instagram as the #1 photo-sharing app.”

“We’re launching our new photo-sharing app today.” So what? “”Our new photo-sharing app is the only to provide combine a sepia filter with an octagon shape.”

Company mission/vision and how your product is differentiated from competitors are important parts of a well-crafted pitch, make sure to include it with the hard facts of what you are announcing.

Lack of social proof

Reporters need to know that you are a legitimate startup that is worth writing about. Absent a very good funding announcement or some kick-ass user metrics you need some social proof as a signal to a reporter that you are with covering.

Did you get Peter Thiel to make an angel investment?

Did you work somewhere that everyone has heard of?

Were you in a well-known incubator?

Did you get Shaq to use your app?

This is probably where a lot of good startups hit a snag. Social proof is hard and fleeting. It has to be relevant and timely. Social proof also changes from reporter to reporter and probably day to day.

Inbox overload

Depending on the publication, reporters get anywhere from dozens to hundreds of pitches a day. As a matter of simple arithmetic, many reporters can only cover a small portion of the pitches they receive. If you’re a smaller startup hustling for coverage it’s simply hard to compete.

So in the end, you can do everything right and your email pitch might not even be read by the reporter who it’s perfect for.

You’re pitching the wrong person

The simple fact of the matter is if you’re sending your SAAS product pitch to a mobile reporter they will just put in the trash pile. They won’t helpfully tell you that you have contacted the wrong reporter, they don’t have the time. You simply need to make sure that your pitch is also sent to the appropriate reporter or you are wasting your time.

~~~~~~~~~~~~~~~~~

If you’ve read this far, chances are that you’ve done everything right. Nothing I said here should be new to startup founders who’ve done their research. That being said, you’re still not getting (enough) press coverage. Well, that’s a problem that we’re trying to solve at my new startup PressFriendly. So if you’re interested in getting more press coverage. Click here to sign up for our beta.

It’s time to call BS on the cult of growth hacking

I watched in amusement  as Andrew Chen called Growth Hackers the new VP of Marketing. Growth hacking (definition) is an important part of the marketing mix, but not a replacement for other aspects of marketing (content, SEM, PR, channel, partnerships, branding.)

I had a little chuckle when I discovered that a quick LinkedIn search showed that I’m connected to 116 Growth Hackers.

Then I saw this tweet:

I’ve seen growth hacking conferences and a few other e-books on the same topic, so a new book wasn’t a surprise. I’m a pretty skeptical person, so I clicked through to Ryan’s twitter profile to find out more about this person who presented themselves as an expert on growth hacking.

Turns out that Ryan probably isn’t a growth hacker at all. He is a tireless self promoter, who was a Director of Marketing for the sketchy American Apparel. Oh and he also worked closely with pickup artist and bro-author Tucker Max (who is, of course, also a growth hacking convert.)  So this is what growth hacking has become? A vehicle for self-promotion and the flavor of the day in bro-culture?

Even then, I was willing to let go and just laugh.

Then I came across this tweet and post:

Read the article, it’s horrendously hyperbolic, written by somebody who clearly doesn’t really know what growth hacking is beyond a few examples. It seems clear that Ryan hasn’t built products and considered the trade-offs enabling virality and a elegant user experience. His view of growth hacking doesn’t give a second thought to how to build sustainable growth for a company.

So what troubles me is that the notion that growth hacking = marketing/pr becomes adopted as gospel, especially among startup founders.

The fact of the matter is that growth hacking, when done well can be a very important part of a user acquisition/growth strategy.  Perhaps it can be your only method of growth early on. At a certain point however, you can’t rely upon growth hacking alone for user acquisition and you certainly don’t want to abandon PR just because you have a positive viral coefficient.

Growth hacking is not easy and has a lot of dependent factors. You need to make sure that your growth hack attracts the right type of user. You need to direct new signups to a user experience that engages customers and makes them brand loyal. My fear is that with the wrong people pushing growth hacking that it will become SEO 2.0. Spammy with an awful reputation. Unfortunately, we’re already going down that path.

We need responsible growth hacking from people who understand marketing and what it takes to grow and build a sustainable business. The notion that growth hacking will replace marketing and pr is crazy.

Let’s be clear, growth hacking will not help when an employee makes a dongle joke in front of Sheryl Sandberg and Marissa Mayer.

A SMS invite is not going to get the Fortune 500 CIOs to consider your product.

Signing a deal with the NFL is not a growth hack. It’s PR and traditional marketing.

Growth hacking is part of the marketing mix. It may be a big part up front, but there’s a lot of other things necessary to create a Dropbox, an Uber or an Airbnb. If someone tells you “growth hacking = marketing” then they’re selling you Kool-Aid from the Jonestown batch.

UPDATE: Somehow I missed this, but Semil Shah wrote a great piece on re-thinking growth. Link.

UPDATE 2: Link to the Hacker News discussion.