Calculating your Marketing ROI – a practical guide

Calculating your Marketing ROI sounds super simple, right? You just take the income, divide it by the marketing expenses, and boom! – there’s your marketing ROI. Super simple – or is it?

Well, the truth is, when you google “marketing roi” you will find that there is a huge number of different definitions out there. So many, in fact, that Forbes Magazine has called this situation “ROI Anarchy”.

In this article, we’ll look at what a marketing ROI calculation should take into account in order to deliver meaningful insights to evaluate a campaign.

read more

How selection bias will skew your activity mailing results

Here’s an interesting thing that I came across when evaluating a mailing for a client that I need to share.

Fallacy01

The observation

So what we’re looking at here is the revenue for a given group of customers from 06/2014 to 12/2014. (Note that the data presented here is completely fictional recreated in a spreadsheet, but follows patterns similar to the original data. See below for more details.)

This group of customers had been selected due to its inactivity as defined by the revenue in 06/2014 being smaller then a certain threshold (again, the actual selection was a lot more refined that is described in this model).

The mailing was sent out on 01/07/2014 to the group of customers. So we’re looking at a total revenue of $46,837 in 06/2014 and a subsequent jump in revenue to $271,950 in 07/2014.

At first sight, that seems to be great news – the mailing worked, revenue increased drastically, everything fine.

However, when I looked at preceding months, the following pattern emerged:

Fallacy02b

So our group of customers had a high revenue 01/2014 – 05/2014.  Then in 06/2014 – the month that was used to determine whether or not a customer was active – the revenue suddenly drops by about 80%, only to be back at the original level the following months.

Now this seems rather odd. In fact, it looks a lot like there’s something wrong with the analysis.

But as it turns out, it’s actually perfectly correct. What we’re observing is due to what I call the selection bias.

read more

How to tell if your results are significant – a practical guide

Marketers frequently face a situation like this: In a survey it is found that 57% of women prefer product A, while 60% of men prefer product B.

In this article I will show how marketers – using only simple statistical analysis tools available in Microsoft Excel – can quickly and easily decide whether or not they can draw meaningful conclusions from such a result, or whether they may be making fatal mistakes by interpreting random noise as valid data.

stats mofo

Marketers frequently face a situation like this: In a survey it is found that 57% of women prefer product A, while 60% of men prefer product B.

Some marketers will just go “Great, statistics prove that women prefer product A, and men prefer product B. We’ll market product A to women then and product B to men.”.

But is this really always a valid conclusion? Couldn’t it also be that the difference is purely coincidental? After all, we haven’t asked all people, but only a subset of people: those participating in our survey. So maybe if we took another sample, and asked different people, the results would be different? May well be!

Statistics to the rescue!

As is often the case, statistics can provide a solution. Before delving into the details, let’s look at another, more formalized example. Dice!

Suppose we take two dice, and we want to know if one of them is loaded, i.e., we want to know if one of the dice yields better values than the other. Let’s start by throwing them 10 times each. This is what the results may look like:

Example 10 dice

Well. What do we get? Let’s look at the mean value for each die. As a reminder, if the two dice were fair dice, there would be an equal likelihood of one in six for each number to turn up. More formally, the expected value would be 3.5 (=1/6*1+1/6*2+…+1/6*6).

So what do we get for our dice? For die 1, the mean is (4+4+4+…+3)/10 = 4.40, the mean for die 2 is (5+2+1+…+1)/10 = 3.20.

So is die 1 better than die 2? Well the average is higher, of course, but as you will intuitively suspect, 10 throws is quite a small number of throws to draw any meaningful conclusions.

read more

How to get 2,000 likes in 12h

Deezer UK today pulled quite a trick today: They got about 2,000 (from 499,508 to 501,503) likes in less than 12h.

So here’s what happened: this morning Dezzer tweeted to their 3,200 followers (@DeezerWorld (2,574 followers), @deezeruk (549 followers), @DeezerIRL (95 followers), but not by @deezer with 271,709 followers)

#Deezer is currently at 499,508 likes on Facebook. Go on, help us reach a half a million milestone. on.fb.me/orBrlU (→)

read more

Social media campaign for new Tom Waits album

The latest Tom Waits album “Bad As Me” (ANTI) has been launched last year with quite a remarkable social media campaign that deserves some credit:

[nggallery id=2]

Here’s the order of events

  • Aug 16, 2011: First announcement of upcoming news on Aug 23 (reminder on Aug 22)
  • Aug 23, 2011: Video clip with Tom Waits announcing new album release on Oct 25, contains snippets of new songs
  • Aug 30, 2011: Title track “Bad as me” free stream; pre-order CD, deluxe CD, LP
  • Sep 27, 2011: Single “Back in the Crowd” available as (paid for) download
  • Oct 04, 2011: Track “Back in the Crowd” free stream
  • Oct 11, 2011: Announcement: Full album for streaming available on Oct 17. Mailing list signup. Reminder on Oct 14 (“the only way you will be able to hear the new album (…) before it is released”)
  • Oct 17, 2011: Full album available for streaming; only available to people on mailing list (who received an invite code by email). People with invite codes can invite 5 friends (see below)
  • Oct 18, 2011: Interview on pitchfork.com
  • Oct 18, 2011: Invites code available until Friday (Oct 21)
  • Oct 21, 2011: “Listen with Friends”: Set of local record stores listening parties (US only)
  • Oct 24, 2011: Album available in stores (CD, deluxe limited edition CD (+3 songs), LP
  • Nov 01, 2011: Interview on npr.org
  • Nov 08, 2011: Video for “Satisfied”
  • Nov 23, 2011: Deluxe limited edition CD “available again”
  • Jan 03, 2012: “Best album of 2011” award by Metacritic
    (Unfortunately, as of today, the album is not yet available for streaming on Spotify.)

Here’s a few more screenshots from the invitations campaign:

[nggallery id=1]

read more