Category: Research

Thursday, 27 July 2006

Nielsen Analytics’ Economics of Podcasting

by Garrick van Buren 1 Comment on Nielsen Analytics’ Economics of Podcasting

Earlier this year, Nielsen Analytics surveyed a bunch of people in Las Vegas and found 102 people that described themselves as ‘regular podcast listeners’.

Thankfully, those 102 (77 men / 35 women) represented.

Some interesting bits

  • 61 people (50 men / 11 women) said they “always fast forward” advertisements
  • The survey found that the average length of the podcasts being listened to was 44 minutes. (Dave, isn’t that how long your walk is?)
  • The cumulative total of all monthly downloads for a given podcast series can hit 2 million. (No mention of how many were failed or actually listened to.)
  • Training, education, and other business-to-business oriented podcasts make more sense than entertainment podcasts.

Nielsen’s entire report is $1200 (I’d like to link it, but I can’t find a url that works). Now that’s how to make money podcasting.

Sunday, 25 June 2006

Project Launch: Minnesota Judicial Branch Website

by Garrick van Buren Comments Off on Project Launch: Minnesota Judicial Branch Website

I’m happy to see the new Minnesota Judicial Branch site launched this month (old one’s available for a while here if you want to compare and contrast). Looks like it’s a somewhat ‘soft’ launch – as they migrate content from the old to the new.

Last year, I helped the Court Information Office with the redesign – specifically in the information architecture. Their goal was common – consolidate a number of separate sites – in this case, 10 district court sites and the state court itself – into a single, easy to understand and maintain, information system. All while still being clear about which level of the site you’re at (state or district).

Like all information consolidations projects I’ve worked on, the first step is asking: “What types of information is everyone publishing today?”. In fact, the answer resolved many of the information architecture issues. Asking the sites’ visitors what they were looking for answered some more.

There’s two new bits I’m most pleased to see made it through to the launch:

  1. The find-a-district by ZIP code (also by map or County). Chances are you know your ZIP code more readily than your county or judicial district. Made it easy to use that as a key.
  2. The ‘How Do I….?’ – global menu item for the most common questions asked by Minnesota citizens (Adopt a child, Change a name, etc). When we asked visitors why they were at the site, the vast majority of their answers started with ‘How do I….’. Only natural to have a top-level section filled with those common questions.

Congratulations to the Court Information Office on the launch. I’m glad to see it out in the world.

Friday, 31 March 2006

A Use Case for Identity XML – Demographic Surveys

by Garrick van Buren 3 Comments on A Use Case for Identity XML – Demographic Surveys

Stowe Boyd’s running a reader survey. I’ve followed Stowe from Get Real to /Message and thought I’d check out the survey.

Standard demographic stuff; age, gender, household income, zip code, employment status, profession, internet usage, etc. Those common questions attempting to build an anonymous picture of people without actually getting involved with them.

Reading through the questions, the myth of blogs-as-conversation fell away. Stowe doesn’t know who I am – or you are – at all.

If he did, he wouldn’t need to ask these questions – because we’ve already answered them. All of us. Somewhere – if only at the BackBeat Media survey, or in our My.Yahoo.com.

I’d don’t mind giving this info. It’s just annoying to answer the same question twice. I’d much rather just point a URL at the survey.

In the same way I’d prefer to point a URL at my current photo than upload it _again_ to another website (43things, Stikipad, Eventful, or Amazon, Technorati etc).

I’m wondering if there’s an XML specification (or something like it) for the basic identity info all these surveys (read marketers) want. For example:

<BirthYear>1974</BirthYear>
<Gender>M</Gender>
<ZIP>55418</ZIP>
<ChildrenCount>1</ChildrenCount>

I could spin a file with this info, host it, maintain it, and provide brief glimpses into for the right price (so could you).

Yes, this is the Customer as Silo idea, feels like there’s some intention economy connection as well. No, I didn’t complete the survey.

Wednesday, 2 November 2005

Our Memories Are Poor, Measure in Context

by Garrick van Buren Comments Off on Our Memories Are Poor, Measure in Context

“It isn’t as astonishing the number of things I can remember, as the number of things I can remember that aren’t so” – Paine, Mark Twain: A Biography, 1912

This Sunday, the Vikings are playing the Lions. I’m confident they’ll be keeping score throughout the game. Yard by yard, play by play. Rather than having the refs remember who played better and declaring an arbitrary score on Monday morning,

There’s a huge, often un-acknowledged difference between what people say they do and what they do. This delta widens with time. If you’re looking for accuracy – metrics need to be captured in context. If you’re looking for fiction – then making up what someone did two days ago is as valuable as asking them to remember (making it up just might be more interesting).

PodcastAlley, Podcasts.Yahoo.com, and a number of the other podcast directories offer ranking and voting for “your favorite podcast”. There are three problems with this;

  1. The metrics at different sites don’t know about each other – diluting the value of each of them. For example, what does it mean to be #8 at PodcastAlley and simultaneously #23 at Podcasts.Yahoo.com?
  2. All these systems rank unrelated podcasts against each other – just like Arbitron or Nielsen Ratings. The only people interested in how, for example, meatloaf ranks higher than lawn mowers are advertisers. This ranking doesn’t help the listeners’ enjoyment or the producers improve (in fact it could be detrimental to both).
  3. I still have to remember to vote, at one site or multiple (just like Arbitron). The action of voting and ranking is separate from listening, so I don’t.

I’ll let Mark Ramsey wrap this one up for us:

“The diary methodology is woefully inadequate to meet the challenges of measurement in our industry going forward.”

“If we want the advertising community to place any credence whatsoever in our measurements, then we are obliged to use measurement methodology which inspires credence.”

Wednesday, 19 October 2005

Mobile China Pilot Study Out From JaredResearch

by Garrick van Buren Comments Off on Mobile China Pilot Study Out From JaredResearch

Jared Braiterman, friend-of-a-friend I met when he was in Minneapolis on another research study, just sent a message on his latest project – Mobile China.

“Mobile China examines mobile technology and youth culture in the world’s largest technology market. The pilot study focuses on the role of generational change, gender and nationalism in shaping Chinese mobile usage. Mobile China is a multi-year endeavor.” – Jared Braiterman

He just published a brief 5-page visual pilot study at;
jaredRESEARCH Mobile China Pilot Fall 2005.

My favorite quote from it:

“Everyone knows that Chinese branded headsets have more radiation and lower quality.”

He’s looking for affiliates and sponsors, so if you like where he’s heading, drop Jared a line.

Tuesday, 21 June 2005

Anonymous Responses Are Useless

by Garrick van Buren 3 Comments on Anonymous Responses Are Useless

One of my current projects has a major survey component. The survey ends with:

“If you’d be open to follow up questions, enter your email address below.”

There’s about a 60 / 40 split on responses with emails and those without. The responses without email addresses have skipped questions, irrelevant answers, and are generally unusable. This is so much the case, that I’ve found it a better use of time to check for an email address first – then read the response.

It’s interesting that people comfortable with being contacted give useful answers, while those providing non-useful responses don’t provide a way to be contacted.

Conventional wisdom on requiring accountability has it backward. Accountable people want to be responsible for their actions. Those that aren’t don’t. Forcing it doesn’t change anything.

On a related note, perhaps my observation is related to Ben Hammersley’s explanation of why wikitorial died.

Wednesday, 23 March 2005

Fake Data to See if Anyone’s Paying Attention

by Garrick van Buren Comments Off on Fake Data to See if Anyone’s Paying Attention

When putting together a prototype for usability testing, it’s best to use realistic data. If you’re evaluating the readability of a search results screen, put in the actual results. If you’re evaluating a check-out process, make all the information throughout the entire process real.

Then, after, tweak the data just slightly. Make it humorous, make it unrealistic, throw in a knock-knock joke.

I’m a big fan of this. It’s an excellent way to find out what people pay attention to and what they completely disregard.

In his post, Amazon’s Time Machine, Seth Godin ponders:

“Why don’t they slip in ridiculous items or funny descriptions? It’s not like they’re going to run out of shelf space or have a problem with inventory.”

It’s an interesting question. Ridiculous items or funny will polarize customers. Some will love it, some will hate it. It’s a big company that can walk away from disenfranchised customers.

Sunday, 17 October 2004

5 Tips For Better Customer Interviews

by Garrick van Buren 3 Comments on 5 Tips For Better Customer Interviews

The easiest way to collect interesting, usable data from a research effort is to blend into the background of the subject’s life. Media journalists know this – that’s why they’re embedded in the presidential campaigns and in the military actions.

To give honest unselfconscious, response, subject’s need to be comfortable with researchers – as peers, as collegues, as one of them.

Susan Orlean describes this necessity in her recent interview on MPR’s Midmorning. For good data, she schedules at least a week to blend into the background.

The need to commit time to really see into someone’s life is echoed in Tod Maffin’s the Art of the 10-Hour Interview.

The projects here at Working Pathways move much more quickly than those in either of the articles. We are often charged with capturing usable data with less than hour per respondant. With that in mind, here are 5 tips we use for “becoming invisible” in under an hour.

  1. Make Good Small Talk.
    The weather, the traffic, a recent news item, the goal is to find common ground quickly. You probably share a handful of similarities -find a a couple. Share the joy of meeting someone new is this small world.
  2. They’re the Expert.
    Whatever you’re talking about, they know more than you. Chances are their situation and challenges in whatever you’re talking about are unique. Use the little you know on the subject to probe and show them you can speak the lingo, not that you could take over their job.
  3. Pay Attention.
    Everything is a conscious decision – body language, intonation, language, word selection, wardrobe, facial expressions, everything. Each movement betrays their personality and honest, unguarded emotions. Picking up and following up on these cues is where the good data lives.
  4. Always Be Curious.
    There’s nothing worse than an interviewer uninterested in the respondant. If you don’t need or want another interview, cancel it. Otherwise you’re wasting everyone’s time and money.
  5. Keep the recording devices out of hand and sight.
    This is not to be sneaky or misleading. As an ethical researcher, all participants should be aware and give concent to recording. Have an assistant be responsible for the recording equipment. This recommendation is so;
    1. You can focus all your attention on the conversation and not recording.
    2. Your respondant can focus on the conversation, not being recorded. Thereby reducing the chance of them being self-conscious or saying what they think ther researcher wants to hear.

Wednesday, 11 August 2004

How Not To Do Customer Research

by Garrick van Buren Comments Off on How Not To Do Customer Research

We do quite a bit of customer and employee research here at Working Pathways. From in-depth 1-on-1, deep dive, interviews to quick email surveys to observational studies – our expertise runs the gamut. Whatever the study, each participant involved is 1. screened and qualified and 2. receives some level of compensation for their time and insight.

With that in mind, here’s how not to conduct a customer research telephone study:

  1. Don’t tell recruits who’s sponsoring the study. That’ll just skew the data and hey, why do participants need to know anyway?
  2. Don’t tell recruits why you’ve called them. That’ll skew the data also, say ‘satisfaction with products you may or may not have used’.
  3. Don’t tell recruits how long the interview will last. Participant’s time isn’t valuable, we can use as much of it as we’d like.
  4. Don’t compensate them. They should be happy just talking to us.
  5. Be surprised when nobody wants to talk with you.

In our experience, the most insightful research comes from passionate customer, they want to share their experiences with your products. You won’t get the valuable stories through a dispassionate qualitative satisfaction survey on products they haven’t used. They only come out when you respect your customers and consider your time with them a business appointment – pre-scheduled on both parties calendars. Like a business appointment, in this mode – one party compensates the other for the interaction.

Friday, 11 June 2004

You Expect What You Pay For

by Garrick van Buren Comments Off on You Expect What You Pay For

Recently – a collegue recounted his experience selling a fully-capable product with a price less than half its competitors.

“After the presentation, the customer turned to the sales representative and asked, ‘What’s wrong with it?'”

This, more than 78 years after J. Walter Thompson’s research for Pond’s Cold Cream proved that price and quality are directly correlated in customers’ minds

Compare the potential customer’s – “What’s wrong with it?” – with the following from JWT’s 1926 Pond’s customer research:

“Reasonable in price, used by everyone, many women had begun to think that they could not be as good as creams that were more costly or that were imported.”

This strategy also works in the service industry, illustrated by this exchange between a collegue and an industry leader:

“How to do you get clients to buy into your recommendations?”

“Charge more.”