Pandora Radio and Predictive Analytics

The fall lecture series on disruptive innovation wrapped up at Linda Hall last night with Nolan Gasser, the chief architect and musicologist behind Pandora’s Music Genome Project.

The talk was good, as all of them have been, though maybe a bit less organized around a central theme. There was quite a bit of dabbling in advanced music theory (that left many attendees scratching their head) as well as some very accessible music playing, from Coltrane to Zeppelin to Pachelbel. Clearly, Glasser possesses a unique genius.*[1]

Amidst the offshoot thoughts spurred by the talk were some audience questions about predictive analytics. Pandora is, at root, a big pop culture analytics experiment. Each track is scored on a scale for 100s of attributes. There are millions (billions?) of tracks. Those are a lot of data.[2]

Three thoughts on these data and how they are analyzed:

1. The human element has not been automated.

Each Pandora track is coded by a human being. I found this really surprising. And sort of gratifying. I remain suspicious of (though optimistic about) highly automated analytics, especially social media monitoring and natural language processing programs. That the Pandora model, especially given its mathematical and theoretical basis, has not completely handed the data inputs over to an algorithm made me feel better about my suspicions.

2. There are critical differences between things that can’t be predicted…

Someone asked if the model could be adapted to predict future radio hits. Gasser was doubtful. It’s a great idea, and maybe it is possible, but it’s important to be realistic about the limits of your data, your algorithm, and your knowledge of the human framework that underlies all our decision-making.

3. …and those which can be predicted.

The beauty of Pandora radio, of course, is that it predicts what songs you might like and adapts its model based on your inputs. Pandora works not simply because it has a massive, static database but because that database is modified with individual inputs.

Another place where Pandora succeeds is in measuring its audience analytics. Gasser mentioned briefly at the end that Pandora has really redefined online marketing with its ability to deliver targeted advertising. I would love to hear more on this topic.

(Aside from analytics, this talk opened up a whole bunch of questions for me on aesthetics. Gasser works from the premise that we are biologically hard-wired to like music, and that there are mathematical properties that make us like some music more. Yet he is unwilling to reduce taste to an equation. I asked him a question about this, not very well, but there was one piece of his response that I particularly liked. Basically: “We’re all hard-wired to laugh, too, but each person’s laugh is different.” I thought this was an interesting take on the matter of taste. But I’d love to ask some follow-up questions on this topic as well.)

[1] Maybe all genius is unique…a topic for another day. Anyway, he played lots of music on a keyboard, along with a few clips from his laptop. But the versatility and the music were both enjoyable.

[2] Sorry, I take small pleasure in data being a plural.


Comments (1)

Video Analytics Are the New Future

In a post a couple weeks ago, I referenced the SKC Tech Summit as an example of a conference outside my field by trade but inside my field of interest. Though it wasn’t an explicit theme, the idea of a “video revolution” certainly loomed large. Topics like telemedicine and virtual education were highly visible throughout the event.

Market research and customer insight was not—although companies like TASKE emphasized an analytical approach to call center software that included rudimentary service satisfaction surveys. Not market research, per se, but a good indication of how integrated insights are adopted by non-traditional market research departments.

Video has been a staple of focus groups for years, of course, and focus group facilities have mostly used advancing technology to help present the work to those who can not be there in person rather than change the way the work is done. QualVu does a great job innovating around video—exploring new techniques and making it more accessible, more diverse, and more entertaining on the report-out side. But as the name suggests, it continues to be primarily qualitative.

What struck me at the SKC conference was the potential for qual-quant hybrid techniques through the usage of video analytics. There are some facial recognition and eye tracking software programs out there that edge towards this idea, but I’m talking about teaching computers to analyze visual motion data, rather than just text/voice or still images.

Considering that text analytics and still image analysis are still immature, I expect video analytics to be at least two technology leaps away—but it’s not too early to get in position to lead in this area.

Take large scale in-home ethnography as an example. QualVu does in-home “ethnography” on a small scale. I put ethnography in quotes, because the respondent is pretty interactive with the camera—it’s more about them showing than you observing. And small scale because the analysis is intensive enough (and qualitative enough) that traditional qualitative sample sizes are both effective and practical.

But imagine setting up three video cameras in 1,000 kitchens for a week. Or two days a month in different households for a tracking study. Quant sample size. Multiple cameras to capture different angles. The client could be a CPG company, a grocery store, a cookware company—you name it. Sophisticated visual analytics could potentially tell you:

  • What’s your morning coffee routine?
  • How much time do the kids spend in the kitchen?
  • How many of those fancy knives do you actually use?
  • Are you drinking wine while cooking dinner?
  • Are Ziploc bags a replacement for Saran Wrap or Tupperware?
  • How many trips to the pantry are required for each meal?

Yes, you can ask things like “Do you clean as you go or wait until the meal is finished?” on a survey, but that sort of misses the point of ethnography. You don’t always know the right question to ask. It’s less about testing hypotheses than uncovering latent needs and motivations. Pervasive video makes observation of people in their natural environments possible in a way that it has never been before.

“Pervasive video” immediately suggests Big Brother (uh, maybe not this one) and privacy concerns. Big Brother or no, privacy has changed. I won’t even get into the respondent confidentiality aspect that consumes much of the traditional market research world. The fact is, individuals have shown an increasing willingness to broadcast their lives that shows no sign of abating, regardless of what Chuck D thinks.

The technology, on the other hand, in not there yet. Talking to people at Cisco, video analytics is on the radar, but not yet a high priority. The more pressing analytic need is to search for text in video, search-engine style. This technology—think automated transcription—is available but still a bit clunky. And of course, creating the broadband and hardware infrastructure to easily facilitate web-based video transmission is a project that is also just underway.

But as we know, technology moves quickly, and I have no doubt video analytics are coming. A five- to ten-year time horizon would not surprise me.

And once we have sophisticated visual analytics, the Big Data explosion will reach another order of magnitude. Potential applications, that may be easier to imagine than voluntary in-home surveillance, include:

In-store video cameras that can provide comprehensive insight into the shopping process, and combined with web analytics, give great insight into online/offline integration for retailers

  • Evaluating B2B performance and client relationships as videoconferencing skyrockets
  • Mobile streaming/life casting—which is already happening but will become more comprehensive as video becomes easier and cheaper to upload

And for those outside the Kansas City area, know that Google is launching a Fiber network here in 2012 that promises to make uploading (and downloading) video much, much faster.

What other applications do you see for insights and analytics as video becomes more prevalent? How far off do you think we are from video analytics?

Leave a Comment

Google, Pandora, and Digital Cameras: Innovation at Linda Hall

Up until about six months ago I had never heard of the Linda Hall Library. Despite its location in the middle of the UMKC campus, it is actually one of the world’s largest privately funded and operated libraries. Founded in the 1940s with a gift from the grain Hall family (rather than the greeting card Hall family), it is expressly devoted to science, engineering and technology.

The Linda Hall Library sponsors an annual lecture series, and the theme this year is innovation. I caught a couple of the spring lecture series on the future of innovation, and I’m really looking forward to the next series, which seems incredibly timely and—given the lead time it surely took to book the speakers—prescient.

The title of the fall series is This Time It’s Personal: Innovation in Your Home. This theme fits perfectly with the Google Fiber initiative—which has established Kansas city as a test market for what ultra high-speed Internet in the home might mean.

The opening lecture, by the author of The Googlization of Everything (And Why We Should Worry), will take a more guarded approach to Google, in contrast to much of the communal giddiness about the corporate giant’s affection for Kansas City.

Additional lectures are, at least nominally, about the invention of the computer, the digital camera, and Pandora Internet radio. From my experience at the spring lectures, I would expect these topics to be a starting point to explore some pretty interesting and provocative issues around the ideas of innovation and discovery.

And the speakers behind the topics are no slouches—the guy talking about the invention of the digital camera is actually the inventor of the digital camera and the Pandora talk will be given by Pandora’s “Chief Musicologist” emeritus, the guy hired in 2000 to architect the Music Genome Project. (The computer talk is given by novelist Jane Smiley, who also happens to have written a book on the topic.)

I’m not sure how these events are typically marketed. Ivy League alumni clubs were certainly involved in the spring, and that reflected in older, tweedier crowds than I typically see at innovation-based events around town. Remarkably, all these talks are free and open to the public. Tickets are required, and apparently they do sell out. Put it on the calendar, and remember Linda Hall.

And it’s worth noting that, even if you can’t make it to the lectures, the Linda Hall is worth a visit for the rare book room. They have an enormous collection of rare and first edition scientific books dating back to the 15th century. And you can actually handle them and read them! Seriously, if you’re any sort of bibliophile, this place is an absolute treasure trove.

Comments (3)

Authenticity Is a Two-Way Street

Eric Melin and Mike Brown have a couple good posts on Scott Monty’s visit to SMCKC last week, which was really a pleasure to be a part of. He deftly mixed in social media truisms (“business strategy, not social strategy”) with original and inspiring campaign executions.

But a couple things struck me beyond the straight social components of the presentation.

First, Scott was the consummate brand representative for Ford and served as an unusual example of the critical relationship between social media and authenticity.

Usually, “authenticity” in social media means that a brand lets its hair down and interacts with people as people. It means cutting the corporate brand-speak and actually engaging. As Scott himself pointed out, people want to be spoken to like human beings.

Neither Average Joe nor hipster guru

Still, you never got the feeling that Scott Monty was the average guy, just keeping it real with the customers. Nor did you feel like he was some hipster creative marketing guru. You felt like he was Ford—a precise blend of heritage, comfort, forward thinking, and approachability. But also that he was genuinely, authentically Scott Monty.

Companies always want to hire good people, but in a social world, hiring the kind of people you want to be is more important than ever.

Another thing that stood out is how Ford uses conventional market research tools in addition to digital metrics to measure the effectiveness of social media campaigns and understand how they work.

Surveys may be out of vogue in a world of sentiment ratings and Klout, but Ford measures trust, quality perception and favorability ratings to understand how social media can have an impact beyond the sliver of its customers who follow @FocusDoug on Twitter or Like him on Facebook.

If social media truly is intended to support broader business strategy, it’s important to take a holistic view of insights and analytics, and it’s great to see Ford really taking that to heart.

Leave a Comment

Showing Up in Unexpected Places

It can be a great learning experience to go where you don’t belong.

I attSKC Logoended a Tech Summit sponsored by SKC Communications a couple weeks ago. I don’t need AudioCodes’ Networking VoIP hardware or TASKE’s call center management software. I felt just a little guilty talking to vendors on the floor hoping to make a sale or meet a prospect.

But I loved the exposure to a whole new set of products and ideas that might not directly touch the work I do, but spur new ways of thinking about how maybe they should. I enjoyed hearing a vision of the future from companies like Cisco and Avaya.

Unified communications has huge implications for market research, insights and strategy—from analytics to CRM to surveys and focus groups. I’ll write more on that in a future post.

But the most important lesson is how much you can learn by stepping outside of your professional niche and spending some time learning about other cool things, simply following your curiosity and taking advantage of odd opportunities and happenstance meetings.

Comments (1)

Three New (Old) Reflections on Data and Analytics

The advance of technology is often perceived as the new continually replacing the old, both in terms of products and human expertise. Implicit in this Inc. Trend Watcher piece is the ability of technology to foster mentorship and knowledge transfer between “old school” experience and young talent. (h/t to @RockhillStrat)

Key quote from “NextGen” market researcher Tom Anderson reflecting on the 2011 MRIA Conference: “We need to become more than traditional researchers while retaining the methodological principles which have served us well for many years.” (emphasis mine) These principles are a huge value-add to new wave analytic approaches, but the key is effectively communicating that value.

@joegermuska posted this critique of McKinsey’s health care study with a caution to journalists to be more stats savvy. But the caution is warranted for anyone who interprets and relays data. There is a lot of upside to Big Data, but a lot of potential for misinformation as well. And you don’t necessarily need to be a stats geek to navigate data, but you do need to have a firm grasp of what to look for to make sure your data is saying what you think it’s saying.

Leave a Comment

Curation, The Mix Tape, and Digital Immigrants

I’ve been preoccupied with curation lately, and it has struck me in a number of different contexts—like the other day listening to Pavement’s Slanted and Enchanted.

My first introduction to Pavement was on a tape a college roommate made—an early ‘90s indie rock compilation. This guy was a careful master of the mix tape, sometimes by label, sometimes by artist, sometimes by mood. Everyone used to make mix tapes—a cultural trope that defined a generation.

People still make mix tapes, of course, but they’re no longer tapes. I compiled some songs for my family last Christmas—downloaded a handful mp3s I didn’t already own, click-and-dragged together a playlist, burned it onto several discs. The whole process took maybe an hour.

I don’t know how carefully kids these days organize their musical tastes, how much they tailor the compilation according to their audience or the desired results. There may be just as many quality “mix tapes” out there now as there ever were. But they float in a much larger sea of playlists, Pandora stations, and iPod shuffles that make things easy but not necessarily better.

The ability to easily access whatever we need, or to explore something new without even knowing what we want—these opportunities offer endless possibility. But they don’t necessarily reinforce the discipline of sorting, prioritizing, organizing, and composing.

These are the challenges of the information age: identifying high-quality inputs, matching them to the right needs, and presenting it all in a way that makes it relatable to your audience.

And there is no shortcut to cultivating the skills needed to meet them.

Leave a Comment

More on New Data Sets (A Nod to Gary King)

The previous post on curation referred to the enormity of data that continues to be available for processing. Among the many useful bits of info in this presentation by Harvard prof Gary King is the run-down he gives on slides 21 and 22.

While his approach is certainly academic, the company that has sprung from his work–Crimson Hexagon–certainly is some evidence of commercial application.

The other key points for me are the need for tech and human methods of analysis to be used in a complementary way. So much momentum seems to be toward tech solutions, the human role (the curator, if you will) is paid too little heed. In fact, that’s how the linked presentation ends:

“Will we wait to be replaced?” Or will we adapt?

Leave a Comment

What Can Market Research Learn from Journalism?

Market research and journalism have a lot in common. The requisite curiosity, persistent investigation, and knack for storytelling are threads that connect my own biography—from college newspaper editor to history grad student to researcher by trade.

The journalist, of course, occupies a pop culture space of much greater visibility; and the savvy researcher can look there for hints of the future.

This recent Mashable piece on curation struck a particular chord for me. The basic premise is this:

“Over the past few weeks, many worries about the death of journalism have, well, died. Despite shrinking newsrooms and overworked reporters, journalism is in fact thriving. The art of information gathering, analysis and dissemination has arguably been strengthened over the last several years, and given rise and importance to a new role: the journalistic curator.

With a torrent of content emanating from innumerable sources (blogs, mainstream media, social networks), a vacuum has been created between reporter and reader — or information gatherer and information seeker — where having a trusted human editor to help sort out all this information has become as necessary as those who file the initial report.”

There are some important parallels, the most important of which is an increasing load of content that is user generated, free, and growing exponentially.

Like consumers of the news, many businesses are ill-equipped to manage the torrent of information that is flowing their way, learning on the fly how to use the bevy of new tools available to help manage it. Like consumers of the news, businesses increasingly expect cheap information and have a hard time evaluating the quality of the source.

In terms of skill set, the best researchers should be able to incorporate curation pretty seamlessly into their portfolio. The very words “curate” and “research” suggest the combination of art and science that has defined market research as a discipline. The ability to apply quantitative discipline to qualitative learning (and conversely, to explore nuances of data in an unstructured way for deeper insights) is critical to using Big Data.

But it requires a shift in orientation, and a different paradigm of what you can and cannot control.

The shift in journalism has taken a painful toll on many of the employees as big media companies struggle to adapt. The research industry historically operates a bit behind the curve.

What else can researchers learn from what’s happening in the media biz?

What Can Market Research Learn from Journalism?

Comments (1)

No Man’s Blog on Problems with Social Media Monitoring

Nice post here by Asi Sharabi on some of the shortcomings of social media monitoring services.  If you sum up his analysis to “they don’t work like they’re supposed to, they take too much time, and they’re too expensive,” it comes off as a little trite.  He didn’t sum it up like that, of course, but that’s the gist of it.  I haven’t seen as many services in action, but I don’t think he’s too far off-base.

The comments here are worth reading, because a few things emerge.

1) He gets little argument.  Comments tend to agree with his anaylsis.  It’s unclear whether it’s a surprised agreement or recognition of familiar but unarticulated sentiment—I think a little of both.

2) As Jeff Scott points out, given how many monitoring companies have responded, “we know they’re drinking their own kool-aid.”

3) It’s a nascent technology.  Sure there will be growing pains, but data are there and knowing how to use them is going to be important.  You’ve got to start somewhere.

4) What really sparked my interest was the observation of the intersection between market research and social media monitoring, both in the problems they address and the tools they use.

How the value proposition can be worked out remains to be seen, but I think monitoring services have utility, even in their current state.  I have not seen a service that will autonomously deliver meaningful results, but in the hands of a capable user, even the current technology—with all its limitations—can be valuable.

I thought Brian Johnson really nailed it:

You didn’t really state what your goals in using those platforms. While there are use cases for monitoring (crisis management, directly engaging influencers, etc), I believe that the real value is thinking of social media like a massive dataset, much like what CRM has evolved to. It’s an incredible dataset, when you sit down and think about it, offering based on the sheer volume, authenticity, and real-time nature of the data. There is amazing value to be had by performing in-depth analytics on that data and using it to inform strategic marketing decisions. Far greater than simply counting how many times your brand is mentioned, and whether it’s good or bad.

Two examples  from a couple local (Kansas City) vendors (who weren’t mentioned in Asi’s hit list).  I’ll say upfront that these reflections are impressionistic.  I don’t have deep experience with either, but I’ve learned a bit about them, and here’s what I was left with:

Social Radar/Infegy’s game is mass data aggregation.  By collecting feed data (as in RSS) and warehousing it, they are able to maintain a remarkably consistent database.  Will it still include spam?  Sure.  Will it miss some important (non-RSS) conversations?  Yep.  But the theory is that you’re accumulating so much information and in a consistent manner such that trends over time will still be meaningful.  It’s not foolproof.  I’m unaware of any studies indicating that variation in spam conversation correlates with actual conversation, but it also seems a reasonable hypothesis.  There will be error in any dataset; acknowledgement and consistency seem a step in the right direction.   Though such monitoring wouldn’t meet all needs, I can see how it would meet some.  The other advantage of the warehousing approach is that from the moment you start, you can look at historical data.  Social Radar seems like a good approach to quantitative analysis.

Spiral 16’s approach, on the other hand, achieves consistency by limiting the universe.  Whether their precise algorithm for identifying the right ecosystem is the best one, I’m not sure; but the case for doing a restricted search is compelling.  Find the relevant web.  Even if you miss some sites and conversations, monitoring an 80% accurate ecosystem can have value.  And categorizing types of conversations (traditional media, blog, video, etc.) seems especially useful.  There may be some back-end work to make sure these categorizations are accurate but once defined, there’s a lot of value in knowing how conversation about your brand is happening.

Leave a Comment

Older Posts »