I’m currently undertaking research to probe demand shaping in the long tail. Interestingly, an HBR article presents another alternative to explain demand (rather than supply) effects of the long tail: network theory.

Duncan Watts and Steve Hasker, in Marketing in an Unpredictable World (you’ll need a subscription), explain why it’s difficult to build formulas to predict hits. Namely, social influence creates very complex interactions that are hard to parse and predict in a reliable way. This is interesting because, as Chris Anderson claims, the “free” digital supply of the long tail removes the barriers that shaped the markets of scarcity of old economics, leaving behind a market of abundance. Perhaps, beyond decreasing friction and cost of supply, new forms of communication technology have also turbocharged mechanisms of social influence.

The article is based on research published in Nature of a study of the roles social influence plays in driving aggregate consumer demand. The findings “suggest that the success of a particular entertainment product cannot be explained by any measure of intrinsic quality or even by ‘appeal’ — the fit between the product’s attributes and consumers’ preferences.” At face value this may be an artifact of the constrained markets of scarcity in that the supply is artifically small and therefore true demand is unable to emerge. While the study design does limit choice sufficiently that this may be the case, there is another paradoxical implication.

Matthew J. Salganik, Peter Sheridan Dodds, and Watts report in the Nature study: “when individual decisions are subject to social influence, markets do not simply aggregate pre-existing individual preferences.” What may be happening is that when social influence is apparent in a system of choice, true individual preference/demand is not revealed. This seems obvious enough — any high schooler can verify the claims of peer influence and social editing. However, this may cast shadows on the democratic nature of the filters, discovery engines, rating systems and aggregation platforms of Web 2.0.

Social networking, social ranking and rating, and other collaborative forms of content generation and communication may be governers to long tail and niche growth. If I make it easier for social influence to take effect — the most diggs on digg.com, the most listens on last.fm, or the most popular books on Amazon, all made more viral through email, IM, blogs, social networks, message boards, etc, peer influence may take on more powerful and demand-stifling forms.

Therefore, some of the same technology that is helping to create the long tail may be making it HARDER for individual’s to 1) understand what they desire and 2) act on that desire — due to hypercharged social influence mechanisms — those same systems and filters that drive demand into the long tail.

I’ve been watching Rockstar Supernova over the last five or six weeks. I was surprised by both the quality of the performances, but also how much I like the band members, notably Gilby and Jason. Then, as the votes started coming in, and I started to see who was in the bottom three in votes, I started to question the premise. Then, last night, Tommy Lee made a comment about the fans voting and buying tickets, so they had to listen to the votes.

I know there are issues with the voting system already–and there has been controversy around the system during American Idol seasons. I think there is another more simple question to be answered. Are Rockstar viewers, and more so, the voters, Rockstar Supernova fans? If the show’s live audience is representative of the viewer base, I’d say definitely not. If Supernova is looking for an Audioslave version of Motley Metal Roses, I don’t think they’re watching the show, and I surely wouldn’t trust them to be voting multiple times for their favorites.

I’ll keep probing on this one for hard numbers–but my take: Rockstar Supernova is good evidence of the disconnect between TV audiences, participatory audiences (go with your web2.0 badness!), mobile, and web users. Then lay a taste/preference overlay, ROCK, over all that. It’s a recipe for mediocre… but I guess it is good entertainment. Just not the way to do a rock audition.

What better model do we have for open sourcing (besides the cost of the books) than cookbooks? Programming books, not unlike the 242 at O’Reilly, proudly use the metaphor made famous by Betty*. Those books capture much of the commonly agreed upon principles, standard methods, and terminology… but they are just the beginning. Communities spring up around sharing and improving applications that begin with the cookbook recipes.

You’d think that a model that came long before the Web would have flourished by now, but real recipe hacking hasn’t really blossomed. Sure, you can search and store recipes, but the wiki or digg-like communities of participation haven’t formed in mature ways. What might a mature one look like?

The recipe would be divisible at the ingredient level–where the corollary could be tracks in a playlist. Each ingredient has an associated amount and state, like a track has length, bpm, artist, and album, at a minimum. These ingredients could then be built into playlists, or recipes, but also live in clouds of a folksonomy. This kind of organization could really allow for a community of learning, experimentation, and preparation optimization and customization.

Some of the sites worth checking out, if you’re in the recipe hacking state of mind:
CtrlAltChicken: Alex Albrecht of diggnation fame hosts a self-proclaimed geek cooking videocast
NPR: Food Hackers Make High-Tech Geek Cuisine
Wired: The Thermochemical Joy of Cooking
Cooking for Engineers
Microformats: Recipe Examples
khymos.org - molecular gastronomy and the science of cooking
My del.icio.us cooking tags.

*Funny enough, Betty is a persona, General Mills turning a persona inside-out way back when… not quite true to the Hacker ethic, yet…

A note from the author: I thought the content of this post would clearly indicate this was a work of fiction… but the web has never been good at capturing irony. It has been dugg… and rumors have begun. I must be watching too much Colbert. Regardless, I can dream of a better iTunes–one that takes advantage of many of the features something like last.fm could enable. Read on. And as you read, imagine the voice of Colbert reading this to you… Further updates will be added to the end of the post.

In a completely unexpected move, Apple insiders say they have quietly acquired the social music site last.fm. What can we expect from the acquisition? A user friendly iTunes. Here are some of the new features iTunes will roll out:

SmartTransitions
Track sequencing algorithms will tap in to the collective dj prowess of the masses. Sequencing of songs will be rated based on how many users play tracks adjacently in playlists, etc, with appropriate weighting of best rated playlists, most novel combinations, and other methods not yet disclosed.

GenreFolksonomies
No longer will iTunes tracks be chained to single categories. Users will be able to create multiple tags across all track variables, as well as at the album, artist, and playlist levels. This intelligence isn’t tied to individual users either.

SocialMeta
Users will be able to share meta tags on all tracks and music. Find groups and users who have meaningful tags–adopt them based on the same smart playlist selection criteria and up the intelligence of your track data in a few clicks.

iMixCasting
No longer are your iMixes just a vehicle to promote sales of files on iTunes. Make your smart playlists podcasts and streaming stations automatically. Users subscribe and tag your casts and stations.

PersonaSettings
The magic of the new algorithms won’t just be social, you’ll also be able to use them to gain better intelligence about your own listening habits. Let the software create your listening personas automatically, or define your own through mood, context, etc. All iTunes settings get stored in the personas including track display variables, equalizer settings, airtunes destinations, and playlist and track tag criteria/filters.

PodTagging
Another useful new feature comes via the iPod. If the iPod is your primary music player you’re currently not capturing a good amount of meta data for your tracks. New iPod software will allow you to quickly tag and rate anything being played–including video, podcasts, playlists, and tracks.

We’ll share more information as it comes in.

Update 2: A couple forums chat about this as well. Interesting take on this post. I’m lucky I didn’t post it during the week. This makes me wonder how the majority of people read blogs. That’ll be for another post.

I’m in the middle of playing with a service called Dapper. As I understand it at the moment, the app allows me to click elements on a web page–and works best if there a few similarily constructed ones–and use the information as building blocks. In other words, it’s a non-programmer interface for the data of the web–which then can be remixed, mashed, and grouped. It’s roll your own API.

So what’s the big deal? I’m not sure there is a big deal in that anything I’m doing with the service can already be done hacking the html, xhtml, or xml and writing code around it. However, it does democratize API building by inviting more people to hack the web. But here’s my cut on it: Nothing I create using this will be as good as a novice to expert could make using Perl, PHP, Ruby, Python, etc.

However, there is value to this–and in the Long Tail frame of mind, what publishing dapplications might do is lead real developers to create more robust and usable apis. Dapper could be a discovery engine for api demand–which then developers can use to create actual solutions for.

I may have more thoughts on this as I play with the service. But for now, if nothing else, it’s a wicked creative idea, if nothing else.

I’ve been alternating between fiction and non the last couple of months, churning through more business and innovation books than I care too.

The current non-fiction book: The Long Tail: Why the Future of Business Is Selling Less of More. I’m generating my own take on all this business. Anderson makes some good points–but I think he’s omitting a big part of the story. But more on that later. I’m conducting some research to test my hypothesis–with some interesting results early on.

In the fiction world I’ve just finished The Geographer’s Library and have started in on David Mitchell’s latest, Black Swan Green: A Novel.