jump to navigation

Digg and the wisdom of crowds June 29, 2006

Posted by dr. gonzo in Technology.
trackback

Lorelle got me thinking about this. So let’s wax philosophical about the wisdom of crowds.

From Evolving Trends:

“Crowds are not wise. Crowds are great as part of a statistical process to determine the perceived numerical value of something that can be quantified. A crowd, in other words, is a decent calculator of subjective quantity, but still just a calculator. You can show a crowd of 200 people a jar filled with jelly beans and ask each how many jelly beans are in the jar. Then you can take the average and that would be the closest value to the actual number of jelly beans.”

Of course, the folks in these posts are talking about social bookmarking, specifically Digg (Read the Evolving Trends entry–great example near the end of the post).

While crowds may have a compromised sense of wisdom in some cases, historical examples abound–look no further than Nazi Germany–modern Western society is essentially based on the wisdom of crowds.

If crowds hold no wisdom why has democracy become so popular?

It seems to me that after 200+ years of democracy in the United States that if it had failed completely we would have returned to the autocratic state of allowing one “benevolent” leader to decide things for everyone based on his/her individual preferences.

In the article on Evolving Trends Fawzi points out that:

“It’s my suspicion that the staff at digg, not just the users (we’re not gullible), have ‘buried’ this story, i.e. censored it, along with all the other anti-digg stories.”

This is somewhat pardoxical. The Digg community censors anti-Digg opinions, no surprise there. But on the opposite end of the spectrum, for instance a news site run by one gatekeeper/editor the information will be censored to fit the views of that individual.

So what to do?

There are obvious flaws in both models. I suppose the question is do we prefer a group consensus or the opinions of one individual?

Comments»

1. Lorelle VanFossen - June 29, 2006

Excellent commentary on the flaws in both the post and the models. It is always easy to think “conspiracy theory” when dealing with a few examples. If Fawzi had several months or even years of testing Digg to see the ebbs and flows in the Digg process, such as when most people check Digg, add to Digg, and hang out with Digg during a day, week, or month time period which impacts post traffic and attention based upon when it was released, and more comparative resources and statistics, then we’d have more information to make a better determination on the truth behind the “wisdom of the crowd”.

Personally, dealing with Slashdot, Wired, Digg, and other social bookmarking services and traffic drivers over the past few years, I’ve found that some posts sit for days before they “catch on”, while others I think will be awesome, even with “inspired” eye-catching titles, go flat. Others do amazingly well for the first few days, and then fall flat, only to boom again, and again, and again. I have several articles which have skyrocketed months later after someone “found” it again and submitted it to Digg.

There are people who are addicted to Digg and other social bookmarking services like it’s a gambling or auction. Let’s bid on the winner and help drive them to the top! Those “diggers” will have more influence over the crowd than the casual digger, but do they control things overall? Dealing with crowds is an iffy business.

Without more clear, statistical research, assumptions can only get us so far. True, in general, trying to make everyone happy all the time doesn’t get us much. The US democracy has become a process of listening to the masses and letting them decide, but only as much as the individual in control agrees with the masses. If the masses had that much control, abortion would never be threatened, nor would gay rights or gay marriage, women would get equal pay for equal work, and Hussein would still be running his county and the people of Iraq wouldn’t be happy but better the devil you know than the ones unleashed by war and terrorism.

Even those in power don’t trust the masses, but a Digg conspiracy? I don’t think they have that much time nor energy, and one thing I do know is that controversy brings a lot more attention than niceness and playing fair.

It will be interesting to watch this, won’t it?

2. dave - July 2, 2006

Apologies in advance if I am taken as a mad raving long-posted lunatic. But I felt almost obliged to respond to this post of yours for reasons contained in the last paragraph below.

The ‘Wisdom of Crowds’ is a book by New Yorker magazine columnist James Surowiecki — pretty far from being an academic paper or anything! Hey, to be fair to the guy, he doesn’t mean to say that wisdom comes from crowds or that all crowds are wise — or anything like that at all; in fact he gives a list of specific conditions that have to be met before a crowd can be called wise — and which would exclude the Nazis and the abundant historical cases you hinted at.

You make the leap from that to ‘democracy’… how bizarre! In the USA there’s been a type of government that has certain aspects that might be labelled as ‘democratic’ — some of the population can and do vote for one of whatever’s available for them to vote. Because of the state-by-state system, most people voted against the winner, and it matters little anyway. How does that relate to a wise crowd? Beats me!

You say “If crowds hold no wisdom why has democracy become so popular?” — Which is pretty funny because ‘democracy’ is nothing but a popularity contest!

Is it not true that, statistically, MOST people are stupid, most people are ugly, most people are going to hell — so what’s so good about being popular or part of the masses? You are either a leader or a follower, a boss or a worker, a have or a have-not. Bizarrely, isn’t the American Dream all about excelling out of the masses, being special, winning and so forth? STANDING OUT FROM THE CROWD?

Is ‘democracy’ so popular anyway? Well, name me two countries that have the same democracy then. Take the United Kingdom — there parliament is reigned over by a monarch who is also the head of the national church in a marriage of church and state. Her advisors are all referred to as Hons and Rt Hons, and she bestows knighthoods still. The House of (unelected) Lords includes bishops of the Church, and then there are small parliaments and assembles which have limited powers and different types of democracy — in NI they have proportional representation, for example… in Gibraltar and the Falklands (etc) they have no democracy, just a monarchy.

You say “It seems to me that after 200+ years of democracy in the United States that if it had failed completely we would have returned to the autocratic state of allowing one “benevolent” leader to decide things for everyone based on his/her individual preferences.”

lol! you are talking about modern politics — all of which are pretty recent experimental ideas — capitalism, communism, democracy, federal states — all within this 200 year period. For the thousands of years humans have been around the arrangements were more organic, more natural: imperialism. Even today, I cannot pass a children’s section in a bookstore without seeing stories of beautiful PRINCESSES and Kings and Queens — and that goes right across cultures too… show me a country or culture where they do not have that history!

Sociologically speaking, when conditions get bad enough, with a ‘spark’ or ‘trigger’ event comes a revolution — lol! maybe that is the ultimate wisdom of crowds (common sense), and that does not depend on democracy or indeed any particular form of government. Why do you think people march in protest rallies when they are supposed to have a working democratic system of governement? In that respect I trust the majority view, because I know human nature is always corruptible.

Presidents and Idols are made, music and products are sold, and the key reason why marketers can manipulate crowds so much is because of the first law of marketing: “Nobody believes they can be manipulated by marketers all that much.” — yep, it only happens to OTHER people. (http://tinyurl.co.uk/06as)

Kurt Vonnegut in his novel Cat’s Cradle’ coined the term ‘GRANFALLOON’ as meaning a group of two or more people who feel a bond because they share some circumstance that, beneath it all, has little to no real significance… believe it or not this forms the basis of a tremendous amount of very persuasive marketing campaigns. And Americans are especially susceptible — they retain their origins — Irish, Scottish, Greek, Jewish, whatever — and will actually trust and identify with people with a shared background!

But what of Digg, Slashdot and Wired? Nothing to do with democracy, but everything to do with THE SCIENCE OF NETWORKS.

About 30 years ago, Sociologist Mark Granovetter started it all with his ‘Strength of Weak Ties’ and Malcolm Gladwell’s book ‘The Tipping Point’. — lead to the present theories of Albert-Laszlo Barabasi, and Mark Buchanan. ‘The Wisdom of Crowds’ is a pop-part of this little-known academic field. (Surowiecki for example, knows it is possible to be too connected and too well-informed for inclusion to a wise crowd. He talks of connection cascades and power laws.) — Barabasi can explain how Google came to be the most popular search engine on the Internet and what it would take to dismantle the Al Qaida terrorist network. Whereas Buchanan explains the spread of infectious disease, how riots form, and why the rich always seem to get richer.

Myspace, Bebo, Bit Torrent, Digg and more all depend on the science of networks — therein lies their beauty, their real power and together with an understanding of the influence of the Rev Thomas Bayes controvertial form of statistics, you get an understanding of important they are in countering politics, media influence and marketing.

… er this post and your blog are part of that. :-)

3. dr. gonzo - July 4, 2006

Wow! These are long posts. I read them and will respond at some point, just busy writing a lot lately. Apologies dear readers. : )

4. evolvingtrends - August 23, 2006

You, Lorrell, the other long winded commentatir and myself are implicitly saying to each other (whether we agree or not) that our individual wisdom is worthy of being spoken.

I’m more concerned when that individual wisdom no matter how flawed gets replaced by crowd concensus.

It’s easier to point the flaws in each others differing arguments than to point the flaws in whatever we happen to agree on.

Humans are flawed but I’ll take my judgment over croiwd judgment any day.

:)

Marc


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: