Boys with Toys

Along with others, I also read Clay Shirky’s Power Laws, Weblogs, and Inequality. However, unlike most others, my reaction to Clay’s newest gem was to go, “What a load of hooie”.

Don’t get me wrong, I think Clay’s sharp as a tack and smart as a whip. (Do I need any other weapon-like metaphors to make my point?) He’s a great speaker, and knows his technology, and loves what he does, and I respect that. But he has one failing in regards to his viewpoints as to social gatherings: he’s an elitist. He believes there will always be an ‘elite’ grouping within any society, something I don’t necessarily discount; however, from his writing and actions, he also tends to facilitate the mistaken belief that social groupings must follow fixed statistical patterns that support a static elite and that we must all behave as the statistics dictate. And I say, what a load of hooie.

Clay references Pareto’s work in wealth distribution, showing that 20% of the people control 80% of the wealth. He writes:

Power law distributions, the shape that has spawned a number of catch-phrases like the 80/20 Rule and the Winner-Take-All Society, are finally being understood clearly enough to be useful. For much of the last century, investigators have been finding power law distributions in human systems. The economist Vilfredo Pareto observed that wealth follows a “predictable imbalance”, with 20% of the population holding 80% of the wealth. The linguist George Zipf observed that word frequency falls in a power law pattern, with a small number of high-frequency words (I, of, the), a moderate number of common words (book, cat cup), and a huge number of low-frequency words (peripatetic, hypognathous). Jacob Nielsen observed power-law distributions in website page views, and so on.

Clay equates these versions of the Pareto curve with weblogging popularity, as a measure of weblogging elitism. In his first figure, which I copied under Fair Use and replicated here, he shows a curve that plots the number of incoming links as a function of popularity. This is, Clay assures us, a demonstration of weblogging popularity as mapped to a power law distribution.

figure1.gif

(Clay pulled the figures from NZ Bear’s old blogging ecosystem work, an effort that is now defunct, and still alive and well as the Blogging Ecosystem.)

At first glance, Clay’s diagram does demonstrate the traditional curve that marks both Pareto and power law distributions. However, Clay pulled his data from a tainted source, and then compounded the error by an extrapolation that hasn’t been born out in observed behavior.

First, the tainted data. NZ originally polled his ‘starting’ weblogs based on his own viewing patterns, which tend to reflect his warblogger interests. This created a bias towards warblogging weblogs. As NZ wrote at the time:

So, after a few days of screwing around with lots of different tools, I found a way to do it. The methodology, in a nutshell, is this: I started with a fairly large list of about 175 blogs; mostly, I stole from Instapundit and Vodkapundit’s lists, since they are pretty comprehensive, especially when taken together. Then, I built a process to do the following:

– Download the front page of each blog to my local machine
– Scan through each page and extract every link (URL) found in the HTML of the page
– For each of the original list of blogs, scan through the total link list and count how many links go to that blog
– Sort the list of blogs in descending order of their number of inbound links, and include the number in parentheses next to the blog link

NZ’s work was never based on the random sampling necessary in order to make a sound statistical measurement. Tainted data leads to a tainted statistic.

However, even if NZ’s earliest work had been based on this sampling, Clay’s extrapolation about ‘links’ forming a power law distribution is not borne out by an examination of the existing Blogging Ecosystem, which shows that the power law distribution tends to favor tools and mainstream media links over weblogs. Of the top ten link earners, only two, Scripting News and Boing Boing belong to webloggers. The rest belong to Moveable Type, Blogger, CNN, Google, and so on.

If we were to start with untainted data and then filter it to exclude anything other than weblogs, the results are not as static as Clay’s hypothesis would suppose. He wrote:

However, though the inequality is mostly fair now, the system is still young. Once a power law distribution exists, it can take on a certain amount of homeostasis, the tendency of a system to retain its form even against external pressures. Is the weblog world such a system? Are there people who are as talented or deserving as the current stars, but who are not getting anything like the traffic? Doubtless. Will this problem get worse in the future? Yes.

Using my own behavior as a guideline, perfectly acceptable if I view myself as a statistical subject, I started out linking primarily to the more well-known webloggers. However, over time, I found other weblogs and webloggers who I tended to read more and more and appreciate more than the so-called elite webloggers. Most of these people I met in my comments, and in comments on other weblogs. As I added more of these people to my blogroll, and linked to them in my postings, I tended to link to the elite bloggers less and less because I found that I just didn’t read them as much. In other words, as my experience level increased in weblogging, my reliance on linking to a set group of elite bloggers decreased.

If you look at my blogroll now, you only find a few of what can be termed ‘elite bloggers’ (if elitism is a measure of incoming links as measured in the Blogging Ecosystem and Technorati and elsewhere). My blogroll reflects what is an unmistakable human trait — my tastes have changed, my interests have matured, some people have quit, while others have gone in directions I’m not interested in pursuing.

What Clay doesn’t factor into the equation is that unlike Pareto’s work, based on a closed system with finite resources, weblogs are neither closed, and links are neither finite nor fixed.

Even without all these statistical games, Clay’s observations are just not borne out by practice. Quoting his conclusion:

At the head will be webloggers who join the mainstream media (a phrase which seems to mean “media we’ve gotten used to.”) The transformation here is simple – as a blogger’s audience grows large, more people read her work than she can possibly read, she can’t link to everyone who wants her attention, and she can’t answer all her incoming email or follow-up to the comments on her site. The result of these pressures is that she becomes a broadcast outlet, distributing material without participating in conversations about it.

Meanwhile, the long tail of weblogs with few readers will become conversational. In a world where most bloggers get below average traffic, audience size can’t be the only metric for success. LiveJournal had this figured out years ago, by assuming that people would be writing for their friends, rather than some impersonal audience. Publishing an essay and having 3 random people read it is a recipe for disappointment, but publishing an account of your Saturday night and having your 3 closest friends read it feels like a conversation, especially if they follow up with their own accounts. LiveJournal has an edge on most other blogging platforms because it can keep far better track of friend and group relationships, but the rise of general blog tools like Trackback may enable this conversational mode for most blogs.

In between blogs-as-mainstream-media and blogs-as-dinner-conversation will be Blogging Classic, blogs published by one or a few people, for a moderately-sized audience, with whom the authors have a relatively engaged relationship. Because of the continuing growth of the weblog world, more blogs in the future will follow this pattern than today. However, these blogs will be in the minority for both traffic (dwarfed by the mainstream media blogs) and overall number of blogs (outnumbered by the conversational blogs.)

What a load of hooie. Or as Dave Winer says, rightfully, and more diplomatically, Clay doesn’t understand weblogs.

What Clay doesn’t take into account is that many of the so-called A-List, or head bloggers, the ones that primarily link and comment, have always been the type of blogger who primarily links and comments. This isn’t a measure of their popularity as much as it is that’s how they started their blogging and that’s how they continue it. There are just as many webloggers who don’t have as many incoming links but are the “link and comment” type of weblogger.

This type of weblogging is a matter of preference, not time or popularity.

Clay also mentions that the ‘long tail’ of webloggers, those with the least links, will always be the ‘conversational’ bloggers. By this, I’m assuming that Clay means those webloggers who talk about their life, their interests and events in their lives, and who get into cross-blog and comment style of conversations.

What a load of hooie. I can’t count the number of times I read Dave Winer talk about what he had for dinner, or about his illness, quitting smoking, and his father’s illness. There’s been many a time I’ve gotten into cross-blog and comment debates with Dave, and others who are currently in the ‘Pareto head’.

In fact, about the only popular bloggers who never get into cross-blog or comment conversations is Andrew Sullivan and Wil Wheaton. To be honest, no real loss.

Looking at the top 100 weblogs in Blogging Ecosystem or Technorati — if you filter out the tools and the major publications, the vast majority of people in the top slots are all conversationalists.

A person not having comments does not mean they don’t get into conversations. Many a so-called non-conversational and popular weblogger has spoken up in comments, mine and others, more than once. I’ve even had a cross-blog conversation with the Great Pundit, Glenn Reynolds himself, a couple of times. Mark Pilgrim, Dave Winer, Anil Dash, Chris Pirillo, VodkaPundit, Chris Locke, Jon Udell, John Robb, Jason Kottke — these are ‘popular’ webloggers (as measured by incoming links in the systems that measure these sort of things) and you couldn’t shut any of these people up if you tried because they want to be part of the conversation.

Most of the webloggers with the highest incoming number of links thrive on conversation. It’s our drug of choice.

As for this “At the head will be webloggers who join the mainstream media…” This reminds me so much of the parable of the elephant and the six blind men. If you only read Glenn Reynolds, your view of weblogging is that webloggers link and comment and then get good jobs as journalists. If you only read Dave Winer, your view of webloggers is that they link and comment, write an occasional longer essay, and get a job at a prestigious university. If you read Doc Searls mainly, your view of webloggers is that they’re professional journalists who link a lot, but also write a lot and tend to lose things a lot (which is unfortunate).

If you only read Boing Boing, your view of webloggers is that they link and comment and write science fiction, which they publish online for free access. If you only read Jon Udell, all webloggers are technical.

But where does Mark Pilgrim fit into this? Mark’s an all over the board blogger and he’s a ‘popular’ blogger from the statistics. How about Big Pink Cookie? Christine is about as conversational and personal and connected with her audience as you can get, and she’s popular. What about Anil Dash? Boy, can’t beat Anil for getting in and mixing it up with his audience. Anyone forget the time when Anil took on Little Green Football? How about Steven DenBeste? How about Michele from Small Victory? Or Davezilla for that matter, who’s one of the most eclectic people in weblogging?

Do any of these people — do any of us — fit into the statistical cookie cutters that Clay used in his effort to bake us into his weblogger cookies?

(Speaking of which, since this is a weblog: if you were a cookie, what type of cookie would you be?)

Two years from now, if I were to write this again, chances are that I’d be using the names of weblogs that don’t even exist now. Why? Because two years ago, many of the weblogs I just quoted didn’t exist.

Clay’s extrapolations based on statistical observations about webloggers is not validated by the empirical behavior of the webloggers. Or, in layman’s terms: We blow Clay’s hypothesis all to hell and gone. Clay has too much invested in his beliefs in static social patterns to open his eyes and look at what we’re doing. And that’s okay because we’re too busy doing what we’re doing to be all that concerned.

Archived with comments at Wayback Machine

Get Used to Disappointment

A friend told me last week that disappointment is part of any friendship. I have to agree with him because any relationship between people that is something more than the barest superficial association is going to have times when one, or both, is disappointed in each other.

The same has to hold with our friendships we make with each other through these weblogs; a connectivity that is almighty strange at times, but basically boils down to human behavior, digitalized. If we become disappointed in people in real life, how can we become less so in the virtual? Virtual connectivity is a conduit, not a transformation device.

This last week has led to disappointment for me and others, which I take in some ways to be a positive, not a negative, experience. I’m finding that the people who I have set up to be bigger than life, are actually human, doing things and saying things I don’t like, or approve of. My heros had feet of clay, but I didn’t see the footprints until they stomped about, in big oversize boots, all over their weblog pages.

Well, didn’t that just blow my dewy eyed view of things all to hell and gone? About time, too. I was beginning to think I was the only imperfect being out here on the boards.

Jeneane has been writing about the loss of the Columbia and saying things that are truthful, but not necessarily easy. Things such as she doesn’t feel the sorrow others do at the loss of the Columbia crew; that she would keep pieces of the shuttle if they had fallen into her yard.

This was a disappointment for Liz, who wrote:

I was shaken, deeply, by this. I’m appalled by the belief that profiting from tragedy–no matter how removed you feel from that tragedy–is a legitimate expression of “capitalism.” I’m trying to imagine how Jeneane’s daughter would feel, years from now, if her “money for school” was acquired through the sale of this debris. I’m wondering if Jeneane’s belief that “anything that lands in her yard is hers” extends to human remains–heck, those are probably worth even more, right? Likely to fetch a bundle on ebay from collectors.

Why this makes me so angry, I’m not sure. I suppose it’s because it comes from someone’s whose writings I trust–someone who writes so beautifully about her relationship with her daughter, her frustrations with injustice. It’s hard to reconcile this self-described “slimey” statement with the person I feel as though I’ve come to know through her writing.

Jeneane responded to Liz with a frank, honest discussion, which I appreciated. She also apologized to me, saying:

To Shelley, I’m sorry for commenting on what were such beautiful tributes on your site. I know this is a deep loss for you because you believe in all that is space exploration, and because you have a deeper heart than you like to admit. I should have kept my insensitivity over here.

It is true that the loss of the Columbia was a very deep loss for me because of my passionate interest in space exploration, and astrophysics. (And because the loss of good people doing good things always disturbs me.) But Jeneane shouldn’t apologize because I provided a forum for comments, and she expressed her view.

If I wanted you all to agree with me, I wouldn’t provide comments, I would provide the following:

 Shelley, you’re so right, and smart, too.
 Shelley, you’re so right, and beautiful as well as clever.
 I agree with you Shelley, not as much as yesterday, but not more than tomorrow
 Shelley, I agree with you and I love you, marry me
 Shelley, I agree with you, I lust after you, have sex with me
 Shelley, you’re so smart. Run for president.
 Shelley, you’re so smart, I have a job that’s perfect for you and that pays a million a year.

Jeneane definitely bucked against the general sentiment with her statements. However, spending one’s time echoing the sentiments of the world around you, whether it be the sentiments of the country you live in or the sentiments tracked by Daypop, is a lie. We can’t all agree on the same things, feel the same way.

Even the clannish warbloggers have been disagreeing more and more lately, as different facets of each of their personalities, other than those associated with going to war against Iraq and other assorted general Arab countries, begin to surface. Just because you’re a warblogger doesn’t mean you’re a Buffy fan, or that you support Bush, or that you even agree with who should be bombed, and when.

This isn’t the usual metablogging crap — this is people who have come to know each other through a weblog, learning about deep differences in each other’s viewpoints and coming to grips with those differences. This is about as human as it will ever get here. Not pretty, not eloquent, not classy, and definitely not made up of people holding hands around the campfire in some great exploration of new connectivity (can’t you just hear the orchestra building with this one?). No, it’s messy, sad, disappointing, and real human behavior.

wKen wrote something recently that stuck with me a bit. He said the following:

So my basic rambling round-about point is that I rarely talk about bad things in my life on my blog, not because nothing bad happens to me, but because I don’t want to dwell on negative things. I deal with them the best that I can and move on. That doesn’t mean that I’ll never whine or complain about anything, but I try not to make bitching and moaning my main focus. It just isn’t productive, and also not very entertaining for others to read.

Don’t think I’m trying to tell anyone else what to write about, because I’m not. Blogville is big enough for all the bitching and moaning anyone cares to publish. I’m just offering a little bit of my own experience, and suggesting that it might work for some other people the way that it has worked for me. My life isn’t even close to perfect, but it isn’t half-bad either. So, while it may be cathartic to pour all of one’s sorrow online and have group hugs from virtual friends, making a habit of it may not take you where you want to go in life. I’m just saying…

I hear what wKen is saying — if one devotes one’s weblog to bitching and moaning, a person will never change their focus in life from the negative to the postive. Pool’s too big to only paddle about the shallow end.

If I disagree with wKen, it’s that I think spending most of one’s time talking about only the positive things in one’s life is a way of hiding behind your weblog, carefully forming the picture we want to show people, never quite showing the truth. Weblogging from behind a one-way mirror — I see you, but you can’t see me!

There’s no wrong in this, but I didn’t come here to read about saints. I came to read about real people, having good times and bad, telling us how they feel not what they think we want to hear. This means at times I’m going to get disappointed, because these real people are not going to live up to my expectations. Sadly, this means I’m also going to disappoint people at times, when I don’t live up to their expectations.

I imagine that the more ‘traditional’ webloggers, those who focus primarily on the dispassionate “link and comment”, must grow weary of the rest of us coming in, dripping real humanity over their nice clean monitors. All those footprints made of clay.

Archived with comments at the Wayback Machine

Once we believe in ourselves…

Another e.e. cummings poem has been coming to mind lately, spurred on by the discussions about the ‘proper’ amount of sorrow we should feel at the loss of the Columbia crew. Proper sorrow. What is that? I do not know what proper sorrow is.

cummings once said:

We do not believe in ourselves until someone reveals that deep inside us is valuable, worth listening to, worthy of our trust, sacred to our touch. Once we believe in ourselves we can risk curiosity, wonder, spontaneous delight or any experience that reveals the human spirit.

For me, I can’t think of a better good-bye to the Columbia and its crew (Rick Husband, William McCool, Michael Anderson, David Brown, Kalpana Chawla, Laurel Clark, and Llan Ramon) than that quote, and the following poem, If I believe:

if i believe
in death be sure
of this
it is

because you have loved me,
moon and sunset
stars and flowers
gold creshendo and silver muting

of seatides
i trusted not,
one night
when in my fingers

drooped your shining body
when my heart
sang between your perfect
breasts

darkness and beauty of stars
was on my mouth petals danced
against my eyes
and down

the singing reaches of
my soul
spoke
the green–

greeting pale
departing irrevocable
sea
i knew thee death.

and when
i have offered up each fragrant
night,when all my days
shall have before a certain

face become
white
perfume
only,

from the ashes
then
thou wilt rise and thou
wilt come to her and brush

the mischief from her eyes and fold
her
mouth the new
flower with

thy unimaginable
wings,where dwells the breath
of all persisting stars

e.e. cummings, from Tulips and Chimneys

A goodbye to Columbia, but never to space; and never to wonder and the unquenchable human spirit of the child within.

Archived with comments at Wayback Machine

Blanc Mange

If we look hard enough, we can find the lowest common denominator among us, and we can beat down the peaks and fill in the valleys and take comfort in the sameness among us.

And one spark of beauty, one ray of true art, can multiply, like the loaves and the fishes, to feed the millions. And when we shake the dirt of this ball of mud from our feet, no one will be left behind. We’ll all travel faster than the speed of light, because that is our destiny and destiny cannot be denied.

But there will be no room for difference on the flight, it will be crowded. We must all turn and breath in synch. That’s okay, though. As long as we’re all together. All the same, each holding our one spark of beauty, the last ray of true art.

So far left

Doc Searls has me pegged when he says that I’m a Lefty, though I agree with him that being libertarian is the only way to go with the Net. (Or better yet, anarchy on the Net, all the way.)

How far left am I? I’m so far left that I’d come full circle and run into Glenn Reynolds’ back if all those warbloggers kissing Glenn’s butt weren’t in the way.