Year: 2004

In comments to my post,WeblogTweaks: Tool Independence Jacques Distler wrote:

There are things that one can do in WordPress that one cannot (easily) do in MovableType. And there are things one can do in MT that one cannot (easily) do in WP.

The day that our tools become interchangeable is the day that one can truly say that innovation has died.

If the developers of weblogging software (and, in the case of MT and WP, the legion of plugin authors that surround them) are not producing cool new features that no one else has, then they are not doing their jobs.

It is deeply wrongheaded to hope for (let alone celebrate) the homogenization of this technology.

Leaving aside Jacques’ equating of tool independence with homogenization, I want to focus for a moment on his underlying assumption about the importance of innovation to most webloggers.

Two or three years ago, innovation was the heart and soul of weblog tool development primarily because most of the functionality to support weblog writing in early times was fairly primitive. In addition, a significant proportion of the people who weblogged then were just as interested in the nature of the medium, as they were in publishing their thoughts online. New innovations almost invariably broke new ground, and generated excitement that caused ripples felt throughout most of weblogging environment. This excitement, in turn, fed the innovation. as those who developed the gadgets and whatnots received plentiful feedback, encouraging them to yet more feats of development magic. It was bewildering and exciting and like flemings to the sea, we flocked from tool to tool based almost solely on innovation.

But then the innovations became to develop costs, such as over-aggresive aggregators that queried RSS feeds by the minute; or spammers blitzing our comments. Seeing our names at the top of a Google search begin to lose it’s novelty, and we stopped paying much attention to articles that talked about weblogging. The demographics of weblogging has changed and today’s average weblogger is less interested in wizbang technology or the coolness of what we’re doing then in having a tool that upgrades without breaking; allows us to reliably write to the weblog and to do so easily and quickly with a mimimum of problems; have the resulting page look good; and not have to wake up every morning and spend time cleaning out spam.

This change in priorities regarding our tools is reflected in recent discussions. A few years back, most discussions about web technology were on this new gadget, and that new functionality–it was all very positive and exciting and even the most technology shy webloggers would be moved to participate from time to time. For the last year and a half, though, most of the buzz about weblogging technology has focused on comment spam and elimination thereof. The ragged echo of voices you hear are tired, and frazzled, and sound more like beseiged castle dwellers holding back a foe, than brave adventurers into new territory.

As for excitement – the last major excitement related to weblogging technology was when Six Apart introduced it’s licensing scheme with 3.0, and Dave Winer closed down weblogs.com. Neither of these events had anything to do with ‘innovation’.

Webloggers are just not as interested in shiny and new as they were at one time. Shiny and new attracts attention most of us aren’t sure we want.

Now, I’m not saying this to discourage innovation, and couldn’t even if I wanted to–the nature of the human beast is such as to always see what is done, and how it can be made better. But most of the innovation in weblogging lately has been directed more at the geeks than based on any real and expressed need of weblogging tool users. This is well and good, too, as most advances in technology are based on putting new and geeky technology out on the market and then showing the people how this is something they wanted all along and just didn’t know it.

But at some point, there must be a connect between the technology and the users, or the innovation is bound to fail. History is littered with the desiccated remains of brilliant ideas that no one wanted.

But I digress. Returning to Jacques equating weblogging tool independence with tool homogenization, and from there, to the death of weblogging innovation. It’s funny, but his statement is an echo of something that I wrote once, a long time ago, about web standards. and Mozilla.

Challenge your assumption that all Internet services are provided by a Web server and consumed by a browser. Challenge your assumption that chaos within a development environment is a bad thing. And challenge your assumption that standards must take precedent over innovation.
…
Somewhere along the way…standards became less of a means for providing stability and more a means of containment. In some cases, standards have become a weapon used to bludgeon organizations for practicing the very thing that started the growth of Web applications in the first place: innovation.

This old article I wrote for O’Reilly was inspired by the the WSP’s criticism of the Mozilla team’s focus on creating an infrastucture rather than just providing a standards-compliant browser. Stop wasting time, they said. Give us standards compliance and then you can go play.

Thanks to the Mozilla group’s resistence to the outcries of the time, we now have Filezilla for FTP, and Thunderbird for email, not to mention Firefox and all it’s wonderful, marvelous extensibility to which I have become addicted. That time spent on geeky stuff back then has born fruit now and we’re all giddy with the possibilities.

This would seem to support Jacques statement of the importance of innovation not being suppressed, but but not how tool independence leads to homogenization, which in turn suppresses innovation.

Because of it’s focus on infrastructure, Mozilla did lose users, people who switched to other more standards compliant browsers; however, this didn’t stop it from continuing to innovate. And Mozilla eventually did develop that standards-compliant browser the WSP wanted. Now, many of those people who switched earlier are returning to Mozilla, or more specifically Firefox, leading other browser vendors to look more closely at their own products.

If anything, innovation is more closely associated with vendor independence than dependence. After all, you don’t have to work as hard with a captive audience (beer vendors at ballgames have long understood this association).

Making it difficult for webloggers to change tools does not encourage innovation. If anything, more of Six Apart’s recent innovative work–such as dynamic PHP-based pages–came from people switching or threatening to switch, to other tools, than from happy customers politely putting in enhancement or bug fix requests.

No, all that tool dependence does is make people frustrated because they feel trapped into using one tool. Eventually, even if the tool they’re using does improve, they won’t see it, or even acknowledge it because that feeling of being ‘captive’ to the tool becomes a mighty powerful filter that influences their preception of the tool from that point on. Microsoft’s experiences with Windows is a good example of this.

But can tool independence limit innovation, as Jacques implies? No more so than with any other use of technology. Take the washing machine industry for example.

A washing machine’s basic functionality is very simple: provide a waterproof tub; fill it with water; agitate the contents of the tub a bit; empty the water and spin the tub to get some of the excess water from the clothes. All washing machines share this same functionality.

Now, Washing Machine A, in order to capture more of the market share, develops a new machine that allows different temperatures. This generates a lot of interest because we all know that red socks washed at hot temperatures dye other things in the wash, pink. However, the creators of Washing Machine A, in their enthusiasm, rushed their product to market, and it breaks down a lot. Some customers, frustrated by sopping up water on the floor, or holding up shredded nighties, move to Washing Machine B, which has less temperature options, but is more reliable. New customers, hearing that A, though nifty, breaks down a lot also decide to buy B. Other people, though, who have a lot of red socks and really need these many different temperature options, and who sleep in the buff anyway, stay with A.

Still, Vendor A loses customers. Alarmed by this, A improves the reliability of their product. It finds and closes the tiny black hole that is eating up one of the red socks with each load; they slow down the agitation of the spin cycle so that the clothes aren’t beaten until their threads begin to disintegrate.

Some of Machine A’s old customers that switched to B move back to A because of the increased reliability. New customers, hearing that the old problems have been fixed, buy A this time instead of B. While this is happening, though, Washing Machine vendor B, inspired by A’s customer’s interest in access to different water temperatures, and alarmed in turn about the loss of its own customers, adds an improved temperature control, but separates the temperature control by wash and by rinse – pretty exciting stuff.

Some of A’s loyal customers that stayed through the reliability issues, see ’shiny’ and ‘new’ with Machine B, version 2.0, and make the switch. So Vendor A adds a new innovation–a device that adds softener at the right time so you don’t have to do it yourself. Machine B counters by providing a glass front so that customers can stare at the clothes as they go through the cycles, mesmerized by the action.

About this time, vendor C, who has been focusing on televisions, sees there’s a lot of money to be made in dousing clothes with water and shaking them up a bit. So they enter the market, adding yet more incentive for innovation. Seriously alarmed now, Washing Machine vendor A, in a fit of inspiration, and encouraged by government tax rebates, introduces a new model that is very energy efficient, hoping to take advantage of ‘green’ interests. Unfortunately, the new model costs signifantly more than other models that are less energy efficient, but providing the same functionality, and A takes a real hit in the marketplace.

Vendor B also wants to have those government rebates, but when they introduce their energy efficient machine, they also load it with a bunch of new options so that people can see something new for their increased costs, above and beyond the energy savings.

Yet through all of this crazy innovation, and marketing strategies, the basic functionality of the machines is the same: water, movement, rinse, repeat. It is because of this shared common functionality that customers can switch between products, choosing the features they want, or the reliability they need, while being assured that this basic, needed functionality is provided.

Returning to weblogging tool technology: it is a shared functionality and a minimum data model between products that leads to many of the innovations we have come to take for granted. It is this that allows us to have syndication feeds that can be consumed by all aggregators regardless of tool manufacturer and innovation. It is this that allows one weblog to ping another, or to ping notification aggregators such as weblogs.com. It is this that allows desktop editors or email clients, or even cellphones to post to different weblogging products. And it is this sameness that can be exploited to make it easier for webloggers to switch to a different product.

Being able to switch relatively easily led to customer demands that led to competition among the washing machine vendors, and it can do the same with weblogging technology. It is customer demands that lead to support of multiple syndication feeds rather than just one; it is customer interest in, and demand for, comments that led to them being incorporated into most weblogging tools, including the recent addition of comments to Blogger; and it is customer demands that are now driving much of the work on comment spam prevention.

If enough tools support an innovation it becomes commonplace and hence a de facto standard; eventually it forces enhancements to the underlying commonly shared behavior and data between tools. This then ups the minimal level of necessary functionality for all products, and the innovative cycle begins again.

Rather than homogenize weblogging, by enabling webloggers to move relatively easily between weblogging tools, and encouraging them to so move when they’re unhappy with their existing products, we’re giving them a say in the the direction that technology takes–that intersection between innovation and use I mentioned earlier. This, in turn, leads to better products, leading to happier webloggers, who are encouraged to write more about things that interest them–such as washing machines, and what happens to all those socks that disappear.

The first challenge to moving a weblog is getting a snapshot of your weblog data in a format that can be imported into the new tool. To create an export that works with most tools, at least for moment, you’ll want to export your existing weblog’s data using the Movable Type import/export format.

WordPress doesn’t have a MT export built into the tool (more on this later); as I mentioned in a previous post, I used Scott Hanson’s new WordPress to MT export script to export my posts, categories, comments, and other data. Once I copied his file, I edited the parameters given in the script, providing the same username, password, and database name I added to the wp-config.php file. I also edited the file to use the default line break within Movable Type, __default__ , rather than ‘markdown’, which is the text format tool currently set as default in the import tool. These items are easily found in the script using a text editing tool.

When I load the export script page into a browser, the exported data prints out to the page, in a technique made popular by Movable Type. Once the export is finished (a message displays at the end of the page), I used the browser’s File/Save As functionality to save the page to a local file called bb.export.

(Another approach is to use the Unix wget utility on the command line as follows: wget http://somedom.com/import.php. This saves the exported data as ‘import.php’ and you can the use the data as needed. )

After the file was created, I FTP’d it to a newly created sub-directory, import within the MT cgi-bin directory. All that was needed at that point was to open up my MT installation, select import/export, and pick the import option–choosing to use myself as default author. the default post status of ‘publish’, and since all the entries had categories, I didn’t need the default category. I also have titles for all entries, so didn’t need to fill in the start and end title HTML fields, either. Clicking the link to do the import should load the data, and the data migration of the tool porting is finished.

(See the Movable Type documentation for more information about importing data into a MT weblog.)

A regular WordPress supporter, Carthik also has a version of a WP-to-MT export tool that uses WordPress global variables for the database variables. You can access it here. Carthik had his export tool finished before Scott started working on his, but had withheld publication because he’s now working on what he and Matt Mullenweg, the lead WP developer, are calling a “lossless XML export”–a import/export format that is going to be included with WordPress 1.3, and licensed as GPL for others to use if they wish.

One reason that the WordPress folks are creating this new format is that there has been problems with the existing MT format in the past. I have exported and imported data several times using this format and haven’t had issues recently, but others have had problems, specifically with fairly fragile points of breakage in the scripts such as dependence on a dashed line to separate entries. When I first used the import format to move from Blogger to MT, the import kept stopping as it would run into a sequence of dashed characters and the import functionality thought, “Well, that’s it – she’s done.” Once I edited for this problem, another would surface, making my move from Blogger to MT the most painful tool move I’ve done.

However, there is no denying that the MT export/import format is the most widely supported format in most weblogs. To have tool independence, in this case you need to depend on a specific tool import/export format…at least until enough vendors can support a replacement.

Technical issues of clean transformations aside, a challenge with a new universal format is the underlying data model each tool shares. For instance, a ‘post’, that has 0 or more ‘comments’ and at least one ‘category’ is more or less a standard model of data across all weblogging tools. However, beyond this simple core model, each tool does differ widely.

For instance, Movable Type supports keywords but not key/value pairs. Keywords are just a listing of terms associated as values to the weblog, while key/value pairs have both a term and an associated value. WordPress supports key/value pairs, and I use these in the ‘about this entry’ box in the top-left of the page. When I moved the data to the Movable Type test site, I lost this key/value pairing. Even if a new export format included these key/value pairs, there would be no place to receive it in the target weblogging tool–in this case Movable Type. The most we could do is strip the key portion of the pair and just take the value, and this would defeat the usefulness.

Now, if we use something like RSS or Atom to act as the transport medium, it might seem as if these would then ensure a common data model because most tools support one or more of these these feed syntaxes. The assumption is that if the tool supports the feed, they have to support the data that provides the feed, and therefore a minimum data model is guaranteeed. Right?

Well, not necessarily….

If a syndication feed supports complex or hierarchical categories but these are optional, and one weblog tools supports them, but another doesn’t, using the syndication feed to export the data from one tool to the other will result in loss of data; using XML won’t improve this situation, or prevent the loss of data.

That’s where the MT import/export format comes in handy at this time. The power behind it isn’t in it’s syntax, which is problematical–but in the underlying data model. The MT format has, by virture of it’s wide usage, defined a minimal shared model that most tools support. An XML-based version of this model could then provide a more robust import/export format. This is a win/win for all tools, and one that we as customers should encourage.

However, until a good majority of tools support the use of this XML format, whether it’s based on RSS or Atom or even something entirely new – the de facto standard for most tools now is the existing MT regular text-based format. This is what I will be using for the rest of these writings.