Month: September 2004

Categories
Technology Weblogging

Tool independence: The export format

The first challenge to moving a weblog is getting a snapshot of your weblog data in a format that can be imported into the new tool. To create an export that works with most tools, at least for moment, you’ll want to export your existing weblog’s data using the Movable Type import/export format.

WordPress doesn’t have a MT export built into the tool (more on this later); as I mentioned in a previous post, I used Scott Hanson’s new WordPress to MT export script to export my posts, categories, comments, and other data. Once I copied his file, I edited the parameters given in the script, providing the same username, password, and database name I added to the wp-config.php file. I also edited the file to use the default line break within Movable Type, __default__ , rather than ‘markdown’, which is the text format tool currently set as default in the import tool. These items are easily found in the script using a text editing tool.

When I load the export script page into a browser, the exported data prints out to the page, in a technique made popular by Movable Type. Once the export is finished (a message displays at the end of the page), I used the browser’s File/Save As functionality to save the page to a local file called bb.export.

(Another approach is to use the Unix wget utility on the command line as follows: wget http://somedom.com/import.php. This saves the exported data as ‘import.php’ and you can the use the data as needed. )

After the file was created, I FTP’d it to a newly created sub-directory, import within the MT cgi-bin directory. All that was needed at that point was to open up my MT installation, select import/export, and pick the import option–choosing to use myself as default author. the default post status of ‘publish’, and since all the entries had categories, I didn’t need the default category. I also have titles for all entries, so didn’t need to fill in the start and end title HTML fields, either. Clicking the link to do the import should load the data, and the data migration of the tool porting is finished.

(See the Movable Type documentation for more information about importing data into a MT weblog.)

A regular WordPress supporter, Carthik also has a version of a WP-to-MT export tool that uses WordPress global variables for the database variables. You can access it here. Carthik had his export tool finished before Scott started working on his, but had withheld publication because he’s now working on what he and Matt Mullenweg, the lead WP developer, are calling a “lossless XML export”–a import/export format that is going to be included with WordPress 1.3, and licensed as GPL for others to use if they wish.

One reason that the WordPress folks are creating this new format is that there has been problems with the existing MT format in the past. I have exported and imported data several times using this format and haven’t had issues recently, but others have had problems, specifically with fairly fragile points of breakage in the scripts such as dependence on a dashed line to separate entries. When I first used the import format to move from Blogger to MT, the import kept stopping as it would run into a sequence of dashed characters and the import functionality thought, “Well, that’s it – she’s done.” Once I edited for this problem, another would surface, making my move from Blogger to MT the most painful tool move I’ve done.

However, there is no denying that the MT export/import format is the most widely supported format in most weblogs. To have tool independence, in this case you need to depend on a specific tool import/export format…at least until enough vendors can support a replacement.

Technical issues of clean transformations aside, a challenge with a new universal format is the underlying data model each tool shares. For instance, a ‘post’, that has 0 or more ‘comments’ and at least one ‘category’ is more or less a standard model of data across all weblogging tools. However, beyond this simple core model, each tool does differ widely.

For instance, Movable Type supports keywords but not key/value pairs. Keywords are just a listing of terms associated as values to the weblog, while key/value pairs have both a term and an associated value. WordPress supports key/value pairs, and I use these in the ‘about this entry’ box in the top-left of the page. When I moved the data to the Movable Type test site, I lost this key/value pairing. Even if a new export format included these key/value pairs, there would be no place to receive it in the target weblogging tool–in this case Movable Type. The most we could do is strip the key portion of the pair and just take the value, and this would defeat the usefulness.

Now, if we use something like RSS or Atom to act as the transport medium, it might seem as if these would then ensure a common data model because most tools support one or more of these these feed syntaxes. The assumption is that if the tool supports the feed, they have to support the data that provides the feed, and therefore a minimum data model is guaranteeed. Right?

Well, not necessarily….

If a syndication feed supports complex or hierarchical categories but these are optional, and one weblog tools supports them, but another doesn’t, using the syndication feed to export the data from one tool to the other will result in loss of data; using XML won’t improve this situation, or prevent the loss of data.

That’s where the MT import/export format comes in handy at this time. The power behind it isn’t in it’s syntax, which is problematical–but in the underlying data model. The MT format has, by virture of it’s wide usage, defined a minimal shared model that most tools support. An XML-based version of this model could then provide a more robust import/export format. This is a win/win for all tools, and one that we as customers should encourage.

However, until a good majority of tools support the use of this XML format, whether it’s based on RSS or Atom or even something entirely new – the de facto standard for most tools now is the existing MT regular text-based format. This is what I will be using for the rest of these writings.

Categories
Critters outdoors Photography

A cat’s perspective

Recovered from the Wayback Machine.

Hi, I’m Zoe. This is my first time writing to a weblog, and I hope I do it right. I’m not sure how many cats write weblogs. I saw a weblog written by a dog today, but all it wrote was variations on feed me and smell my butt. Seems rather limited, somehow.

Shelley is my pet person, though I think she sometimes gets the idea she runs things around here. But as you can see here in my most recent photo, I take my supervision of Mom’s care quite seriously.

cat who means business

Mom was going to write about her road trip yesterday, but she had a complicated day today and was tired; so I sent her to bed and decided to write her post for her.

Yesterday was sunny and Mom decided a walk among the lakes might be nice. She went to Busch Conservation Area, which has 35 lakes– some swampy, some clear and all different sizes. One thing they all have in common is they’re all full of fish. Nice, juicy, tasty fish. Mom said the waterfowl lake was full of ‘fingerlings’, which I guess is a small fish. Just my size, I thought, but did she bring me one? Not a bit of it! Not one lousy, stinking fish.

She did bring home photos, though what good these are, I don’t know. You can see them at Tin Foil Project, if you’re of a mnd for that sort of thing. Mom says a good photograph helps the viewer smell the scents on the air, feel the summer heat against their cheek, and hear the fish jumping in the water. All I can say is Mom must not be a good photographer because it just looked like a bunch of gray stuff to me.

(But I like the cursor. I like to chase the cursor. Wheee…there it goes! Quick, quick before it moves away. Jump! Now! Death to Cursor!)

Mom spent the afternoon driving from lake to lake, getting out at and walking around some of the bigger ones. She said it was a pretty hot day, and she had to keep the windows up and the A/C on to keep the dust of the road out. The road was in bad shape, too, and she thought she would hang her car up a time or two.

“Kitty, Golden Girl might not be an SUV but she can handle rough roads with the best of them”, she said when she got home. Of course, if she’d busted an axle or flattened her tire, she’d be moaning and groaning, and feeling damn sorry for herself today, wouldn’t she?

I crawled up into her lap to get some neck scritches and Mom took this as me wanting to hear more about her trip. No, I just want more neck scritches. However, everything in this house comes with a price tag. if you want your neck scritched, you have to listen to a story. It’s like those places that feed you when you’re poor, but you have to hear a sermon, first.

At one lake Mom had to drive past waist high weeds on a poor track to get to it. As she was moving slowly along, these hard, black things started flying into side windows and windshield. Turns out it was some kind of big, black, shiny bee-like creature, and she figures she must have driven right into some kind of feeding ground, or perhaps even into their nest. She was mighty glad she had those windows, up, she said.

(Yeah, yeah, Mom. It’s just bugs. Who care. More scritches, less chatter. )

When she got back yesterday, she went online to check out the weblogs like she does most days. She’d been following a lot of stuff lately having something to do with being a woman and not being seen or heard or something like that. It doesn’t make sense to me, a cat, but Mom refers to it as being invisible on still water. Must be a human thing.

I kind of wish Mom wouldn’t get involved in this stuff because it upsets her. She ends up writing something here or in other’s comments, but doesn’t feel like she’s heard when she does. Then she gets both angry and sad, and forgets all about my snack.

Today was tech day, though, and Mom was hard at work on code, humming under her breath as she typed away. She was working at something called “Movable Type”, and why it’s called that when the type doesn’t move, not even a wiggle, I don’t know and believe me, I’m an expert on moving things. She was pushing stuff into it from something else called “Wordpress’, and that one makes sense as the words do seem pretty squished and flat on the screen.

Then this evening she read something in another weblog that surprised her and, she said, made her feel invisible again. She was pretty somber for awhile; just sat and stared out the window as it got darker, and I was beginning to worry that I was going to miss out on both evening cuddle and chase the feather-that-is-dead.

But someone else wrote something that also surprised her, but this time it made Mom smile. It was a good smile, too. Sometimes I don’t see enough of it, and I’m not sure all this “Movable Type” and “Wordpress” and ‘asshole-rss’ (what is an ‘asshole-rss’?) and people writing things and doing things that make each other somber is such a good thing.

But then there was that smile at the end. And I got my cuddle, time with the feather-that-is-dead, and even an extra scritch. So maybe this pressed word stuff is okay.

Categories
Technology Weblogging

Weblog tweaks: tool independence

Recovered from the Wayback Machine.

Once upon a time, a weblogger was defined by the tool they used as much as the type of material they published and the style of writing they favored. We were, and many still are, a fiercely loyal bunch when it comes to the software that help us publish our thoughts to the world.

But things happen.

We barely brushed the afterbirth from comments before the spammers hit; people became successful writing the software, or got tired of writing the software, and that changed the dynamics between tool giver and tool taker; open source tools came, and went, and then returned, either with a whisper or a shout. Over time, the packs that moved en masse from one tool to another begin to splinter here and there, and the tenuous bonds of loyalty between tool maker and tool user began to fade. Now, to most people, weblogging software is just that…software. And moving between tools is less a declaration of independence than a simple convenience.

When changing tools, instead of:

I am moving from Movable Type to WordPress now!

Hear me! I am moving now! Watch!

Are you watching?

Here I go!

It becomes:

Oh, I forgot to mention last week, I moved to WordPress.

There will always be core groups of people loyal to one type of software or another; more power to them because they also provide the core support for the product and we all need a little help from time to time. But for most of us, all we want is to write in peace and not have to clean our buffers of 1001 spam messages every morning.

Of course, no one wants to spend their time, moving their weblogs from tool to tool. However, If you truly want to focus on your writing (or your photos, or your garden stories and recipes) then you have to be willing to move your weblog to a new tool rather than stay with one that makes you unhappy. A tool that pisses you off is going to claim a whole lot more of your attention than the day or so to move it.

It’s also not good for the tool makers or vendors to have people continuing to use their tool because they, the webloggers, feel ‘trapped’ into using it. These people will express their unhappiness: probably frequently, and most likely noisely. There’s enough customers for all the tool providers, and none of them needs an unhappy customer. Moving to a different tool could be a far, far better thing for your customer to do, then they have ever done before. I know that my previous weblog tool providers have certainly felt that way about me.

However, rumor has it that porting your weblog between tools is a major undertaking similar to, as I wrote someone recently, dying and being re-born. Painfully. And it’s true that moving your weblog between tools is not the easiest thing in the world–but it doesn’t have to be overly difficult either.

I have moved several people through a variety of tools and environments. I have written a couple of hundred tutorials and tips for the various weblogging tools I’ve used, including how to move your weblog from Tool A to Tool B. But I’ve never tried to separate out the commonalities between the tools when I’ve created these writings–what it is about each tool that is very similar. That’s what I hope to do in this set of essays I call my WeblogTweaks – the last of my writing on weblogging technology.

First up: Moving from WordPress to Movable Type.

I created a brand new MT installation this morning, and I’ve finished the port of the entries using Scott Hanson’s WP to MT export script. In addition, since I’m moving from a PHP-based tool, I’m preserving the dynamic nature in the new environment, and have incorporated the new MT PHP configuration into this weblog; the pages you see are being served dynamically. I’ve also preserved the URLs from WordPress to MT–no redirect or fancy foolin’ will be needed to maintain the permalinks.

Now, I’m working on the sidebar. In my current weblog, much of this is fed by WordPress specific plugins. Whatever functionality can be handled by tags will be replaced by tags, but there are some functions unique to my specific setup. Rather than drop them, I hope to convert them into MT style plugins using the new PHP-based plug-in API.

(And the transparent calendar looks rather nifty on the faintly patterned background. Hmmm…)

But that’s for tomorrow, as is the detailed writeup on the work I did today. I’m beat and I want to write something non-techy before bed.

Caveat: I’m not recommending people move from tool A to tool B just because they can; nor am I recommending any one tool over another. Each tool has good points and bad, and both are relative to the person using the tool and their specific needs. And here’s a hint: all weblogging software is buggy. It is the nature of software–there is no magic fairy that sprinkles pixie dust on weblogging software that makes the bad stuff go away. If there was, Microsoft would have hired him or her a long time ago.

Categories
Weblogging

Bad mans find good woman

I’ve been hit severals time recently with comment spams. In fact the frequency of attack has really picked up as WordPress has become more popular.

I don’t use any form of blacklist, but controlling the spam is still pretty trivial. There’s one throttle in place that doesn’t allow more than so many posts in a short period of time–over ten posts in a minute, and over fifty a day; anything beyond these limits is automatically moderated. I can easily increase the number of comments per minute or day if I suddenly gain political blogger status.

(The ten posts in a minute throttle is new, so please send me email if you have problems.)

I also have code in place to automatically put my comments into moderation at 20 days. Due to the increased frequency of attack, I’ve changed this to put any comments older than five days into moderation. Since most activity falls within this five day period, this shouldn’t be too restrictive.

With these protections in place, I still get the comments. But no build is happening, so it’s just a straight database access and an email. And since I can’t get more than ten of these beasties at a time, the mail is no burden. Once a day, I then go into WordPress, go to the comment power-editing page, search on whatever is common on all the spams, and then mass delete the bunch.

If there’s one drawback to this, when my throttle is in place, your comments may end up in my moderation queue. But unless you’re selling me online gambling, I’ll let you through.

This spam throttling is one of my favorite WP modifications. If I had to pick any five modifications I’ve done to WordPress that has more than paid for the time to maintain these between version upgrades, I would pick the following:

  1. My comment spam throttling. Without having to resort to IP or other blacklisting, I have it controlled and managed with a minimum of effort. I am looking at putting this into a plug-in for WP 1.3
  2. My fullpage preview. There’s nothing like previewing your post within the environment it will be viewed at when published. The only way this can be a plug-in is to use DHTML to modify the page objects on load. Since I’m not fond of using DHTML for anything to do with navigation, I prefer to customize the page.
  3. My comment editing. As I posted comments in the last few days in other weblogs, I really regretted I couldn’t edit the comments after the fact. My damn typos. Because of this, I’ve come to have a deep appreciation for my own comment editing feature. I think all sites that offer comments, should provide comment editing. Best thing of all, this can be a drop-in modification. I need to package it accordingly for 1.3.
  4. My moderated comments customization. Being able to turn moderation on an off selectively rather than globally is a wonder. Again, this can be a plug-in, but would require DHTML to modify the document and this doesn’t ring my bell. So it stays a customization.
  5. My other customizations are tied for usefulness, but I like my comment/trackback split, and my talkback feature (both plug-ins, which I need to link at the WP support wiki). I also like my Insert post status that allows me to publish an entry, without it being part of the overall site navigation. This is particularly useful for About pages and other content that you want static. I also like my static page implementation, though I think the one that comes with WordPress now might be better (have to check it out and see).I also like my new “about this entry” with notes annotation that I just added. It adds an element of fun.

Why do technologists like weblogging? Because the tools are a tweaker’s paradise.

Categories
Weather

Guess who’s coming to dinner

Dave Rogers is also less than impressed with the new iMach G5 from Apple. However, he has other things to think about now – such as a possible category 5 hurricane by the name of Frances that is shaping up, from all computer models, to hit near his home. If I were a betting woman, I would bet on this storm hitting between Jacksonville and Daytona Beach, with the first winds starting Thursday night/Friday morning.

(I don’t think this will impact service for this site, but note that my host’s (Hosting Matters) servers are in that area. However, if there is a problem with the servers, I do NOT want Annette, Stacey, and the gang worrying about making their way into the shop to fix the problem. Let the sites stay down – Michelle Malkin will understand. She’s a compassionate conservative.)

I’ve been watching this beast for several days, as it got stronger and more determined. The shear is low and the water is hot, and the previous activity was long ago enough so that no churn is bringing up the cooler water. And though there is a possibility of the storm being blown further north or south, even the more cautious weather forecasters are saying that it looks like it will hit the Florida this Friday. And it doesn’t look to weaken.

Hopefully those in mobile homes and along the ocean will use shelters, but they’re already maxed out with Charley. I guess one good thing about all this, if you can say anything is good about Florida being hit with two such major hurricanes in such a short time, is that support people are already there, entrenched, to help out with this one.

It’s frustrating to sit here knowing that this storm is heading towards people I know and there’s not a damn thing I can do to help them. I did sign up at the Red Cross for disaster help, but haven’t had my training yet. When I called to volunteer to help in Florida, I was gently and kindly told that our state is only sending in those with years of experience, and a newbie like me would be more trouble then help.

It’s been said that we webloggers can pool ourselves into a viable force for good. But what good can we do against Mother Nature, who always manages to demonstrate that no matter how clever we are, the Lady always wins?

However, folks in that area have good warning, and they know what to do. Including not standing out in winds in excess of 120MPH, yelling ‘hee YAH!’ at the top of their lungs, while they take pics to post.

Not that I would do something like this if I lived there. Not me.

I’ll be watching the storm, and keeping the folks there in my most positive thoughts.