Categories
Technology Weblogging

WordPress and bug databases and communication

Recovered from the Wayback Machine.

The folks behind WordPress have made a step in the right direction by first of all, not deprecating old functions without going through a formal deprecation process. This is to ensure that people have time for a release or two to modify a now deprecated function or global variable before it is pulled from the application.

They’ve also just started a bug database, and this could be good – if they understand that a bug system is a two-way communication process. It’s not always about code.

entered a bug in the database yesterday about the fact that if you turn off magic quotes in your .htaccess file, but you try and edit a comment and save it, it causes the SQL to crash. The reason for this is that the text of the comment isn’t ‘escaped’ – slashes put in front of quotes contained in the text to tell the database to treat these are characters, not the end of the database string.

Well, the response was to point to a change record in CVS in the newest releases of the code an d say, ‘Does this fix it?’

Well, how the hell do I know? I’m using 1.2. What if I can’t read code? Wouldn’t an English description acknowledging my problem and the solution have been better? With a little note about when it will be released? Such as, “This fix will be released in the 1.2.1 bug fix release”.

Bluntly, the WordPress development crew is not happy with me because I’ve been pushing them pretty hard for the last month. What I’ve been saying is that software is only 50% code – the rest is documentation and infrastructure, quality testing, and communication. Particularly communication.

Oh, you don’t need these things if your code is used by hackers or a small group of friends. But if you want your application to be used by strangers who don’t code – you can’t force them into learning code to communicate, or having to beg pretty please in order not to piss off the development people.

I’ve gotten a lot of flack for my criticisms of past weblogging tools. I stand by these criticisms, every single one of them. I’m not, now, going to play ‘touch not the programmer’ just because the source code I’m now using is open source. If anything, I want the open source solution to work, so will be harder, not easier, on the team behind the product. Is this unfair? What’s fair? Not being critical because this just isn’t done in weblogging?

Unfortunately, the team doesn’t see that I’m attempting to help them succeed over time–by realizing that an application is more than just code. My own frustrations aren’t helping, and have led me to become increasingly confrontational. My bad.

What I’ve been trying to say is that a successful application is demonstrated by the trust of those who use your product, and based on understanding what the users are saying and making an attempt to communicate in their language. It’s slowing down development if it means that the end product is more stable or secure. It’s releasing bug fixes, and developing a plan for future development and communicating it. Most of all, it’s realizing that people can’t be grateful forever. Eventually, the product will have to stand on its own, not on gratitude.

(Six Apart is learning that one rather painfully now.)

In short: a successful application is only 50% code.

But I’m also not going to trade my ‘kicking the baby squirrels’ tag for one of ‘kicking the open source baby bunnies’, either. If I’m not helping and all I’m doing is making people pissed, then time to stop what I’m doing.

Disappointing, though.

update

I’m glad I posted this and had conversations offline and on. It helped me refine, at least for myself, why I’m pushing at the WordPress group so much.

Much of it goes back to the concept of the Coders Only Club, and having to put code down at the door in order to ‘purchase’ the right to be heard. I want this open source application to belong , as much or more, to the non-geeks as it does to the geeks. I think we could have a lot of fun together.

However, the project is young, the developers are trying, and they are most likely getting tired of ducking from me, and my beating about their heads with my ‘help’.

update
General agreement is that I was too harsh on the WordPress developers. Guilty as charged. It’s their project – up to them how they manage it.

As for myself, I’m just going to maintain my own code at this point, and doing my own enhancements.

Sigh of relief for the WordPress team and their so-loyal fans. One less pain in the butt to worry about.

Categories
Technology Weblogging

Possible WordPress Magic Quote fix/workaround

I posted some suggestions on the WordPress magic quote problem at the support forum, here.

Before running the application, make a mysqldump backup first, using the following:

mysqldump -u username -p databasename > wpdatabasebackup.sql

When prompted for password, enter password.

And this concludes the last of my posts on WordPress.

Categories
Technology

Survival guide to LAMP: First, a word from our sponsor…free beer

L is for Linux, A is for Apache, M is for MySQL, and P is for PHP/Python/Perl

Open source applications such as WordPress (functioning under a GPL license) are gaining a lot of fans and followers in the last few years. After all, there is an inherent goodness to projects that are free enough to quote Mark Pilgrim. Thanks to increasingly polished and stable applications like Linux, MySQL, Apache, PHP, and various other products operating under some form of open source license, people are also becoming more comfortable using these applications. They’re no longer just for Dew Drinkers.

But everything comes with a price, and open source is no different. So today I want to talk about the concept of open source, GPL, and free…as in beer.

Free, as in Beer

First, to clarify a misconception: open source is not free source. Open source has to do with whether the source code can be modified or distributed without having to get specialized permission from the originator of the code–but the person who wrote it is free to charge for it.

Free software, on the other hand, is software that supports what are known as the four freedoms: a person can use the program, modify it, copy, or distribute it, freely. In addition, if a person or organization creates a derivation of the original software, they can also attach a more restrictive license to the derived product, or charge for it, or even encapsulate their own modifications under a closed source copyright. The BSD license, so called because much of its use is in relation to the BSD operating systems such as FreeBSD or NetBSD, is an example of a four freedom or 4F type of license. So is the Apache license.

However, there are licenses that go beyond even the free software license when it comes to ensuring future openness of derivations, and these are known as copyleft free software licenses. They not only provide support for the four freedoms, but also demand that any derivation of the work also carry the same type of license. This means that the derived work must be freely available, can be modified, and, in turn distributed and new distributions can never have a more restrictive license. The GNU General Public License (GPL) is just such a license, and applications such as Linux and WordPress are GPL software.

Does a GPL license sound like a sucker bet? I mean, why would any commercial organization use GPL source? How can you make money?

Easy–by adding value on top of the core functionality of the open source application, while respecting the GPL that governs the source, itself.

MySQL supports a dual license; there is the GPL licensed version of MySQL that can be freely downloaded, and used within any application. However, a stipulation of the license is that whatever application uses the MySQL must also abide by the GPL license agreement and be freely distributable and open. If the organization doesn’t want to share their source, then they can buy a nice commercial license. This ‘dual license citizenship’, if you will, is one of the most brilliant I know of for bringing to market a product commercially while still keeping the product source open.

Linux is also GPL source, and there are many variations of Linux you can choose from on the market. Yet for all the freedoms associated with the license, there are several companies who have become very successful with Linux as their core product(which has, unfortunately, attracted the attention of SCO, otherwise known as the Scummy Crappy Organization). These companies honor the license by providing access to the source for the version of Linux they package, while making money by adding all sorts of bells and whistles, customer support, and nice installation CDs and other software packaged with the operating system.

Of course, companies don’t have to necessarily make the GPL controlled source easily available. Before April I could download Redhat’s source directly. However, recently the company made a statement that you can’t get the Redhat code directly. Instead, we can download and install the Redhat style of Linux through the Fedora Project. Basically what the company is saying is here’s the Linux–but no implied support that comes with the Redhat name. In addition, the FedoraLinux is ‘cutting edge’; in hacker terms, this means ‘use at your own risk, but don’t come crying to us if you trash your computer’.

There are no limitations on Fedora–use it however you want, whenever you want, and on as many machines as you want; but it’s not an ‘official’ Redhat release, which means you can’t count on company support, and any of the goodies they package with the commercial product aren’t packaged with Fedora.

(If you become miffed at Redhat for making this move–and some people have–there’s another distribution or two of Linux from which you can choose. For instance, Debian is one of the most popular and community supported versions of Linux.)

Regardless of company support or not, the key fact on GPL products is that the source be available for copy, distribution, modification, and use, and this openness persists forever. And a day. That’s why Mark Pilgrim writes about WordPress:

WordPress is Free Software. Its rules will never change. In the event that the WordPress community disbands and development stops, a new community can form around the orphaned code. It’s happened once already. In the extremely unlikely event that every single contributor (including every contributor to the original b2) agrees to relicense the code under a more restrictive license, I can still fork the current GPL-licensed code and start a new community around it. There is always a path forward. There are no dead ends.

No dead ends. And free. Free…as in beer.

Not really free

Free…as in beer. What’s not to like? However, what applies to the source, does not apply to the people behind the source. Never confuse the two, because if you start treating the community behind the GPL software as ‘free…as in beer’, don’t be surprised if you find yourself and your software without that community someday.

As I watched Movable Type people move to WordPress, not to mention users of other software, I noticed that most people were friendly and interested and appreciative of any help given, and even generous with the WordPress development effort. But not all, and that’s a risk.

The advantage of using a commercial product is that you can expect a certain level of support for the product, and that includes, at times, taking heat when you make mistakes, or make a sudden change in direction in where the product is going.

I wrote a while back about Oracle’s ‘bite the bullet’ moment, when the company changed the underlying architecture between versions 5.0 and 6.0 and the flack the company took because of it. However, all Oracle could do was say, “Yes, we knew this hurt you all, but we had to make this change to ensure the product is viable in the future. We’re sorry.” What the company could not say was, “Well, don’t like it? Stuff it!” As it was, the company barely survived–not allowing their customers to vent would have been a disastor.

There’s an implication that you can’t ‘vent’ with open source, or should I say ‘free’ source, applications because, after all, you’re only getting what you pay for, and you’re not playing for the application. There’s also a risk that if you vent too much, the developers will say, “Well, don’t like it? Stuff it!”, and you’re out of a support and development team.

Therein lies the hidden cost to free source: you can’t expect the same level of support, or the same level of abstration of objectivity with a free source application that you can with one bought and paid for in cold, hard cash. Where the latter is a transaction, and therefore a formal relationship is established between you and the organization providing the application; the former is a some nice people providing neat software for thousands of their closest friends.

Donations, rather than clarifying the roles between developer and user just muddy them up even further. When Six Apart released Movable Type 3.0 and the new license, almost everyone who was critical of the new license prefaced their remarks with, “I donated to Six Apart for the use of Movable Type”. Donations add a monetary element to the relationship between organization and user, but it’s not formalized as a fee would be. What the donation buys is also widely open to interpretation: some people consider a donation to be nothing more than a thank you gift, and others think that donating to an open source developer makes them into a form of serf.

The source may be free, and you may not pay for it, but you still get frustrated. If you’re using version 1.0 of the product and upgrade to version 2.0 and it breaks half your site, you’re going to be unhappy. If you can’t find the documentation you need to use the tool properly, you’re going to be unhappy. And if you find out that the reason your site is broken is that the developers made underlying changes in the functions you use, without notice, you’re going to be very unhappy.

Free source or no, there are certain things you can’t do with a product and maintain any form of credibilty with your customers–yes customers–moving forward.

Different realm, different coin

Mature open source environments such as that in which Apache operates know that you have to maintain credibility in your product if you want people to continue to use it. When this organization made the move from Apache 1.3 to 2.0, they did so with a great number of warnings about what you can expect if you upgrade. In addition, tool developers like PHP also issued a number of warnings about which of their tools was compatible with what version of the web server. Because of some of these issues, many sites, including my own, still operate under 1.3, and the Apache organization still maintains an active thread in support of the 1.3 server.

If Apache had decided, instead, to make the move to 2.0 with a minimum of warning, and then just drop the 1.3 branch in a few months, the organization’s credibility would have suffered. Since, in come ways, the coin of the realm in the open source movement is both credibility and respect, there is a degree of accountability even with ‘free software’. There is–or the application joins the tens of thousands of unused and barely known open source software applications that litter the Internet.

But Apache has gone beyond the early days of being an open source application, and is organized enough now that no one person is responsible for it’s maintenance or development. In addition, the organization also acts as a buffer between the developers and the users, so that we can rant and rave about it without key people being hurt or offended and leaving in a huff; leaving you with your now unsupported software tied around your ankles like a girdle who’s elastic has snapped.

The same can be said for Linux, and MySQL’s commercial roots provides enough abstration (not to mention that it, too, is now widely used). As for PHP or Python or Perl or any of the other programming languages that are open, these are so common now that people bitching about them are more gnats in a swamp than anything seriously annoying.

Commercial applications have a monetary impartiality, and big open source efforts have density. That just leaves the smaller open source applications that have achieved some or even a great deal of success. Applications like WordPress.

Get Tough, Baby

When I first went to WordPress over a month ago, I went out on the support lists like a plague. I pushed them on everything from delivery schedules to coordination of documentation. They must have thought I was a platinum bitch, and in some ways I was. But I also knew that if there was ever an mass exodus to WordPress, the friendly atmosphere in which the WordPress organization has been maintained to this point, was going to be stressed and strained.

At the time, I had a feeling that there might be some move to WordPress because of TypeKey, and the lateness of Movable Type 3.0, not to mention the problems we had with comment spammers. In addition, the time felt ripe for change–we’d about sucked the innovation out of Movable Type, and more than a few of us were frustrated how our code was lost with each new implementation of the product. There are a whole lot of tweakers among us, but you can only tweak so far with closed source.

Based on all this, I knew that it was only a matter of time before the numbers flocking to one of the more popular GPL-licensed weblog tools began to change the dynamic in which this project operated. I didn’t expect the push that the organization would get from Movable Type’s new license strategy – not to mention being slashdotted twice in one week–an event that has finally kicked the WordPress server down to its knees.

You might say that in a GPL world, success isn’t always a blessing.

(Slashdot is worse than a Denial of Service (DOS) attack at times – it’s comparable to someone making a mistake on the ad for your house, listing it at 35,000 instead of 335,000 and then you watch as the resultant mass coming to the open house tears your home down around your head. But the good news is: If the house survives, there might be a buyer in that mess of humanity.)

The last few days when I went out on the support lists, I hit it, again, like a plague; but this time it had nothing to do with the future of WordPress, and everything to do with my own frustrations migrating from 1.02 to 1.2. Without complete plugin documentation, incorrect magic quote management, and especially rumors of deprecated or broken functions that aren’t documented, the migration process from 1.02 to 1.2 has been painful–a pain not shared by new users who are just starting with 1.2. Which of course, only adds to the sense of umbrage.

But at the same time, I also have to remember: this is free source. I didn’t have to pay. The project won’t die. It’s supported by people who donate their time for development, documentation, and even money to support the servers. I really like the software in spite of the current glitches.

But I still feel frustrated.

What’s a growing free source application to do?

Matt and the other WordPress developers must be dazed by all the new users and attention, and they deserve the spotlight. But the spotlight, just as with free source, comes with hidden costs: there are too many users of WordPress, now, for it to continue to be a small intimate open source development effort. If WordPress is going to continue to succeed, things are going to have to change, and the development team are going to have to adapt. The easy going environment in which they’ve operated to this point won’t survive the numbers that have moved, and will continue to move, to WP. It’s no longer just about the free source, the code.

The keys to a successful open source software effort can’t be found in lines of code or pretty screens; it exists in the relationship between those that create the source and those that use it.

Categories
Technology Weblogging

Did the port thing

I wrote this before running into the magic quote problem in last post

I created another WordPress weblog today by copying the database and files from this weblog, and then once created, upgraded all the files to 1.2. I needed an environment identical to this one to make my edits so that upgrading this site would be relatively risk free.

To create the second database, I used mysqldump to download a copy the contents of the existing one, created a new database, and then loaded the data from the dump. Once I copied the files, I modified the wp-config.php file to point to the temporary database, and started to work.

I received my first comment spam within an hour of creating the duplicate weblog, and received a comment by someone who came in through Google within two hours. And during my effort, the original weblog was hit by a comment spam attack, which played havoc with my effort (me wondering what I was doing wrong to be generating all these ‘approve comment’ requests in the inbox). However, thanks to the attack, I found I had an error in my crapflood protection; fixing it should prevent manage these attacks in the future.

The first edit I made was to modify the upload.php file to incorporate a modification that wraps a thumbnail image with a hypertext link to the larger photo. I then tried this on a photo of another major storm that hit St. Louis today.

The next modifications were to copy the edited wp-comments.php and wp-comments-post.php files, and replace the top part of the wp-comments.php with the new code in the upgrade (everything above the line). All my edits I could save, as is–including the live preview from Chris Davis. There’s also a new plugin for WordPress 1.2 that provides a preview page for comments, and I may add that as an option for people who don’t have Javascript enabled.

The next change was to add the entry into menu.php for my Switch blogs multiple blog handler, in addition to using the install-multi-php file I wrote to create the table. All that was left then was copying the switch.php file into the wp-admin directory.

Following the previous discussion on multiple weblogs, I also deleted the wp-images, wp-contents ,and wp-includes directory for the new installation and created a symbolic link to their counter-parts in the first test WordPress 1.2 weblog directory location. By doing so, the plugins I’ve downloaded or created for my other WordPress directories are now available for this one.

Include the my_recent_comments list that feeds the sidebar–my processing includes links to comment authors web sites and a few other odds and ends not provided by any existing WP function or plugin.

After making sure comments work, I then decided to have some fun and play with the CSS for the WordPress administration pages. I didn’t change much: added some background color, and some border effects for the buttons. I also changed the background color for the ‘look at me’ events such as deleting a post. Instead of that glaring red, I have a nice dark blue. The darkness is alert enough – never depend on color to ensure that a person pays attention to what they’re doing.

I rather like my new WordPress Admin look. It’s a newer, kinder, gentler WordPress…that just happens to work the same no matter what pretty pretty we do to the CSS.

Currently, WordPress supports moderation for all posts, or none. I’ve always liked turning moderation on by item, which I implemented in WordPress 1.02. To carry this forward, after upgrading the database during the 1.2 upgrade process, I had to use phpMyAdmin to add the ‘moderated’ option to the comment_status field in the wp_posts table.

The code to manage moderated comments in wp-comments.php didn’t need to change. But if you’re interested in adding this modification to your setup, first of all, look for the following line using your text editor:

<?php if (’open’ == $post->comment_status) { ?>

And replace it with:

<?php if ($post->comment_status ==’open’ || $post->comment_status == ‘moderated’ ) { ?>

After the line to create the “Leave a Comment”, add the following, but edit the message to whatever you want:

<?php

if ($post->comment_status==’moderated’)
echo(“Use your own moderation message here, complete with HTML formatting”);
?>

Save the file, and then open wp-comments-post.php and look for the following:

if ( ‘closed’ == $wpdb->get_var(“SELECT comment_status FROM $tableposts WHERE ID = ‘$comment_post_ID’”) )
die( __(’Sorry, comments are closed for this item.’) );

With the following:

$commentstatus = $wpdb->get_var(“SELECT comment_status FROM $tableposts WHERE ID = $comment_post_ID”);
if (’closed’ == $commentstatus)
die(’Sorry, comments are closed for this item.’);

Then look for the the following line:

$wpdb->query(“INSERT INTO $tablecomments

And insert the following code before this line:

if ($commentstatus == ‘moderated’) {
$moderation_notify = true;
$approved = 0;
}
else
$approved = 1;

Save the file. The last change then is to add the ‘moderated’ option in the advanced editing form, as shown in the following screen shot. Rather than talk you through this, a copy of my changes files is contained in zip file at the end of this writing.

Wordpress 1.2 Screenshot of moderated modification

This is the only change on this page–I also have added a link to my preview page.

Currently WordPress uses the same page, index.php, to serve all requests: archives, category, individual pages and so on. I don’t necessarily like the same look with each page, so I copied it into category.php and individual.php.

The pages themselves don’t need to change, other than to modify the look into what you prefer for these pages. But to ensure they’re called, I needed to modify my .htaccess file to point to these pages, rather than index.php:

RewriteRule ^archives/([0-9]{4})?/?([0-9]{1,2})?/?([0-9]{1,2})?/?([0-9a-z-]+)?/?([0-9]+)?/?$ /individual.php?year=$1&monthnum=$2&day=$3&name=$4&page=$5 [QSA]
RewriteRule ^archives/category/?(.*) /category.php?category_name=$1 [QSA]

To create my preview page, I copied individual.php to preview.php and made one modification to the code: it now calls a file called wp-blog-draft-header.php rather than wp-blog-header.php. This new file is an exact copy of the old one except for one change:

Look for:

$where .= ‘ AND (post_status = “publish”‘;

And replace with:

$where .= ‘ AND (post_status = “draft”‘;

Save the file, and now I have a preview page. You can take a look at it with a post kept in draft state.

WordPress 1.2 does have a page preview located at the bottom of the edit page, but I like the ability to preview the writing within the context. Especially when using my photos, I need to know how they look in context.

The file upload, individual moderated comments, and full page preview (and the new look and feel) are the first phase of modifications. The second is to add another page and menu option to manually generate a static page from any given page; a plugin that can be used to statically generate the index.php and syndication feeds; and selecting comments by date range to easily delete a comment spam attack.

You can get a copy of the files discussed in this writing here. If you decide to play with any of this, make sure you back your files up first.

Categories
Technology Weblogging

Slash problem in WordPress 1.2

As you may have noticed, I upgraded this site to WordPress 1.2 yesterday. You may have noticed because WordPress 1.2 has what I consider to be a very serious bug in that it ‘escapes’, or adds a slash, in front of all apostrophe’s in both comments and RSS. This is commonly referred to as the ‘magic quote’ problem with PHP. (I’m writing a LAMP essay on this for later.)

The rumors of workarounds at the support center talk about adding filters to filter out the slashes, and I did this for comments. But then, thanks to some friends, I found out that the RSS is also featuring escaped slashes. In fact, the built-in page preview is escaped, and the only thing not escaped is the text in the main post.

 

Since there are new users of WordPress 1.2 that haven’t said anything about this problem, I’m wondering if it has to do with the upgrade from WordPress 1.02 to WordPress 1.2. Or maybe they have a fix. Or maybe they don’t know the problem exists. Or maybe they got the secret decoder ring to wear while writing to their weblog. Mark did you get the secret decoder ring?

I don’t know what’s happened in WP to generate this problem, but you might want to consider postponing a port or upgrade to WordPress 1.2 until an official fix has been created. Having to add these strip slash filters isn’t the best way to deal with this problem.

For those who have ported to WP 1.2 and are having problems, I’ve added filters to the syndication feeds, and the comments that seem to work with the comments at least. We’ll see how it does with syndication.

Add the following to wp-comments.php:

add_filter(’comment_text’, ’stripslashes’);

Make sure to add this into an existing PHP block.

Add the following to the PHP code in the syndication feeds:

add_filter(’the_excerpt_rss’, ’stripslashes’);
add_filter(’the_content’, ’stripslashes’);

Let’s say this post is a good test whether these work or not.

More later on the adventures of upgrading and also what hacks I’ve added.

Update

And when I first posted this, I had an opening PHP block, <?php> …<?> that I didn’t convert to HTML escaped characters included in the code. This caused WordPress to fail. This should not happen–the results might look funny, but the application should not fail.

But the stripslashes seem to be working on new entries within the syndication feeds.

Second update

Let’s do this the easy way: add the following to your .htaccess file:

php_flag magic_quotes_gpc off
php_flag magic_quotes_runtime off

This solves the problem without having to add all the filters on new or newly saved entries! The slashes are still in the database for the old entries.

In fact, this probably explains why others do not have this problem–either magic quotes are turned off for their installation, or they have these entries in their .htaccess file. However, it does seem as if the magic quote escaping is happening in the upgrade process. Yes? No?

Or was it that WordPress 1.02 and before handled this ‘badly’, and WP 1.2 handles the situation correctly, but the data is already ucky? I didn’t check the data before I did the upgrade (more fool me).

(Yes, that’s a tech term – ucky. )

More later in a LAMP essay on ‘magic quotes’.

Evil things.