Categories
Technology

The importance of degrading gracefully

When I first went to work at the dot-com, Skyfish, too many years ago, I was faced with an application that had a partial interface, little back end development, and that had cost the investors 1.5 million dollars I believe it was (might have been 2 million — hard to keep track in those days). Where did the money go was the question, and the answer was: it went on building lots of graphics and a really fancy Dynamic HTML (or DHTML or HTML and script) page navigation.

Well, the first thing we had to do, after firing the design company, was pull the DHTML out and revert to plain links for the menu. Why? Because DHTML menus don’t degrade gracefully.

I first heard about DHTML from Scott Isaacs at an invitation-only author introduction to the technology held at Microsoft’s campus. I remember when he–or actually, I think it was the program manager who did the demo–clicked on a name in a web page, and the space beneath opened up and more content was shown.

I was floored. I was astounded. I was in love. Up to that point, the only dynamic component of a web page was BLINK or an animated GIF, and neither of these was particularly helpful from a professional stand point.

In the months that followed, as the technology of CSS-P, as it was known then, was released, I spent an amazing amount of time working with it; I was sure it was going to revolutionize everything we knew about web page design. In fact, it was at that time that I became heavily involved with ASP, and between the two–DHTML for the front and ASP for the back–I felt that this was it: we could close the book on innovation, tell the other contenders they could go home now.

Well, it wasn’t long before cracks in this little nirvana started to develop. Whatever cooperation existed between Microsoft and Netscape at the rollout of CSS-P died a rather painful death, and we started having to deal with a thing called “cross-browser compatibility”–making stuff work in multiple browsers.

Microsoft really drove the browser market, too. Too fast and too far, and sucking people in right and left away from the Netscape browser, Navigator, to IE. The frenzied pace splintered the browser market and left us with a legacy of non-standard, proprietary extensions that haunted us for years. And then when Microsoft had us, they dumped us.

Compatibility across browsers wasn’t the only problem; we also had to worry about making pages work across browser versions. I remember about two years ago, when someone using Navigator 4.x asked me to change something in my weblog and I said, enough is enough, I’m no longer supporting a browser that was released six years ago. Now, I feel that way about IE.

Compatibility issues aside, other problems started to pop up in regards to DHTML. Screen readers for the blind disabled JavaScript, and still do as far as I know (I haven’t tried a screen reader lately). In addition, security problems, as well as pop-up ads, have forced many people to turn off JavaScript–and keep it off.

(Search engines also have problems with DHTML based linking systems.)

The end result of all these issues–compatibility, accessibility, and security–is a fundamental rule of web page design: whatever technology you use to build a web page has to degrade, gracefully.

What does degrade, gracefully, mean? It means that a technology such as Javascript or DHTML cannot be used for critical functionality; not unless there’s an easy to access alternative.

For instance, the main use I make of JavaScript and DHTML in my weblog is the live preview and spell check for comments. Now, neither of these is critical for people wanting to leave comments, and this means my site meets one requirement of the fundamental rule: the page can degrade. However, I made a design error when I added the live preview text area and Check Spelling button, in that I didn’t degrade gracefully: the Check Spelling button still shows, as does the live preview area. If JS is not enabled, these should not show. (It’s now an item on my to-do list.)

Whatever holds for DHTML also holds for Ajax. Some of the applications that have been identified as Ajax-enabled are flickr and Google’s suite of project apps. To test how well both degrade, I turned off my JavaScript to see how they do.

Flickr was a delight and an example of a web page that not only degraded gracefully, if it were a dancer, it would be the prima ballerina. As an example of this, there is an option on each photo page for the photo owner to add tags. Clicking the option instantly opens up a set of text boxes to add new tags. This is using DHTML, and very handy.

However, when you turn JavaScript, and hence DHTML, off, this option isn’t there, but you can easily edit tags and other information by clicking the edit link below. Flickr used DHTML to enhance the user experience, but never once built a dependence on the technology to drive the user experience. More, if you access the page without JavaScript, you’ll never know that you’re ‘missing out’ by doing so. Lovely use of technology.

Google’s gmail, on the other hand, did degrade, but did not do so gracefully. If you turn off JavaScript and access the gmail page, you’ll get a plain, rather ugly page that makes a statement that the primary gmail page requires JavaScript, either turn this on, get a JS enabled browser, or go to the plain HTML version.

Even when you’re in the plain HTML version, a prominent line at the top keeps stating how much better gmail is with a Javascript enabled browser. In short, Google’s gmail degrades, by providing a plain HTML alternative, but it didn’t do so gracefully; not unless you call rubbing the customer’s nose in their lack of JS capability “graceful”.

You don’t even get this message with Google Suggest; it just doesn’t work (but you can still use it like a regular search page). As for Google Maps? Not a chance–it is a pure DHTML page, completely dependent on JavaScript. However, Mapquest still works, and works just as well with JS as without.

(Bloglines also doesn’t degrade gracefully — the subscription is based on a JavaScript enabled tree. WordPress, and hence Wordform, does degrade gracefully.)

If we’re going to get excited about new uses of existing technology, such as those that make up the concept of Ajax, then we should keep in mind the rule of degrading gracefully: Flickr is an example of a company that understands the principle of degrading gracefully; Google is an example of a company, that doesn’t.

Update: As Doug mentions in comments, flickr is dependent on Flash. If Flash is not installed, it does not degrade gracefully.

Sigh, there goes my prima ballerina.

Categories
JavaScript

AJAX: The all-purpose cleaner

I’ve been hearing quite a bit about Ajax, the new wonder technology lately, and finally decided to take some time to check it out.

I followed the link to the Adaptive Path essay on Ajax, and started reading through the writing until I came to the diagram showing an “Ajax Engine”, as part of the new innovation. Being a geek like most other geeks, I then stopped reading and starting looking around for the download button for this new Ajax Engine. Not having found anything I could download, I returned to the essay and continued reading.

What I found is something that sounded strangely familiar. In fact, this ‘new’ technology sounded just like DHTML–Dynamic HTML–except that there was a reliance on a non-browser safe object called the XMLHttpRequest: an object to manage asynchronous XML access from the server from within web pages. An object invented by Microsoft for IE and implemented in Firefox and Safari, but not an objects that’s guaranteed to be cross-browser safe to use.

I admit, I was very confused by this time. I wrote on DHTML in 1998, in fact wrote the book whose cover you see in this post. I have hundreds of examples of DHTML, most several years old. I’ve even used the request object a time or two, but wasn’t necessarily overjoyed by it: I really didn’t like using proprietary objects when I couldn’t find the same functionality in other browsers.

In the FAQ attached to the essay, I did receive some clarification:

Q. Did Adaptive Path invent Ajax? Did Google? Did Adaptive Path help build Google’s Ajax applications?

A. Neither Adaptive Path nor Google invented Ajax. Google’s recent products are simply the highest-profile examples of Ajax applications. Adaptive Path was not involved in the development of Google’s Ajax applications, but we have been doing Ajax work for some of our other clients.

Q. Is Adaptive Path selling Ajax components or trademarking the name? Where can I download it?

A. Ajax isn’t something you can download. It’s an approach — a way of thinking about the architecture of web applications using certain technologies. Neither the Ajax name nor the approach are proprietary to Adaptive Path.

Q. Is Ajax just another name for XMLHttpRequest?

A. No. XMLHttpRequest is only part of the Ajax equation. XMLHttpRequest is the technical component that makes the asynchronous server communication possible; Ajax is our name for the overall approach described in the article, which relies not only on XMLHttpRequest, but on CSS, DOM, and other technologies.

I then read Dare Obasanjo’s excellent tap-tap that yes, the emperor is naked, and that Ajax really is Dynamic HTML with the use of the proprietary XMLHttpRequest object.

(Of course, I’ve used the term DHTML and Dynamic HTML, and in Dare’s write-up, even this is a renaming of an existing concept, so mea culpa in that regard.)

Wow, I didn’t know that I was ahead of the times when it comes to technology. I feel so super cool right now. To celebrate, I decided that I would go through all my burned CDs and recover my old DHTML applications in addition to old write-ups. As I find them, I’ll post them into posts labeled “Ajax” so that you know you’re supposed to jump up and down. (The old Well, if Google uses it then it must be OK thing.)

Starting with one of my favorites: The DHTML Menu Button from Hell, used to demonstrate the benefits of using DHTML to build menus. But if you don’t like that one, then how about Pick-A-Pair–a favorite script of porn sites the world over. Betcha can’t win the triple game.

Of course, this isn’t really Ajax, which requires the use of XMLHttpRequest. Let’s call it “Aja”, instead. Or perhaps “web safe Ajax”. But I’ll see if I can dig up something proprietary for you to see — back when these samples were built, years ago, I tended to focus on standards-based objects and cross-browser compatibility. I now kick myself for seeing that this wasn’t the way of the future.

You know, that’s why we women in technology never get ahead. We just don’t understand the right way of doing things.

update

Ooops, pointed to the wrong DHTML button example. I’ve updated the page to the right one.

Categories
Technology Weblogging

The Survival Guide to LAMP: MySQL and Saving the Pig

In the last two weeks, two WordPress weblog sites have had their sites suspended or moved to interim servers because of performance issues. In both cases, the ISPs who hosted the sites (different companies) sent snapshots of the MySQL processes that caused the problems with the emails.

I worked with one of the sites offline, but the owner of the second site, Ampersand, posted the copy of the email he received at the WordPress support site. I grabbed a sample of it, as follows:

| 2073 | theenn2_amptoons | localhost | theenn2_MT | Query |
4 | Copying to tmp table | SELECT alas_posts.*,
MAX(comment_date) AS max_comment_date FROM alas_comments,
alas_posts WHERE alas |
| 2078 | theenn2_amptoons | localhost | theenn2_MT | Query |
4 | Copying to tmp table | SELECT alas_posts.*,
MAX(comment_date) AS max_comment_date FROM alas_comments,
alas_posts WHERE alas |

Plain as dirt what the problem is, eh?

Both of the weblogs are WordPress, but the SQL that generated the performance hit differed. With one, it was the latest comment plug-in; with the other, it was the SQL to support a category listing. In addition, this isn’t specific to WordPress–it could occur with the PHP-based version of Movable Type, ExpressionEngine, or any other MySQL based tools that have dynamic access.

Ostensibly, there is something wrong with these two sites. However, they’re only representative of what we’ll most likely see more of in the future. Our weblogging tools are becoming increasingly sophisticated, the data richer and more complex, the functionality modular and extensible by every person with a text editor and a yen–all packaged up in one reusable, standard, one-size-fits-all package. To make it all even more interesting, these applications are being installed on systems where you can get all you can eat for $5.88 a month, which means that a lot of sites need to be hosted on each server in order for the company to break even.

Things are bound to start breaking. But hey, it’s all fun, until you get that email that says your site is a pig, and it’s just been sent to the butcher.

Please! Won’t someone save the pig!

Now that we’ve established that the world is out to get our weblogs, let’s focus back on these problems, and in particular, the information that the ISPs sent.

In both cases, the messages contained a phrase “Copying to tmp table” and then what looks like a standard SQL query. If you look for this phrase in any of the WordPress code, you won’t find it–it’s a MySQL process that only shows up when you run SHOW PROCESSLIST within MySQL, or access the same from PHPMyAdmin.

Now, depending on your ISP’s tech person and how proficient they are with database optimization, you may be told that “Copying to tmp” is a ‘bad’ thing and shouldn’t happen, and therefore something is wrong and the code is crappy. Well, this isn’t true.

MySQL optimizes queries before they’re executed, to get the maximum amount of data in finished format with the minimum amount of processing and time. Part of the optimization can be to build a temporary table to hold an intermediate set of data before finishing the query. In addition, if the order the data (the sorting sequence) is on one column, but it’s grouped on another, or on a column in a different table, MySQL uses a temporary table.

There is a function, EXPLAIN, that provides information about how MySQL will execute a query. Developers use this in order to fine-tune the SQL so that the use of ‘expensive’ operations are avoided. If you have access to PHPMyAdmin, and run a query, the option to run EXPLAIN is provided with the results. Still, you can only tweak a query so much, and sometimes even the optimal SQL results in MySQL creating a temporary table.

When the use of a temporary table is always bad is when MySQL doesn’t have enough memory to hold all of the contents of the temporary table; the tool then needs to copy the contents to disk. Anytime MySQL has to go to the disk rather than memory, performance takes a hit. This shows up in the processes as:

“Copying to tmp table on disk”

However, what showed up with both of the weblogs that had problems is:

“Copying to tmp table”

An open question at MySQL asks whether this is the tmp table or memory, but from the impact to the servers, we can probably assume it’s to disk.

If it is, then the problem could be that tmp space allocated isn’t enough, and MySQL is having to write to the disk frequently. Or it could be, depending on your type of MySQL, that the maximum space allocated for memory is less than the maximum size allocated for the tmp space. Or other variations of settings at the database level.

Or it could be a badly written plugin, or too many plugins, or a cheap host that doesn’t allocate enough space for the sites hosted, or less than optimum SQL query, or even a trackback attack

Get used to sleeping with the pig

Ampersand’s site, Alas, a Blog gets between 1000 and 2000 visitors a day. It’s a popular site and gets lots of comments, and spam, of course, so several plug-ins were installed to help contain it. Evidentally, it was one of these plug-ins that started to have problems, because the query is not part of the WordPress code. In particular, if you look for max_comment_date in WordPress, you don’t find it.

However, with the second weblog, the query is within list_cats, a built-in WordPress function, and looks like the following:

SELECT cat_ID, cat_name, category_nicename, category_description, COUNT( wp_post2cat.post_id ) AS cat_count
FROM wp_categories
INNER JOIN wp_post2cat ON ( cat_ID = category_id )
INNER JOIN wp_posts ON ( ID = post_id )
WHERE post_status = ‘publish’
AND post_date_gmt < ‘2005-03-18 20:48:04’
AND cat_ID <>1
GROUP BY category_id
LIMIT 0 , 30

While Ampersand’s weblog problem was being discussed in the WordPress forum, a third person also had the same problem — most likely in a completely different area.

Bacon

With the growing number of ‘cheap’ hosting sites, and an increasing use of sophisticated PHP applications and MySQL queries, not to mention the extensibility of the tools, we’ll see more of this problem–especially as we demand more and more from our tools. Think about it: how many plugins are you using with your WordPress installation?

So what can you do? As a starter, you might want to look at that ‘good’ deal you get from your host. Not all inexpensive hosts cut costs and have less than optimal installations, but you’re less likely to have a host that will patiently work through problems with you if you’re only there for the 5.88 special.

You could also trim the fat by dropping plugins that you really don’t need, and make sure that whatever tools or applications or plugins you use are fully baked, i.e. have been through a healthy bug fix period.

If a problem does occur, make sure to file a report with the developers of whatever tool you’re using, providing all the information the host provides. The SQL used in the tool may not be optimum, and being informed of problems provides necessary feedback.

Hopefully this hasn’t been more info than you want or need (”too much sharing!”). At the least, if this situation comes up, you’re not going to be as intimidated when your host sends you an email that tells you …your site is a pig, fix it, or we’re kicking you off.

Categories
Technology Weblogging

Another update

I was asked to help a group weblog with its new look a week or so ago and it ended up being more work than I expected. However, I finally finished that work today — really, really finished– and have started my final descent for the first release of this product.

I found a couple of things I need to change. First of all, WordPress creates page titles the first time you save a post. However, I think the page title should match the post title up and until you publish a post. But, there’s a difference between a title being generated and one manually entered, so if go this approach, I’ll need to add a flag to the database.

However, I’m tired of numbered posts appearing because I forget to title a post when I save a draft, and then forget to change the post SLUG before publishing.

Some more clean up in Admin to pull all SQL separate from the processing.

And I’ve finally taken my first shot at integrating RDF API in as an extension to support the metadata effort. That one isn’t going as well as I would like — but it is optional, and if need be, I’ll leave it off the first release.

Oh, I do love my interface, and my comment management, but magic quotes got screwed up again.

All in all though, it’s movin’ along.

Categories
Technology

MT free servers

I’m in the middle of preparing my annual “Burningbird’s Bash of Etech” presentation. It will have the usual: laughs, tears, and passionate outrage and sad reflection in equal measure–not to mention, intense inspection of photographs with an accompanying “is the one with a shaved head, tattoos, bulgy chest, and eye liner a boy or a girl?” discussion. You know. The usual.

First, though, thanks to Rogi, I found out my host has created its first “Movable Type” free server, named Circe. (Hee, good name.) The thread to discuss this can be found here.

I’ve been watching the processes the last week or so, on the server and in my logs, and what I’m seeing is a lot of hits for trackbacks. And I mean a lot. Many are to Movable Type weblogs but the WordPress weblogs are getting their share, too. Now, because of whatever spam protection you have, these may not be showing up in your pages; but the pings are creating a significant strain on the database, which in turn strains the CPU and the disk I/O.

For instance, if I had trackback enabled, even with spam protection in place, each trackback requestion would still generate, at a minimum, four requests to the database and over thirty function calls. This isn’t that big a deal–until you multiply that several times a minute, and across many, many weblogs. Then repeat this daily, once in the morning and once in the afternoon, because that’s how often it’s happening.

Of course, since I’ve pulled every aspect of trackback from Wordform, the most that happens is that the web server returns a “404″ whenever one of my pages is accessed with “/trackback” appended.

I have no doubts that if this much activity is happening with WordPress, which is relatively stripped down as these types of applications go, much more is happening with the increasingly complex comment spam management in Movable Type. In addition, as Annette details in the HM thread — CGI applications such as MT spawn a new thread for each trackback request. I can say that the most limited resource on a server are these threads of execution.

I most likely will ask that my site be switched to Circe, as soon as I can. And to be honest, I’m feeling pretty damn smug for deciding to yank trackback out of Wordform, right about now. For you MT folks — if you’re not running the PHP version by now, you should be. And you also need to start pressing Six Apart into providing a PHP-based comment and trackback management system.

Or switch to new software.

Or continue dealing with problems.

Your choice.