Category: Technology

Categories
Technology Weblogging

Recent Comments Trackback – Introduction

Recovered from the Wayback Machine.

My weblogs show a Recent Comment/Trackback list that I’ve implemented using SQL and PHP rather than MT tags. The main reason I didn’t use tags is that I filter comments to showing only those that are on posts 30 days old or newer. This helps focus comments on current conversations, and also helps cut down some of the comment spamming problems.

An additional requirement for my Last Comments/Trackbacks list to intermix the comments and trackbacks into one list, showing the most recent items regardless of type of comment — local or remote. There are plug-ins to use to do some of this, but I like to keep my fingers into the PHP/SQL world.

To manage this, what I did was use what is called a SQL union. A SQL union operates pretty much as it sounds: it creates one set of data that’s the union of the result of two separate queries, and this data is what’s returned to the PHP program for processing. Unions have been around in Oracle and Sybase and other databases for some time now, but only added to MySQL in version 4.x. Luckily most of us are using 4.x.

The query string I’m using in my PHP process is:

 

$sql = ‘( SELECT tbping_id, tbping_source_url, tbping_title, entry_title, entry_id, blog_archive_url, tbping_created_on, 1 \’flag\’, category_label FROM mt_entry, mt_tbping, mt_trackback, mt_blog, mt_placement, mt_category WHERE entry_id = trackback_entry_id AND trackback_id = tbping_tb_id and entry_blog_id = blog_id AND entry_status = 2 AND placement_entry_id = entry_id and placement_is_primary = 1 and category_id = placement_category_id ORDER BY tbping_created_on DESC LIMIT 20 ) UNION ( SELECT comment_id, comment_url, comment_author, entry_title, entry_id, blog_archive_url, comment_created_on, 2, category_label FROM mt_comment, mt_entry, mt_blog, mt_placement, mt_category WHERE entry_id = comment_entry_id AND entry_blog_id = blog_id AND entry_status = 2 AND placement_entry_id = entry_id and placement_is_primary = 1 and category_id = placement_category_id and TO_DAYS(NOW()) – TO_DAYS(entry_created_on) <= 30 ORDER BY comment_created_on DESC LIMIT 20 ) ORDER BY 7 DESC LIMIT 20 ‘;

 

As you can see, this query is not necessarily for the faint at heart, or someone who isn’t familiar with SQL. I could at this point just tell you to copy and past this into your own page. However, if you’re like me, you don’t necessarily like using technology without having a better understanding of exactly what it is you’re doing, and why.

In this multi-part roll-out of the MySQL/SQL for Poets weblog, I’m going to cover all the different components of this query, including providing a basic introduction to the SELECT statement, using functions with queries, and ending with the UNION and how this query is used within PHP to provide the recent comments/trackbacks list.

Categories
Internet

Transferring domains

Netcraft started a bit of a panic with an ill-considered writing about a new ICANN ruling that if a registrar or domain owner doesn’t respond within five days to a request, the site can be transferred automatically. I noticed that Dorothea just wrote on it, as have others including Kottke. But as Kottke found out the ruling doesn’t make your domain as vulnerable as it would first seem.

How it would work: I’m right in the middle now of transferring four domains from Dotster to GoDaddy. I’ve initiated the appropriate requests and punched in the appropriate codes. The transfer hasn’t completed yet. Why? Because Dotster is known to be a problem when it comes to you moving your domains away from the company. They don’t seem to want to let them go.

In the past, any registrar could make it extremely difficult for you to transfer, until you finally give up and continue to pay them to maintain the domain registration–what amounts to legal and technical extortion.

However, now, registrars can’t drag their corporate feet. Unless a dispute occurs, the registrar must respond to a legitimate request from another registrar in five days, or the transfer automatically proceeds regardless. Not another person, a registrar. A registrar that is responsible for making sure that all the of the appropriate authorizations are in place to prevent someone from hijacking your site.

Now remember, a registrar has certain obligations they have to meet to continue being a registrar, including getting the appropriate authorizations in place to ensure that someone can’t steal your domains. Registrars that don’t quickly lose their registrar status, and whatever dubious transfer is initiated is terminated. If you want to move a domain, they have to contact the domain owner and inform them of the request, and ask the owner to respond either confirming or declining the move.

(I also had to provide a code to move a .info domain.)

Much of the hand waving on the part of registrars in this is let people know about this change, which is significant. It also highlights the reason the contact information for the domain has to be up to date (though you can use third-party companies now to hide your information from casual WHOIS users).

However, a great deal of the hand waving is marketing and registrar intimidation. By all means, lock your domains–extra precautions should help (but check out the policy for unlocking the domains, first, before you push that button). It’s more important, though, to ensure that your contact information for your domain contains an up to date email that will ultimately resolve to you, so you can dispute a domain request if one comes up unexpectedly. And to ensure that you’re given a timely reminder when your domain is about to expire.

Now, we’ll see if Dotster complies with this new ruling, because it’s fast approaching the time when it has to allow the transfer.

Categories
Burningbird Technology

You might see this

Recovered from the Wayback Machine.

You won’t see this. It doesn’t exist.

Or at least, it doesn’t exist – yet – in the new Burningbird home, but does in the old.

I had hoped to be farther along in the transfer, but limitations in the software in the shared environment are slowing things. Rather than being able to log into each system using SSH and FTP files directly, I’m having to download them to my PC and then upload them to the new site. The first of many limitations I’ll face moving from the dedicated server to the shared environment.

However, once moved, the limitations will be offset by not having to worry about constant software upgrades or installing patches. A good trade, all in all.

I did implement the new file system naming here in the old site to make sure it would work on the new. Worked beautifully. You have to love Movable Type templates.

Yes I am aware that my recent comment/trackback functionality is broken with the new naming system. I am looking at solutions, or the possibility of using something else on the new site. You’ll need to click through to the individual page for now to see these items.

I’ve also added a link to the PostCon RDF file for each individual page. Right now all the link does is pull up the RDF meta file. When I’m finished, it will provide either a raw RDF feed, or a prettified HTML feed. I still want to see what RDF vocabularies created elsewhere I can incorporate in. If you see anything, let me know. The files are created by a template – let me know if you want a copy.

I’m itching to get into the design on some of the pages, I’m in a re-design mood. First things first, get everyone safely moved and working, and then I can play. Since I have 9 webloggers, and 15 weblogs (10 of which are MT), with about 23 domains, I think I have enough to do.

I can’t believe what a difference cleaning out the old Burningbird made – from 3000+ entries down to about 1100 or so. Most of these were drafts never posted, or entries used for the photo blogs. And the new naming system is based on category and title – I don’t care much for the date-based directory.

The kicker is going to be doing a string substitution in the database to replace the old photograph URLs with the new one for mirrorself.com. Can’t do that, though, until the DNS change for the domain makes it through. I have a feeling this is going to be slow, and we’re going to be in for interesting times this week.

DNS. You can see start to see increasing problems with DNS. For those who asked about my prediction for a rough year next year, here’s what I wrote in an email (with some edits):

Why bad? It’s a contentious US election year, with a much more web enabled populace, not only in the US but throughout the rest of the world. More software is being built with holes in it, managed by more web holders who know little or nothing about web technology. ISPs, to stay competitive, are adding more people to individual servers so they can cut costs. And more sites like slashdot and other sites that serve up mass traffic attacks act as a DoS because we just can’t handle any variation in established access patterns. Nothing has room to give.

As more and more webloggers voice an opinion on political issues, or religion, or even spammers, more and more sites will become the targets of DoS attacks. But this doesn’t just burden the site – it burdens the entire system.

Other forms of ‘junk’ connectivity are being outlawed, including junk mail and phone solicitation. The only avenues left to the swarms determined to separate us from our money is door to door, and your computer.

Add to this an increasing privacy issue: with governments becoming more aware of how web enabled their populaces are. My site being scanned by the California tax organization is just one example. And other governments will block, but they’ll use our own paranoia to do so. For instance, some of the original comment spammers had a Chinese IP address, and people were blocking entire networks of Chinese readers because of it.

Which leads to blacklisting. Blacklisting is going to grow as a problem, which means huge blocks of IP addresses are going to get into SPEWs and others lists like this, tainting them so they can’t be used again. IP addressing is enough of a problem now without this.

(Do you know that once an IP is ‘dead’ to a spammer, its released back for some poor old soul to use for their legitimate site? Do you know how long it can take to clear an IP address from all the lists?)

To protect against the bad guys we’re loading our systems down with software that checks this list or that, lists of which are growing exponentially in size. Each of us doing so burdens already over-burdened CPUs on machines holding more and more people, each added to keep costs down in a Net, which is becoming increasingly cheaper to get into, but still being supported on the same architecture that existed years ago.

Used to be you needed to be a smart hacker to cause problems. No longer – not with today’s new user friendly destructiveness. There is DoS software you can download and run without any programming experience. There is software you can get from the W3C that will allow you to post comment spam. And people are hurting themselves – they still won’t stop opening attachments!

How about IM, chat rooms, IRC, moblogging, audio files, video files – do you think that bandwidth grows out of thin air?

We webloggers have to accept that our own actions are adding to the increased burden – very few sites update, check, or send bots out like webloggers. Tell me, how many times were your index.xml and index.rdf files accessed in this last hour? We’re putting significant burdens on the system in comparison to our numbers.

And all of this hits us at our most vulnerable spot – the DNS and the routers.

All in all what we have is a badly educated populace using the Net more and more, buggy software, smarter hackers, and a great deal of overreaction. And just to make things fun – lets put voting online.

I was glad to see I’m not the only one talking about the problems of blacklists by URL or IP – Mark Pilgrim has also covered it, and many of the same concerns I’ve had. (And I love Mark’s new header.)

I’ve already had two ‘poisen pill’ blacklist entries with URLs for weblogs.com and fda.gov. Mark talks about reactions to his writing that are similiar to what I’ve had – why am I saying this negative stuff? Where’s my solutions?

You want a solution? Drop your weblog, sell your computer, have the electrical company turn off electricity to your home, or better yet – move to a cabin on a mountaintop somewhere. Use paper and pen, and get yourself a carrier pigeon – the spammers haven’t gotten to them yet.

I’m not dooming and glooming to scare people away from the Net, but more to get people to realize that comment spammers happen, down times happen, s__t happens – but overreaction just makes it worse.

And knowledge. Knowledge is power in this environment. The more you know, the better equipped you’ll be to ride the rough tides without getting wet. Speaking of which, I added a new For Poets site: MySql/SQL for Poets.

Back to work. Hoo-rha.

Categories
Burningbird Technology

Semantic web, live and in color

I’m taking advantage of this server move to make some pretty drastic changes in my own sites. For instance, I’m not going to try maintaining the old numbered system for my Movable Type page names because, to be blunt, it’s a mess.

What with my recent tax evasion weeding out, and my habit of splitting weblog entries across different sites, the numbering is completely out of whack. Enough to bother even me, virtual slob that I am; for the anal among you, it would be enough to send you into a coma.

I contemplated a weblog redesign — something all new. My first thought was to put a big graphic at the top of a half naked man, but then I thought, what does this tell people about my weblog? So I discarded that idea. The For Poets sites has a look I like and I considered using it with this weblog, but displaying different photos every time you access the page. However, this idea is too much like Jonathon Delacour’s and I don’t want to steal his mojo.

Besides, I like my weblog look. I’m used to it. It suits me and what I write about. I may, however, change the look of the photo blogs, and I’m definitely changing the rest of the sites, such as burningbird.net.

The photo blogs are going to their own domain, mirrorself.com. All my photographs are going to this domain, and you can imagine how interesting this is going to be with all the embedded photos I have in my pages, and the number of photo blogs I have (each with hardcoded absolute URLSs). The For Poets weblogs are also going to their own Movable Type installation, and will be using the new page naming system. There aren’t that many For Poets weblog entries so doing redirects could be handled manually. However, with Burningbird and the rest of my stuff, we’re talking a significant impact. I have an application that’s currently tracking requests for missing resources and all I can say is, you sure can tell I’ve been online a long, long time, and that I move things around a lot.

One challenge with splitting my weblogs into completely different MT installations is my current comment/trackback facility. Normally this goes across all the weblogs; through this approach, to be blunt, I own Blogdex, as a comment for one post is repeated across all weblogs and robots see this as a fresh link to the post. I’ve been in the top Blogdex ranks every weekend for two months (weekends are slower linking times.) I’m trying to decide if I’ll find a way to work across databases, or to be kind to Blogdex.

To handle the Burningbird weblog reorganization, I’m putting my little PostCon application into full gear. The only part missing on the application is the forms-based front end that allows you to create a PostCon RDF file from scratch. I really don’t like doing forms-based development — I like working backend stuff. However, I don’t need to have the forms-based component right now. It would be handy, but I don’t need it.

(What would be nice is a generic forms application that can be used to define a data model, automatically create the forms, and then record data to create the serialized RDF/XML files. Wait a sec, I do! It’s called Protege. I’m using Protege for my PostCon pages that aren’t being generated through Movable Type.)

I integrated PostCon into Movable Type some time ago, but now I’m increasing the integration and am using pieces of PostCon, as well as Movable Type to handle the redirects — from old numbered pages to the new page system. More than that, though, is that each page now has its own particular history — what did the resource used to be named, what is it now, who wrote it, what’s it about, and linkage info. All in a machine readable format, that can also be viewed by people pushing a button on each individual page and seeing the ‘hidden’ page self-description. There’s a little FOAF in this, as well as a few other odds and ends RDF vocabularies that I’m absorbing.

I’ll be writing all this up in my Semantic Web for Poets site. I hope to show that the semantic web starts small, and starts when each of us takes a little bit of extra time to record just a little bit of extra information that could be helpful down the road. Yes, PostCon uses RDF. But it also uses plain old, Perl, too, and is served through Apache, and run on Linux. The entire Internet did not have to be rewired in order to use it.

For those who like moving parts, yes, there’s even some moving parts, though my weblog still doesn’t talk to my toaster.

Caveat on all of this, though: There is going to be some major changes and expect a rough week for my sites. Not for anyone else — the other weblogs should move with a minimum of fuss and bother.

Categories
Technology

State of Greek: Interchangeable Parts

I had promised to return and finish the State of Geek series, but I haven’t been in much of a mood for it. I have such mixed feelings about ‘geek’ lately. I am once a geek, but I am also not a geek — one foot in, one foot out.

The good news about the job market would seem to preclude these writings because, as it seems, the problems are all gone — hail, hail, the jobs all here. But you know, and I know, these jobs are not geek jobs. No, the hot degree now is in Business — the degree we laughed at when we trotted out into the work force with our hot and heavy tech credentials.

(Where were the laurel leaves and the whispered, “Thou art mortal. Thou art mortal.” in those days?)

America has become a service economy, which means we export raw material and import finished product and most people are employed facilitating this whole process. But among the moving parts, don’t count on tech or manufacturing. And the smug bunch holding up their biotech degrees? Remember those laurel leaves — you’re next.

All in all, this is not a healthy situation for a country to be in, but it is a short-term cost effective solution for corporations barely able to keep up with their bonus payments and still show inflated profits each quarter.

Still, I am less a geek now than I am a writer or photographer. Why should I care that the geek jobs go overseas? You might say my geek job caught that ship two years ago, and two years is a long time to stand on the pier, waving Bye bye. Bye bye.

Then I read stories such as a recent one in the Mercury News (thanks to Head Lemur and Ralph Poole):

Avinash Vashistha, managing director at San Ramon-based offshore consulting firm NeoIT, loves telling the story of asking a Silicon Valley executive this year which jobs he could offshore.

“Could you move this person’s job?’’ asked Vashistha.

“Oh, no,’’ the executive said. “I couldn’t move her job. She’s been here for 25 years. It would take eight people to do her job.’’

“Very well, we’ll hire eight people to replace her,’’ Vashistha said.

NeoIT calculated that the company could hire eight people to replace that one longtime employee and still save 20 percent by moving the entire division overseas, Vashistha said.

There is become two types of people in the world — those who control and those who work. When we, who work, become nothing more than cheap, non-differentiated interchangeable parts to those who control, then there’s a lot more at stake than some geek jobs in the States.