BurningbirdGord, at poeticgeek.net.
All this writing makes me questions Shelley’s humanity. I have come to the conclusion that she is actually a sophisticated robot using a markov generator to make compelling entries. There is no way a human being can consistently produce witty, helpful, and well-written articles. I swear that women like Shelley make me question the benefits of being gay.
Thank you, Gord. That is the best compliment I’ve ever had.
Recovered from the Wayback Machine.
Though at this time I’m running WordPress 1.02 on my main weblog, and hadn’t planned on adding more hacks until moving to 1.2, I did hear your calls– like the far off sad sighs of birds hidden among the dense growth of the forests I walk:
We need comment preview! Please, we need preview!
Who am I to tempt the fate of birds that can fly overhead. Ladies and gentlemen, you now have comment preview.
But I expect tests. Lots of tests.
Consider this an open comment thread. Discuss anything you want.
Anything?
Anything.
L is for Linux, A is for Apache, and P is for PHP
Before providing instructions for two of the tweaks I made to WordPress 1.2–multiple weblog emulated support and generating static pages–I need to spend some time on the issue of permissions and writing to directories from a PHP application.
Typically, when a web page is accessed from the Internet, it starts a processing thread in the operating system to allow the web server to serve that page. If you have followed this series and bravely embraced SSH, you can log into your site now and take a peek at these processes just by typing the Unix command, ‘ps’ (short for process status). To see processes other than just your own, and to get a nice, full listing of information, use the command options of ‘e’ (for every process), and ‘f’ (for full listing):
$ps -ef
You’ll get a lot of stuff back, but some of what you get back should look similar to the following:
nobody 19905 24708 0 16:58 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL
nobody 20054 24708 0 16:59 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL
nobody 20059 24708 0 16:59 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL
nobody 21016 24708 0 17:00 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL
nobody 21018 24708 0 17:00 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL
nobody 21019 24708 0 17:00 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL
nobody 21022 24708 0 17:00 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL
nobody 21095 24708 0 17:00 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL
These are threads serving web requests. The first column in the output is the ‘owner’ of the process, and as you can see, requests to the Apache web server are owned by ‘nobody’.
When an Apache web server is started, it’s started by the special user known as ‘nobody’. By restricting the environment in which ‘nobody’ operates, only a narrow window of accessibility is allowed into the system, to prevent or at least slow down malicious activity. After all, ‘nobody’ owns any directories, or belongs to any groups–what can it do?
(That’s not to say ‘nobody’ owns all web page accesses. A CGI-based application, such as Movable Type, can operate with expanded permissions in an environment (implemented through the application of a web server add on, such as suexec) that redefines the owner associated with the web request process: instead of the generic ‘nobody’, the page runs within a process controlled by the actual owner of the web site. This means that when you access a page from within these cgi-bin directories, they operate with all the permissions of the owner of the subdirectory.)
Though that nobody owns most of the web requests on your system, but since you’re not a system admin or webmaster, what does this mean to you? A great deal, as a matter of fact, if you’re using applications built on PHP. Any action within PHP that results in output to a directory requires setting permissions on the directory to allow that nefarious user, ‘nobody’, write access. Since ‘nobody’ is really anybody that means setting a directory to wide open write access.
If the PHP application allows file uploads into this directory, the file name can be manipulated in such a way that it exposes files that normally shouldn’t be read–such as the /etc/password file containing user/passwords for the server. In addition, other combinations of file names and actions can result in directories being deleted, or sensitive material being placed in system that when accessed via web server can result in odd behavior. At best.
Having a directory open to global write access is a system vulnerability. However, before you run to your server to wipe your PHP weblogging software from it, take a deep breath, relax, and access your administrative pages for your software. The first thing you should hit is a login page to provide a username and password.
PHP applications that have some form of file upload almost always have some form of password protection to keep that infamous ‘nobody’ from accessing the page, and hence uploading the ‘bad stuff’. Though the server sees the access as ‘nobody’, the application ensures that the access is from ’somebody’.
Of course, this isn’t a totally reliable solution–someone could also run a sniffer on the network, grab your password if you don’t access your weblog tools using SSL security, https, and then log into your system and do havoc. But then, they can do this with those cgi-bin applications I mentioned earlier, too.
(There is a version of suexec being circulated about for PHP called phpsuexec. However, with the limitations associated with it, including running PHP as a CGI application, I can’t see its use spreading very quickly. )
The short end to this long story is that write access and file and directory permissions are always an issue when working with PHP applications. However, by restricting global write access to as few a directories as possible, wrapping authorization about the software that does the writing, and then ensuring that good user permissions are maintained by the system administrator for our computers, unless we get specifically targetted by some of the more clever of the bad guys, we should be safe. And let’s face it: nothing keeps out the really clever bad guys, regardless of what we do, other than unplugging the machine from the Net.
However, having to enable global write access for directories where uploads or changes are going to land also means there’s some extra work for you when installing the software. PHP-based weblogging software such as WordPress almost always require that at least one directory is set to global write (usually detailed in the installation instructions). Not a problem you think, but when you create the directories, they’re initially created without global write permission. Unfortunately, since you’re in a fever of anticipation about getting the software up and running, you tend to forget to change the permissions and get an error such as this:
Sorry, I can’t write to the directory. You’ll have to either change the permissions on your WordPress directory or create your wp-config.php manually
More likely, you’ll get a less friendly message, such as the following:
Warning: fopen(../wp-config.php): failed to open stream: Permission denied in /home/…/wordpress2/wp-admin/install-config.php on line 122
If you’ve installed PHP applications before, you know what’s wrong; but if you haven’t it may take some help from support folk to figure out what’s going on. Well, until now – now you know why you’re getting those errors.
Are you curious as to why the global write isn’t set when you first create the directory? Of course you are. It’s kind of like the seventh Harry Potter book of Unix knowledge.
When creating a directory for the first time, there’s a basic set of permissions given it by default that have been predefined for our user accounts by our system administrators. This is called the umask or user mask, named that way because setting permissions can be seen as a masking operation.
Just think of file and directory permissions as a filter with three holes: small, medium, and large. If you throw a bunch of rocks into it and all the holes are open, all the rocks will fall through. However, if you cover the large and medium holes using masking tape, then the only rocks falling through are the small ones.
Masking tape. Masking. Mask. User mask. Umask. And here you thought that Unix terms were bizarre.
Anyway, no system administrator worth her salt would ever define a umask that automatically sets directories to write enabled: not unless it’s her last day of work and she’s just won the lottery. And she’s a malicious bitch to boot.
No, you, as just any old user on the system, will have to change the global write permissions using the chmod command, and has luck would have it, I’ve already written how to use this command.
(Or you can have your FTP program change the permissions for you if you’re bypassing the hacker track on this series. )
Just remember to keep those file write errors in mind as you read the next few LAMP essays – not that there’s any doubt you’ll forget this essential element after this essay. Why, I bet you never forget write permissions on a directory ever again.
Recovered from the Wayback Machine.
Last week Liz Lawley wrote what I thought was a very thoughtful piece, both at her weblog and at Many-to-Many. She wrote:
I think we’re watching a significant moment in weblog history. Justified or not, the anger among MovableType’s users will push many of them to new tools, and has permanently changed the perception of SixApart by its customers. The users have spoken, and the landscape has shifted.
I agree with her completely. More than that, I think that this experience ultimately will prove healthy for all parties involved.
Years ago when I worked at Boeing, I worked very closely with the Oracle folks in the area because we were one of Oracle’s biggest customers. About that time Oracle released a new version of their flagship database product: Oracle 6.0. It generated a considerable amount of discussion on our floor, not to mention a lot of sleepless nights because unlike previous versions, 6.0 was a change in the architecture of the product, not a feature release. And boy was it a change, going from an old partition system to the new tablespaces, and providing a new fangled thing called row level locking..
Later on I was chatting with one of my friends at Oracle about the product, after the dust had settled and we were finally past the move. I remember saying something about how it takes guts to pull the rug out from under your customer’s feet with a new architecture. He just shook his head and said I didn’t even know the full truth of how much of an impact this move had on the company. He said the customer reaction was so severe that Oracle came within a half a step of having to declare bankruptcy, and going out of business.
It was a bite the bullet moment for Oracle.
I was reminded of this story while reading the criticism directed at Six Apart, and realize that this was Six Apart’s bite the bullet moment. All software products have to, at some or another, go through a bite the bullet release. Microsoft did so with .NET. IBM did so with DB2 (and Linux). Oracle, Sun, every company that has a maturing software product will at some time or another, have to re-think it’s architecture or strategy or consumer base and possibly issue a bite the bullet release.
If the company survives, they’ll look back on the moment, realize what they did right and wrong and hopefully be a better, stronger company as a result. If the company doesn’t survive, well, this too is a growth experience.
Just as Oracle’s customers did long ago, Six Apart’s customers are also having to bite the bullet with this release, jarred out of their complacent dependence on regular, no or low cost staircase upgrades, and forced into reviewing what they do, and don’t want from their environments. If the Six Apart crew has learned, hopefully, the value of good communication, we’ve learned that we can separate the technology from the people and make decisions about what’s best for us, overall, without having to be worry about the personal consequences–after all, the technology we use is a tool, not membership in a clan.
When the dust is settled, we’re all going to be a bit more mature, and our environments are going to be a bit richer; but we’re not going to be moving in the same direction. Some will stay with Movable Type, others move back to Blogger, or on to new environments like ExpressionEngine or WordPress or Textpattern or any of the other wonderful tools that exist – too many to list.
As Liz said, the landscape has shifted, and I think this is good; we’ve all been moving in lockstep too long. We need diversity, and not just in our technology.
Perhaps I should have stopped reading Many-to-Many at this point, otherwise I wouldn’t have read Clay Shirky’s amazingly condescending writing today. Rather than focus on the gentle, even slightly melancholic reflections of Liz’s post, or focus on the fact that Six Apart’s recent experience is a wonderful demonstration of how not to communicate with customers, he responds with the following:
First, most of the analyses have focussed on the users, as if MT were a word processor whose main value was to individuals. Seen in this light, the users complaining about the changes are behaving childishly.
However, that’s what users always do in this situation — the reaction is baked in. The problem is not with these particular users, it would be with any group of users in a similar situation. Weblogging tools are community enablers, and when you create community, you engage people’s emotions. Period. Community membership precedes rationality, both historically (all higher primates are social) and literally (children attach to their families before they can talk.)
The dilemma for people who build communal tools is this: if you want something that hooks people emotionally, you cannot have rational users, and vice-versa. And when you build a tool that helps create a social fabric, changes to the tool trigger social anxieties. Always.
…
This is not to say that MT shouldn’t charge for their product — we use it here, and I’m assuming we’ll upgrade when the time comes. It is to say, though, that because MT has succeeded in creating social value, you cannot expect users to act rationally to change. If you want users to really care about a piece of social software, they will invest in it emotionally. If you change the bargain they think they are operating under, even if that bargain is merely implicit and obviously unsupportable and even if you have the absolute and unilateral right to change it, they will freak out.
According to Clay, this really isn’t about money. It’s about the fact that we users are regressed infants, crying out when the bottle is taken away. Or is that chimps losing a banana?
It is impossible for me to understand how Clay can disregard what many of us have been saying so completely as to not only miss the mark, but to do so in about the most offensive way possible. But then I had to look at who he socializes with in the social software arena, to better understand where Clay is coming from: he’s used to interacting with people who are comfortably situated, and therefore has no idea–none– about how the difference between $70.00 and $150.00 (or $700.00!) can generate such a reaction.
After all, have we not spent the last year listening to the social software people as they talk about this trip to London and that trip to Zurich? How many conference reports have we had to sit through, or photos of dinners where all the faces looking amazingly alike from event to event? How many posts focused on this new iPod, or that new cellphone?
Didn’t Dave Winer demonstrate this so aptly? Calling us ‘childish’ because we reacted in shock to the license prices, while saying that after all a dinner costs $100.00, a hotel $150.00 –why are we bitching about software that costs $70.00?
These people, they don’t have a clue about how the rest of us live. They don’t know that for most of us, the difference between $70.00 and $150.00 is the difference between making a car payment or not; paying for tuition or your kid’s dentist bill; or paying one’s health insurance premium; or even making the rent or buying food.
Dinner cost $150.00? My big treat is to take my roommate and myself out for a concrete at the frozen custard place, and I can tell you, we drive ourselves, do not take a cab, and it costs less than $10.00. It also doesn’t come around that often, either.
Liz gave us the benefit of the doubt, that we were complaining about the cost because many of us could no longer afford to use the product, and we were given no warnings that such price increases were just around the corner. And she did so gracefully, in such a way that there is no loss of dignity–that we’re all shocked about the costs, we’re all in this together.
What Clay has done, is rubbed our noses in the fact that there are those that have, and those have not; and then made an assumption that everyone is a ‘have’ and therefore the complaints were about emotional investment not the cost.
Next time Clay, leave your assumptions at home with your Gucci case, next to your new iPod and the tickets from your last trip. You’ll excuse me as I go back to the free software us poor folk use.
P is for PHP
Right about now I imagine all those folks who downloaded the promotional free versions of ExpressionEngine given out over the weekend are finding something out: not all PHP is ‘open’. PHP source code can be encrypted so that the source code cannot be viewed, much less edited or copied. From the bit of open code within the EE files, it would seem that pMachine, Inc. (parent company for EE) is using Source Guardian’s encryption technology to protect their code. Encryption adds an extra step of security, but more than that, protects a company’s proprietary technology.
Which *blushes, kicks dirt, looks sheepish* I should have remembered before making the statement about EE.
My apologies for being wrong about ExpressionEngine’s use of encryption. Now, who has a copy that they’ll allow me access to so I can peek at the code? I am intrigued by the module architecture associated with the tool.
These source code protection systems are rather ingenious. How they work is that they provide client side applications that encrypt the PHP files and generate matched decoder files that are then included as part of the install of the source code. The call to load these decryptor files is in ‘plain view’ so that the PHP installation can run this part of the code without any additional software. Once the encryption functionality been loaded, it acts as a filter, providing the processing necessary to decrypt the code and allow it be to processed. You can see these files in a subdirectory included with ExpressionEngine labeled ‘ixed’.
Of course, for this type of system to be successful for webloggers, it must run in most environments without having to make any administrative changes on the server, and this depends on how secure the PHP installation is. For instance, if the server is running PHP in what is known as PHP safe mode, and has disabled dynamic module loading (the ability to load new PHP modules in at run time, which is what Source Guardian needs), then the application will fail and the only way for it to run would be for the system administrator’s to either disable that aspect of PHP safe mode or, what is more likely, to add an entry to the PHP configuration to allow this application to run.
(Note for Hosting Matters users: Hosting Matters servers are not running safe mode. Why? Safe mode is a pain in the butt, and breaks most of the really great open source PHP applications. As the PHP developers themselves have said, security should reside in the server management and operating system, not in the language tools. )
I think the fact that PHP can be encrypted is a surprise for some people because it’s such a major component of the whole open source movement. Myths aside, yes, you can use LAMP technologies, including PHP, to create closed source, proprietary, protected, copyrighted, encrypted applications. Contrary to any expectations you might have because of my strong, verbal support for open source technologies, I don’t believe there’s anything wrong with proprietary software–not as long as good faith is maintained between the company publishing the software and the customers using it.
Is there a risk that if the company folds, the customer is stuck with something that is no longer supported? Sure, but the same can be said of open source projects that never inspire any new community effort; Source Forge is littered with these.
I prefer using open source products as much as possible for my own site and home use because as I’ve said before, I’m a tweaker. I like to tweak the code, and I like to make the tweaks available for others to use. But I see nothing wrong with people using closed source applications. If I felt that all closed source applications were wrong, I’d have to give up my beloved Adobe Photoshop. I’ll sacrifice my two laptops out the window before I let you take Photoshop from either one of them.
(Fly, TiBook! Fly!)
And you couldn’t hold a gun to my head to get me to have anything to do with C code, again, open source or not. No, I take it back. You couldn’t hold a gun to my cat’s head, and get to me to work with C code, again.
This isn’t to say you can’t tweak in ExpressionEngine. It provides a ‘plugin’ environment, just as Movable Type, WordPress, and Textpattern do. The only difference with these applications is you can’t hack the internal code in ExpressionEngine even if you wanted to; you can with Movable Type but you shouldn’t because of license and future compatibility; and you’re welcome to with Textpattern and WordPress, and can distribute the hacks legally–as long as you and the other users of your hacks are aware that your code may be overwritten in future versions of these products, so you’re better off trying to work within the plugin environment as much as possible.
Speaking of WordPress, hacking, and plugins, back to work…