It seems like only yesterday when I upgraded to Ubuntu 14.04, but it’s been several years and it’s time now to move on to Ubuntu 16.04.
Once before I’d tried to upgrade Ubuntu to a new major release in place. In other words, upgrade my current installation.
It didn’t go well.
Over time, your server can get crufty. It can work perfectly fine…up to and until you try to upgrade the major OS release. Then you find out that settings may not be ideal, software combinations may not be healthy, or something just plain doesn’t work. Three times I tried to upgrade the OS and three times it crashed and burned.
Thankfully I had backups so I didn’t permanently lose my sites, but it was a painful experience.
Linode has been my VPS (virtual private server) host since 2010, and will most likely stay my VPS host until the day I die. One reason for my loyalty is the company’s affordability. Another is the excellent tech support, and how easy it is to bring up, and take down, a new VPS.
I chatted with the folks at Linode about possibly hiring someone to debug the upgrade problem, but they had a better idea: just spin up a new Linode VPS, do a fresh install of software, and move my stuff over. I’m only charged for the hours I have the second VPS, and the charge is fractions of a penny per hour.
Good advice. What I do now when I want to make a major change to my server is ramp up a new Linode with the new operating system. Then I can start with a fresh install of the software I want to use…sans the cruft. Once I’m finished moving the old server contents over to the new one, I delete the old server.
Setting up and securing the Site
After creating the new VPS with Ubuntu 16.04, I had to perform some basic setup and server security. I logged in via the web-based SSH client so I could access the SSH RSA key value. Then I logged in using the new IP address using Putty. I doublechecked the SSH key value to make sure I was accessing the same site.
I provided my server’s hostname, the name the machine is known by internally:
hostnamectl set-hostname phoenix
I updated the hosts file, adding my ‘seed’ domain, the FQDN (fully qualified domain name) domain I use for the new server until I’m ready to move my other domains over. I maintain two domains solely for background work, such as moving to a new server. That way I can work with the new server before switching the DNS records for my public-facing domains. I also use the background domains for working on web applications and testing code.
Linode pushes the editor nano in its documentation, but I’m vi/vim, all the way:
vim /etc/hosts
I also named my server and set the timezone:
dpkg-reconfigure tzdataNext, to secure the server I created a username using useradd and added the name to the sudoer list, so that I can use the name for root privileges.
adduser example_user sudoI then modified the /etc/ssh/sshd_config file to turn off root access via SSH. You never want to use root to log into a system.
I did leave SSH password enabled rather than using a generated key to access my server, as I access my server from multiple computers. But I changed the port number for SSH so that hackers can’t use the standard port of 22 to break into my site. I prohibit port 22 access via the firewall, which I’ll get into the next section.
Lastly, I installed Fail2ban, which monitors system access for hacking attempts. If someone tries to access my site so many times in a row and fails, it blocks them.
Setting up a Firewall
Firewall setup is not trivial. In Ubuntu, you have to configure iptables in order to customize your firewall. However, today you don’t have to configure iptables directly, because there’s an app for it. The application is called UFW, for uncomplicated firewall, and it does all the tricky bit work now.
To start, turn on all outgoing and turn off all incoming connections.
sudo ufw default allow outgoing
sudo ufw default deny incoming
Now, I can open just the connection ports I want, and know there aren’t any sneaky ones hanging around ready to trip me up.
I created an opening for my custom SSH port and ones for port 80 (default HTTP access), and 443 (default HTTPS access). That’s it.
sudo ufw allow http
sudo ufw allow httpsSince my sites support IPV6 as well as the older IPV4 web addressing system, UFW creates a port opening for each. When I type in sudo ufw status, I get:
To Action From
-- ------ ----
22 DENY Anywhere
**** ALLOW Anywhere
80 ALLOW Anywhere
443 ALLOW Anywhere
22 (v6) DENY Anywhere (v6)
**** (v6) ALLOW Anywhere (v6)
80 (v6) ALLOW Anywhere (v6)
443 (v6) ALLOW Anywhere (v6)Update:
I inadvertently tested my firewall when I spun up a Node application, and couldn’t figure out for a moment why I was having problems accessing it. Once I reached that “D’oh!” moment, I added a couple of port numbers for Node development and testing.
Installing LAMP using Tasksel
Over time you learn that you don’t need as much software as you think you do. But when you run WordPress, you’ll need is a web server and a database, as well as PHP.
I thought about going with Ngnix instead of Apache with this fresh install. The environment would be LEMP (Linux-Ngnix-MySQL-PHP) instead of LAMP (Linux-Apache-MySQL-PHP). However, Ngnix just didn’t offer enough of what I need to make the overhead worth it for me. I’ve been using Apache since it was a pup; I’m not going to kick it out just because it’s getting old dog.
Installing LAMP in Ubuntu isn’t all that complicated, but the process has been simplified with the Tasksel application. One command, only, and the latest production releases of Apache2, MySQL, and PHP are installed.
sudo tasksel install lamp-serverHow easy is that?
Once I installed the application, I added the WordPress database user to MySQL. I also imported my weblog database that I had dumped using mysqldump.
mysqldump -u databaseuser dbname > dbname.sqlAnd then imported the database into the new MySQL installation:
mysql -u databaseuser dbname < dbname.sqlUpgrade Apache 2.2 to 2.4
The new Apache installation ended up being the most work. The last time I had upgraded my Ubuntu I had installed Apache 2.2. The stable Apache installation is now 2.4, and yes, there are differences.
The major stumbling blocks had to do with changing the 2.2 directives over to 2.4 for my virtual hosts. Thankfully, Apache provides a page that simplified the process. Still, it was an interesting couple of hours getting all the virtual host files created. A password-protected subdirectory ended up taking the most time until I figured out the proper directives to use.
The virtual host file for burningbird.net is:
<VirtualHost *:80>
ServerAdmin shelley.just@gmail.com
ServerName burningbird.net
ServerAlias www.burningbird.net
DocumentRoot /home/****/public_html/
<Directory />
Options FollowSymLinks
AllowOverride None
Require all denied
</Directory>
<Directory /home/****/public_html>
Options FollowSymLinks
AllowOverride All
Require all granted
DirectoryIndex index.html index.php
</Directory>
ErrorLog /home/****/logs/error.log
CustomLog /home/****/logs/access.log combined
</VirtualHost>
And the virtual host file for the password protected subdirectory is:
<VirtualHost *:80> ServerAdmin shelley.just@gmail.com ServerName blipdebit.com DocumentRoot /home/****/public_html/ <Directory "/home/****/public_html/"> Options Indexes FollowSymLinks AuthType Basic AuthName secure AuthUserFile "/****/****/.htpasswd" Require user **** </Directory> ErrorLog /home/****/logs/error.log CustomLog /home/****/logs/access.log combined </VirtualHost>
Replacing the ‘****’ with the actual directory names, of course.
So now the software is ready, all I have to do is copy over my WordPress installation. Using SFTP from the old server transferring the files to the new made this process a snap. The following command is what I used to copy over an entire directory:
put -r directory_name
I also made a backup of all my static HTML files on my PC. I didn’t move them over because I’m incorporating support for HTTPS in my sites. One of the problems with statically generated HTML files is they’ll contain mixed content: references to resources such as CSS and JS files accessed via http:// instead of https://. If you access mixed content with HTTPS, you get a nasty warning.
Dealing with a Apache 2.4/Wordpress Multisite Quirk
One quirk that popped up when I accessed my multisite WordPress is an error in the Apache log file:
AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use ‘LimitInternalRecursion’ to increase the limit if necessary. Use ‘LogLevel debug’ to get a backtrace.
The solution was to comment out two lines in .htaccess. So far, everything seems to work. Knock on wood.
# RewriteRule ^(wp-(content|admin|includes).*) $1 [L] # RewriteRule ^(.*\.php)$ $1 [L]
Changing the DNS zone files and eliminating SMTP support
Time to change the DNS records for my domains. Instead of modifying the DNS records using the DNS manager at Linode, I moved the DNS records directly to my domain name registrar, Namecheap. Doing this, I can eliminate the need for installing an SMTP email server.
In my opinion, the nastiest piece of software to manage on a server is Postfix, the most popular SMTP (email) server. It’s tricky to configure and difficult to protect from hacking and abuse.
Frankly, I don’t want to deal with it anymore. It’s more trouble than it’s worth. I don’t need email alerts from Fail2ban since I have logs I can review. I do have email addresses featuring my domain names I need to support, but all I’m doing with the email addresses is forwarding them on to Gmail accounts.
Luckily there’s a simple solution: have my domain name registrar handle email forwarding for me. I’ve used Namecheap as a registrar for years. One of the services it provides is DNS management, including email forwarding. And, unlike other domain registrars (I’m looking at you, GoDaddy), you don’t have to be hosted at Namecheap in order to use this service. In fact, Namecheap even provides this service if you are using a domain registrar that doesn’t provide this service.
(I’m as loyal to Namecheap as I am to Linode, and this is just one of the many reasons why. Plus, Namecheap are the good guys. )
I moved all my DNS zone files to Namecheap, and added the appropriate email forwarding. I also added redirects for subdomains that I had removed when I removed the static HTML files. Of course, now I get on average over 700 404 requests a day. Over time, I’ll add redirects or just let the 404s fall where they may.
I just don’t get worried about 404s nowadays. They’re like gray hair—an inevitable consequence of the passage of time. I’ll manually load old posts I want to keep into WordPress.
I added A records for my IPv4 addresses, and AAAA records for the new IPv6. I used CNAMEs for the working subdomains. Namecheap automatically adds a TXT record for email forwarding.
Up and Running
Once the DNS modifications were made and propagated, I could access my web sites on the new server. Best of all, the performance has improved significantly, and most of crap traffic was eliminated by getting rid of the email server.