Author: Shelley Powers

Categories
Technology

Moving servers

It was time for me to upgrade my version of Ubuntu, from 18.04 to 20.04. I upgraded my software, thought I had a clean site, and tried to upgrade in place. After all, upgrading from 16.04 to 18.04 was a simple one line command.

Moving from 18.04 to 20.04 was not so simple, and the upgrade failed. Time to do a manual build of a new server and port my site to it. Which also ended up being more complicated than I thought it would be.

Moving to 22.04

First, if I was going to go through all the work, I was going with the latest Ubuntu LTS: Jammy Jellyfish, otherwise known as 22.04. I spun up a new Linode instance of 22.04 and set to work.

The LAMP installation went very well. I ended up with not quite the latest Apache, since the absolute latest wasn’t supported on 22.04 yet. However, I added the famous Ondřej Surý repository and I was good to go:

sudo add-apt-repository ppa:ondrej/apache2 -y

MySQL is 8.0.29 and PHP is 8.1.

All that certbot stuff

I had manually built a new server when I went from 14.04 to 16.04, but times have changed. That move was pre-HTTPS, pre-HTTP/2, pre-HSTS (HTTP Strict Transport Security), well, basically, pre-everything. I had the support in my existing server, so I know my pages and installation are clean. But the sheer amount of work to set it up again was a bit daunting.

Thankfully, since I had made these moves in the past, my site was already clean. All that I needed to worry about was installing certbot to manage my Let’s Encrypt digital certificates.

You’d think moving a server wouldn’t be that unusual, but neither Let’s Encrypt nor certbot cover what to do when your certificates are on one server and you need to set them up on another. Searching online gave me two options:

– copy everything and change symbolic links for the certificates

– just install new certificates on your new server, and delete the old

And that’s when things got sticky.

Where am I and who is managing my IP address?

When I made the move to 16.04, I was manually setting up my network configuration using ifupdown and editing the /etc/network/interfaces file. But when I went to 18.04, netplan was the new kid on the block for network configuration.

The problem is, I had one foot in both camps. So when I tried to access a test page on the new server, it failed. I certainly couldn’t run the certbot web validation for installing a new digital certificate if I couldn’t even serve a simple page on port 80.

In addition, Linode has the ability to manage network configuration for you automatically, so if you change servers and IP addresses, you don’t have to do a thing. But when I tried to turn it on, even SSH no longer worked. I had to restore the site from a backup.

It took a bit of painful digging around, but I finally upgraded my network configuration to netplan, and only netplan. I could now use SSH again, and install a new digital certificate for my test domain. But then, things got tricky again.

I hate the old propagation thing

When I created the new Linode server, I installed it in the Atlanta data center rather than the Dallas center I was using with the old. After all, Atlanta is now only a couple of hours away.

But doing so meant when I switched, I had to update my name registrar to set my DNS entries to the new server IP addresses. This is a pain, in itself, but it’s also a bit of a fragile time when trying to determine if my site will work on the new server. After all, you don’t want to permanently change your IP address only to find out your site doesn’t work, and then have to change it back. And digital certificates kind of mean you have to have all or nothing.

Thankfully, Linode had a peachy keen workaround: swap IP addresses. If two servers are in one data center, you can swap the IP address between them.

Of course, doing so meant I had to migrate my existing site to the new data center and change the DNS entries, but still, it would be worth it to be able to switch back and forth between servers when making major modifications. And the migration should be a painless button click from the Linode cloud control manager.

So, I migrated my old Linode VPN to Atlanta, and then tried to swap the IP addresses. Crash and Burn.

IPv4 and IPv6

What I didn’t know about the Linode IP swap facility is that it only swapped the IPv4 address, not the IPv6 address. So when I did the following

ip -a

My IPv4 address reflected the new server, but my IPv6 address reflected the old, and everything was just broken. Again.

The only recourse at this point was to bite the bullet, make the move to the new server, do the DNS propagation, and then deal with the digital certificates. I put up a warning page that the site might be off for a time, had a coffee. and just made the move.

After the move, I thought about doing the Let’s Encrypt digital certificate copying like some folks recommended, but it seemed messy—sort of like the network configuration issue I had just cleaned up.

I used certbot to do a new installation, and the move was flawless. Flawless. This is really the only clean way to move your site to a new server when you’re using digital certificates:

– Make sure you site can support port 80, at least temporarily

– use certbot to generate new digital certificates for your site(s)

– delete the old server and certificates

Five Years. I’m good for five years.

So here you are and here I am: on the new server with all new software on the data center closest to me, with clean, uncrufty network configuration and sparkly digital certificates.

Best of all?

Jammy Jellyfish has standard support until April, 2027. I’m good for five years; ten if I want extended support. And who knows where I’ll be in ten years.

Probably back here, doing it all over again.

 

 

 

Categories
Just Shelley

The story of the check valve

There’s a story behind this older post. Bit long, but might be fun if you have nothing else going on.

Municipal size check valve with engineer standing in front for perspective
Our place in O’Fallon was in the unincorporated county area, which means it’s a hit or miss what kind of service you get. Originally, the homes in the subdivision had septic tanks, but our plot couldn’t be developed until there was a sewer line, because of the steep hill in back.
When the O’Fallon gravity line was added, our house was barely in range, and they took the lateral sewer line to the manhole, and terminated it in the manhole.

Now, terminating a lateral sewer line in a manhole is a bit of a no-no, but since we were at the start of the line, they thought it would be OK.
The problem is, Lake St. Louis, next door, grew too fast and there were issues with its sewer. So they drilled a _force_ main from Lake St. Louis and terminated it at the manhole for our gravity main.

This is engineering insanity, and actually illegal in most of the country. But such is the quality of sewer management in St. Charles county, Missouri.

What would happen is every time it rained over a certain amount, typically about 1 1/2 inches to 2 inches in a 24 hour period, the water seeping into the force main would overwhelm the man hole and would force the flow back to the next outlet.

Our house.

So when it rained, we couldn’t use our toilets, and if it rained enough, raw sewage would back up into our shower and bath.

I was polite about complaints at first, but then got tired. So, every time this would happen, I would call O’Fallon and tell them we had a sewage backup. They’re required by law to come out to check it out, and it also gets recorded as a sewage backup for the environmental powers that be.

I wasn’t a squeaky wheel…I was a full out bitch. There wasn’t anyone in the sanitation department of two different systems that I didn’t contact by email, or in person.

Water District 2 (force main) blew me off, but O’Fallon took it seriously. The assistant supervisor explored options everywhere for what they could use to protect our lateral sewer line.

In the end, they had the company that developed the check valves you see here, develop a custom made one just for the end of our lateral line in the manhole.

When they installed it, the force main was shut down temporarily and two crews came out: one for the manhole to install the device, and one further down to send fresh air into the manhole.

It installed in about 10 minutes. That was it. And it worked beautifully.

From then on, no matter how much rain, no backup. And we could continue using our toilets, since even when the valve closes, it could still expel liquids out.

When we sold the house, the day before closing, we stayed in a hotel because everything was packed up. That night was one hell of a storm—over 5 inches of rain fell. When we got to the house, the sanitation crew was out washing down our neighbor’s yard because the sewage had overflowed into their yard. It was a mess.

The neighbor across the street came over and said for the first time, they had backup into their house and into their washing machine in the basement. He knew about our previous problems, and he and a couple of other neighbors thought our place would be destroyed by the sewage, and right before closing.

Nope. Not a drop. Clean as a whistle.

These things look simple. No whirring lights, no gadgetry, just plain rubber. But they are brilliant.

And if your wheels don’t squeak nothing good will happen.

 

Categories
Places

Growth isn’t always good

I moved to Savannah from Missouri a couple of years ago. The city I moved from, O’Fallon, is the fastest growing city in Missouri. It shows, and not in a positive way.

We had little in the way of parks, few sidewalks, no bus system, and wall-to-wall stores and shopping centers (and offices). The roads are a mess, and the water/sewer utilities are a disaster.

One of the more famous parks, the Katy Trail which uses the old railroad system to create a walking and biking trail that crosses the state, was damaged when the county decided that heck yes, a developer can build mega-priced homes on the cliffs overlooking the trail.

It’s not a pleasant place to be solely because the county and the city value growth over quality of life.

So take it to heart when I warn you that growth isn’t always a positive thing.

Housing is an issue already in Savannah, as is strains on the necessary utilities to support a growing community. Savannah is already constrained by being bordered by an immovable object, Fort Stewart—not to mention the rivers, flood plains, and marshes. And I’ve heard many folks complain of too much development around already overcrowded roads and freeways. Big big houses on tiny matchbook sized lots, and all of it too expensive for the average citizen.

Savannah is not considered a good place to retire because of lack of adequate medical care.

The real question is: what does Savannah want to be when it grows up?

Does it want to be a major port with lot after lot full of containers and warehouses? Does it want to be an industry hub? Or does it want to be a quaint tourist town with lovely homes and parks and waterfront activities? How about being a new Hollywood or artsy academic center with SCAD?

All of these aren’t mutually exclusive…but it takes care and caution and planning to do it right. What I’m seeing now is less careful planning, and more of what happened to O’Fallon in Missouri. And once you’ve done the damage, there’s no going back.

Categories
Political

Putin: Russia’s Trump

Putin and Trump shaking hands
Vladimir Putin and Donald Trump meet at the 2017 G-20 Hamburg Summit

Whatever we want to say about Putin from this moment forward, let’s all agree to no longer use the word ‘shrewd’ to describe him. Because his invasion of Ukraine was anything but shrewd.

Continue reading “Putin: Russia’s Trump”
Categories
Critters Legal, Laws, and Regs

Feld Entertainment Circus Elephants Finally Find a Home

Take equal parts dealing with Trump as President and then COVID and you have an explanation for why I didn’t notice that there was a piece of outstandingly good news beginning in 2020.

The Feld Entertainment/Ringling Brothers elephants I had written about in the past have finally found a good and loving home in a sanctuary in northern Florida.

For years I followed the animal welfare court cases related to Feld Entertainment’s indifferent and callous care of the elephants in the organization’s famous circus. I had though of writing a book on the longest of the court cases, Animal Welfare Institute (formerly ASPCA) vs. Feld Entertainment. Even today, one of my most popular web pages is a PDF listing incidents with elephants in circuses in North America.

When Kenneth Feld retired the Ringling Brothers Circus, and the circus elephants, they were out of sight out of mind but we all knew their care would continue to be indifferent, their future dismal.

Now, the elephants run free at the White Oak sanctuary, in a lovely home with lakes and forests and they’re no longer chained up for days at a time on cold concrete.

Though Feld Entertainment is attempting to paint themselves in the best light with this move, they sold the elephants to the sanctuary; they did not give the elephants to the sanctuary. I suspect the reason why is equal parts Feld’s daughters telling him to move on and the cost and upkeep of the elephants when Feld Entertainment was hit by COVID losses. Regardless of the underlying reasons behind it all, we can celebrate the fact that Feld no longer has elephants and the elephants, and we, can all move on to something better.