Someone told me once that some kinds of dogs that are bred for herding will herd clouds of dust if you keep them inside for too long. The herding instinct is so deep inside their minds that they can’t help but act on it, even if it’s not really productive. I think sometimes that people can be like this, too. Some people, it seems to me, will make art, exercise, build things, or do whatever feeds their soul no matter what situation they’re in. To them, their universe is filled with sheep, and they were born to herd.
This is how I feel about computing systems. Now, I don’t claim to be great at it, and I don’t think I have the drive that many people do—let alone dogs bred for herding—but it sure seems like everything I do eventually leads back to making computers do stuff that pleases me.
Earlier this month, I went back to where I grew up for about a week and half. Since I moved away about seven months ago, I’ve kept in touch with my family there, but I’ve also learned a lot about the world simply by living somewhere slightly different. Returning was a whole process of emotion for me that I won’t go into here, but while I was away I thought I’d try to make sure my home laptop was accessible remotely. You see, my laptop is quite large, and I had already decided to leave it safely on my desk rather than lug it through airport security, as well as all the places I’d visit while I was away. I brought a much smaller machine instead with the expectation that I’d be able to reach back to my home network whenever I wanted.
The way I planned to do this was simple enough. Some months (or years, maybe?) ago, I set up a VPN on my personal Linode server and added my machines to it. After a fair amount of fighting with the config files—this was, after all, the first time I’d done this—I had a reasonably workable system that let me access my machines from practically any place I had a connection. In fact, I’m using it even as I type this via a Google Cloud Shell.
Being a decent engineer—so I thought—I knew that this system had a single point of failure: the VPN connection. In the past, I’d already had issues where the OpenVPN daemon on my machines would just kind of stop working, and I’d have to restart the service. I knew this wouldn’t be possible remotely, so I came up with a “cunning plan”. I’d forward a random high port on my home router to port 22 on my machine, thus allowing me to access the machine in an emergency. I tested it out, and everything worked fine. Great, I thought, now I’ll be safe no matter what. I enabled remote access on my router and set a strong password.
So, off I went. I flew back to my hometown, and after getting settled, one of the first things I tried was to connect to my laptop. Just as planned, I got a shell. I was pleased, perhaps too much, with my design.
It wasn’t long before that pleasure evaporated. Later, maybe the next day, I tried to log in again. Nothing happened. Uh oh. I pulled the ripcord and tried my remote access trick. To my surprise, my machine was listed as disconnected. I messaged my girlfriend to ask her to check the machine for power. She confirmed that it was on. I realized in that moment that something unexpected had happened: my machine had lost its connection to the router.
This, of course, ruined the whole system. Without a connection to the router,
the machine was totally inaccessible. Why it had disconnected, I still don’t
know. I’d already had problems with the WiFi daemon on BunsenLabs, the OS I was
and am running, so maybe it has something to do with that. In any case, it
seemed likely to me that the machine had somehow lost its connection to my
network and latched onto one of the many open
xfinitywifi networks in the
I asked my girlfriend to power cycle the router. My hope was that either (a)
xfinitywifi that my machine chose was the one offered up by our router,
and that power cycling it would knock it off that network and force it back
onto the proper one, or (b) that I was wrong—which I’d now realized again was
possible—and something else entirely was going on, and that power cycling
would reset things enough to get back to a recoverable state.
My girlfriend dutifully reset the router once, twice, and nothing changed. It was becoming clear that I’d be ten days without my main machine (and she’d be ten days without our Minecraft server, which runs on it). I thought about having her log in, or even power cycle the machine itself, but I didn’t want to have to try to relay my passwords over the phone. She assured me that it wasn’t a big deal, but I still felt pretty dumb.
I learned then that single points of failure can bite you hard, and that it’s
important to look for them everywhere. I still haven’t figured out how to tell
nmcli to only connect to one network, but that would probably help. I also
considered running a cable from the router down to my machine, but that would
be pretty involved. In truth, I haven’t got a good solution to this problem,
other than that I might switch to stock Debian soon on that machine and see if
it plays better with my WiFi hardware.
When I came back, I also learned that I was at least correct about how the
system failed. I logged into the idle machine, checked the WiFi daemon, and
sure enough it was connected to some random
xfinitywifi, doing nothing
valuable. I cursed the machine then, but I only had myself to blame.
Lesson learned: never trust a computer if you can avoid it, especially when you program it yourself.