THE BIG ONE

Total website redesign.
Multiple new blog posts.
Routine updates on several pages.
Too much to list, even if I remembered it all.
Remind me not to put this git repo on any portfolios.

1 files changed, 7 insertions, 7 deletions

diff --git a/blog/posts/grow_spikes/index.md b/blog/posts/grow_spikes/index.md
index 4bf7c61..6b7247a 100644
--- a/blog/posts/grow_spikes/index.md
+++ b/blog/posts/grow_spikes/index.md
@@ -6,7 +6,7 @@ When I switched reverse proxies to Caddy last month, I figured it was time to up
 
 This guide assumes you already have Caddy working and are familiar with the basics of its use. It also assumes you're running Ubuntu Server 24.04 LTS, since that's what I'm using. I hear some of the filepaths are different depending on whether you're on a Debian-based distro or not, so adjust accordingly if you're on, like, NixOS or whatever. Although, NixOS users probably don't need my help.
 
-## robots.txt
+# robots.txt
 
 `robots.txt`, if you're not familiar, is a text file you serve from the root of your website that tells crawlers and other bots whether they have your permission to access your site. I didn't bother with this last time, because like user agent strings it's basically reliant on the honor system to work, and most of the bots I'm interested in blocking are scummy enough that they'd probably ignore it anyway. This time around, though, I'm planning to give it some teeth (which we'll get to later) so it's actually important here. Here's the relevant excerpt from mine:
 
@@ -23,7 +23,7 @@ Disallow: /slop/
 
 The first two lines tell all bots (`User-Agent: *`) not to look at anything (`Disallow: /`). You can stop there, but if you want, you can also allow specific user agents, for example your favourite indie search engines, by following that with a directive specific to them. `Allow: /` tells the bot it's allowed to crawl your whole site, and `Disallow: /slop/` lets it know to avoid a subdirectory you'd rather it not crawl. There are a few reasons you might want to do this, but I'm using it to spare the crawlers I like from the trap I'll be setting in that subdirectory.
 
-### Defining this universally in Caddy
+## Defining this universally in Caddy
 
 Serving a plain text file, like most things in Caddy, is pretty simple to do in your caddyfile:
 
@@ -61,7 +61,7 @@ yoursite.example.com {
 
 If you do all that and reload Caddy, you should be able to go to yoursite.example.com/robots.txt and see the `robots.txt` file you entered. Now on to the fun part.
 
-## Nepenthes
+# Nepenthes
 
 [Nepenthes](https://zadzmo.org/code/nepenthes/) is a tarpit for misbehaving scrapers, named after a species of carnivorous pitcher plant. There's a few of these around now, but they all seem to work in pretty much the same way: generating, as the developer of [Iocane](https://iocaine.madhouse-project.org/) so eloquently puts it, "an infinite maze of garbage" for scrapers to get trapped in.
 
@@ -69,7 +69,7 @@ In other words, it craps out an infinte series of procedurally generated pages t
 
 Obligatory warning here: Nepenthes and programs like it are intentionally malicious software designed to cause harm, and will also probably get your site delisted from any search engines whose crawlers get caught in it. Proceed only if you're comfortable with all that.
 
-### Setting up Nepenthes
+## Setting up Nepenthes
 
 Setting up Nepenthes is pretty simple if you're comfortable self-hosting stuff in general, and well-documented on its website. I'm lazy so I used the provided Docker image. If you're not familiar with Docker, here's the series of commands to spin it up identically to how I did (adding `sudo` as needed):
 
@@ -106,7 +106,7 @@ chmod +x load.sh
 ./load.sh beemoviescript.txt http://localhost:8893/train
 ```
 
-### Pointing Caddy at it
+## Pointing Caddy at it
 
 Simply add a `reverse_proxy` directive to the `bans` snippet you made before (you did make it, right?):
 
@@ -126,7 +126,7 @@ Simply add a `reverse_proxy` directive to the `bans` snippet you made before (yo
 
 That will create a subdirectory called /slop/ on every website you use that snippet on that points to your shiny new tarpit. The `header_up` directive is there to tell Nepenthes what sudirectory it's running under. It's important to have this, or it won't work.
 
-### Setting the trap
+## Setting the trap
 
 Now all that's left to do is add a link to the tarpit on your website, ideally every page. You can make this invisible to the user using CSS (in fact, I'd recommend doing this) and the scrapers won't care — they don't look at that anyway. They'll just see a yummy juicy link in the HTML, follow it, and fall directly into your trap.
 
@@ -134,7 +134,7 @@ If you've got processing power and bandwidth to burn and you're content to just
 
 I was initially content to do this, too, until some dipshit on Alibaba Cloud with like 300-odd IP addresses in the ranges `47.82.0.0/16` and `47.79.0.0/16` showed up and hammered my Nepenthes instance so hard it hit some built-in resource usage limit in Docker and couldn't even respond to queries to its statistics endpoint anymore and I decided it was time to intervene. Here's what I came up with.
 
-## Integrating this with Fail2Ban
+# Integrating this with Fail2Ban
 
 [Fail2Ban](https://github.com/fail2ban/fail2ban) is a program that scans log files for failed authentication attempts and bans the offending IP addresses. You should already have this set up. If you don't, go set it up right now. It's probably packaged for your distro. Go do it. Do it now. `sudo apt install` that bitch. I'll wait. It takes like two seconds. While you're at it, set up key-based auth for your SSH server and turn password login off.