Block bad bots in Nginx

Bots hitting your sites(especially for dynamic pages) can cause extensive resource (CPU, Memory, Connections) usage. It’s simple to drop or block badbots like MJ12bot, AhrefsBot in Nginx without using Modsecurity.

Step 1.) Create a badbot.conf in your Nginx directory.

/etc/nginx/conf.d/badbot.conf

Step 2.) Edit /etc/nginx/conf.d/badbot.conf and add the following content(modify highlighted part if you want to add or remove bots)

map $http_user_agent $bad_bot {
    default 0;
    ~*^Lynx 0; # Let Lynx go through
    libwww-perl                      1;
    ~*(?i)(AhrefsBot|DotBot|80legs|360Spider|^BackDoorBot|GalaxyBot|MJ12bot|MegaIndex|python-requests|scanbot|ZmEu|SemrushBot|^majestic12|Nimbostratus-Bot|^WallpapersHD) 1;
}

Step 3.) To enable the badbot block, add one of following to your server block

# Send 403 Forbidden error
if ($bad_bot) { return 403; }

OR

# Drop the connection
if ($bad_bot) { return 444; }

Step 4.) Run the following command to verify your Nginx configuration is valid.

nginx -t

You should get an output that looks something like this:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Step 5.) Reload Nginx and you’re all set!

service nginx reload

Source: https://github.com/mitchellkrogza/nginx-badbot-blocker