Several years ago, after noticing the system overhead of the standard Joomla Web site administrator login page during a password attack, I added a secondary Apache password to the administrator page using a .htaccess file stored there:
AuthName "Secured Area"
AuthType Basic
AuthUserFile /etc/passwd.joomla
Require valid-user
Two passwords are now required to gain access, but the first one does not require launching PHP, so it consumes much less CPU time. For a microserver like mine (an Intel Atom CPU on a fanless Mini-ITX motherboard), that's a big deal. The gateway password also provides more security, much like disallowing root remote logins to a Linux server. I do this on all my servers - first login to a non-privileged account, then use su to gain privileges. An attacker must guess the non-privileged account name, then the password, and finally the privileged account (root) password.
Usually an attacker will try a few passwords and then go away forever. On May 11, 2016, I noticed that there were a large number of attacks from separate hosts. Each one would try to login using the user name admin and an unknown password (Apache does not save failed passwords). Since there is no such user, that would fail and the host would try five more times for a total of six.
A moment later, there would be another set of six attacks. Sometimes two attack sequences would intermingle, but there was essentially no overlap between any of them. Clearly this was a coordinated attack.
92.113.138.200 - - [15/May/2016:03:18:50 -0700] "GET /administrator/ HTTP/1.1" 401 492 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
92.113.138.200 - admin [15/May/2016:03:18:50 -0700] "GET /administrator/ HTTP/1.1" 401 492 "http://chapman-consulting-sj.com/administrator/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
92.113.138.200 - - [15/May/2016:03:18:50 -0700] "GET /administrator/ HTTP/1.1" 401 492 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
92.113.138.200 - - [15/May/2016:03:18:51 -0700] "GET /administrator/ HTTP/1.1" 401 492 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
92.113.138.200 - admin [15/May/2016:03:18:51 -0700] "GET /administrator/ HTTP/1.1" 401 492 "http://chapman-consulting-sj.com/administrator/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
92.113.138.200 - admin [15/May/2016:03:18:52 -0700] "GET /administrator/ HTTP/1.1" 401 492 "http://chapman-consulting-sj.com/administrator/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
178.120.30.191 - - [15/May/2016:03:18:56 -0700] "GET /administrator/ HTTP/1.1" 401 492 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
178.120.30.191 - - [15/May/2016:03:18:56 -0700] "GET /administrator/ HTTP/1.1" 401 492 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
178.120.30.191 - - [15/May/2016:03:18:56 -0700] "GET /administrator/ HTTP/1.1" 401 492 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
178.120.30.191 - admin [15/May/2016:03:18:56 -0700] "GET /administrator/ HTTP/1.1" 401 492 "http://chapman-consulting-sj.com/administrator/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
178.120.30.191 - admin [15/May/2016:03:18:57 -0700] "GET /administrator/ HTTP/1.1" 401 492 "http://chapman-consulting-sj.com/administrator/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
178.120.30.191 - admin [15/May/2016:03:18:57 -0700] "GET /administrator/ HTTP/1.1" 401 492 "http://chapman-consulting-sj.com/administrator/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
I continued to watch the attack logs until the afternoon of May 15, 2016, when the attacks halted. I wrote a little script to count all of the attacks and obtain the list of attacking IP addresses:
#!/bin/bash -f
cat chapman-consulting-sj-access_log-20160515 \
chapman-consulting-sj-access_log | fgrep "GET /administrator" | \
sed -e 's/ .*//' | sort | uniq > all_addresses_sorted.txt
This says "find all of the /administrator page retrievals (i.e. the breakin attempts), strip off everything past the IP address at the front of each line, sort the resulting list of IP addresses, remove duplicates, and write the result to a text file."
Over the course of five days, there were 304,499 distinct attempts to retrieve the administrator page from 28,774 unique IP addresses. One of them came from me, as I tried to verify that the Web site had not been compromised, and a scattered few were individual attacks.
So why me? Why attack the Web site I use to advertise my consulting business, and why direct 28,000+ hosts to do so? There is no private customer data such as credit card records stored on it, so even a successful breakin wouldn't earn any money. I own the server and can rebuild the site from backups if it gets compromised (again - it was taken down by a zero-day Joomla attack in December 2015).
Clearly, the attackers didn't know anything about my site other than it has a password-protected administrator page. Identifying Joomla- or WordPress-based Web sites is straightforward; I see attempts in the logs all the time. Someone's automated attack system found my site and decided to attack it.
The six attempts per IP address may have been intended to evade intrusion detection systems. It's fairly easy to block a single IP address that attempts to break in thousands of times. It's much harder to block IP addresses from around the world. This was pretty sophisticated.
Oddly, the attack was quite primitive in one way - every probe tried the same user name admin. Unlike Unix/Linux hosts, the privileged management account for a Web site does not always have the same name. The ordinary system breakin attempts I see try a wide range of account names - certainly root, but a lot of other names as well. For some reason these attackers did not.
Clearly this was an attack network with a lot of power behind it. I ran reverse IP address lookups on a few of the attackers and found they were mostly in Russia and Ukraine. Most likely they were part of a botnet, a centrally controlled network of compromised computers, directed from who knows where. Apparently, computers in former Soviet republics are often not kept up to date, so they are vulnerable to malware infection. Now I've seen what some of that malware does.
(If you are interested in studying this botnet, download the list of IP addresses here.)
With over 300,000 attempts and over 28,000 IP addresses, the average attacking host tried to break in about 11 times - a little less than two rounds (with six attempts) each. I did verify that some hosts came back again, but I'm not bored enough to spend that much time analyzing the attack profile. For me, security is something that takes away from productive time - it is not my full-time job.
Those 28,000+ hosts weren't directed only at me, either. The attack ran for almost 400,000 seconds; with about one probe per second per attacking host, six probes per attack round, and two attack rounds, each computer in the botnet could have attacked about 33,000 Web sites (400,000 / 12) during that time. Now the botnet could be attacking another 33,000 Web sites.
This assumes only one probe per attacker is active at a time; with two simultaneous probes active (still few enough that the owners of the compromised computers wouldn't notice a network slowdown), the botnet could attack 66,000 Web sites in five days. With a few more probes per second active, hundreds of thousands of Web sites could be attacked simultaneously. The criminals attacked my low-value site because they had enough computers in that one botnet to do it. And who knows how many other botnets are out there, running the same malware?
So be on your guard. Use secondary passwords and non-obvious administrator account names on your servers, both for Web sites and (when possible) the host machines. There are a lot of criminals, with even more computers, out there trying to break in.
Update July 21, 2017:
Antivirus company ESET posted a blog entry suggesting that the attack came from the Stantinko Botnet, based on the User-Agent signature in the Apache logs. They estimate that it has 500,000 machines. A full white paper analysis is available from the post.
