servers

Servers of Hacker News

I did a little experiment:

  • For about a month I scraped the front page of NH
  • For every link of the front page I looked at the “server” header value and stored it for every unique URL.
  • Repeat, every hour.

Goals of experiment, in order of importance

  • Have fun
  • Satisfy my curiosity
  • Try to find servers I have never heard of (achievement unlocked!)

Results

v1

 

Full breakdown of server count (total count = 2415)

Apache total : 995

Apache : 558
Apache-Coyote/1.1 : 66
Apache/1.3.41 : 2
Apache/1.3.42 : 2
Apache/2 : 4
Apache/2.0.52 (Red Hat) : 5
Apache/2.2 : 18
Apache/2.2.11 (Unix) mod_ssl/2.2.11 OpenSSL/0.9.8e-fips-rhel5 PHP/5.2.14 : 17
Apache/2.2.12 (Ubuntu) : 3
Apache/2.2.14 (Ubuntu) : 27
Apache/2.2.15 (CentOS) : 51
Apache/2.2.15 (Red Hat) : 9
Apache/2.2.15 (Red Hat) mod_ssl/2.2.15 OpenSSL/1.0.0-fips PHP/5.3. : 28
Apache/2.2.15 (Scientific Linux) : 2
Apache/2.2.16 (Debian) : 32
Apache/2.2.17 (Ubuntu) : 6
Apache/2.2.22 : 16
Apache/2.2.22 (Debian) : 12
Apache/2.2.22 (Ubuntu) : 50
Apache/2.2.22 (Unix) FrontPage/5.0.2.2635 : 7
Apache/2.2.23 (Amazon) : 6
Apache/2.2.24 : 6
Apache/2.2.24 (Amazon) : 4
Apache/2.2.24 (Unix) mod_ssl/2.2.24 OpenSSL/1.0.0-fips mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 : 5
Apache/2.2.3 (CentOS) : 33
Apache/2.2.3 (Red Hat) : 26

Nginx total : 703

nginx total : 435
nginx/0.7.65 : 6
nginx/0.7.67 : 9
nginx/0.8.53 : 7
nginx/0.8.54 : 10
nginx/0.8.55 : 5
nginx/1.0.11 + Phusion Passenger 3.0.11 (mod_rails/mod_rack) : 10
nginx/1.0.15 : 5
nginx/1.1.19 : 66
nginx/1.2.1 : 21
nginx/1.2.3 : 7
nginx/1.2.4 : 6
nginx/1.2.6 : 64
nginx/1.2.7 : 12
nginx/1.2.8 : 4
nginx/1.2.9 : 3
nginx/1.3.11 : 3
nginx/1.4.1 : 30

GitHub.com total : 207

GSE total : 90

Microsoft-IIS/ (6.0, 7.0, 7.5, 8.0) total : 72

Others total : 

cloudflare-nginx : 62
AmazonS3 : 30
Sun-Java-System-Web-Server/7.0 : 25
HTTP server (unknown) : 22
Google Frontend : 19
WP Engine/4.0 : 19
lighttpd/ (1.4.18, 1.4.31-devel-783a962) : 19
tfe : 17
YTS (1.19.11, 1.20.10, 1.20.27, 1.20.28) : 16
Economist Web Server : 12
WP Engine/1.2.0 : 11
TheAtlantic Web Server : 11
gunicorn/0.14.3 : 11
SSWS : 11
WEBrick/1.3.1 : 10
LiteSpeed : 10
Resin/4.0.34 : 8
thin 1.5.1 codename Straight Razor : 6
gwiseguy/2.0 : 6
ECD (dca/24FD) : 4
gws : 3
NPG Webhosting/1.0 : 3
Varnish : 3
IBM_HTTP_Server : 2
Oracle-Application-Server-11g Oracle-Web-Cache-11g/11.1.1.6.0 : 1
TornadoServer/3.0.2 : 1
Oracle-iPlanet-Web-Server/7.0 : 1
PCX : 1
publicfile : 1
PWS/8.0.15 : 1
QRATOR : 1
R2D2 : 1

Critical analysis : what does this mean?

Not much! Because… (read on)

Notes

  • I’m aware that the “server” header is not 100% reliable to determine the server type.
  • If 5 articles from the same organization’s website (github.com, cnn.com or google.com, etc.) made the front page in a day, that’s 5 “counts” for a single server.
  • I didn’t count all the single, weird server instances – including empty header values.
  • I didn’t scrape much metadata with this yet, so it’s hard to see it in full context.