Category Archives: Server

Simple bandwidth monitoring on Linux

Posted on by
Reply

If you want a simple lightweight tool to monitor the network traffic in and out of your server vnstat might be just what you need. It keeps hourly, daily and monthly records and provides simple estimates of your expected use, it is also easy to link up to a web based frontend for fancy charts and reporting.

Installing and configuring vnstat is very simple, firstly install using your standard package manager, for example:

Debian

apt-get install vnstat

CentOS:

yum install vnstat

Then tell vnstat to create a database for the network interfaces you want to listen to (e.g. eth0):

vnstat -u -i eth0

That’s it, wait a few minutes then run vnstat to view a simple console display of the amount of traffic that has traveled though all the interfaces you’re monitoring:

vnstat
 
   eth0 since 01/22/12
 
          rx:  177.59 MiB      tx:  7.78 MiB      total:  185.37 MiB
 
   monthly
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
       Jan '12    177.59 MiB |    7.78 MiB |  185.37 MiB |    0.59 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated       183 MiB |       7 MiB |     190 MiB |
 
   daily
                     rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
     yesterday     12.53 MiB |    1.36 MiB |   13.89 MiB |    1.32 kbit/s
         today      8.28 MiB |     127 KiB |    8.40 MiB |    0.88 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated        --     |      --     |      --     |

You can also get vnstat to dump its output in a programming friendly format (semicolon delimited):

vnstat --dumpdb

If you do want a nicer looking interface or one that doesn’t require shell access have a look at: vnstat PHP frontend

If you need a bandwidth monitoring solution that records the utilization of individual protocols instead of just received and transmitted traffic then have a look at bandwidthd

Setting up a centralised syslog server in the cloud

Posted on by
Reply

This post should help you get a basic syslog server and client(s) up and running in a virtual environment, It will take you through the implementation of a reasonably secure (using rsyslog’s TLS authentication) yet flexible setup useful to most virtual based server architectures I will assume if you’re reading this that you know what syslog is and what it’s used for. (if not have a quick Google then come back)

 Why Setup a Centralized Syslog Server

  • For convenience – If you for instance have a large number of web servers and you need to diagnose a problem on one of them (maybe not sure which one) you only have to check in one place, if you wanted to compile some statistics from all of them or check if they had all successfully completed a software upgrade.
  • For added security – If someone hacks into one of your servers they will probably try and cover their tracks by erasing any log records created by there presence, however if your logs are also sent to another (hardened) server then the logs will still be available to sysadmins.
  • Another very useful reason which only really applies to virtual servers is to help retain the log files from a terminated server (e.g. shut-down due to decreased demand on your application).

Continue reading →

Bash Script to Create new virtual hosts on Nginx each under a different user

Posted on by
Reply

This script is a modified version of the original vhost creator script for nginx posted here (Actually it was a modified version of the original version of this script).

The script below will automatically create a new user on the system and adds the nginx user to the new users group. This allows FTP access to be given to the newly created user and the user will only have access to the their site and not to all vhosts running on your server, however as long as the users group has permission to access the files then nginx will still be able to serve them.

NOTE: This setup only helps lock down access to the vhost directories on a web server hosting only static sites, if CGI of any kind (i.e. PHP) is available then this will also need to be locked down so each user has access to a CGI process or set processes running as that user.

What does the script do:

  • Creates a new system user for the site
  • Creates a new vhost config file for nginx using a basic template
  • Creates a new directory for the site, within the new users home directory
  • Adds a simple index.html file to the new directory to show the site is working.
  • Makes sure the new nginx config syntax is correct before trying to reload nginx
  • Reloads Nginx to allow the new vhost to be detected

Debugging fail2ban not starting

Posted on by
Reply

If when you’re trying to start Fail2ban you just get the following response:

Starting Fail2ban:                                         [FAILED]

And you check in the fail2ban log file (or system log) and find no errors, it is probably caused by your Fail2ban init script writing the output of the fail2ban-client to /dev/null, effectively just discarding the output. The easy way to debug this is to try directly calling the fail2ban-client which will print out any syntax errors found in its config files. Use it like so:

$ fail2ban-client -x start
WARNING 'action' not defined in 'php-url-fopen'. Using default value
WARNING 'action' not defined in 'lighttpd-fastcgi'. Using default value
ERROR  Error in action definition #iptables[name=SSH, port=ssh, protocol=tcp]
ERROR  Errors in jail 'ssh-iptables'. Skipping...

This should then highlight where the errors are in your config file(s) and allow you to resolve them. You could obviously change the behavior of your init script to stop it discarding the output.

“501 5.0.0 Invalid domain name” trying to send email

Posted on by
1

Had this error appear today when trying to send an email using Swift Mailer (on an nginx powered box using PHP FastCGI). It turned out to be caused by the CGI parameter SERVER_NAME, the site was running on a wildcard domain under nginx and as a result the SERVER_NAME param contained the wildcard symbol. (e.g. *.test.example.com) However the SMTP server didn’t like this (and quite rightly so), this is where the Invalid domain name bit comes from.

Adding the following line to my nginx server config for that virtual host fixed the problem.

fastcgi_param SERVER_NAME $host;

The above line just sets the SERVER_NAME parameter to be the same as the host header for the current request. The only problem with this is if the host header in the request is not set then this value can still fall back to the original $server_name one (see nginx manual for more info). However this seemed to fix the problem for me, and I don’t think it should cause any problems as the site is not available directly via an IP address, so host header will always be required.

Scheduling Individual Backups of all MySQL Databases on Linux

Posted on by
Reply

Expanding on from an older post covering a simple usage of the the mysqldump program I finally found some time to write a bash script to control this process and allow all MySQL databases to be backed up separately with one command.

This script uses the MySQL client to pull in a list of all the databases on the server and then loop round and back each one up into its own compressed file (using the gzip program). Once a backup file for all the databases has been created the script then bundles them all into a single tar archive, removing the original individual files. This then allows you to easily keep track of all your backups, by just having one file containing all the data you need to restore all DBs (or just one) to a point in time. A script such as my one could be used to limit the number of backups stored at any one time (to save on disk space).

Continue reading →