Shane Rutter Blog

Tag: server

  • NCDU – Find what’s using all your disk space on linux based system

    NCDU – Find what’s using all your disk space on linux based system

    NCDU is a text-based user interface, it allows you to visually see how and where disk space is being used via an easy-to-use interface. Simple run the command in the current directory to see a directory tree of folders & files and their space usage.

    I use this tool mainly for analysing why a server is running out of disk space, determine the reason why and if it’s simply expanded the disk space or is there unused files / massive logs which can be purged.

    Install

    Run the required command for your linux OS version.

    sudo apt install ncdu

    sudo yum install ndcu

    How to use

    Simple navigate to a directory you want to check and run the command:

    ncdu

    If you want to scan the whole system go to the root directory:

    cd /
    ncdu

    You can also specify a directory if you don’t want to change directory:

    ncdu /home

    Windows Server / PC

    For windows-based machines I use a bit of software called TreeSize, it does the same thing as NCDU, allowing you to see how your disk space is being used on a windows based OS.

    TreeSize on windows PC showing disk space usage
    TreeSize on windows PC showing disk space usage

     

    This is provided as it and may require changes to work for your needs.

     

  • How I deleted 530 million records from a large MySQL database table hitting nearly 200GB

    Came across the need to perform maintenance on a MySQL database which holds millions of records, we needed to remove all data which was 2+ years old which involved deleting around 530 million records from a very large table, hitting nearly 200GB.

    Problem

    Due to the size of the database and transaction logs / table locking, it wasn’t possible to do a bulk delete such as

    This caused the database to become very unresponsive and after hours of running eventually failed with table locking issues. This is because DELETE operations are very costly due to all the checking and transactional logging involved.

    I tried creating a second duplicate table and inserting the data we wanted to keep into this table, however this involved turning the system offline during this process, after some testing locally, was going to take roughly 90 hours to complete, this was not an option.

    Solution

    I created a MySQL stored procedure, which allowed me to split the DELETE query into chunks, stored procedures are not handled as a single transaction, this allowed each delete operation to complete and move onto the next chunk, slowly deleting the data from the active table whilst the table is still in use.

    As it was costly to query against the table or use LIMIT & OFFSET, which would take around 10 minutes to return results, it was not possible to use these as part of the procedure if we wanted this process to complete in a timely manner.

    As the ID where incremental, I was able to query the table until I found the largest ID I was willing to delete up to, in my case it was 530,000,000.

    The below stored procedure is what I ended up using to solve the issue.

    It is recommended a professional under takes these actions, please always backup existing data / files / configs.

  • cPanel – Sender x has an outgoing mail hold. Message will be reattempted later.

    A client was having issues sending emails and receiving a number of alerts regarding delayed emails. After looking investigating a number of outbound emails where queued with a failure reason of:

    The account in question had hit the sending limits configured within cPanel and there emails where being held.

    Pre-Checks

    It is important you pre-check and make sure the hold is not legitimate such as a spam attempt / hacked site, I checked the emails being sent and due to a influx in online orders there email usage per day has significantly increased.

    1st Attempt

    Try running the following commands under sudo / root, replaceing [accountname] with the cpanel account name of the user. Then try send an email or retry sending an email in the queue.

    2nd Attempt

    The above worked for a couple accounts, but for one this did not work. There is 3 files depending on the type of block. Have a look in the following files below and remove the accountname / email address if present.

    I found my user within “/etc/outgoing_mail_suspended_users” and once removed, everything went back to normal.

    It is recommended a professional under takes these actions, please always backup existing files / configs.

  • Ubuntu EC2 configure Amazon Time Sync Service for Amazon AWS

    Having problems keeping your server time in sync? or a project requiring accurate timing? If your using a Amazon EC2 instance you can swap to using Chrony and Amazon Time Sync using the below commands.

    (more…)

  • Linux: Killall command

    It is your responsibility to ensure safe usage of these commands, improper usage could cause damage to services and file systems. We will not be held responsible.

    There is a number way of killing processes on a Linux based OS, killall is one of these methods. It’s always good to be ready to kill an unwanted process or a rogue process which is eating resources.

    Usage

    Kill all instances of the “httpd” process

    Kill all proccesses running under the user “postfix”

    Kill a specific process but its full path

    Command not found?

    It does not always come pre-installed, it’s simple to install.

    Ubuntu / Debian

    RHEL – Red hat / Fedora / CentOS

    This package also contains the following useful programs:

    • fuser – identifies what processes are using files.
    • killall – kills a process by its name, similar to a pkill Unices.
    • pstree – Shows currently running processes in a tree format.
    • peekfd – Peek at file descriptors of running processes.
  • Automatic MySQL database backups and zipping on windows with a bat file

    Automatic MySQL database backups and zipping on windows with a bat file

    It is important you double check your configuration before running this script! We cannot be held responsible for any damage this may cause.

    Need to backup all your databases on a MySQL windows server? Red Olive have created a simple windows batch file to export all database tables and zip, including deleting older backups! It’s quite simple to setup look at their website.

    https://www.redolive.com/utah-web-designers-blog/automated-mysql-backup-for-windows/

    Their version is configured for a US date/time format, I am in the UK with UK based date/time formats, I have made a modified version of the script available below.

    You should setup a scheduled task to run the bat file on a regular interval, the more often you perform backups the easier it will be for you if there is a failure and you require to restore a backup.

    We also advise you look at a process for storing backups at remote location, if you backup to a Dropbox folder your backups will be uploaded to the cloud, protecting you if the server dies or has disk failure!

    Download MySQLBackups Script (UK Timezone)

  • Local website & services remotely accessible via secure tunnels with ngrok

    Need an easy way to allow external network users to access services or websites on your local machine? ngrok allows you to easily create a secure tunnel from ngroks servers to your machine. For a web developer, this is a dream come true, this is a great way to demo a development whilst in the early stages, saves time uploading to a server.

    • Allows access to your locally hosted web server from a unique domain
    • Can use your own domains
    • Works with vHost files
    • Allows testing of website which require an external accessible point such as payment gateways or oAuth
    • Can tunnel other services such as PhoneGap allowing users external to your network to test your mobile app
    • Various API and IoT implementations
    • Easy / quick private networking, VPN benefits

    These are only a short list of what is possible using ngrok. Look at their website for a full list of its powerful features and get started.

    https://ngrok.com/

  • Repairing damaged or crashed MySQL tables

    Always remember, regular backups of your databases are best practice, in the event the below fail, backups may be the only method you have to restore a working table or database.

    Damaged or crashed MySQL tables due to an unsafe server shut downs, disk space limits or general MySQL failures? You will normally see something along the lines of “marked as crashed and should be repaired when using LOCK TABLES” within your MySQL log files.

    MySQL has a number of tools to try and repair crashed / damaged tables. Log into your server as root and run the following commands. Replacing “/var/lib/mysql/” with the location of your DB files.

    Check what tables need repaired:

    myisamchk –check /var/lib/mysql/*/*.MYI

    Try a safe recovery:

    myisamchk –safe-recover /var/lib/mysql/*/*.MYI

    Try a normal recovery (if safe recovery fails):

    myisamchk –recover /var/lib/mysql/*/*.MYI

    Try a force recovery (as a final last step):

    myisamchk –safe-recover –extend-check –force /var/lib/mysql/*/*.MYI

    or:

    myisamchk –recover –extend-check –force /var/lib/mysql/*/*.MYI

  • What is a server

    What is a server

    A server is a computer designed for a specific task such as running a website, a server tends to be powered up more often compared to a regular computer with most servers running 24/7. Servers are a major part of our everyday live, the internet is a collection of hundreds of thousands of servers all working together performing various tasks. (more…)