Shane Rutter Blog

Inline CSS for better emails

The technology behind your next fancy email is so outdated! Limited by HTML and CSS from 5+ years ago it can be a pain getting your email to work and look correctly in all email clients. Not only do you have HTML and CSS version limitations, but the email client also itself have various limitations, for example Yahoo can sometimes strip out any CSS you include in the head of the email via the <style> tags!

Came across this cool tool from Campaign Monitor, which will take your HTML email and inline all your CSS, so certain email clients don’t strip your styling from the head! Give it a shot and let me know how you get on.

https://www.campaignmonitor.com/resources/tools/css-inliner/

1st October 2021

WooCommerce emails add reply to header

A client was having issues with bounce back due to customers entering incorrect email addresses, so I updated the outbound email address used by WooCommerce to “[email protected]”, this meant any bounce backs would go to a null mailbox instead of to their mains “[email protected]” mailbox. However, this meant when customers hit reply, it would reply to the noreply@ mailbox. Using this code, I was able to set a Reply-To header, so customers could still reply to emails.

Code

Drop this into your functions.php file and edit as required.

function woocommerceEmailAddReplyTo($headers = ”, $id = ”, $order)
{
	$replyToName = get_bloginfo(‘name’);
	$replyToEmail = get_bloginfo(‘admin_email’);

	if (is_array($headers)) {
		$headers[‘Reply-to’] = $replyToName . ‘ < ‘ . $replyToEmail . ‘>’;
	} else {
		$headers .= ‘Reply-to: ‘ . $replyToName . ‘ < ‘ . $replyToEmail . ‘>’ . “\r\n”;
	}

	return $headers;
}
add_filter(‘woocommerce_email_headers’, ‘woocommerceEmailAddReplyTo’, 10, 3);

1

2

3

4

5

6

7

8

9

10

11

12

13

14

function woocommerceEmailAddReplyTo($headers = ”, $id = ”, $order)

{

$replyToName = get_bloginfo(‘name’);

$replyToEmail = get_bloginfo(‘admin_email’);

if (is_array($headers)) {

$headers[‘Reply-to’] = $replyToName . ‘ < ‘ . $replyToEmail . ‘>’;

} else {

$headers .= ‘Reply-to: ‘ . $replyToName . ‘ < ‘ . $replyToEmail . ‘>’ . “\r\n”;

}

return $headers;

}

add_filter(‘woocommerce_email_headers’, ‘woocommerceEmailAddReplyTo’, 10, 3);

This code is provided as it and may require changes to work for your needs.

24th September 2021

WordPress include private pages in parent dropdown

WordPress only includes published pages in the parent page dropdown, I ran into an issue where both parent and child page were made private so work could be done to the pages, the child pages when edited lost their parent relationship, due to the parent page no longer appearing within the “Parent” dropdown.

Why would you need this?

In my case we wanted to make some changes to the existing pages but needed to do these changes in private until all where complete, it was not until after we re-published the pages, we found the child pages where no longer link to the parent.

Fix

Let’s add the private and pending pages to the parent dropdown menu, allowing us to pre-create pages and child / parent relationships without having to already have published the pages.

function updatePageParentSelectionDropdown($args)
{
	$args[‘post_status’] = [‘publish’, ‘private’, ‘pending’, ‘draft’];
	return $args;
}
add_filter(‘page_attributes_dropdown_pages_args’, ‘updatePageParentSelectionDropdown’);
add_filter(‘quick_edit_dropdown_pages_args’, ‘updatePageParentSelectionDropdown’);

1

2

3

4

5

6

7

function updatePageParentSelectionDropdown($args)

{

$args[‘post_status’] = [‘publish’, ‘private’, ‘pending’, ‘draft’];

return $args;

}

add_filter(‘page_attributes_dropdown_pages_args’, ‘updatePageParentSelectionDropdown’);

add_filter(‘quick_edit_dropdown_pages_args’, ‘updatePageParentSelectionDropdown’);

This code is provided as it and may require changes to work for your needs.

10th September 2021

SuiteCRM – clear / reindex search index

Came across a problem with a SuiteCRM where the search was not working correctly, returning none or unrelated results. SuiteCRM relies on an indexing service called Lucene, which handles indexing for advanced searches, as in my case, this needs to be full reindexed to work correctly.

1. Lock Files

When Lucene begins indexing it generates lock files, normally found under modules/AOD_Index/Index/Index with the extension .lock, it is save to delete these files if they are old (have an old time stamp), sometimes this is enough to fix the issue, as previous indexing attempts max have failed and left the lock file, stopping future indexing processes from starting. First try, removing these lock files and wait and see if the indexing processes continue / repair them selfs.

2. Full Reindexing

I would recommend you do a full backup of the modules/AOD_Index directory before starting, as well as a backup of the database.

Also temperarly stop cronjobs from running, as we dont want the cron indexer to kick in during this clearing process.

1. Rename index folder (can be deleted after)

From your SuiteCRM root directory:

mv modules/AOD_Index/Index/Index modules/AOD_Index/Index/Index.BACKUP

We are renaming your current (old) index to Index.BACKUP, you can delete this later once you are sure everything is resolved.

2. Empty Table

Now delete all rows from the table aod_index

3. Schedules

Make sure the Optimise AOD Index and Perform Lucence Index are running as part of the schedule.

4. Wait

Make sure you have re-enabled your cronjob and wait for the indexes to rebuild. This will take a long time, but you can monitor the rebuilding process with the below command. Once it has stabilised, it should be complete.

watch du -h modules/AOD_Index/Index/Index

This code is provided as it and may require changes to work for your needs.

27th August 2021

NCDU – Find what’s using all your disk space on linux based system

NCDU is a text-based user interface, it allows you to visually see how and where disk space is being used via an easy-to-use interface. Simple run the command in the current directory to see a directory tree of folders & files and their space usage.

I use this tool mainly for analysing why a server is running out of disk space, determine the reason why and if it’s simply expanded the disk space or is there unused files / massive logs which can be purged.

Install

Run the required command for your linux OS version.

sudo apt install ncdu

sudo yum install ndcu

How to use

Simple navigate to a directory you want to check and run the command:

ncdu

If you want to scan the whole system go to the root directory:

cd /
ncdu

You can also specify a directory if you don’t want to change directory:

ncdu /home

Windows Server / PC

For windows-based machines I use a bit of software called TreeSize, it does the same thing as NCDU, allowing you to see how your disk space is being used on a windows based OS.

TreeSize on windows PC showing disk space usage

This is provided as it and may require changes to work for your needs.

13th August 2021

WordPress disable XMLRPC.php for increased security

I have recently been having a few issues with bots and malicious attacks trying to breach WordPress websites via brute force attacks. From my experience 90% of the websites, I have built or worked on never utilise the xmlrpc service, which is included with WordPress, due to this I decided to simply block any access to this service via Apache, which fixes a number of security related issues.

Code

Place this in your websites .htaccess file to take effect, then simply browser to domain.com/xmlrpc.php and you now get an access denied message.

# Block XMLRPC access
<Files xmlrpc.php>
Order allow,deny
Deny from all
</Files>

1

2

3

4

5

# Block XMLRPC access

Order allow,deny

Deny from all

</Files>

What is XMLRPC?

The xmlrpc.php allows remote access to your WordPress site, it allows various tools and publishing applications to control and manage content from outside of the normal WordPress admin. For example, a publishing tool may allow you to write new blog posts using an alternative tool (instead of the WordPress admin).

Do I need it?

If you do not use any tools / services which utilise xmlrpc, then you do not need it and can apply the block in this post. For example, if you use the WordPress app to manage your website from your phone then you will need to keep XMLRPC working.

Why block XMLRPC?

Due to the nature of XMLRPC, it allows remote services to connect and perform actions on your website, this creates an additional security risk, and it is very common for malicious attacks to target XMLRPC with bruite-force attacks, trying to guess your password and / or access other services exposed by the XMLRPC protocol.

This code is provided as it and may require changes to work for your needs.

15th June 2021

How I deleted 530 million records from a large MySQL database table hitting nearly 200GB

Came across the need to perform maintenance on a MySQL database which holds millions of records, we needed to remove all data which was 2+ years old which involved deleting around 530 million records from a very large table, hitting nearly 200GB.

Problem

Due to the size of the database and transaction logs / table locking, it wasn’t possible to do a bulk delete such as

DELETE FROM table where created_at < x

1	DELETE FROM table where created_at < x

This caused the database to become very unresponsive and after hours of running eventually failed with table locking issues. This is because DELETE operations are very costly due to all the checking and transactional logging involved.

I tried creating a second duplicate table and inserting the data we wanted to keep into this table, however this involved turning the system offline during this process, after some testing locally, was going to take roughly 90 hours to complete, this was not an option.

Solution

I created a MySQL stored procedure, which allowed me to split the DELETE query into chunks, stored procedures are not handled as a single transaction, this allowed each delete operation to complete and move onto the next chunk, slowly deleting the data from the active table whilst the table is still in use.

As it was costly to query against the table or use LIMIT & OFFSET, which would take around 10 minutes to return results, it was not possible to use these as part of the procedure if we wanted this process to complete in a timely manner.

As the ID where incremental, I was able to query the table until I found the largest ID I was willing to delete up to, in my case it was 530,000,000.

The below stored procedure is what I ended up using to solve the issue.

CREATE PROCEDURE DeleteLargeData()
BEGIN
	DECLARE incrementValue INT DEFAULT 10000;
	DECLARE curMaxId BIGINT DEFAULT 10000;
	DECLARE maxIdx BIGINT DEFAULT 530000000;
	
	WHILE curMaxId <= maxIdx DO
		DELETE FROM table WHERE id < curMaxId;
		SET curMaxId = curMaxId + incrementValue;
	END WHILE;
END

1

2

3

4

5

6

7

8

9

10

11

CREATE PROCEDURE DeleteLargeData()

BEGIN

DECLARE incrementValue INT DEFAULT 10000;

DECLARE curMaxId BIGINT DEFAULT 10000;

DECLARE maxIdx BIGINT DEFAULT 530000000;

WHILE curMaxId <= maxIdx DO

DELETE FROM table WHERE id < curMaxId;

SET curMaxId = curMaxId + incrementValue;

END WHILE;

END

It is recommended a professional under takes these actions, please always backup existing data / files / configs.

19th January 2021

cPanel – Sender x has an outgoing mail hold. Message will be reattempted later.

A client was having issues sending emails and receiving a number of alerts regarding delayed emails. After looking investigating a number of outbound emails where queued with a failure reason of:

R=enforce_mail_permissions defer (-1): “Sender their@email.com has an outgoing mail hold. Message will be reattempted later”

1	R=enforce_mail_permissions defer (-1): “Sender [email protected] has an outgoing mail hold. Message will be reattempted later”

The account in question had hit the sending limits configured within cPanel and there emails where being held.

Pre-Checks

It is important you pre-check and make sure the hold is not legitimate such as a spam attempt / hacked site, I checked the emails being sent and due to a influx in online orders there email usage per day has significantly increased.

1st Attempt

Try running the following commands under sudo / root, replaceing [accountname] with the cpanel account name of the user. Then try send an email or retry sending an email in the queue.

whmapi1 unsuspend_outgoing_email user=[accountname]
whmapi1 release_outgoing_email user=[accountname]

1 2	whmapi1 unsuspend_outgoing_email user=[accountname] whmapi1 release_outgoing_email user=[accountname]

2nd Attempt

The above worked for a couple accounts, but for one this did not work. There is 3 files depending on the type of block. Have a look in the following files below and remove the accountname / email address if present.

/etc/outgoing_mail_hold_users
/etc/outgoing_mail_suspended_users
/etc/email_send_limits

1

2

3

/etc/outgoing_mail_hold_users

/etc/outgoing_mail_suspended_users

/etc/email_send_limits

I found my user within “/etc/outgoing_mail_suspended_users” and once removed, everything went back to normal.

It is recommended a professional under takes these actions, please always backup existing files / configs.

5th January 2021

WooCommerce admin filter products by image

A client recently asked me to add a filter to the WooCommerce admin products list so they can filter items based on if the item had an image or not. This allowed them to easily find products which had missing images so they could correct.

Why not take a look how to filter by on sale items?

Code

/*
 * Woocommerce Filter by has image
 */
function custom__woocommerce_filter_by_hasimage($output) {
    global $wp_query;
    
    $selected = 0;
    if (isset($_GET[‘product_hasimage’])) {
        $selected = (int)$_GET[‘product_hasimage’];
    }
    
    $output .= ‘
		<select name=”product_hasimage” id=”dropdown_product_has_image”>
			<option value=””>Filter by image</option>
			<option value=”1″ ‘ . (($selected === 1) ? ‘selected=”selected”‘ : ”) . ‘>Has Image</option>
			<option value=”2″ ‘ . (($selected === 2) ? ‘selected=”selected”‘ : ”) . ‘>Has No Image</option>
		</select>
	‘;
    
    return $output;
}
add_action(‘woocommerce_product_filters’, ‘custom__woocommerce_filter_by_hasimage’);



/*
 * Woocommerce Filter by on has image where statement
 */
function custom__woocommerce_filter_by_has_image_where_statement($where, $query) {
    global $wpdb;
    
    // Only trigger if required
    if (!is_admin() || $_GET[‘post_type’] != “product” || !isset($_GET[‘product_hasimage’]) || (int)$_GET[‘product_hasimage’] <= 0) {
        return $where;
    }
    
    // Is filter applied
    if ($_GET[‘product_hasimage’] > 0) {
        // What condition to check
        $condition = ‘IN’;
        if ($_GET[‘product_hasimage’] == 1) {
            $condition = ‘NOT IN’;
        }
        
        // Query
        $queryStr = ‘
            SELECT ID FROM (
                SELECT ID
                FROM djqns_posts
                WHERE ID IN (
    				SELECT tp.ID
    				FROM djqns_posts tp
    				LEFT OUTER JOIN djqns_postmeta tpm ON tpm.post_id = tp.ID AND tpm.meta_key = “_thumbnail_id”
    				WHERE 1=1
    				AND (
    					tpm.meta_value = “0”
    					OR tpm.meta_key IS NULL
    				)
    				AND tp.post_type = “product”
    				ORDER BY tp.ID
                )
                AND post_type = “product”
                ORDER BY ID
            ) tmp
		‘;
        
        $where .= ‘ AND (‘ . $wpdb->posts . ‘.ID ‘ . $condition . ‘ (‘ . $queryStr . ‘)) ‘;
    }
    
    return $where;
}
add_filter(‘posts_where’ , ‘custom__woocommerce_filter_by_has_image_where_statement’, 10, 2);

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

/*

* Woocommerce Filter by has image

*/

function custom__woocommerce_filter_by_hasimage($output) {

global $wp_query;

$selected = 0;

if (isset($_GET[‘product_hasimage’])) {

$selected = (int)$_GET[‘product_hasimage’];

}

$output .= ‘

<option value=””>Filter by image</option>

<option value=”1″ ‘ . (($selected === 1) ? ‘selected=”selected”‘ : ”) . ‘>Has Image</option>

<option value=”2″ ‘ . (($selected === 2) ? ‘selected=”selected”‘ : ”) . ‘>Has No Image</option>

</select>

‘;

return $output;

}

add_action(‘woocommerce_product_filters’, ‘custom__woocommerce_filter_by_hasimage’);

/*

* Woocommerce Filter by on has image where statement

*/

function custom__woocommerce_filter_by_has_image_where_statement($where, $query) {

global $wpdb;

// Only trigger if required

if (!is_admin() || $_GET[‘post_type’] != “product” || !isset($_GET[‘product_hasimage’]) || (int)$_GET[‘product_hasimage’] <= 0) {

return $where;

}

// Is filter applied

if ($_GET[‘product_hasimage’] > 0) {

// What condition to check

$condition = ‘IN’;

if ($_GET[‘product_hasimage’] == 1) {

$condition = ‘NOT IN’;

}

// Query

$queryStr = ‘

SELECT ID FROM (

SELECT ID

FROM djqns_posts

WHERE ID IN (

SELECT tp.ID

FROM djqns_posts tp

LEFT OUTER JOIN djqns_postmeta tpm ON tpm.post_id = tp.ID AND tpm.meta_key = “_thumbnail_id”

WHERE 1=1

AND (

tpm.meta_value = “0”

OR tpm.meta_key IS NULL

)

AND tp.post_type = “product”

ORDER BY tp.ID

)

AND post_type = “product”

ORDER BY ID

) tmp

‘;

$where .= ‘ AND (‘ . $wpdb–>posts . ‘.ID ‘ . $condition . ‘ (‘ . $queryStr . ‘)) ‘;

}

return $where;

}

add_filter(‘posts_where’ , ‘custom__woocommerce_filter_by_has_image_where_statement’, 10, 2);

This code is provided as it and may require changes to work for your needs.

17th December 2020

WooCommerce delayed webhooks using actions scheduler

Recently been contacted by a user who needed the ability to delay the WooCommerce webhooks by x time, this would allow other processes happening on the site to finish instead of firing the webhook straight away, for example waiting for the payment process to finalise. Delaying webhooks is not common practice, however some time special case arises!

Take a look at actions scheduler if you wish to read more into the scheduler system used by WooCommerce.

Code

The following code allows you to delay several WooCommerces webhooks by a set duration (seconds) and also run in two modes.

Single Webhook, creates 1 single webhook but delayed by x seconds.
Multi Webhook, creates 2 webhooks, the normal webhook created by WooCommerce and a second repeat webhook which runs x seconds later.

function filter_woocommerce_webhook_deliver_async($true, $instance, $arg) {
    // Replace default schedule action? (single web hook fired, with delay)
    // Or schedule a seperate delayed action (default webhook fied with a second delayed web hook)
    $singleWebhook = false;
    
    // Target webhook topics
    $targetWebhookTopics = [
        ‘order.updated’,
    ];
    
    // Delay time in seconds
    $delayTime = 300;
    
    // If order.update, we want to create a seperate delayed webhook via schedule actions
    if (in_array($instance->get_topic(), $targetWebhookTopics)) {
        $payload = [‘webhook_id’ => $instance->get_id(), arg => $arg];
        $group = $singleWebhook ? ‘woocommerce-webhooks’ : ‘woocommerce-webhooks-delayed’;
        
        // Schedule a single action to run in 5 minutes, to cause the order.update to fire (delayed)
        // Make sure to run as_next_scheduled_action so multiple instances are not scheduled if one is already scheduled
        if (as_next_scheduled_action(‘woocommerce_deliver_webhook_async’, $payload, $group) === false) {
            as_schedule_single_action(time() + $delayTime, ‘woocommerce_deliver_webhook_async’, $payload, $group);
        }
    }
    
    return $true;
};
add_filter(‘woocommerce_webhook_deliver_async’, ‘filter_woocommerce_webhook_deliver_async’, 10, 3);

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

function filter_woocommerce_webhook_deliver_async($true, $instance, $arg) {

// Replace default schedule action? (single web hook fired, with delay)

// Or schedule a seperate delayed action (default webhook fied with a second delayed web hook)

$singleWebhook = false;

// Target webhook topics

$targetWebhookTopics = [

‘order.updated’,

];

// Delay time in seconds

$delayTime = 300;

// If order.update, we want to create a seperate delayed webhook via schedule actions

if (in_array($instance–>get_topic(), $targetWebhookTopics)) {

$payload = [‘webhook_id’ => $instance–>get_id(), arg => $arg];

$group = $singleWebhook ? ‘woocommerce-webhooks’ : ‘woocommerce-webhooks-delayed’;

// Schedule a single action to run in 5 minutes, to cause the order.update to fire (delayed)

// Make sure to run as_next_scheduled_action so multiple instances are not scheduled if one is already scheduled

if (as_next_scheduled_action(‘woocommerce_deliver_webhook_async’, $payload, $group) === false) {

as_schedule_single_action(time() + $delayTime, ‘woocommerce_deliver_webhook_async’, $payload, $group);

}

return $true;

};

add_filter(‘woocommerce_webhook_deliver_async’, ‘filter_woocommerce_webhook_deliver_async’, 10, 3);

This code is provided as it and may require changes to work for your needs.

12th November 2020