Category: Programming & Development

Windows 10 Subsystem for Linux Ubuntu

I have always been more of a Windows based developer and used Linux based systems for server technologies, I have tried in the past to move to Linux but I am to fixed in my way and prefer the tools I have on windows! Is that a bad thing, I dont no? however with Windows 10 comes Subsystems which allows you to run a Linux based Ubuntu as part of a subsystem working jointly with your windows 10 operating system.

Im not talking about a VM or dual booting, I am talking about the ability to go to windows command line and type “ubuntu” and a full fledged ubuntu system from the command line starts up and allows you to run various tools, applications and services all within Ubuntu within Windows 10!

Microsoft site has a simple guide to follow to install the linux subsystem here.

1. Enabled windows subsystems for Linux

Powershell
PowerShell

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

1

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

2. Install Ubuntu via Windows Store

Open Microsoft Store and search for Ubuntu and click install

3. Initialise Setup

Open command line and type ubuntu to start the ubuntu instance, it will take you throught a initialisation stage

What next? have play around with Ubuntu on windows. For development purposes I am going to be using it with Docker! allowing me to control my Windows Docker environment from the Ubuntu Linux, as it allows me to use all the Linux based command structures and pipes making the experience even smoother and more powerful.

11th April 2019

Fix for WooCommerce schema data missing brand and mpn

UPDATED 25/10/2021 to comply with latest google changes.

Today I came across and issue where Google Search Console was complaning products “brand” and “mpn” where missing from the products schema / rich snippets data. You can checking using the rich snippet testing tool by Google found here.

https://search.google.com/structured-data/testing-tool

I fixed my issue by adding a WordPress filter using the “woocommerce_structured_data_product” filter option. This allows you to add new data to the already generated WooCommerce data.

This is an example. You will need to change for your implementation.

/*
 * Rich Snippet Data
 * Add missing data not handled by WooCommerce yet
 */
function custom_woocommerce_structured_data_product ($data) {
	global $product;
	
	$data[‘brand’] = $data[‘brand’] = [‘@type’ => ‘Brand’, ‘name’ => $product->get_attribute(‘pa_manufacturer’) ?? null];
	$data[‘mpn’] = $product->get_sku() ?? null;
	
	return $data;
}
add_filter( ‘woocommerce_structured_data_product’, ‘custom_woocommerce_structured_data_product’ );

* Rich Snippet Data

* Add missing data not handled by WooCommerce yet

function custom_woocommerce_structured_data_product ($data) {

global $product;

$data[‘brand’] = $data[‘brand’] = [‘@type’ => ‘Brand’, ‘name’ => $product–>get_attribute(‘pa_manufacturer’) ?? null];

$data[‘mpn’] = $product–>get_sku() ?? null;

return $data;

}

add_filter( ‘woocommerce_structured_data_product’, ‘custom_woocommerce_structured_data_product’ );

2nd April 2019

Twitter Bootstrap 3 Media Queries

Some useful bootstrap 3 media queries for both mobile first and non mobile first development styles, to make modifications to your front end simplier and follow the bootstrap 3 standard breakpoints.

Origional article can be found here.

Min-Width: Refers to everything greater than or equal to the amount given.
Max-Width: Refers to everything less than or equal to the amount given.

/*========== Mobile First Method ==========*/

/* Custom, iPhone Retina */
@media only screen and (min-width : 320px) {

}

/* Extra Small Devices, Phones */
@media only screen and (min-width : 480px) {

}

/* Small Devices, Tablets */
@media only screen and (min-width : 768px) {

}

/* Medium Devices, Desktops */
@media only screen and (min-width : 992px) {

}

/* Large Devices, Wide Screens */
@media only screen and (min-width : 1200px) {

}

/*========== Non-Mobile First Method ==========*/

/* Large Devices, Wide Screens */
@media only screen and (max-width : 1200px) {

}

/* Medium Devices, Desktops */
@media only screen and (max-width : 992px) {

}

/* Small Devices, Tablets */
@media only screen and (max-width : 768px) {

}

/* Extra Small Devices, Phones */
@media only screen and (max-width : 480px) {

}

/* Custom, iPhone Retina */
@media only screen and (max-width : 320px) {

}

/*========== Mobile First Method ==========*/

/* Custom, iPhone Retina */

@media only screen and (min–width : 320px) {

}

/* Extra Small Devices, Phones */

@media only screen and (min–width : 480px) {

}

/* Small Devices, Tablets */

@media only screen and (min–width : 768px) {

}

/* Medium Devices, Desktops */

@media only screen and (min–width : 992px) {

}

/* Large Devices, Wide Screens */

@media only screen and (min–width : 1200px) {

}

/*========== Non-Mobile First Method ==========*/

/* Large Devices, Wide Screens */

@media only screen and (max–width : 1200px) {

}

/* Medium Devices, Desktops */

@media only screen and (max–width : 992px) {

}

/* Small Devices, Tablets */

@media only screen and (max–width : 768px) {

}

/* Extra Small Devices, Phones */

@media only screen and (max–width : 480px) {

}

/* Custom, iPhone Retina */

@media only screen and (max–width : 320px) {

}

1st April 2019

htaccess: Block bad bots for security purposes

Please double check any bots you wish to allow are not in the block list. This is a list of known bots, as well as default user agents.

I have had a scenario where a website was getting scraped by a number of bots, causing excessive resource usage on the server and disrupting sales and bad experience for real customers.

Here is a basic technique to stop those pesky bots, simply add the following to your .htaccess file. You may need to modify the exceptions at the top if you do require certain files to be accessed by bots. Currently it will allow access to robots.txt and acme-challenge for LetsEncrypt.

# BLOCK BAD BOTS START
RewriteEngine On
RewriteCond %{REQUEST_URI} !/robots.txt$
RewriteCond %{REQUEST_URI} !\.well-known/acme-challenge
RewriteCond %{HTTP_USER_AGENT} ^$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EventMachine.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NerdyBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Typhoeus.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*archive.org_bot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*archive.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*github.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*chroot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Jorgee.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go\ 1.1\ package.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go-http-client.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copyscape.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*semrushbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*semrushbot-sa.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JamesBOT.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SEOkicks-Robot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkpadBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*getty.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*picscout.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*AppEngine.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Zend_Http_Client.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlackWidow.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*openlink.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*spbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Nutch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Jetbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebVac.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Stanford.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*scooter.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*naver.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*dumbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Hatena\ Antenna.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*grub.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*looksmart.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZip.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*b2w/0.1.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copernic.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*psbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetMechanic.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*URL_Spider_Pro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CherryPicker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CopyRightCheck.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Crescent.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CCBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ProWebWalker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CheeseBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LNSpiderguy.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailCollector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Alexibot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Telesoft.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*moget.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*TheNomad.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WWW-Collector-E.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*RMA.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*libWeb/clsHTTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*asterias.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*httplib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*turingos.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*spanner.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Harvest.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InfoNaviRobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bullseye.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NICErsPRO.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Microsoft\ URL\ Control.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DittoSpyder.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Foobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorldForumBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SpankBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BotALot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*lwp-trivial.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorld.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BunnySlippers.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*URLy\ Warning.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkWalker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*cosmos.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*hloader.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*humanlinks.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkextractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mata\ Hari.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LexiBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*woobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*The\ Intraformant.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*True_Robot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlowFish.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SearchEngineWorld.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JennyBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BuiltBotTough.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ProPowerBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BackDoorBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*toCrawl/UrlDispatcher.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebEnhancer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*suzuran.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebViewer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*VCI.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Szukacz.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*QueryN.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Openfind.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Openbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EroCrawler.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkScan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Keyword.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Kenjin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Iron33.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bookmark\ search\ tool.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FairAd\ Client.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Gaisbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Aqua_Products.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Radiation\ Retriever\ 1.1.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Flaming\ AttackBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Oracle\ Ultra\ Search.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MSIECrawler.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PerMan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*searchpreview.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*sootle.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Enterprise_Search.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bot\ mailto:craftbot@yahoo.com.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ChinaClaw.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Custo.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DISCo.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Download\ Demon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Express\ WebPictures.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetWeb!.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go-Ahead-Got-It.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Stripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Sucker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Indy\ Library.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Internet\ Ninja.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JOC\ Web\ Spider.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LeechFTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mass\ Downloader.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIDown\ tool.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Net\ Vampire.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetZIP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Octopus.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Navigator.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Papa\ Foto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pavuk.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*RealDownload.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport\ Pro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*VoidEYE.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Sucker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebGo\ IS.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebLeacher.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*wesee.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ eXtractor.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WWWOFFLE.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Xaldon\ WebSpider.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Zeus.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Semrush.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BecomeBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Screaming.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Screaming\ FrogSEO.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SEO.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*AhrefsBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*rogerbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*exabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Xenu.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*dotbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*gigabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Twengabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*htmlparser.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*libwww.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Python.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*perl.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*urllib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*scan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Curl.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*email.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PycURL.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Python-urllib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Pyth.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PyQ.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCollector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopy.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*webcraw.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*webcraw.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot_IgnoreIP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlekkoBot.*$ [NC]
RewriteRule ^.*.* http://www.google.com/ [L]
# BLOCK BAD BOTS END

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

# BLOCK BAD BOTS START

RewriteEngine On

RewriteCond %{REQUEST_URI} !/robots.txt$

RewriteCond %{REQUEST_URI} !\.well–known/acme–challenge

RewriteCond %{HTTP_USER_AGENT} ^$ [OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EventMachine.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NerdyBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Typhoeus.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*archive.org_bot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*archive.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*github.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*chroot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Jorgee.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Go\ 1.1\ package.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Go–http–client.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Copyscape.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*semrushbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*semrushbot–sa.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*JamesBOT.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SEOkicks–Robot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LinkpadBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*getty.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*picscout.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*AppEngine.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Zend_Http_Client.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BlackWidow.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*openlink.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*spbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Nutch.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Jetbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebVac.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Stanford.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*scooter.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*naver.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*dumbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Hatena\ Antenna.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*grub.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*looksmart.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebZip.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*b2w/0.1.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Copernic.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*psbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetMechanic.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*URL_Spider_Pro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*CherryPicker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*CopyRightCheck.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Crescent.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*CCBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ProWebWalker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*CheeseBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LNSpiderguy.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailCollector.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Alexibot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Telesoft.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*moget.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*TheNomad.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WWW–Collector–E.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*RMA.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*libWeb/clsHTTP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*asterias.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*httplib.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*turingos.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*spanner.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Harvest.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*InfoNaviRobot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Bullseye.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NICErsPRO.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Microsoft\ URL\ Control.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*DittoSpyder.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Foobot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorldForumBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SpankBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BotALot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*lwp–trivial.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorld.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BunnySlippers.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*URLy\ Warning.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LinkWalker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*cosmos.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*hloader.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*humanlinks.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LinkextractorPro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Mata\ Hari.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LexiBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*woobot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*The\ Intraformant.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*True_Robot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BlowFish.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SearchEngineWorld.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*JennyBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BuiltBotTough.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ProPowerBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BackDoorBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*toCrawl/UrlDispatcher.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebEnhancer.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*suzuran.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebViewer.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*VCI.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Szukacz.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*QueryN.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Openfind.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Openbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EroCrawler.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LinkScan.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Keyword.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Kenjin.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Iron33.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Bookmark\ search\ tool.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*FairAd\ Client.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Gaisbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Aqua_Products.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Radiation\ Retriever\ 1.1.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Flaming\ AttackBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Oracle\ Ultra\ Search.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MSIECrawler.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*PerMan.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*searchpreview.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*sootle.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Enterprise_Search.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Bot\ mailto:craftbot@yahoo.com.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ChinaClaw.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Custo.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*DISCo.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Download\ Demon.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Express\ WebPictures.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*GetWeb!.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Go–Ahead–Got–It.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Stripper.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Sucker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Indy\ Library.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Internet\ Ninja.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*JOC\ Web\ Spider.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LeechFTP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Mass\ Downloader.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MIDown\ tool.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Net\ Vampire.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetZIP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Octopus.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Navigator.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Papa\ Foto.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*pavuk.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*RealDownload.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Teleport\ Pro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*VoidEYE.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Sucker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebGo\ IS.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebLeacher.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*wesee.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Website\ eXtractor.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WWWOFFLE.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Xaldon\ WebSpider.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Zeus.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Semrush.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BecomeBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Screaming.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Screaming\ FrogSEO.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SEO.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*AhrefsBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*rogerbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*exabot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Xenu.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*dotbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*gigabot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Twengabot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*htmlparser.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*libwww.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Python.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*perl.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*urllib.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*scan.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Curl.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*email.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*PycURL.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Python–urllib.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Pyth.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*PyQ.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebCollector.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebCopy.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*webcraw.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot_IgnoreIP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BlekkoBot.*$ [NC]

RewriteRule ^.*.* http://www.google.com/ [L]

# BLOCK BAD BOTS END

31st July 2018

htaccess: HTTP to HTTPS redirect

It is recommended to you SSL on your website even when not required. Google Chrome and other browsers have begun to flag non-SSL websites as insecure. Don’t lose your visitors due to appearing as an insecure website.

With the introduction of LetsEncyrpt and other free SSL certificate providers, there is no reason for websites to not be secured. Google Chrome is now beginning to flag websites which are non-HTTPS as secure leading to your visitors and customers believe your website is insecure. Make sure your using and SSL and redirecting all non-HTTP traffic to HTTPS using the below htaccess config setting for Apache.

RewriteEngine On RewriteCond %{SERVER_PORT} 80 RewriteRule ^(.*)$ https://shanerutter.co.uk/$1 [R=301,L]

1
2
3

RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://shanerutter.co.uk/$1 [R=301,L]

18th July 2018
Automatic MySQL database backups and zipping on windows with a bat file

It is important you double check your configuration before running this script! We cannot be held responsible for any damage this may cause.

Need to backup all your databases on a MySQL windows server? Red Olive have created a simple windows batch file to export all database tables and zip, including deleting older backups! It’s quite simple to setup look at their website.

https://www.redolive.com/utah-web-designers-blog/automated-mysql-backup-for-windows/

Their version is configured for a US date/time format, I am in the UK with UK based date/time formats, I have made a modified version of the script available below.

You should setup a scheduled task to run the bat file on a regular interval, the more often you perform backups the easier it will be for you if there is a failure and you require to restore a backup.

We also advise you look at a process for storing backups at remote location, if you backup to a Dropbox folder your backups will be uploaded to the cloud, protecting you if the server dies or has disk failure!

Download MySQLBackups Script (UK Timezone)

23rd August 2017
Local website & services remotely accessible via secure tunnels with ngrok
Need an easy way to allow external network users to access services or websites on your local machine? ngrok allows you to easily create a secure tunnel from ngroks servers to your machine. For a web developer, this is a dream come true, this is a great way to demo a development whilst in the early stages, saves time uploading to a server.
- Allows access to your locally hosted web server from a unique domain
- Can use your own domains
- Works with vHost files
- Allows testing of website which require an external accessible point such as payment gateways or oAuth
- Can tunnel other services such as PhoneGap allowing users external to your network to test your mobile app
- Various API and IoT implementations
- Easy / quick private networking, VPN benefits
These are only a short list of what is possible using ngrok. Look at their website for a full list of its powerful features and get started.

https://ngrok.com/
6th August 2017
Laravel, time zones and carbon

Laravel defaults to storing all dates in UTC format, this is highly recommended as UTC time never changes unlike other time zones which have daylight savings. It is best practice to store and read dates in UTC, and then convert UTC to your required time zone for the user interface.

Great article below of a good way to implement time zones using Carbon within your Laravel app.

https://andrew.cool/blog/49/Easy-timezones-in-Laravel-with-Carbon

26th June 2017
Repairing damaged or crashed MySQL tables

Always remember, regular backups of your databases are best practice, in the event the below fail, backups may be the only method you have to restore a working table or database.

Damaged or crashed MySQL tables due to an unsafe server shut downs, disk space limits or general MySQL failures? You will normally see something along the lines of “marked as crashed and should be repaired when using LOCK TABLES” within your MySQL log files.

MySQL has a number of tools to try and repair crashed / damaged tables. Log into your server as root and run the following commands. Replacing “/var/lib/mysql/” with the location of your DB files.

Check what tables need repaired:

myisamchk –check /var/lib/mysql/*/*.MYI

Try a safe recovery:

myisamchk –safe-recover /var/lib/mysql/*/*.MYI

Try a normal recovery (if safe recovery fails):

myisamchk –recover /var/lib/mysql/*/*.MYI

Try a force recovery (as a final last step):

myisamchk –safe-recover –extend-check –force /var/lib/mysql/*/*.MYI

or:

myisamchk –recover –extend-check –force /var/lib/mysql/*/*.MYI

9th June 2017
Easy mobile browser detection for several major languages and web servers

All app’s and websites these days need to work for several devices including desktop, tablets and mobiles. It’s not always simple detecting the type of device being used and it can sometimes be tedious adding support to your app for something which seems very simple.

Does your chosen platform not have a pre-defined way of detecting a mobile browser? Check out Detect Mobile Browsers, has several copy / paste examples for several programming languages.

http://detectmobilebrowsers.com/

Remember Apache, IIS and nginx detection examples also present for doing for pesky redirects.

17th May 2017

Category: Programming & Development

1. Enabled windows subsystems for Linux

2. Install Ubuntu via Windows Store

3. Initialise Setup