Blog

Windows 10 Subsystem for Linux Ubuntu

I have always been more of a Windows based developer and used Linux based systems for server technologies, I have tried in the past to move to Linux but I am to fixed in my way and prefer the tools I have on windows! Is that a bad thing, I dont no? however with Windows 10 comes Subsystems which allows you to run a Linux based Ubuntu as part of a subsystem working jointly with your windows 10 operating system.

Im not talking about a VM or dual booting, I am talking about the ability to go to windows command line and type “ubuntu” and a full fledged ubuntu system from the command line starts up and allows you to run various tools, applications and services all within Ubuntu within Windows 10!

Microsoft site has a simple guide to follow to install the linux subsystem here.

1. Enabled windows subsystems for Linux

Powershell
PowerShell

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

1

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

2. Install Ubuntu via Windows Store

Open Microsoft Store and search for Ubuntu and click install

3. Initialise Setup

Open command line and type ubuntu to start the ubuntu instance, it will take you throught a initialisation stage

What next? have play around with Ubuntu on windows. For development purposes I am going to be using it with Docker! allowing me to control my Windows Docker environment from the Ubuntu Linux, as it allows me to use all the Linux based command structures and pipes making the experience even smoother and more powerful.

11th April 2019

Fix for WooCommerce schema data missing brand and mpn

UPDATED 25/10/2021 to comply with latest google changes.

Today I came across and issue where Google Search Console was complaning products “brand” and “mpn” where missing from the products schema / rich snippets data. You can checking using the rich snippet testing tool by Google found here.

https://search.google.com/structured-data/testing-tool

I fixed my issue by adding a WordPress filter using the “woocommerce_structured_data_product” filter option. This allows you to add new data to the already generated WooCommerce data.

This is an example. You will need to change for your implementation.

/*
 * Rich Snippet Data
 * Add missing data not handled by WooCommerce yet
 */
function custom_woocommerce_structured_data_product ($data) {
	global $product;
	
	$data[‘brand’] = $data[‘brand’] = [‘@type’ => ‘Brand’, ‘name’ => $product->get_attribute(‘pa_manufacturer’) ?? null];
	$data[‘mpn’] = $product->get_sku() ?? null;
	
	return $data;
}
add_filter( ‘woocommerce_structured_data_product’, ‘custom_woocommerce_structured_data_product’ );

* Rich Snippet Data

* Add missing data not handled by WooCommerce yet

function custom_woocommerce_structured_data_product ($data) {

global $product;

$data[‘brand’] = $data[‘brand’] = [‘@type’ => ‘Brand’, ‘name’ => $product–>get_attribute(‘pa_manufacturer’) ?? null];

$data[‘mpn’] = $product–>get_sku() ?? null;

return $data;

}

add_filter( ‘woocommerce_structured_data_product’, ‘custom_woocommerce_structured_data_product’ );

2nd April 2019

Twitter Bootstrap 3 Media Queries

Some useful bootstrap 3 media queries for both mobile first and non mobile first development styles, to make modifications to your front end simplier and follow the bootstrap 3 standard breakpoints.

Origional article can be found here.

Min-Width: Refers to everything greater than or equal to the amount given.
Max-Width: Refers to everything less than or equal to the amount given.

/*========== Mobile First Method ==========*/

/* Custom, iPhone Retina */
@media only screen and (min-width : 320px) {

}

/* Extra Small Devices, Phones */
@media only screen and (min-width : 480px) {

}

/* Small Devices, Tablets */
@media only screen and (min-width : 768px) {

}

/* Medium Devices, Desktops */
@media only screen and (min-width : 992px) {

}

/* Large Devices, Wide Screens */
@media only screen and (min-width : 1200px) {

}

/*========== Non-Mobile First Method ==========*/

/* Large Devices, Wide Screens */
@media only screen and (max-width : 1200px) {

}

/* Medium Devices, Desktops */
@media only screen and (max-width : 992px) {

}

/* Small Devices, Tablets */
@media only screen and (max-width : 768px) {

}

/* Extra Small Devices, Phones */
@media only screen and (max-width : 480px) {

}

/* Custom, iPhone Retina */
@media only screen and (max-width : 320px) {

}

/*========== Mobile First Method ==========*/

/* Custom, iPhone Retina */

@media only screen and (min–width : 320px) {

}

/* Extra Small Devices, Phones */

@media only screen and (min–width : 480px) {

}

/* Small Devices, Tablets */

@media only screen and (min–width : 768px) {

}

/* Medium Devices, Desktops */

@media only screen and (min–width : 992px) {

}

/* Large Devices, Wide Screens */

@media only screen and (min–width : 1200px) {

}

/*========== Non-Mobile First Method ==========*/

/* Large Devices, Wide Screens */

@media only screen and (max–width : 1200px) {

}

/* Medium Devices, Desktops */

@media only screen and (max–width : 992px) {

}

/* Small Devices, Tablets */

@media only screen and (max–width : 768px) {

}

/* Extra Small Devices, Phones */

@media only screen and (max–width : 480px) {

}

/* Custom, iPhone Retina */

@media only screen and (max–width : 320px) {

}

1st April 2019

htaccess: Block bad bots for security purposes

Please double check any bots you wish to allow are not in the block list. This is a list of known bots, as well as default user agents.

I have had a scenario where a website was getting scraped by a number of bots, causing excessive resource usage on the server and disrupting sales and bad experience for real customers.

Here is a basic technique to stop those pesky bots, simply add the following to your .htaccess file. You may need to modify the exceptions at the top if you do require certain files to be accessed by bots. Currently it will allow access to robots.txt and acme-challenge for LetsEncrypt.

# BLOCK BAD BOTS START
RewriteEngine On
RewriteCond %{REQUEST_URI} !/robots.txt$
RewriteCond %{REQUEST_URI} !\.well-known/acme-challenge
RewriteCond %{HTTP_USER_AGENT} ^$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EventMachine.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NerdyBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Typhoeus.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*archive.org_bot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*archive.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*github.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*chroot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Jorgee.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go\ 1.1\ package.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go-http-client.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copyscape.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*semrushbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*semrushbot-sa.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JamesBOT.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SEOkicks-Robot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkpadBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*getty.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*picscout.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*AppEngine.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Zend_Http_Client.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlackWidow.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*openlink.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*spbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Nutch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Jetbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebVac.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Stanford.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*scooter.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*naver.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*dumbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Hatena\ Antenna.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*grub.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*looksmart.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZip.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*b2w/0.1.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copernic.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*psbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetMechanic.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*URL_Spider_Pro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CherryPicker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CopyRightCheck.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Crescent.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CCBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ProWebWalker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CheeseBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LNSpiderguy.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailCollector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Alexibot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Telesoft.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*moget.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*TheNomad.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WWW-Collector-E.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*RMA.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*libWeb/clsHTTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*asterias.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*httplib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*turingos.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*spanner.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Harvest.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InfoNaviRobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bullseye.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NICErsPRO.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Microsoft\ URL\ Control.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DittoSpyder.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Foobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorldForumBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SpankBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BotALot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*lwp-trivial.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorld.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BunnySlippers.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*URLy\ Warning.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkWalker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*cosmos.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*hloader.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*humanlinks.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkextractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mata\ Hari.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LexiBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*woobot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*The\ Intraformant.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*True_Robot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlowFish.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SearchEngineWorld.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JennyBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BuiltBotTough.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ProPowerBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BackDoorBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*toCrawl/UrlDispatcher.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebEnhancer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*suzuran.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebViewer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*VCI.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Szukacz.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*QueryN.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Openfind.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Openbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EroCrawler.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LinkScan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Keyword.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Kenjin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Iron33.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bookmark\ search\ tool.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FairAd\ Client.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Gaisbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Aqua_Products.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Radiation\ Retriever\ 1.1.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Flaming\ AttackBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Oracle\ Ultra\ Search.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MSIECrawler.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PerMan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*searchpreview.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*sootle.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Enterprise_Search.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bot\ mailto:craftbot@yahoo.com.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ChinaClaw.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Custo.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DISCo.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Download\ Demon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Express\ WebPictures.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetWeb!.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go-Ahead-Got-It.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Stripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Sucker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Indy\ Library.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Internet\ Ninja.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JOC\ Web\ Spider.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LeechFTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mass\ Downloader.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIDown\ tool.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Net\ Vampire.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetZIP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Octopus.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Navigator.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Papa\ Foto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pavuk.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*RealDownload.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport\ Pro.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*VoidEYE.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Sucker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebGo\ IS.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebLeacher.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*wesee.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ eXtractor.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WWWOFFLE.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Xaldon\ WebSpider.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Zeus.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Semrush.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BecomeBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Screaming.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Screaming\ FrogSEO.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SEO.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*AhrefsBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*rogerbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*exabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Xenu.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*dotbot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*gigabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Twengabot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*htmlparser.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*libwww.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Python.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*perl.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*urllib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*scan.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Curl.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*email.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PycURL.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Python-urllib.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Pyth.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PyQ.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCollector.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopy.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*webcraw.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*webcraw.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot_IgnoreIP.*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlekkoBot.*$ [NC]
RewriteRule ^.*.* http://www.google.com/ [L]
# BLOCK BAD BOTS END

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

# BLOCK BAD BOTS START

RewriteEngine On

RewriteCond %{REQUEST_URI} !/robots.txt$

RewriteCond %{REQUEST_URI} !\.well–known/acme–challenge

RewriteCond %{HTTP_USER_AGENT} ^$ [OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EventMachine.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NerdyBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Typhoeus.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*archive.org_bot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*archive.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*github.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*chroot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Jorgee.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Go\ 1.1\ package.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Go–http–client.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Copyscape.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*semrushbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*semrushbot–sa.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*JamesBOT.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SEOkicks–Robot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LinkpadBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*getty.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*picscout.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*AppEngine.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Zend_Http_Client.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BlackWidow.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*openlink.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*spbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Nutch.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Jetbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebVac.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Stanford.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*scooter.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*naver.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*dumbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Hatena\ Antenna.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*grub.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*looksmart.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebZip.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*b2w/0.1.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Copernic.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*psbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetMechanic.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*URL_Spider_Pro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*CherryPicker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*CopyRightCheck.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Crescent.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*CCBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ProWebWalker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*CheeseBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LNSpiderguy.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailCollector.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Alexibot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Telesoft.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*moget.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*TheNomad.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WWW–Collector–E.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*RMA.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*libWeb/clsHTTP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*asterias.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*httplib.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*turingos.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*spanner.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Harvest.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*InfoNaviRobot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Bullseye.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NICErsPRO.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Microsoft\ URL\ Control.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*DittoSpyder.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Foobot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorldForumBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SpankBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BotALot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*lwp–trivial.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorld.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BunnySlippers.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*URLy\ Warning.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LinkWalker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*cosmos.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*hloader.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*humanlinks.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LinkextractorPro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Mata\ Hari.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LexiBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*woobot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*The\ Intraformant.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*True_Robot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BlowFish.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SearchEngineWorld.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*JennyBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BuiltBotTough.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ProPowerBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BackDoorBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*toCrawl/UrlDispatcher.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebEnhancer.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*suzuran.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebViewer.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*VCI.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Szukacz.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*QueryN.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Openfind.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Openbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EroCrawler.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LinkScan.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Keyword.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Kenjin.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Iron33.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Bookmark\ search\ tool.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*FairAd\ Client.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Gaisbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Aqua_Products.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Radiation\ Retriever\ 1.1.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Flaming\ AttackBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Oracle\ Ultra\ Search.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MSIECrawler.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*PerMan.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*searchpreview.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*sootle.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Enterprise_Search.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Bot\ mailto:craftbot@yahoo.com.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ChinaClaw.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Custo.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*DISCo.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Download\ Demon.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Express\ WebPictures.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*GetWeb!.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Go–Ahead–Got–It.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Stripper.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Image\ Sucker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Indy\ Library.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Internet\ Ninja.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*JOC\ Web\ Spider.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*LeechFTP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Mass\ Downloader.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MIDown\ tool.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Mister\ PiX.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Net\ Vampire.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*NetZIP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Octopus.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Explorer.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Offline\ Navigator.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Papa\ Foto.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*pavuk.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*RealDownload.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Teleport\ Pro.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*VoidEYE.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Image\ Collector.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Web\ Sucker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebGo\ IS.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebLeacher.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*wesee.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Website\ eXtractor.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Website\ Quester.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WWWOFFLE.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Xaldon\ WebSpider.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Zeus.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Semrush.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BecomeBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Screaming.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Screaming\ FrogSEO.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SEO.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*AhrefsBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*rogerbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*exabot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Xenu.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*dotbot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*gigabot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Twengabot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*htmlparser.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*libwww.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Python.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*perl.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*urllib.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*scan.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Curl.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*email.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*PycURL.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Python–urllib.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*Pyth.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*PyQ.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebCollector.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*WebCopy.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*webcraw.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot_IgnoreIP.*$ [NC,OR]

RewriteCond %{HTTP_USER_AGENT} ^.*BlekkoBot.*$ [NC]

RewriteRule ^.*.* http://www.google.com/ [L]

# BLOCK BAD BOTS END

31st July 2018

htaccess: HTTP to HTTPS redirect

It is recommended to you SSL on your website even when not required. Google Chrome and other browsers have begun to flag non-SSL websites as insecure. Don’t lose your visitors due to appearing as an insecure website.

With the introduction of LetsEncyrpt and other free SSL certificate providers, there is no reason for websites to not be secured. Google Chrome is now beginning to flag websites which are non-HTTPS as secure leading to your visitors and customers believe your website is insecure. Make sure your using and SSL and redirecting all non-HTTP traffic to HTTPS using the below htaccess config setting for Apache.

RewriteEngine On RewriteCond %{SERVER_PORT} 80 RewriteRule ^(.*)$ https://shanerutter.co.uk/$1 [R=301,L]

1
2
3

RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://shanerutter.co.uk/$1 [R=301,L]

18th July 2018
Linux: Killall command
It is your responsibility to ensure safe usage of these commands, improper usage could cause damage to services and file systems. We will not be held responsible.

There is a number way of killing processes on a Linux based OS, killall is one of these methods. It’s always good to be ready to kill an unwanted process or a rogue process which is eating resources.

Usage

Kill all instances of the “httpd” process

killall httpd

1

killall httpd

Kill all proccesses running under the user “postfix”

killall -u postfix

1

killall –u postfix

Kill a specific process but its full path

killall /usr/libexec/postfix/master

1

killall /usr/libexec/postfix/master

Command not found?

It does not always come pre-installed, it’s simple to install.

Ubuntu / Debian

apt-get install psmisc

1

apt–get install psmisc

RHEL – Red hat / Fedora / CentOS

yum install psmisc

1

yum install psmisc

This package also contains the following useful programs:

fuser – identifies what processes are using files.

killall – kills a process by its name, similar to a pkill Unices.

pstree – Shows currently running processes in a tree format.

peekfd – Peek at file descriptors of running processes.
31st August 2017
Automatic MySQL database backups and zipping on windows with a bat file

It is important you double check your configuration before running this script! We cannot be held responsible for any damage this may cause.

Need to backup all your databases on a MySQL windows server? Red Olive have created a simple windows batch file to export all database tables and zip, including deleting older backups! It’s quite simple to setup look at their website.

https://www.redolive.com/utah-web-designers-blog/automated-mysql-backup-for-windows/

Their version is configured for a US date/time format, I am in the UK with UK based date/time formats, I have made a modified version of the script available below.

You should setup a scheduled task to run the bat file on a regular interval, the more often you perform backups the easier it will be for you if there is a failure and you require to restore a backup.

We also advise you look at a process for storing backups at remote location, if you backup to a Dropbox folder your backups will be uploaded to the cloud, protecting you if the server dies or has disk failure!

Download MySQLBackups Script (UK Timezone)

23rd August 2017
Local website & services remotely accessible via secure tunnels with ngrok
Need an easy way to allow external network users to access services or websites on your local machine? ngrok allows you to easily create a secure tunnel from ngroks servers to your machine. For a web developer, this is a dream come true, this is a great way to demo a development whilst in the early stages, saves time uploading to a server.
- Allows access to your locally hosted web server from a unique domain
- Can use your own domains
- Works with vHost files
- Allows testing of website which require an external accessible point such as payment gateways or oAuth
- Can tunnel other services such as PhoneGap allowing users external to your network to test your mobile app
- Various API and IoT implementations
- Easy / quick private networking, VPN benefits
These are only a short list of what is possible using ngrok. Look at their website for a full list of its powerful features and get started.

https://ngrok.com/
6th August 2017
ESP8266 low-cost Wi-Fi full stack TCP/IP MCU (Arduino Compatible)
ESP8266 is a tiny extremity cheap MCU with third-party library making it Arduino compatible. Features all the normal connections you would expect such as GPIO, SPI, I2C, I2C and UART to name a few, the biggest selling point is its size and build in WiFi support with full stack TCP / IP communications.

Great for those IoT projects, connected devices and internet / network projects.

Features
- 32-bit RISC CPU: Tensilica Xtensa L106 running at 80 MHz*
- 64 KiB of instruction RAM, 96 KiB of data RAM
- External QSPI flash: 512 KiB to 4 MiB* (up to 16 MiB is supported)
- IEEE 802.11 b/g/n Wi-Fi
  - Integrated TR switch, balun, LNA, power amplifier and matching network
  - WEP or WPA/WPA2 authentication, or open networks
- 16 GPIO pins
- SPI
- I²C
- I²S interfaces with DMA (sharing pins with GPIO)
- UART on dedicated pins, plus a transmit-only UART can be enabled on GPIO2
- 10-bit ADC (this is a Successive Approximation ADC)
More Information

https://en.wikipedia.org/wiki/ESP8266
20th July 2017
An easy music player using Pi MusicBox and Raspberry PI

Got a spare raspberry pi and like music? Turn it into an easy to use music player with access to Spotify, YouTube, internet radio and much more. Pi MusicBox runs headless allows you to simply plug in speakers and control it via a fancy web interface.

http://www.pimusicbox.com/

3rd July 2017

Blog

1. Enabled windows subsystems for Linux

2. Install Ubuntu via Windows Store

3. Initialise Setup

Usage

Command not found?