AWS LEMP Stacks and EFS Issues

Lesson learned — if you are using EFS on production systems you want to be using provisioned throughput mode.

But, before we get into that, let’s go over the details of this implementation…

Service Configuration

We utilize AWS EC2 instances to run multiple WordPress sites hosted in different directories. The configuration is fairly standard: 2+ servers configured as part of an load-balanced cluster. The servers run from the same image meaning they use the same underlying software stack.

Part of that image includes a mounted EFS (Elastic File Storage) directory , used to share WordPress resources between all nodes in the cluster. The original architecture was designed to host not only the typically-shared wp-content/uploads folder of WordPress via this EFS mount but also the code. The thought was that sharing the code in this way would allow a system admin to easily update WordPress core, plugins, or themes from the typical wp-admin web login. Any code updates would immediately be reflected across all nodes.

EFS Web App Code Hosting – A Bad Idea

Turns out this is a bad idea for a few reasons. First of all, EFS volumes are mounted using the NFS4 (network file storage) protocol — this defines how the operating system handles file read/write operations for a network mounted drive. While NFS4 is fairly robust, the throughput of ANY network drive, even on a high speed AWS data center backbone, is much slower than a local drive such as an EBS volume.

That means that even on a good day every PHP file, JavaScript file, or anything else needed to serve up that WordPress web page are going to be a bit slower than normal.

However, the bigger problem comes to light if you happen to choose the default, and pushed as “the mode to use” by Amazon, EFS throughput mode known as “Burst mode”.

Read More

Converting A Standalone Instance Into An EC2 Load Balanced Cluster

Creating a new web application that resides on an AWS load balanced cluster is easy with the Elastic Beanstalk assistant. That is a great solution if you want to run every web service or application on their own instances. It is not a great fit for complex environments like the one being used for Store Locator Plus®.

Store Locator Plus® has several environments running within the same master domain. Multiple servers and load balancers creates a security certificate nightmare. Not too mention it starts racking up EC2 server fees quickly if they each became their own cluster. The better option is to retain a server instance that allows us to run our SaaS offering, our buy-and-own plugin store, our documentation site, and our demo site from a single disk image. We want to setup a full EC2 Load Balanced Cluster to gain the benefits of horizontal scaling on a server hosting multiple domains and web apps.

While this is easy to do with a single EC2 instance that hosts multiple host names for the domain, making it scalable under load is the trick. It turns out Elastic Beanstalk is not a good fit. Instead we need to build a load balanced cluster “from scratch”. We’ll need to combine a machine image from a running server with a Launch Template. We will need an Application Load Balancer that will have instances attached and detached automatically from an Auto Scaling Group that we will also create.

Our environment also has a configured EC2 instance to run the web application stacks, mostly WordPress, locally on an EBS volume that uses an Amazon Aurora MySQL RDS database in multiple zones for performance and reliability. These two features make it easy to replicate the disk image for the software portion and maintain a persistent DB store across all instances.

Read More

Alibaba An AWS Threat? Not Quite

A recent Seeking Alpha article proposes the theory that Alibaba could threaten Amazon’s AWS cloud services.

Amazon has been my best investment by far.

The article has some great information about the cloud services industry. It includes a lot of facts and figures that ring true. I disagree with this article’s overall assessment, however.

Alibaba will erode AWS market share in Asia but few other regions. They are a Chinese company. I would never use them myself nor recommend the Alibaba cloud solution to my clients because of this.

China has a history of banning content and restricting the free flow of information — especially on the Internet. Shutting down a business on the Alibaba cloud and taking over the assets of anything they deem “inappropriate” or “propaganda” is well within the realm of possibility. Why would any business add that to their risk factors when you have AWS as an alternative.

Not too mention Amazon continues to innovate and introduce services at a dizzying pace. The dollars spent on investment only matters if it yields results. Based on the service offerings alone, Amazon is 10 times further ahead in this space than the closest competitor.

I’m maintaining a hold position on AMZN despite the growing popularity of Azure, Google Cloud, and Alibaba.

Install A SSL Cert On Amazon Linux

Get Your Certificate Signing Request (CSR)

From Amazon Linux:

cd /etc/ssl
openssl req -new -key vim <domain>.<tld>.key -out <domain>.<tld>.csr

Buy Your Certificate

From purchase a cert for either a wildcard or single-host fully-qualified domain name.  It must match the domain identifier . used when creating your CSR.

You’ll need the contents of the .csr file and private key you created above.

Read More

Save Money With RDS / Get Your gSuite Mail Forwarding

Two quick hacks I learned today while doing some general “tech life” maintenance.

Amazon Web Services – Saving Money

The first “hack” is an easy one that I am now kicking myself in my own ass for not picking upon 6 months ago.    This is a $600 oversight that is a LOT of beers-worth of savings.   The trick is simple…  RESERVE INSTANCES.    Especially with RDS.

It turns out that for RDS (and possibly EC2 and other instance types) you can “purchase” a No Upfront Costs reserved instance.   The cost?   FREE.     The savings can be substantial.  For an M4.Large size instance it can be as much as $80/month!

I have been running a M4.Large sized RDS instances for months.    The hourly rate is $0.350.    That runs $3066 per year or about $255/month.

By purchasing a reserved M4.Large instance for 1 year with no upfront fee the rate drops to $0.241/hour or $2,111 annually.   That is $175/month.    A quick $80 for clicking a few buttons and pressing submit.

In my case I opted for the partial up front payment.   In this case you pay $648 today as a one-time charge for that M4.Large reservation for 1 year.     The hourly rate drops to $0.132/hour , $1,156/year or about $150/month.    That is $105 less than what I was paying.  Sweet!

gSuite Email Forwarding

I have been using gSuite / Google for Work / Google for Business or whatever other of a half-dozen names they’ve used in the past 10 years for… well … about 10 years now.     One of the key features I use regularly is the email routing which allows a single domain-wide email address to be forwarded to both in-company gSuite and non-Google users (such as Yahoo! email addresses).     It makes it easy to give customers (or my son’s tennis team) a single short email address that goes to multiple people.

It turns out that the forwarding rules can be written as regular expressions.  Add a gSuite email route for and set recipients to a half-dozen different email addresses and anyone that sends email to skunkworks@ will get broadcast to the team.

It turns out that the regular expressions allow you to do some pretty cool things like send email coming in to skunkworks@(yourdomain|mydomain).com to hit that same group of people whether  the person sends to or on the email.

Pretty cool.

But … this system happens to break a common tenant of nearly every major email system in existence.  It is CASE SENSITIVE.


Yup.   Email to Skunkworks@ will NOT go anywhere and the user will get an “undeliverable” message for everyone not on the Google family of services.

Well it turns out a simple hack can fix that.    add ^(?i) at the start of the expression to make it case insensitive.


%d bloggers like this: