Output Caching for the Internationally Aware

Output caching is highly recommended for high traffic sites. Microsoft offers output caching out of the box for MVC applications through the System.Web.Mvc namespace. With the OutputCache attribute class you can specify certain MVC actions on a controller to return cached outputs. This will greatly reduce server load and improve responstime for large scale applications.

However, whole-section or whole-page output caching for a server-side dynamic site doesn’t meet everyone’s needs out of the box. What if you have certain areas of your site that are user-specific dynamic and, more importantly, have pii that should at least be prevented by accidental access from another user? Enter, donut hole caching, or donut caching. This allows the caching of just specific areas of a site (donut hole caching) or excluding specific areas from cache (donut caching) so you don’t cache areas such as the login information in the top right corner of a page to be viewed by anyone.

DevTrends created a great library for donut caching on MVC. You can access it in NuGet Install-Package MvcDonutCaching. DevTrends also has a much more indepth article on the matter here.

Because there’s already a great article on output caching in MVC already. I’ll focus on internationalization.

There are a couple unfortunate things about DevTrends MvcDonutCaching library 1) It is currently up for adoption and 2) It doesn’t seem to currently have support for the VaryByHeader functionality in Microsoft’s MVC OutputCaching.

OutputCaching: Utilizing VaryByHeader

The OutputCache class has a VaryByHeader property that you can set to allow output cache keys to vary by headers passed from a browser. This is especially useful when your site is translated based on browser culture. The standard browser will send culture information up on a request in the Accept-Language header key. This header’s value is a prioritized list of culture strings. For instance, if my main language is French, but I also understand English, my header might look like this from Firefox:

firefox request headers

You can specify that cache keys take this into consideration and append the header values to the keys so that visits in China don’t influence the output of visitors in France. Below is an example of doing this in the attribute:

1
2
3
4
5
[OutputCache(CacheProfile = "Cache1HourInternational", VaryByHeader = "Accept-Language")]
public ActionResult Index()
{
  return View();
}

By naming the CacheProfile, you also have the benefit of specifying in the config like so:

1
2
3
4
5
6
7
8
9
10
11
12
<caching>
      <outputCache defaultProvider="Memcached">
        <providers>
          <add name="Memcached" type="MemcachedOutputCacheProviderNamespace" />
        </providers>
      </outputCache>
      <outputCacheSettings>
        <outputCacheProfiles>
          <add name="Cache1HourInternational" duration="3600" varyByParam="none" varyByHeader="Accept-Language"/>
        </outputCacheProfiles>
      </outputCacheSettings>
    </caching>

With this, we can reliably output cache all pages from ActionResults that have the OutputCache attribute and the CacheProfile Cache1HourInternational.


How to Build a Resilient AWS EC2 Linux Instance Utilizing Nginx and Node.js (Part 2)

If you followed read my last post, you were walked through how to build a node-nginx EC2 instance on RHEL 6. I had initially started with Amazon Linux before writing that post, and then switch to Ubuntu 12.04 and finally to RHEL6.4 only after not having much, off-the-bat success with the first two. This post, I am going to tackle the same thing, but with Amazon Linux and work out the kinks. Why? Because Amazon Linux is cheaper per hour, optomized for running in Amazon’s cloud, has much better Amazon support, AND if you want to do any kind of automated deployments/installations when launching this instance from an AMI, you can use aws-cfn-bootstrap which isn’t terribly well supported on RHEL6.4 (for instance, if you install on RHEL6.4 your likely to run into some interesting issues trying to reconnect to your instance on port 22 later…).

1. Start an Amazon Linux EC2 Instance

I won’t go into detail about how to do this, as it’s very similar (in fact, nearly identical except for choosing Amazon Linux as opposed to RHEL 6.4) to step 1 in my previous post.

2. Update yum

When I connected to my instance, I noticed a few updates available so I updated them via yum:

1
$ yum update

3. Install nginx 1.4.1

As I mentioned in my previous post, Amazon Linux only has an older version of nginx available via yum by default. Because of this, we’re going to have to get the nginx rpm package and build it ourselves.

First, we’ll need rpm-build and rpm-devel as Amazon Linux doesn’t ship with these out of the gate.

1
# yum -y install rpm-build rpm-devel

Next, we’re going to get the nginx rpm. I’m going to do this in the /usr/local/src/ directory. Also, you’ll note that the url to the rpm that I’m using is specific to nginx 1.4.1. If you want a different version you can see the package list at http://nginx.org/packages/centos/6/SRPMS/

1
2
# cd /usr/local/src/
# wget http://nginx.org/packages/centos/6/SRPMS/nginx-1.4.1-1.el6.ngx.src.rpm

Now, we need to install the dependencies for building nginx

1
2
# yum install zlib-devel pcre-devel openssl-devel
# yum install gcc

Use the rpm tool

1
# rpm -i nginx-1.4.1-1.el6.ngx.src.rpm

Then, we’ll actually build nginx

1
2
# cd /usr/src/rpm/SPECS
# rpmbuild -bb nginx.spec

Finally, we’ll install via yum the locally built package.

1
# yum install /usr/src/rpm/RPMS/x86_64/nginx-1.4.1-1.amzn1.ngx.x86_64.rpm

You should now have an /etc/nginx/ directory and you should be able to run $ sudo nginx and see the following page at your instance url (assuming you opened port 80 in your instance’s security group)

Welcome to nginx

4. Configuring nginx

See step 5 of my previous post, it’s the same here.

5. Installing node

We’re going to install git and download node source and compile it locally. We’re going to checkout the branch for v0.10.13 because at the time of this artical, that’s the current version as noted on nodejs.org. We’ll also need g++ so we’ll install taht as well

1
2
3
4
5
6
# yum install gcc-c++
# yum install git
# git clone https://github.com/joyent/node.git
# cd node/
# git checkout v0.10.13
# ./configure --prefix=/opt/node && make && make install

Now, grab some coffee because this is going to take a while…

Once it’s finished you can add node to your $PATH with the following command

1
# export PATH=$PATH:/usr/local/bin

6. Getting node up and running with nginx

See step 7 of my previous post this step is the same.

7. Setting up supervisord

This is the same as step 8 in my previous post.

8. Baking an AMI

This was something that I meant to cover in my previous post, but didn’t so I’ll cover it here. Although, this is by far the easiest step. In order to create an AMI (Amazon Machine Instance) on AWS you simply right click the instance that you’ve been working on in the EC2 –> Instances list, then click ‘Create Image (EBS AMI)`

Create Image

Once you’ve done this the most important (and required) thing you’ll be asked for is a name. After giving a name, click ‘Yes, Create.’

NOTE: make sure you name it something rememberable like node-nginx_YYYY.MM.DD. This will help you when you start making multiple images a day.

You’re AMI will be ready shortly. With an AMI you have the ability to launch exactly where you left off on any different instance size, security group, key-pair, etc you’d like.

Brilliant.


How to build a resilient AWS EC2 Linux instance utilizing nginx and node.js

I spent lots of time figuring out the best recipe for building out an Amazon EC2 instance that would support nginx as my http reverse proxy and node.js web app builder. These steps detail what I’ve found to be a “winning path.” I will start with the disclaimer that not only am I not a Linux power user, but I am also new to nginx. Hence the amount of time I spent working on different combinations. Please follow these directions at your own risk, I do not provide them with any warranty, just for the sake of perhaps saving another newbie some time and because all the tutorials of sorts that I followed didn’t seem to sync up with my real experiences.

1. Launching an Instance

NOTE: If you’re familiar with launching instances in Amazon’s cloud, you can probably just skim through this until you get to the section on installation and configuraiton.

Choosing an AMI

You’ll start by launching the Request Instances Classic Wizard. I’ve experimented with Amazon Linux AMI 2013.03.1, Red Hat Enterprise Linux 6.4 and Ubuntu Server 12.04.2 LTS. Of the three, I’ve had the best experience with RHEL. Amazon Linux had an outdated nginx in their repo, and I had problems getting Ubuntu configured correctly. Request Instances Classic Wizard

Selecting Instance Details

Once you’ve selected RHEL6 64bit you’ll setup the instance details. RHEL6 (at the time of this post) is in the free teir with the micro instance, but for the sake of time, I’m going to go with an m1.small. Note: most instance details don’t really matter in the setup process as we will ultimately be setting up an AMI (Amazon Machine Instance) and when launching an AMI you get to choose whatever instance details you would like for that specific instance. Requst Instance Wizard:Instance Details

Next we can breeze through the next few screens because the default values suit our purposes for now. I tend to add a helpful name of sorts in the ‘Name’ value (as seen below) since I have quite a few Amazon instances. Request Instance Wizard:Tags

Selecting Key Pair

This section may require a little background knowledge on Amazon key pairs. I recommend you google that a bit before continuing if the following doesn’t make sense to you:

Public/private key pairs allow you to securely connect to your instance after it launches. For Windows Server instances, a Key Pair is required to set and deliver a secure encrypted password. For Linux server instances, a key pair allows you to SSH into your instance. To create a key pair, enter a name and click Create & Download Your Key Pair. You will be prompted to save the private key to your computer. Note: You only need to generate a key pair once – not each time you want to deploy an Amazon EC2 instance.

I choose a key that I had previously setup here. If you haven’t made one, or only have one that you know you shouldn’t use, you should take this moment to create a dev, play, sandbox, or config keypair just to get you through the setup process. Again, this is something that you can change when launching this as an custom AMI later.

Configuring Firewall

This is another area that will get you if you don’t understand the role of the amazon instance firewalls. Just like any other firewall, they block all ports and originating IPs that are not specified as allowed in the firewall policy. These firewall policies are stored in what Amazon calls Security Groups. Wether you’re selecting an already created security group or creating a new one, the key ports that should be open are as follows: Security Group Rules

Review

After reviewing your instance creation details you can launch it, wait a small bit, and then move on to the next step!

2. Connecting to Instance via SSH

Once the machine is launched and in a ready state, we can SSH into it.

  1. Right click on the running instance
  2. Click ‘Connect’
  3. Expand the ‘Connect with a standalone SSH Client
  4. Follow those steps… (they’re pretty good steps, Amazon even tells you practically exactly what your SSH statement should be)

I’m using iTerm on a mac, and if you aren’t you should be too, only because it’s excellent.

3. Turning off RHEL firewall

Ok, so I mentioned earlier that since this is sitting behind an amazon firewall, there really isn’t a need for the RHEL firewall. Heck, Amazon Linux and Ubuntu didn’t even have one enabled by default! So, for our purposes, we’ll disable this. (Note the superuser priveledges, if you’re new to linux, sudo su is the command your looking for. You know you’re super user by the #)

1
2
3
# service iptables save
# service iptables stop
# chkconfig iptables off

4. Installing nginx

First note, nginx is not in the default package repositories, so we’ll have to add it.

Second note, you can download and build nginx yourself if you want the latest features and bugfixes, to do this you can checkout their wiki. However, for our purposes, nginx version 1.4.1 is fine for us (it’s got the included support for web sockets wich is important if you wan’t to use something like socket.io for node.js)

Creating the package repository file for yum

NOTE: I’m going to use nano throughout this tutorial for 2 reasons: 1) It’s the easiest for someone not use to terminal text editing. 2) I think it’s a fine tool for non-complex text editing tasks.

Create a file named /etc/yum.repos.d/nginx.repo and use the following configuration $ sudo nano /etc/yum.repos.d/nginx.repo

Note, this must be sudo because of the permissions required on that directory

1
2
3
4
5
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/rhel/6/$basearch/
gpgcheck=0
enabled=1

Also, if you’re new to nano, after you’ve pasted this text into the editor, you’ll do ctrl+o to write out to the file, then hit enter (because we already specified the file name when we started nano), and then ctrl+x to exit nano. Remember this, you’ll do it again.

To verify that the repo file is correct, you can search for the package yum search nginx and you should get some results

Installing nginx via yum

sudo yum install nginx After executing that command, yum will figure out everything that needs to happen, and prompt you with ‘Is this ok [y/N]:’. Type ‘y’ and hit enter and off you go.

You should see something like this, verifying nginx version 1.4.1 was installed: nginx package download and installation

Verify nginx is working

With the rhel firewall turned off, and the security group firewall configured as mentioned earlier. You should be able to run nginx and see that it works!

First, start nginx: $ sudo nginx you shouldn’t see anything if everything was successful.

If you’ve stuck with me this far, you should be able to navigate to your instance using the public domain name found in the EC2 Instance details on the EC2 area of the AWS Console. You’ll see something like this: public domain name Copy everything from ec2 all the way to .com and paste it in a browser, you should see the following page load! nginx default landing page Success!

5. Configuring nginx

Now that we have nginx up and running, we can familiarize ourselves with configuration. Nginx has a concept of server blocks for configuration that represent ‘sites’ (for the sake of this tutorial). You can have many different server blocks that are nested inside an http block.

Nginx’s configuration structure is setup like so (by default):

  • Master configuration file: /etc/nginx/nginx.conf
  • Server block configs: /etc/nginx/conf.d/*.conf

We’ll want to create a new server block configuration (and delete the default.conf file since we don’t need that anymore). Since I’m going to be setting up node, I’ll call it /etc/nginx/conf.d/node-myapp.conf with the following contents:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#the IP on which your node server is running, 127.0.0.1 for local of course. I chose port 8080 as the listening port for node
upstream node-myapp {
        server 127.0.0.1:8080;
}

#the nginx server instance
server {
        listen 80;
        server_name _;
        access_log /var/log/nginx/node-myapp.log;

        # pass the request to the node.js server with the correct headers and much more can be added, see nginx config$
        location / {
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_set_header X-NginX-Proxy true;

                proxy_pass http://node-myapp/;
                proxy_redirect off;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
        }
}

There are a few things to note in the configuration of this server block.

  1. I set the server_name to _. This means that all nginx server traffic will go here as ‘best match’.
  2. The last 3 lines inside the location section are important for websockets. Which I am utilizing and are only supported in later versions of nginx like the one we’re using in this tutorial.

We can leave the nginx.conf alone for now. To make these

6. Installing Node

Some prefer to download node via git others prefer to use the package manager. In this case, we’re going to install node from a package manager. This is to avoid issues with compiling (this version of RHEL either doesn’t have a C compiler installed or it’s not in a standard location). In order to get node from a package manager we have to enable EPEL. To do this, follow these commands:

$ su -c 'rpm -ivh http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm'

Then insure it’s configured correctly with this:

$ yum repolist

Now you should see epel in that list.

Finally, we can install nodejs via yum with this command:

$ su -c 'yum install nodejs'

7. Getting Node up and running

You’ll need somewhere to put your node assets, and I recommend the following: /var/www/node-myapp.

This will keep a nice consistent experience for your node apps and nginx server blocks.

For the sake of having a node app that does something (if you don’t have one to play with). You can use the following ‘hello world’ code:

1
2
3
4
5
6
7
var http = require('http');

http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Hello World\n');
}).listen(8080, "127.0.0.1");
console.log('Server running at http://127.0.0.1:8080/');

Place this in /var/www/node-myapp/app.js

Now we can test out the whole experience. First, make sure nginx is running and listening to port 80 (you can do something like this $ sudo lsof -i :80). Then start node like so:

1
2
# cd /var/www/node-myapp
# node app.js

Then when you send requests to the public domain for your instance you should see your hello world message! Vioala!

8. Supervisord: taking node to the next level

So now that we know how to setup nginx, and we’ve figured out how to get nginx to proxy to our node app, let’s think about resilliancy. There might be two things that are going on in your mind through this tutorial: 1) node is not a daemonized process, and that’s pretty limitting to me 2) what happens if node crashes??

This is where supervisord comes in handy. It will handle daemonizing your node process as well as reliably monitoring it and restarting it if necessary. Here is how to get supervisord set up in our environment. (for more advanced documentation, checkout supervisord)

  1. Install supervisord: # easy_install supervisor
  2. Setup default supervisor.conf file # echo_supervisord_conf > /etc/supervisord.conf
  3. Add node as program to supervisord.conf for the sake of the tutorial I’ll paste what I put, but the echo_supervisord_conf is pretty extensive and just about as good as supervisord’s documentation
1
2
3
4
5
6
7
8
9
[program:node-myapp]
command=/var/www/node-myapp/node_modules/coffee-script/bin/coffee app.coffee ;this is because my node app is in coffeescript
directory=/var/www/node-myapp
autostart=true
autorestart=unexpected
startsecs=2
startretries=3
exitcodes=0,2
stdout_logfile=/var/www/logs/node-myapp.log
  1. start supervisord: # supervisord -c /etc/supervisord.conf

SUCCESS!

Now, when you reboot your machine, supervisord will be configured to start automatically need to be started, and because we set autostart to true for our node-myapp program, it will also start as a daemon automatically as well. It’s also worth noting that nginx will start at reboot also, but you may want to consider adding nginx as a process that’s monitored by supervisord as well. There are many more complicated things you can do with supervisord now that you have this foundation.

Good luck!

Basic Troubleshooting

Supervisord

Supervisord is controlled by supervisorctl. You can enter into this control tool by just starting # supervisorctl. This will give you a supervisor prompt. Typing help will give you a list of commands to guide you through troubleshooting, and status will give you some insite into wether or not your app is even running correctly (helping identify errors in the supervisord.conf program section).

Nginx

When you make changes changes to the server blocks while nginx is running, you may have to reload it. To do this, use the command # nginx -s reload. You can pass other signals after the -s flag as well to aid in troubleshooting.

No 502, no anything…

Check to make sure your security group is set up correctly with the correct ports opened. If not, you may not even see anything when you’re attempting to access your ec2 server by public domain via a browser. NOTE: rules can be added to a security group that take place nearly instantly while an instance is up and running, but security groups cannot be added or removed after an instance is launched.

Also, check to make sure the RHEL6 iptables are turned off so the local firewall isn’t causing you some heartache because you’ve assumed it’s not on in the first place.