Part 2/3 - Deploying with rsync - Using builds

In the previous video we saw an example of a basic deploy to either a LAMP or LEMP stack using rsync.

Whilst that process may be good enough to complete your very first deploy, beyond this you likely need something a little more robust.

As we covered in the previous video, our approach was to copy all the files from our development environment up to our production server, overwriting whatever we had in the live directory at the time.

For a good few reasons this shouldn't be considered a viable deploy strategy on a site with more than a few tens of visitors a day.

There are plenty of things that can go wrong, and rolling back may not be easy, and almost certainly won't be quick.

Not to mention, you're probably going to break your site for your visitors whilst you're uploading the new content.

Overview of Improvements

Our revised strategy is to make use of two new concepts:

• A 'builds' directory
• symlinks / Symbolic Links

Previous we would run an rsync against our development directory, and copy the full contents up to the live directory:

rsync --exclude '.git' --exclude=var -avzh \
  /home/chris/Development/symfony-deploy-test-site \
  root@104.236.215.199:/var/www/crvfakeexample.com

As mentioned, this presents a number of problems, not least of which is taking on an elevated level of risk that we can, quite easily, mitigate.

Our new strategy will be to break our deploy down into two steps:

• rsync our new build up into the /var/www/crvfakeexample.com/builds directory
• Switch the live symlink to point to the desired build in the builds directory

To give a more concrete example, let's imagine we have three builds up on our server:

cd /var/www/crvfakeexample.com/builds

ls -la

total 20
drwxr-xr-x 5 www-data www-data 4096 Nov 11 09:52 .
drwxr-xr-x 4 www-data www-data 4096 Nov 11 09:49 ..
drwxr-xr-x 2 www-data www-data 4096 Nov 11 09:52 20171015-1005
drwxr-xr-x 2 www-data www-data 4096 Nov 11 09:52 20171019-1431
drwxr-xr-x 2 www-data www-data 4096 Nov 11 09:52 20171109-0949

I'm using a basic strategy of naming my build based on the approximate time I uploaded the files / folders.

An alternative strategy may be to use a git commit hash, or a tag name, or any other setup you prefer.

Let's imagine that our very latest build - 20171109-0949 - should be considered the Live build.

If we go up one directory, we can see how this might work:

ls -la

total 16
drwxr-xr-x 4 www-data www-data 4096 Nov 11 09:53 .
drwxr-xr-x 4 root     root     4096 Nov 11 09:43 ..
drwxr-xr-x 5 www-data www-data 4096 Nov 11 09:52 builds
lrwxrwxrwx 1 root     root       21 Nov 11 09:53 current -> builds/20171109-0949/

We would also need to update our nginx or Apache site config to use the directory:

/var/www/crvfakeexample.com/current

As our root.

Note that the current directory isn't really a directory at all.

It is a symlink. Or, a shortcut.

The idea here is that now all we need to do is change the directory that the shortcut / symlink is pointing too, and we can make any build live with the minimum of fuss.

There is a subtle problem with this setup that will become evident as we continue through this process. We will address this in the next video.

One quick last point: at this stage everything is done manually, by us. There exist many tools to automate this / similar processes. We will get onto these shortly, but for now, understanding how things work at a basic level will stand you in good stead for understanding the more complex, automated approaches to come.

Ok, that's the overview, let's see how the heck we can make use of this.

Implementing The Improvements

Regardless of whether you are using Apache or nginx, the first thing we need to do is to change up our directory structure.

I'm going to suggest you start from an empty directory here - as though you have just built a fresh LAMP or LEMP stack. You can do this process with existing files and folders, but it is more complex.

From your server:

cd /var/www/crvfakeexample.com

mkdir builds

chown www-data:www-data builds

We make the new build directory, and we change the ownership to www-data so our webserver user can access the contents appropriately.

Next, let's edit the Apache / nginx configs accordingly:

Revised Apache Vhost Config

# /etc/apache2/sites-available/crvfakeexample.com.conf

<VirtualHost *:80>
    ServerName crvfakeexample.com
    ServerAlias www.crvfakeexample.com

    DocumentRoot /var/www/crvfakeexample.com/current/web
    <Directory /var/www/crvfakeexample.com/current/web>
        AllowOverride None
        Order Allow,Deny
        Allow from All

        <IfModule mod_rewrite.c>
            Options -MultiViews
            RewriteEngine On
            RewriteCond %{REQUEST_FILENAME} !-f
            RewriteRule ^(.*)$ app.php [QSA,L]
        </IfModule>
    </Directory>

    # uncomment the following lines if you install assets as symlinks
    # or run into problems when compiling LESS/Sass/CoffeeScript assets
    # <Directory /var/www/crvfakeexample.com/current/>
    #     Options FollowSymlinks
    # </Directory>

    # optionally disable the RewriteEngine for the asset directories
    # which will allow apache to simply reply with a 404 when files are
    # not found instead of passing the request into the full symfony stack
    <Directory /var/www/crvfakeexample.com/current/web/bundles>
        <IfModule mod_rewrite.c>
            RewriteEngine Off
        </IfModule>
    </Directory>
    ErrorLog /var/log/apache2/crvfakeexample_com_error.log
    CustomLog /var/log/apache2/crvfakeexample_com_access.log combined
</VirtualHost>

Be sure to run an Apache reload after changing this config:

service apache2 reload

Revised nginx Site Config

# /etc/nginx/conf.d/crvfakeexample.com.conf

server {
    listen 80 default;
    server_name crvfakeexample.com www.crvfakeexample.com;
    root /var/www/crvfakeexample.com/current/web;

    location / {
        # try to serve file directly, fallback to app.php
        try_files $uri /app.php$is_args$args;
    }

    # DEV
    # This rule should only be placed on your development environment
    # In production, don't include this and don't deploy app_dev.php or config.php
    location ~ ^/(app_.*|config)\.php(/|$) {
        fastcgi_pass unix:/run/php/php7.0-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $realpath_root;
    }

    # PROD
    location ~ ^/app\.php(/|$) {
        fastcgi_pass unix:/run/php/php7.0-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $realpath_root;
        internal;
    }

    # return 404 for all other php files not matching the front controller
    # this prevents access to other php files you don't want to be accessible.
    location ~ \.php$ {
      return 404;
    }

    error_log /var/log/nginx/crvfakeexample.com_error.log;
    access_log /var/log/nginx/crvfakeexample.com_access.log;
}

Be sure to run an nginx reload after changing this config:

service nginx configtest
service nginx reload

Build Process

You may want to improve your build process.

I'm going to keep mine the same as before.

The tweak I will make is to rsync up to the builds directory, rather than directly in to the site root.

From my local PC:

rsync --exclude '.git' --exclude=var -avzh \
  /home/chris/Development/symfony-deploy-test-site \
  root@104.236.215.199:/var/www/crvfakeexample.com/builds/20171111-1008

As we are using the root user to rsync up the files, the end result will be that the directory:

/var/www/crvfakeexample.com/builds/20171111-1008

is owned by user and group of root.

We must, therefore, log in and recursively change the ownership back to www-data. From the server:

chown -R www-data:www-data /var/www/crvfakeexample.com/builds/20171111-1008

This is exactly the sort of command you will inadvertently forget to run. It must be done for each new build.

The human factor says you will forget this, at least once.

Unfortunately, there's no fail safe in place if you do this. If you forget to change the ownership, and update your symlink as described below, you will 500 error your site. Until, that is, you realise your mistake, and fix it. Always fun in production.

Going Live

We've configured our webserver, and we've uploaded our latest build.

Now, we need to put this build Live.

From the server:

cd /var/www/crvfakeexample.com

ln -s builds/20171111-1008 current

Awesome.

We now have our latest build Live.

A New Release

At a guess, a large part of why you are investing the time into learning about ways to improve your deploy process is because deploys happen often, and are usually quite nerve-wracking.

Let's see how we can push some new code live.

We will make a simple change to the home page CSS, and then put that into live.

I'm making my change in the app.scss file:

/* assets/scss/app.scss */

.jumbotron {
  background-color: #1c699d;
}

Make whatever change you like. You don't need to make a CSS change. Making a HTML change would be easier. However, this way we get to - briefly - see Symfony's approach to webpack, via Encore.

yarn install
# ...

yarn build

yarn run v1.3.2
$ ./node_modules/.bin/encore production
Running webpack ...

 DONE  Compiled successfully in 9448ms                                                                                                       10:52:20

 I  27 files written to web/build
    + 192 hidden modules
Done in 9.96s.

You can substitute these commands out for npm install, npm build.

Now, let's push up our changes:

From my local PC:

rsync --exclude '.git' --exclude=var --exclude=node_modules -avzh \
  /home/chris/Development/symfony-deploy-test-site \
  root@104.236.215.199:/var/www/crvfakeexample.com/builds/20171111-1056

Note the extra inclusion of: --exclude=node_modules

The node_modules directory will have been created during yarn install / npm install, but need not be included in our deploy.

Permissions next. So important. So easy to miss.

From the server:

chown -R www-data:www-data /var/www/crvfakeexample.com/builds/20171111-1056

And now, the go live command:

cd /var/www/crvfakeexample.com

ln -sfn builds/20171111-1056 current

Note the extra flags: -sfn. RTFM if at all curious.

Rollback

Life isn't always roses.

Things can and will go wrong during a deployment.

What matters is how easily you can rollback to a previous known good state.

When using this approach, rolling back to a previous release is as simple as updating the symlink to point to a previous build. Essentially it's the exact same process going forwards as it is going back:

chown -R www-data:www-data /var/www/crvfakeexample.com/builds/{some previous build}

And for me, that brings confidence to proceedings.

At this point we may be done. It depends on your list of post-deploy tasks.

Uh-oh, Not Quite

There is a gotcha here.

If we are logged in during the time of a deploy, we will be roughly booted out after the deploy.

Why?

Find out in the next video.