Part 3/3 - Deploying with rsync - Shared var

In the previous video we saw how we could improve the original rsync deploy process to enable multiple builds to exist on our server, and give ourselves a way of quickly swapping out the live site code for any of these builds using a symlink.

We saw there were a few processes in which we must manually intervene:

• The build and upload process from our local PC
• Changing the ownership of the newly uploaded build
• Updating the symlink

Each of these processes can, and likely will, go wrong.

Again, deployment tools exist to aid us through these processes, and we will get to them shortly.

Problem: Logged Out On New Deploy

Whilst each of the areas listed above is a potential problem waiting to happen, we could write some scripts or processes to minimise the probabilities of mistakes happening.

There is one greater problem though, depending on your site's complexity:

var

Symfony uses the var directory for some write-based operations during the execution of code on your webserver.

This presents problems, and is why we had to go through the pain of setfacl / permissions tweaks.

Fabien mentioned (look for the section 'Temporary files under var/' - I can't link directly, sorry) that the longer term plan for Symfony is to remove the use of var/cache as a writable directory in future versions of Symfony.

For right now, we need to make sure we can write to it. That was the purpose of that permissions change earlier.

Symfony will create some sub-directories under the var/ directory:

• cache
• logs
• sessions

Maybe others, too, specific to your project.

Right now we have a problem:

Our var/ directory, and as such all our logs and user sessions, exist solely for the lifetime of any current build.

There's a bunch of reasons why it's less than optimal to lose the existing var/ directory contents on each new deploy.

This could include 'losing' all your prod logs (this can be a good thing, depending on your point of view), but more problematic in my experience:

If we lose the cache directory, then when using our demo app, we will also be logged out due to CSRF token generation changes (to the very best of my knowledge).

This is when we get logged out on new deploy.

If you're kicking your user's out of their active session every time that you deploy, then expect to start receiving plenty of support requests. Not. Good.

Solution: Shared var/ Directory

The solution to our problem will be to mirror what other deployment tools do:

Creating a shared directory, move the var contents from our Symfony site root into that shared directory, and symlinking the var/ directory before switching the current symlink across.

It's actually fairly easy to accomplish. It's just yet another manual step that we may either forget, need to script (potential bugs to fix), or just generally find annoying / slows us down per deploy.

From the server:

mkdir -p shared/var
chown -www-data:www-data shared/var

mkdir -p allows you to make a directory and nested subdirectories all in one go. Without the -p flag, the command will fail as the entire path must otherwise exist.

We will re-run our var/ directory permissions setup here:

cd shared

HTTPDUSER=$(ps axo user,comm | grep -E '[a]pache|[h]ttpd|[_]www|[w]ww-data|[n]ginx' | grep -v root | head -1 | cut -d\  -f1)
setfacl -dR -m u:"$HTTPDUSER":rwX -m u:$(whoami):rwX var
setfacl -R -m u:"$HTTPDUSER":rwX -m u:$(whoami):rwX var

Now, let's take the easy road and imagine we are on a brand new, freshly installed server, doing our very first deploy using this approach.

We create a new build on our local, and rsync it up to the server:

rsync -avzh \
  --exclude '.git' \
  --exclude=node_modules \
  --exclude=var \
  /home/chris/Development/symfony-deploy-test-site \
  root@104.236.215.199:/var/www/crvfakeexample.com/builds/20171111-1228

We do still need to sort out the permissions immediately.

From the server:

chown -R www-data:www-data /var/www/crvfakeexample.com/builds/20171111-1228

And now, the extra step.

We will symlink the contents of:

/var/www/crvfakeexample.com/shared/var

to our build's var directory.

# split over multiple lines for readability
ln -s \
  /var/www/crvfakeexample.com/shared/var/ \
  /var/www/crvfakeexample.com/builds/20171111-1228/var

# one liner
ln -s /var/www/crvfakeexample.com/shared/var/ /var/www/crvfakeexample.com/builds/20171111-1228/var

Now we can make this build live:

cd /var/www/crvfakeexample.com
ln -sfn builds/20171111-1228/ current

Ok, now here comes the test.

First, is our shared/var directory getting used?

cd shared/var/
ls -la

total 36
drwxrwxr-x+ 5 www-data www-data 4096 Nov 11 12:35 .
drwxr-xr-x  3 www-data www-data 4096 Nov 11 09:49 ..
drwxrwxr-x+ 3 www-data www-data 4096 Nov 11 12:35 cache
drwxrwxr-x+ 2 www-data www-data 4096 Nov 11 12:35 logs
drwxrwxr-x+ 3 www-data www-data 4096 Nov 11 12:35 sessions

Yes, it sure looks like it is.

Next, if we log in to our web app, and then do a new deploy, are we still logged in?

Let's see. From my local machine:

rsync -avzh \
  --exclude '.git' \
  --exclude=node_modules \
  --exclude=var \
  /home/chris/Development/symfony-deploy-test-site \
  root@104.236.215.199:/var/www/crvfakeexample.com/builds/20171111-1236

And then from the server:

cd /var/www/crvfakeexample.com

chown -R www-data:www-data builds/20171111-1236

ln -s /var/www/crvfakeexample.com/shared/var/ /var/www/crvfakeexample.com/builds/20171111-1236/var

And switch the symlink:

ln -sfn builds/20171111-1236/ current

And cool, browsing the site shows we are still logged in.

Wrapping Up

This deploy process, whilst very manual, is essentially how the automated tools do it.

When doing things by hand we have a bunch of things that could go wrong. Automated deployment tools aim to solve these problems by masking them away from us, usually allowing us to hook into, or modify them by using their custom DSL.

They also take care of house keeping chores such as deleting builds older than some configurable value - maybe keeping the last 7 builds on disk, or whatever.

Again, we will get to this.

I don't advocate doing all of this by hand.

There's too much that can - and therefore will - go wrong.

Even some of the less obvious stuff like forgetting to delete old builds, filling up your servers hard disk unexpectedly. And so on.