Using Fabric for WP Engine WordPress Deployments

One of my first tasks at INN was to familiarize myself with the tools and technology the team currently uses.

What I noticed was that our deployment process was pretty darned manual. We host a number of our WordPress sites on WP Engine and while they allow deployment via git and access via SFTP, they don’t give us shell access to the server. So our deployment process up until this point has involved an FTP client, dragging and dropping files for transfer, maybe picking and choosing parts of our code to deploy based on what we've been hacking on. This approach works, but is fragile and requires a lot more thinking than I'm typically willing to do. Remember, laziness is a virtue.

Requirements

Having a scripted deployment process means that the steps to deploy your code will be the same no matter who has the responsibility. Keeping a repo with your deployment scripts means the process is documented and can be critiqued and iterated upon.

We’re using Fabric because I am familiar with it. By all means, use Capistrano or some other deployment tool. Write a few shell scripts if that’s what you’re comfortable with. Do what you can to put the software to work for you.

A sound deployment strategy meets these criteria:

Extensibility

Want to deploy via git or sftp? Deploy a specific branch? Want to minify and compress assets before deployment? Maybe tag a commit for rollback purposes? Your deployment strategy should be extensible enough to accommodate these things and require little to no input from you. Write a script once and put it to work.

$ fab staging branch:exciting_new_feature deploy

Push button

Type the deployment command into your shell and hit enter.

$ fab staging master deploy

Rollback

If something goes wrong, issue another command to revert to a previous state and deploy.

$ fab staging rollback deploy

Fabric, git-ftp and WPEngine

Fabric is a wonderful Python library that allows you to script away application deployments and remote system administration tasks. It provides a way to wrap up what would otherwise be complex git commands/workflows in simple, easy to remember, fab commands.

Some of the things our fabfile is set up to handle include:

  • Use git-ftp to deploy files to WP Engine via sftp.
  • Check the commit hash of currently-deployed code and automatically tag it is as a rollback point.
  • Allow us to checkout and deploy a specific branch to a specific environment (i.e. master or stable to staging or production).

Why use git-ftp? In our experience, git deployment for WP Engine has been difficult to use and unreliable. Also, WP Engine requires an entire WordPress install as the basis for your repository, which isn’t how most of our repos are structured. Git-ftp allows us to work around these issues.

However, if we want to move to a git push-based deployment strategy or expand our toolkit to work with environments outside of WP Engine, adjusting or adding to our fabfile is trivial.

Usage

If you want to use our deployment tools with your project, you can follow the instructions in the README included in the repo. The README holds all the details regarding software prerequisites and how to install them.

If you’re already familiar with Python, Fabric and have your dev environment set up, you can find the “quick start” steps below.

These tools were meant to be added as a submodule in your project repo. This is based on the way Ryan Mark structured the Chicago Tribune’s deploy tools. To start:

$ git submodule add https://github.com/INN/deploy-tools.git tools

Then copy the files from tools/examples to the root of your project:

$ cp -Rf tools/examples/* ./

Finally, edit `fabfile.py` to use your WPEngine host, username and password:

# Environments
def production():
"""
Work on production environment
"""
env.settings = 'production'
env.hosts = [‘mywpinstall.wpengine.com’, ]
env.user = ‘mywpinstallusername’
env.password = ‘mywpinstallpassword’

After that, add and commit your changes. At this point, you should be able to deploy:

$ fab staging master deploy

If you want to perform a dry run beforehand to see exactly what files will be transferred:

$ fab staging master dry_run deploy

If you want to see a list of all available commands:

$ fab -l

Available commands:

branch                Work on any specified branch.
deploy                Deploy local copy of repository to target environment.
dry_run               Don't transfer files, just output what would happen during a real deployment.
master                Work on development branch.
path                  Specify the project's path on remote server.
production            Work on production environment
rollback              Deploy the most recent rollback point.
stable                Work on stable branch.
staging               Work on staging environment
verify_prerequisites  Checks to make sure you have curl (with ssh) and git-ftp installed, installs them if you do not.

Todos

Future plans include adding commands to handle asset minification and compression.

If you'd like to contribute, you can fork the code here.

The INN Guide to Creating A Data Driven Map With TileMill

TileMill is a free-to-download application for Windows and the Mac that will let you build beautiful, data driven maps.  If the bland sameness of Google maps is giving you ennui, TileMill might be a good alternative for you.   There are many beautiful stylesheets to make TileMill maps visually distinct, and you can control the colors, line widths, and much more about your map using Carto, a CSS-derived language.  Here's a beautiful map of Montreal using a watercolor-like stylesheet:

montreal_watercolor_map

What follows is a detailed, highly-illustrated guide to creating your first map in TileMill.  We will be using real data from the Rhode Island Data Hub, a catalog of public data about education, employment, health, and much more from Rhode Island.

Continue reading "The INN Guide to Creating A Data Driven Map With TileMill"

Finding Data During The Government Shutdown

Reporters across the country are running into a frustrating effect of the federal government shutdown: Many .gov websites have been shuttered, blocking access to some go-to data sources such as the Census Bureau.

But there is some hope of finding the data you need. Investigative Reporters and Editors set up a tracker for reporters to contribute or find information about alternative places to find popular datasets. And IRE's NICAR Database Library remains open throughout the shutdown. (Any INN members interested in data from the library can contact me at denise@ire.org.)

Check out these tips on tracking down data offered by IRE staffers Liz Lucas and Tony Schick:

In partnership with the Reynolds Journalism Institute, IRE offers free Census data to browse or download at census.ire.org. You can also visit the Census Reporter website for more Census-related information.

NICAR maintains an array of federal datasets to keep you warm during the shutdown. Check out these favorites:

  • OSHA Workplace Safety data -- how much does OSHA get done even when they are at full staff? Look at inspection and accident records by industry, location, or business.
  • National Bridge Inventory -- this data has a wealth of information on bridges in the United States, including their status (are they structurally deficient?) and a sufficiency rating.
  • IRS Exempt Organizations -- peruse a listing of all organizations granted tax exemption by the IRS.
  • Medical Device Reports -- sift through records that includes information about problems that occurred with medical devices, including injuries, deaths and/or product malfunctions.
  • Look through the rest of the 40+ databases offered at the NICAR Database Library.

The Donald W. Reynolds Center for Business Journalism also compiled a list of alternative data sources (including NICAR data), available here. Information portals still available include EDGAR, PACER and FRED.