NCZOnline - The Official Web Site of Nicholas C. Zakas

Subscribe to NCZOnline - The Official Web Site of Nicholas C. Zakas feed
The Official Web Site of Nicholas C. Zakas
Updated: 2 hours 30 min ago

Scheduling Jekyll posts with Netlify and GitHub Actions

Mon, 09/30/2019 - 20:00

Last year, I wrote about how to schedule Jekyll posts using Netlify and AWS Lambda[^1]. I used that approach from the moment I wrote that blog post up until today. What changed? In the past year, GitHub has introduced GitHub Actions[^2], a way to run container jobs triggered by different GitHub events. One of those events is a schedule defined in start cron format. So now instead of using AWS to schedule a cron job to deploy my static site, I use a GitHub Action.

For the sake of completeness, I’m duplicating some of the relevant content from my original post.

Configuring Jekyll

By default, Jekyll generates all blog posts in the _posts directory regardless of the publish date associated with each. That obviously doesn’t work well when you want to schedule posts to be published in the future, so the first step is to configure Jekyll to ignore future posts. To do so, add this key to Jekyll’s _config.yml:

future: false

Setting future to false tells Jekyll to skip any posts with a publish date in the future. You can then set the date field in the front matter of a post to a future date and know that the post will not be generated until then, like this:

--- layout: post title: "My future post" date: 2075-01-01 00:00:00 ---

This post will be published on January 1, 2075, so it will not be built by Jekyll until that point in time. I find it easier to schedule all posts for midnight so that whenever the site gets published, so long as the date matches, the post will always be generated.

Generating a Netlify build hook

One of the things I like about Netlify is that you can trigger a new site build whenever you want, either manually or programmatically. Netlify has a useful feature called a build hook[^3], which is a URL that triggers a new build. To generate a new build hook, go to the Netlify dashboard for your domain and go Site Settings and then to the Build & Deploy page. When you scroll down, you’ll see a section for Build Hooks. Click “Add build hook”, give your new hook a name (something like “Daily Cron Job” would be appropriate here), and choose the branch to build from.

You’ll be presented with a new URL that looks something like this:

https://api.netlify.com/build_hooks/{some long unique identifier}

Whenever you send a POST request to the build hook, Netlify will pull the latest files from the GitHub repository, build the site, and deploy it. This is quite useful because you don’t need to worry about authenticating against the Netlify API; you can use this URL without credentials. Just make sure to keep this URL a secret. You can see the URL in your list of build hooks on the same page.

(Don’t worry, the build hook URL in the screenshot has already been deleted.)

Storing the build hook as a GitHub secret

Along with GitHub Actions, GitHub introduced a new feature that allows you to store secrets[^4] for each repository. Each repository has its own secret store that allows anyone with write access to store key-value pairs of sensitive information. Each key is written once and is never shown in the UI again but you can read that information from within a GitHub workflow file.

To find the secret store for your repository, click on the Settings tab at the top of the repository page, then select Secrets from the left menu. Type a name for your secret (for the purposes of this post, I used netlify_build_url) and paste in the value. Click the Add Secret button to store the secret.

With the Netlify build hook stored safely in the secret store, it’s time to create the GitHub workflow file.

Setting up the GitHub Workflow

GitHub Actions are triggered by workflows[^5] defined within your GitHub repository. Workflow files are defined in YAML format and must be stored in the .github/workflows folder of your project for GitHub to automatically detect them. An action starts up a container or virtual machine and runs any number of commands on it. You can choose to use MacOS, Windows, or Ubuntu environments to run the commands. You only need a way to make HTTP requests in order to trigger the Netlify build hook, so the Ubuntu environment (with curl available) is an easy choice.

Each workflow is triggered by one or more events specified by the on key. To create a cron job, define the schedule array and include at least one object containing a cron key. For example:

name: Netlify Deploy on: schedule: - cron: "0 15 * * *"

This workflow is triggered at 3pm UTC every day of the week. All of the POSIX cron syntax is supported in workflows, making it easy to translate existing cron jobs.

Keep in mind that the cron job schedule is always relative to UTC time. Make sure to take into account your time zone when determining your schedule.

The next step is to set up a job to be run on Ubuntu. To do that, create a jobs object. Here’s an example:

name: Netlify Deploy on: schedule: - cron: "0 15 * * *" jobs: build: runs-on: ubuntu-latest steps: - name: Trigger Netlify Hook run: curl -X POST ${{ secrets.netlify_build_url }}

Each property of the jobs object is a separate job to be run (in order). The name of the job doesn’t really matter as long as it’s unique (build is a common job name). The runs-on property specifies the environment to run the command and steps is an array of commands to execute in the environment. There’s only one step in this example, which is triggering the Netlify hook. The name property should be set to something meaningful because it is displayed in the GitHub interface. The run property is the actual command to run. In this case, the command is a curl POST request to a specified URL, which is represented by a secret value. When the job executes, the Netlify deploy is triggered.

Conclusion

I switched this website over to use this GitHub Action approach as soon as I got access to the GitHub Actions beta. So far, I haven’t seen any difference in the end result (publishing my blog daily) and being able to modify the cron job within the website repository streamlines my work. I currently have this website being autogenerated every morning, and that includes pulling in new data via various APIs and publishing future-dated posts.

While I enjoyed experimenting with AWS Cloudwatch and Lambdas for scheduling future posts, I now feel that GitHub Actions is a better solution.

References
  1. Scheduling Jekyll posts with Netlify and AWS
  2. About GitHub Actions
  3. Netlify Webhooks - Incoming Hooks
  4. GitHub Actions - Creating and using secrets
  5. GitHub Actions - Configuring a Workflow
Categories: Tech-n-law-ogy

Securing persistent environment variables using ZEIT Now

Mon, 09/02/2019 - 20:00

I’m a big fan of ZEIT Now1 as an application hosting provider. The way the service abstracts all of the cloud computing details and allows teams to focus on building and deploying web applications is fantastic. That said, I had a lot of trouble setting up secure environment variables for my first application to use. I was used to other services like Netlify2 and AwS Lambda3 exposing environment variables in the web interface to allow secure transmission of important information. When ZEIT Now didn’t provide the same option in its web interface, I had to spend some time researching how to securely set persistent environment variables on my application.

For the purposes of this post, assume that you need to set two environment variables, CLIENT_ID and CLIENT_SECRET. These values won’t change between deployments (presumably because they are used to authenticate the application with OAuth). As such, you don’t want to manually set these environment variables during every deployment but would rather have them stored and used each time the application is deployed.

Setting environment variables in ZEIT Now

According to the documentation4, there are two ways to set environment variables for your ZEIT Now project. The first is to use the now command line tool with the -e option, such as:

now -e CLIENT_ID="abcdefg" -e CLIENT_SECRET="123456789abcdefg"

This approach not only sets the environment variables but also triggers a new deploy. The environment variables set here are valid only for the triggered deploy and will not automatically be available for any future deploys. You need to include the environment variables any time you deploy, which isn’t ideal when the information doesn’t need to change between deploys.

The second way to set environment variables is to include them in the now.json file. There are actually two keys that can contain environment variables in now.json:

  1. env is used for environment variables needed only during application runtime.
  2. build.env is used for environment variables needed only during the build process.

Whether you need the environment variables in one or both modes is up to how your application is built.

Be particularly careful if your build process uses the same JavaScript configuration file as your runtime, as you may find both the build and runtime will require the same environment variables even if it’s not immediately obvious (this happened to me). This is common with universal frameworks such as Next.js and Nuxt.js.

Both the env and build.env keys are objects where the property names are the environment variables to set and the property values are the environment variable values. For example, the following sets CLIENT_ID and CLIENT_SECRET in both the build and runtime environments:

{ "env": { "CLIENT_ID": "abcdefg", "CLIENT_SECRET": "123456789abcdefg" }, "build": { "env": { "CLIENT_ID": "abcdefg", "CLIENT_SECRET": "123456789abcdefg" } } }

The environment variables in now.json are set for each deploy automatically, so this is the easiest way to persist important information for your application. Of course, if your environment variables contain sensitive information then you wouldn’t want to check now.json into your source code repository. That’s not a great solution because now.json contains more than just environment variables. The solution is to use now.json with project secrets.

Using ZEIT Now secrets

ZEIT Now has the ability to store secrets associated with each project. You can set a secret using the now CLI. You can name these secrets whatever you want, but the documentation4 suggests using lower dash case, Here’s an example:

now secret add client-id abcdefg now secret add client-secret 123456890abcdefg

These commands create two secrets: client-id and client-secret. These are automatically synced to my ZEIT Now project and only available within that one project.

The next step is to reference these secrets inside of the now.json file. To specify that the value is a secret, prefix it with the @ symbol. For example, the following sets CLIENT_ID and CLIENT_SECRET in both the build and runtime environments:

{ "env": { "CLIENT_ID": "@client-id", "CLIENT_SECRET": "@client-secret" }, "build": { "env": { "CLIENT_ID": "@client-id", "CLIENT_SECRET": "@client-secret" } } }

This now.json configuration specifies that the environment variables should be filled with secret values. Each time your application is deployed, ZEIT Now will read the client-id and client-secret secrets and expose them as the environment variables CLIENT_ID and CLIENT_SECRET. It’s now safe to check now.json into your source code repository because it’s not exposing any secure information. You can just use the now command to deploy your application knowing that all of the important environment variables will be added automatically.

Summary

The way ZEIT Now handles environment variables takes a little getting used to. Whereas other services allow you to specify secret environment variables directly in their web interface, ZEIT Now requires using the now command line tool to do so.

The easiest way to securely persist environment variables in your ZEIT Now project is to store the information in secrets and then specify the environment variables in your now.json file. Doing so allows you to check now.json into your source code repository without exposing sensitive information. Given the many configuration options available in now.json, it’s helpful to have that file in source control so you can make changes when necessary.

References
  1. ZEIT Now 

  2. Netlify 

  3. AWS Lambda 

  4. ZEIT Now - Environment Variables and Secrets  ↩2

Categories: Tech-n-law-ogy