Creating a Pelican powered iPython notebook blog

by Morten V. Pedersen
in misc

During the last couple of months I've started to use the iPython notebook for more and more stuff. Inspired by Python Perambulations by Jake VanderPlas, I decided to also try to use it for blogging. In the following I'll try to document how I've setup things.

Tools involved

  • Pelican: Pelican is a static site generator, written in Python.
  • iPython: What we will use to write our notebooks in.
  • Docker: Container for the tools
  • Fabric: Automates tasks like building and publishing the site.

Installing Pelican

On the GitHub README.md for the Pythonic Perambulations blog Jake VanderPlas recommonds using Pelican 3.3+. Fortunatly Ubuntu 14.10 ships with 3.4.0 (run: sudo aptitude show python-pelican to check). So installation is easy:

sudo aptitude install python-pelican

Hosting the site

Here I will again follow the approach taken by Jake VanderPlas for his Python Perambulations blog.

The basic idea is to create two GitHub reporitories. The first contains the Pelican project files, notebooks, and other material used when generating the site. The second contains the generated html files etc. With this approach we can use GitHub Pages for the second repository to make the site available on the web.

The first repoitory I named mortenvp-pelican and the second mortenvp.github.io which according to the GitHub Pages documentation is the mandtory name for a personal pages repository.

Generating the initial project

In this case I just follow the Pelican quickstart section to setup a basic site.

git clone <git-url-to-mortenvp-pelican.git>
cd mortenvp-pelican
pelican-quickstart

The quick start will ask you a bunch of questions, these were my answers:

Welcome to pelican-quickstart v3.4.0.

This script will help you create a new Pelican-based website.

Please answer the following questions so this script can generate the files
needed by Pelican.

    
> Where do you want to create your new web site? [.] 
> What will be the title of this web site? blog.mortenvp.com
> Who will be the author of this web site? Morten V. Pedersen
> What will be the default language of this web site? [en] 
> Do you want to specify a URL prefix? e.g., http://example.com   (Y/n) 
> What is your URL prefix? (see above example; no trailing slash) http://blog.mortenvp.com
> Do you want to enable article pagination? (Y/n) 
> How many articles per page do you want? [10] 
> Do you want to generate a Fabfile/Makefile to automate generation and publishing? (Y/n) 
> Do you want an auto-reload & simpleHTTP script to assist with theme and site development? (Y/n) 
> Do you want to upload your website using FTP? (y/N) 
> Do you want to upload your website using SSH? (y/N) 
> Do you want to upload your website using Dropbox? (y/N) 
> Do you want to upload your website using S3? (y/N) 
> Do you want to upload your website using Rackspace Cloud Files? (y/N) 
> Do you want to upload your website using GitHub Pages? (y/N) Y
> Is this your personal page (username.github.io)? (y/N) y
Done. Your new project is available at /home/mvp/dev/mortenvp-pelican

Plugin enabeling iPython notebooks

Support for including an iPython notebook is available in the Liquid-style Tags plugin. Following the README in the Pelican Plugins we add the pelican-plugins repository as a submodule:

git submodule add git@github.com:getpelican/pelican-plugins.git

and activate the liquid-style Tags:

PLUGIN_PATHS = ['pelican-plugins']
PLUGINS = ['liquid_tags.img', 'liquid_tags.video',
           'liquid_tags.youtube', 'liquid_tags.vimeo',
           'liquid_tags.include_code', 'liquid_tags.notebook']

You can choose the location for the notebooks by specifying:

NOTEBOOK_DIR = 'notebooks'

Including the notebook CSS in our choosen theme

At the top of the pelicanconf.py add:

from __future__ import unicode_literals
import os

Which enables us to use (copied from here):

# The theme file should be updated so that the base header contains the line:
#
#  {% if EXTRA_HEADER %}
#    {{ EXTRA_HEADER }}
#  {% endif %}
# 
# This header file is automatically generated by the notebook plugin
if not os.path.exists('_nb_header.html'):
    import warnings
    warnings.warn("_nb_header.html not found.  "
                  "Rerun make html to finalize build.")
else:
    EXTRA_HEADER = open('_nb_header.html').read().decode('utf-8')

Choosing a theme

I'm not a big fan of the default Pelican theme, so instead I chose to use the Pelican-sober theme.

Since the EXTRA_HEADER section (as described above) isn't present in the Pelican-sober theme's templates/base.html, we need to add it. To do this I forked the repository to my user and made the changes there.

Once this is done we can use our chosen theme by editing the THEME variable in pelican.conf (read about this here), so to summarize:

  1. Select a theme and ensure that it has the EXTRA_HEADER section.
  2. Add the theme git repository as a submodule to our site repository git submodule add git@github.com:url-to-repo.git
  3. Add a THEME = 'pelican-sober' vaiable inside pelicanconf.py

Adding a post

Adding a post is pretty easy, create a simple markdown file with some basic tags. As an example constent/first_post.md:

Title: My fist post
Date: 2015-02-26 15:00

# About
Here we can write the blog post using markdown

Including a notebook

To include a notebook called filename.ipynb copy it to the content/notebooks/ directory and add it to a post using:

{% notebook filename.ipynb %}

Generating the site

I generate the site using Docker and Fabric.

Docker

Initially when I wanted to generate the site I got and error Incomplete format

Searching the web the error seems to be related to a Mardown issue with the pelican pugin. The issue was resolved, however comments later indicated that it did not work for everybody, including me.

One user confirmed that the error occured with Markdown 2.5 and that downgrading to 2.4 solved the issue. To verify this fix I decided to install pelican and the dependencies in a Docker container to not interfer with the system packages already on my system.

The Docker container I created for this purpose is available here. The container is invoked using fabric described next.

Fabric

Is a tool that can be use to automate performing some tasks either on the local or a remote machine. The pelican-quickstart already creates a fabfile for generating and deploying the site. I just adapted it to use If you check the fabfile in the repository for this site you will see how I use the docker container to generate the content and then move it to the gihub.io repository.