Programming land

Why versioning Kubernetes replication controllers?

Mon, 01 Feb 2016 02:00:00 -0600

I was surprised to see in all official Kubernetes examples that replication controllers had a version in their name (foo-vX) and had two separate labels with their name and version (app: foo and version: vX). I found no official explanation of why such naming, but discovered it after having to update an application.

One of the strenght of Kubernetes is to allow smoothly deploying an update with no downtime, deploying pods (transient units) of the new application version one at a time and deleting pods from the previous version at the same time. Semantically that operation is performed by replacing a replication controller by a new one, there is no notion of version in Kubernetes itself. For it to be able to perform the update, the replication controller must thus:

have a different name (Kubernetes must know which to kill and which to deploy)
have at least one label which differs (Kubernetes must be able to identify which replication controller a pod belongs to)

In order for services (persistent identities) to transition automatically to new instances, they must use a selector which includes of both replication controllers. Services on the other hand are not subject to rolling updates, they are permanent identities, so to my knowledge do not need versioning.

The recommended pattern is the simplest to achieve both goals:

version in the replication controller name makes it different each time it needs to change
replication controller name label provides a stable way to include all versions of the replication controller in the service
replication controller version label makes a difference in labels each time it needs to change

The service is thus declared as:

apiVersion: v1
kind: Service
metadata:
  name: foo
  labels:
    app: foo
spec:
  ports:
  - port: 80
  selector:
    app: foo

Before the update, replication controller looks like:

apiVersion: v1
kind: ReplicationController
metadata:
  name: foo-v8
  labels:
    app: foo
    version: v8
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: foo
        version: v8
    spec:
      # pod spec

After the update it is transformed to:

apiVersion: v1
kind: ReplicationController
metadata:
  name: foo-v9
  labels:
    app: foo
    version: v9
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: foo
        version: v9
    spec:
      # updated pod spec

The transition is started using kubectl rolling-update foo-v8 -f foo-controller.yaml.

Let's Encrypt and Kubernetes

Mon, 01 Feb 2016 01:00:00 -0600

My personal services like feeds reader and read it later service have been running for a few months on Google Cloud Platform using Kuebernetes. I started using self signed certificates which work fine for using them on desktops, but I could not use them on my phone or tablet without configuring Android keystore. Since the public availability of Let’s Encrypt I decided to use widely trusted certificates instead, also hoping it would simplify setting them up. My architecture is as follow: a single service of replicated webservers listens to the public network and redirect requests to internal services. HTTPS is used on the public network but internally I fallback to HTTP, individual services will not need to manage their own certificates.

------------          -----------------         -------------
|          |          |               |         | service 1 |
|          |          |   webserver   |         |-----------|
| browsers | -HTTPS-> |               | -HTTP-> | service 2 |
|          |          | (n instances) |         |-----------|
|          |          |               |         | service 3 |
------------          -----------------         -------------

The way this is declared is to have a single type: LoadBalancer service for webservers, backed by multiple pods of a replication controller. These pods are a raw Nginx containers configured with a secret containing configuration to make them point to internal services which represent the actual application servers. When needing to create a new host to be serviced or modify one, the config is modified, a new secret generated and a new replication controller to replace the previous one with the only difference being the secret used.

Let’s Encrypt requires to show a file at some place on the public website to verify ownership. The default working for Let’s Encrypt is to have access to the webserver to dynamically create this resource, but this is quite impractical in a Kubernetes environment. For me the certificates needed to be stored in another secret which would be linked to webservers. My process is thus to start Let’s Encrypt manual process, manually create identity validation resource using an Nginx rule, validate the identity, create the secret containing the certificate, create Nginx HTTPS listener and eventually deploy the working server.

I start by executing letsencrypt-auto certonly --manual and entering the target host name. Upon request toverify identity, I create a simple Nginx configuration only responding to the requested resource:

server {
  listen 80;
  server_name example.com;
  location = /.well-known/acme-challenge/aaaaaaaaaaaaaaaaaaaaaaaaaaa { 
    return 200 "bbbbbbbbbbbbbbbbbbbbbbbb"; 
  }
}

I use a script to generate a secret containing this configuration (using openssl for base64 encoding). Each time I change configuration I need to increment the version in the script, run it and create the secret using kubectl create -f web-configuration.yaml.

#! /bin/bash

cat > web-configuration.yaml << EOF
apiVersion: v1
kind: Secret
metadata:
  name: webconfiguration-v1
type: Opaque
data:
  example.conf: $(openssl enc -A -base64 -in conf/example.conf)
EOF

I can then start the controller replicating Nginx instances configured using the secret with kubectl create -f web-controller.yaml:

apiVersion: v1
kind: ReplicationController
metadata:
  name: web-v1
  labels:
    app: web
    version: v1
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: web
        version: v1
    spec:
      containers:
      - name: nginx
        image: nginx:1.9.10
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort: 80
        - containerPort: 443
        volumeMounts:
        - name: config
          mountPath: /etc/nginx/conf.d
          readOnly: true
      volumes:
      - name: config
        secret:
          secretName: webconfiguration-v1

And the public service with kubectl create -f web-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: web
  labels:
    app: web
spec:
  type: LoadBalancer
  ports:
  - port: 80
    name: http
  - port: 443
    name: https
  selector:
    app: web

This creates a load balancer that I need my domain to point at. Once this is booted and replying at the URL Let’s Encrypt requires, I validate the identity. This gives me a privkey.pem and fullchain.pem files that I can now use to create my certificates secret, again using a script creating the resource that I can then deploy to Kubernetes with kubectl create -f web-certificates.yaml:

#! /bin/bash

cat > web-certificates.yaml << EOF
apiVersion: v1
kind: Secret
metadata:
  name: webcertificates-v1
type: Opaque
data:
  example.com.key: $(sudo openssl enc -A -base64 -in /etc/letsencrypt/live/example.com/privkey.pem)
  example.com.cert: $(sudo openssl enc -A -base64 -in /etc/letsencrypt/live/example.com/fullchain.pem)
EOF

I can now add HTTPS server to my configuration. I added configuration recommended by securityheaders.io to enable strict transport security, preventing sites from being displayed in iframes and setting up restrictive content security policy. I also enabled HTTP2. All content is actually redirected to the application server service on HTTP using proxy_pass.

server {
  listen 80;
  server_name example.com;
  location = /.well-known/acme-challenge/aaaaaaaaaaaaaaaaaaaaaaaaaaa { 
    return 200 "bbbbbbbbbbbbbbbbbbbbbbbb"; 
  }
}

server {
  listen 443 ssl http2;
  server_name example.com;
  ssl_certificate /etc/certificates/example.com.cert;
  ssl_certificate_key /etc/certificates/example.com.key;

  add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";
  add_header X-Frame-Options "SAMEORIGIN" always;
  add_header X-Xss-Protection "1; mode=block" always;
  add_header X-Content-Type-Options "nosniff" always;

  location / {
    proxy_pass http://backendservice:80;
  }
}

I can generate again configuration (incrementing the -vX in the name) and create it in Kubernetes, then update the controller with the additional secret (also changing its version):

apiVersion: v1
kind: ReplicationController
metadata:
  name: web-v2
  labels:
    app: web
    version: v2
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: web
        version: v2
    spec:
      containers:
      - name: nginx
        image: nginx:1.9.10
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort: 80
        - containerPort: 443
        volumeMounts:
        - name: config
          mountPath: /etc/nginx/conf.d
          readOnly: true
        - name: certificates
          mountPath: /etc/certificates
          readOnly: true
      volumes:
      - name: certificates
        secret:
          secretName: webcertificates-v1
      - name: config
        secret:
          secretName: webconfiguration-v1

In order to add new site I repeat the process by adding a new configuration to validate identity, deploying it, validating identity, adding the additional certificates to the secret, adding the additional HTTP configuration to the backend, deploying the updated controller.

Next step is to automate renewal. I am not planning into putting certificate renewal inside the web containers because each new instance would use a fresh certificates, which would prevent setting up public key pinning in the future and could mess up with identity validation with multiple container instance running at the same time.

git add --patch

Sat, 23 May 2015 04:00:00 -0500

I’ve been using git and more generally version control for a while but always struggled for a use case: you start working on something very specific but you end up doing several changes more or less related to the original goal. Three very common usages are

the boy scout rule: when spotting something I don’t like like variable name or outdated documentation I usually fix it right away, but I don’t want it to appear under the same label as the original change, I want a dedicated commit
debugging: I usually add some tracing calls to see what happens locally, for example when working on a Flux app my dispatcher is always printing actions on my developer console, but this should never go to the repository
non atomic changes: functionally modifying something usually technically impacts several aspects, but after finishing the work I usually realize how I could have split the work in smaller tasks after which the build would pass, making each individual commit more understandable

Another thing I usually do is to review any change I am going to commit which requires repeatedly calling git status and git diff [file].

For all this cases I have a large change set in the working repository and I want to stage part of them. git add [file] allows to stage only some of the files, which is already nice. git stash allows to reset the working directory, work and commit some changes, and then come back to the original task, pretty neat for applying the boy scout rule.

But often I need to include only a subset of changes in a given file, and for that git add --patch (doc) is really nice. This command will prompt for each identified diff, which to stage or to ignore. So what I now do is git add . --patch and git asks me for each deleted file if I want to remove it and for each difference in each modified file if I want to include the change: I just need to hit y or n and it moves to the next difference. That also solves the reviewing issue, I am prompted for each difference and choose whether to include it right away. I also tried interactive mode (git add -i) but always went to patch mode, so better skip the first screen.

With this workflow I spend less time on git operations and feel safer by including the exact changes I want, without compromising on commit size.

Tips using Amazon SDK from the browser

Wed, 20 May 2015 07:00:00 -0500

For my small home backup I am building I wanted to use Amazon Web Services javascript SDK. The cool part of using it is that I can easily reuse the same code from both the small CLI and the frontend I am building, and since I am planning to use other other services than S3 their integration will be very easy, no need to specify credentials again.

But the drawback is the size of the resulting file. By default browserify is instructed by AWS to load a bunch of services when requiring the root module. Loading services one by one does not look like an easy option because of the way the API is built, but one can select the services to include (and their versions) in the bundle using environment variable AWS_SERVICES as documented on the official page:

AWS_SERVICES=s3-2006-03-01 browserify webapp/app.js --outfile _gh-pages/app.js

This still produces a very large package so in the end I might switch to smaller dependencies targetting specific services or even craft HTTP requests myself.

The second issue I encountered was OPTIONS HTTP requests failing from the browser with 403 Forbidden. Those requests qre actually sent by S3 SDK prior to making actual API related requests to ensure that CORS is correctly configured for the current host. In my case I was missing allowed headers in my bucket. I thus set the following configuration:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <AllowedMethod>HEAD</AllowedMethod>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

This can be set from the console, under bucket permissions, or by using the SDK itself from Node.js (which I now do on each synchronization).

Executables using Docker containers

Sat, 09 May 2015 07:00:00 -0500

I often have issues when using latest software versions like for fast moving projects like Node.js. Since I am using Debian and that hosted packages are often stable versions, I need to install those packages manually. Some projects provide prebuilt packages to download from their website but others require manual installation with all dependencies, hoping that those will not conflict with the system and requiring to remember all manually installed dependencies when cleaning up the system. Latest versions of fast moving projects often change by nature, meaning I often need to manually update them.

Docker’s execution model looked interesting for isolating those in a container rather than installing them on the host system, then being able to start this container in milliseconds when needed. I saw people have been using that strategy for some time and gave it a try. The good news is that Docker being so popular those days, there qre endless up to date images for most projects.

The simplest thing people are doing is just to create an alias that boots a container with a volume to the host working directory and executes the command inside, all arguments simply being passed down:

alias node="docker run -it --rm -v \`pwd\`:/pwd -w /pwd node:0.12.2-slim node"
alias npm="docker run -it --rm -v \`pwd\`:/pwd -w /pwd node:0.12.2-slim npm"

Adding this alias in my .bashrc solves my two issues:

upgrading to a newer version just requires to change the image version in my alias
cleaning up is just asking Docker to prune unused images

The first issue I encountered was that all files being created by the command were created as root. Solving that requires to have a user defined in the container with the same identifier than the host user, and running the container with this user. I thus created a script which creates a local image with the user properly set inside, and changed aliases to use this image instead:

docker run --name node-with-user node:0.12.2-slim /bin/bash -c \
  "groupadd -f -g $(id -g) dummy && useradd -u $(id -u) -g dummy dummy"
docker commit node-with-user local/node-with-user
docker rm node-with-user

alias node="docker run -it --rm -v \`pwd\`:/pwd -w /pwd -u dummy local/node-with-user node"
alias npm="docker run -it --rm -v \`pwd\`:/pwd -w /pwd -u dummy local/node-with-user npm"

Again upgrading to another version is just changing the image in the file and running it again.

The second issue I had was that this does not respect npm caching. npm is keeping all downloaded binaries locally and only checks metadata against the server when installing the same package over and over. For this I decided to use docker volumes, I created a data container that would keep the cache and reuse its volumes in running containers. In the generated image I had to create the folder containing npm cache with the correct user. For convenience I also created an alias not mounting cache volumes so I can force fresh installs if needed:

docker run --name node-with-user node:0.12.2-slim /bin/bash -c \
  "groupadd -f -g $(id -g) dummy && useradd -u $(id -u) -g dummy dummy && mkdir -p /home/dummy && chown -R dummy:dummy /home/dummy"
docker commit node-with-user local/node-with-user
docker rm node-with-user

docker inspect npm-cache > /dev/null
if [ $? -eq 0 ]; then
  docker rm npm-cache
fi
docker create -v /home/dummy --name npm-cache local/node-with-user

alias node="docker run -it --rm -v \`pwd\`:/pwd -w /pwd -u dummy local/node-with-user node"
alias npm="docker run -it --rm --volumes-from npm-cache -v \`pwd\`:/pwd -w /pwd -u dummy local/node-with-user npm"
alias npm-fresh="docker run -it --rm -v \`pwd\`:/pwd -w /pwd -u dummy local/node-with-user npm"

The volume is set to the whole home directory to also include .npmrc that allows logging to npm.

That organisation is quite limiting as a new data container is created each time one creates a bash session, I thus had to split this in two scripts:

one creating the image and data container each time it is invoked
in .bashrc I kept a test for the data container existence (calling the other script if not) and aliases

#!/bin/bash

# Create a container with a user having the same id than the local one and rights to edit npm cache directory, create an image out of the container and delete it (this is were node version is chosen)
docker run --name node-with-user node:0.12.2-slim /bin/bash -c \
  "groupadd -f -g $(id -g) dummy && useradd -u $(id -u) -g dummy dummy && mkdir -p /home/dummy && chown -R dummy:dummy /home/dummy"
docker commit node-with-user local/node-with-user
docker rm node-with-user

# Create a data container keeping npm content
docker inspect npm-cache > /dev/null
if [ $? -eq 0 ]; then
  docker rm npm-cache
fi
docker create -v /home/dummy --name npm-cache local/node-with-user

docker inspect npm-cache > /dev/null
if [ $? -ne 0 ]; then
  source ~/.config/node-setup.sh
fi

alias node="docker run -it --rm -v \`pwd\`:/pwd -w /pwd -u dummy local/node-with-user node"
alias npm="docker run -it --rm --volumes-from npm-cache -v \`pwd\`:/pwd -w /pwd -u dummy local/node-with-user npm"
alias npm-fresh="docker run -it --rm -v \`pwd\`:/pwd -w /pwd -u dummy local/node-with-user npm"

Upgrading to a new version now requires that I change image version in the standalone script and runs it once.

I had the same working for Maven:

#!/bin/bash

# Create a container with a user having the same id than the local one, create an image out of the container and delete it (this is were maven version is chosen)
docker run --name mvn-with-user maven:3.3.3-jdk-8 /bin/bash -c \
  "groupadd -f -g $(id -g) dummy && useradd -u $(id -u) -g dummy dummy && mkdir -p /home/dummy && chown -R dummy:dummy /home/dummy"
docker commit mvn-with-user local/mvn-with-user
docker rm mvn-with-user

# Create a data container keeping maven content
docker inspect mvn-cache > /dev/null
if [ $? -eq 0 ]; then
  docker rm mvn-cache
fi
docker create -v /home/dummy --name mvn-cache local/mvn-with-user

docker inspect mvn-cache > /dev/null
if [ $? -ne 0 ]; then
  source ~/.config/java-setup.sh
fi

alias mvn="docker run -it --rm --volumes-from mvn-cache -v \`pwd\`:/pwd -w /pwd -u dummy local/mvn-with-user mvn"
alias mvn-fresh="docker run -it --rm -v \`pwd\`:/pwd -w /pwd -u dummy local/mvn-with-user mvn"

Even if pretty satisfied by the setup I encountered a few limitations. The obvious one is that it consumes more space on disk than just installing packages: my hard drive is way large enough to accomodate it. It also consumes a lot of bandwidth on setup to download all filesystem layers. Execution time does not seem to be affected by running inside containers.

The real limation I found was that those executables are actually isolated, they cannot rely on the presence of other executables on the system. For most cases this is exactly what I wanted: the software I write does not implicitly depends on things installed on my system. It is howerver a problem for npm commands, I used to have a few helpers relying on git to deploy to Github pages:

{
  "scripts": {
    "deploy-pages": "cd _gh-pages && git add --all && git commit -m \"Update site\" && git push",
    "setup-pages": "git clone --depth 1 -b gh-pages $(git config --get remote.origin.url) _gh-pages"
  }
}

The biggest problem with third parties is encapsulation

Sat, 30 Aug 2014 07:00:00 -0500

Last A List Apart issue features an article about how to safely manage third parties: Dependence Day: The Power and Peril of Third-Party Solutions. The main discussion is around determining whether it is best to develop a feature or to use a third party. One of the criteria is vitality, the risk of the dependency being abandoned.

For external providers this is obviously very important because it is likely it will end up hosting data, so if the provider disappears data is gone with it. But for linked libraries I think the criteria is much less important given a proper abstraction is built. The pain we all suffer to migrate to an alternative, or even to a newer version of the original one, is mostly the result of poor respect of encapsulation.

It is simplest done for technical libraries, generally one needs to make a few calls to the library in order to get a task done. Converting an object to JSON in Java can be done with the Gson library. The first time the application needs to serialize, one import the library in the project and simply add lines:

Gson gson = new Gson();
String content = gson.toJson(myBean);

Some time later another developer needs to serialize to JSON in another part of the app, and he simply imports Gson at that place and makes the same sequence of calls. And trouble just got in, because at this point the cost of migrating to another library just doubled. The actual answer should be to create a new utility class with a method expressing the need rather than a technical implementation:

import com.google.gson.Gson;

public class Json {

  public static String serializeBean(Object bean) {
    Gson gson = new Gson();
    String content = gson.toJson(bean);
    return content;
  }
}

This completely hides the underlying library which can be switched at will. There is still one issue here to ensure a new third party can replace the hold ones: it must handle all use cases. And this can be managed with tests. The first developer may only need beans with primitive properties, thus it adds tests for primitive values are added and appropriate javadoc:

/**
 * Serializes beans to JSON.
 *
 * <p>Bean may only have primitive properties.</p>
 *
 * @param bean bean to serialize.
 * @return JSON representation of bean
 */

The second one may need to handle lists and sets, thus it adds new tests and updates the javadoc accordingly:

/**
 * Serializes beans to JSON.
 *
 * <p>Bean may only have properties:<ul>
 *    <li>primitive</li>
 *    <li>collection of wrapped primitives</li>
 * </ul></p>
 *
 * @param bean bean to serialize.
 * @return JSON representation of bean
 */

Once switching, tests proves whether the new library fulfills the need in seconds.

This was obviously a very simple example. A more realistic one is around databases, there are so many applications riddled with SQL or MongoDB queries everywhere, which become specific to the database provider or version over time. This should never occur with a proper encapsulation of the database behind a functional abstraction. Contrary to what 2000’s MVC frameworks enforced, I am not recommending the usage of a big full fledged ORM, but rather creating ad-hoc abstractions as the need arise within the app, possibly using different datastores for different modules:

public interface PonyStable {

  void feed(Pony p);
  void stroll(Pony p);
}

class SqlPonyStable {

  public void feed(Pony p) {
    execSql("UPDATE Pony ...");
  }

  public void stroll(Pony p) {
    execSql("UPDATE Pony ...");
  }
}

By dogmatically constraining the third party to a few classes with a clear contract, it becomes easier to switch to a new version or an alternative. Since both sides have a clear contract testing is also much easier: most of the application can be tested against mock data while a real database can be setup for the tests of the wrapper.

Data synchronization with Couchbase

Wed, 16 Jul 2014 07:00:00 -0500

I’m interested in synchronization of data to (mobile) web applications for a while and recently watched Sync is the Future of Mobile Data on InfoQ by J. Chris Anderson, presenting Couchbase Mobile.

The solution looks very interesting for use cases that fit exactly in their model: state-based replication of full documents. I also relly like the idea that concurrent versions of documents are natively managed and conflicts to be explicitly handled by the application. But to me there are a few drawbacks to their approach compared to the library I am building.

The first is that it does not support operation-based synchronization where instead of pushing the JSON object and looking for difference a functional operation is pushed to other nodes. Not having that option means that one needs to handle batch updates in a different way or they may result in having to push a large bag of data to the server. It may also be harder to impose complex access rules, here if the user has the rights to modify the object it can do whatever it wants with it by sniffing the network, even breaking application invariants. Mathsync library only manages pulling from the server and application developers have to choose how they push data to the server which can then easily enforce invariants.

A second issue is about synchronizing the whole document to the client. Usually there are fields which are usually set by the server and not meant to be sent to the client. It usually is of no harm, but sometimes sensitive fields may force one to split a logical document in multiple parts, which look to me a going agains the framework. Even when safe, it means the payload transferred to the phone is increased.

I guess they aknowledge those limitations, for example my second concern is mentionned in the talk, because it allows to build powerful applications with very few thinking (much easier than my own library, especially since the library support for serialization is awful for now). But for more complex applications it may be interesting to have more flexibility, at a higher cost.

GPX traces in Jekyll

Fri, 27 Jun 2014 12:17:00 -0500

I used to display gpx traces of my running using a Jekyll/Octopress plugin I made quite some time ago, but after the migration to pure Jekyll I preferred to display GPX traces with no plugin at all.

I just declare traces as a Jekyll collection, drop .gpx files in _traces to which I set a custom layout in add yaml front-matter. The custom layout which imports Leaflet and parses the xml in javascript directly in the browser.

## Collection

The collection is declared in _config.yml and set to generate standalone content:

collections:
  traces:
    output: true

Then I drop .gpx files in _traces and simply add a front-matter with the custom layout:

---
layout: trace
title: "Marathon des Alpes Maritimes 2013"
---
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<gpx xmlns="http://www.topografix.com/GPX/1/1" creator="byHand" version="1.1"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd">
  <trk>
    <name>Marathon des Alpes Maritimes 2013</name>
    <trkseg>
      <trkpt lat="43.695063" lon="7.269926">
        <ele>60.5</ele>
        <time>2013-11-10T06:56:37.267Z</time>
      </trkpt>
      <!-- more -->
    </trkseg>
  </trk>
</gpx>

## Layout

The layout is a bit more complex, it needs to

import Leaflet (which I personally checkin in my blog repository rather than using the hosted version to be standalone)
drop content (which is the gpx trace) in a script tag rather than in the body
display the expected page with an empty div in where the map is to appear
include javascript code to initialize the map from the embedded gpx content

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title>{{ page.title }}</title>
    <meta name="viewport" content="width=device-width">

    <!-- Leaflet -->
    <link rel="stylesheet" href="/css/leaflet-0.7.3.css"">
    <script src="/js/leaflet-0.7.3.js"></script>
    <script type="application/gpx+xml" id="gpx-source">{{ content }}</script>

    <!-- Custom CSS -->
    <link rel="stylesheet" href="/css/main.css">
  </head>
  <body>

    {% include header.html %}

    <div class="page-content">
      <div class="wrap">
        <div class="post">
          <header class="post-header">
            <h1>{{ page.title }}</h1>
          </header>
          <article class="post-content">
            <div id="map" style="height: 40em;"></div>
          </article>
        </div>
      </div>
    </div>

    {% include footer.html %}

    <script>
(function () {
  function htmlCollectionMap(collection, cb) {
    var transformed = new Array(collection.length);
    for (var i = 0; i < collection.length; i++) {
      transformed[i] = cb(collection.item(i), i);
    }
    return transformed;
  }

  function pointToLatLng(point) {
    return [parseFloat(point.getAttribute('lat')), parseFloat(point.getAttribute('lon'))];
  }

  var gpxSource = document.getElementById('gpx-source').textContent;
  var parser = new DOMParser();
  var gpx = parser.parseFromString(gpxSource, "text/xml");
  var colors = ['red', 'blue', 'green'];

  var bounds;
  var map = L.map('map');
  htmlCollectionMap(gpx.getElementsByTagName('trkseg'), function (segment, idx) {
    var points = htmlCollectionMap(segment.getElementsByTagName('trkpt'), pointToLatLng);
    bounds = bounds ? bounds.extend(points) : new L.LatLngBounds(points);
    L.polyline(points, { color: colors[idx % colors.length] }).addTo(map);
  });
  map.fitBounds(bounds);
  L.tileLayer('http://otile3.mqcdn.com/tiles/1.0.0/map/{z}/{x}/{y}.png', {
    attribution: 'Tiles Courtesy of <a href="http://www.mapquest.com/" target="_blank">MapQuest</a> <img src="http://developer.mapquest.com/content/osm/mq_logo.png">',
    maxZoom: 18
  }).addTo(map);
})();
    </script>
  </body>
</html>

For my case I also added custom tags in the front-matter like the duration and the type of race. The layout displays them directly in the served HTML and it is indexed.

## Github pages

Not using a custom plugin theorically allows to directly use Github pages to generate the site and simply push sources to there. Though the version of Jekyll currently supported by Github pages is too old so in the meantime I use the following makefile to manually push to it:

init:
  test -d _gh-pages || git clone --depth 1 -b gh-pages git@github.com:xxx/yyy _gh-pages

build:
  cd _gh-pages && git rm -rf .
  jekyll build
  cp -R _site/* _gh-pages
  cd _gh-pages && git add .

deploy:
  cd _gh-pages && git commit -m "Update" && git push

## Likes

What I really like in this solution is being able to display traces with natively in Jekyll, no real hack there. An important aspect to the solution is that the gpx track itself is embedded in the page, not need to fetch it asynchronously.

On the bad side, it may be harder to use that to include a trace in the middle of a post, in my case traces are standalone items, but the solution may not apply if mixed with text.

JSSophia #11

Sat, 24 May 2014 03:31:00 -0500

This week we had a chance to have a new event of the JSSophia user group, hosted by CrossKnowledge, well organized by Olivier and Corinne. Really cool to see the crowded room each time!

## Cordova

The first talk was given by Erik Jan de Wit about Cordova and how easy it is to convert an application written with web technologies inside native wrappers for most mobile platforms. I especially like the advice not to try replicate the exact look and feel of the platform but rather create your own recognizable style. It was also cool to highlight the aim at following WebAPI so that once those are standardized and included in webviews, there will not even be a need for native wrappers.

It was then followed by lightning talks. I took the challenge to introduce Browserify in less than 5 minutes.

Browserify lightning talk

I started by showing the difference between CommonJS modules initially made for Node.js and were designed synchronous, and AMD more targeted at browser environments and designed to allow for lazily loading modules from the server asynchronously.

// CommonJS
var math = require('math');

exports.increment = function(val) {
    return math.add(val, 1);
};

// AMD
define('increment', ['math'], function (math) {
  function increment(val) {
    return math.add(val, 1);
  }

  return { increment : increment };
});

I personally build code which is aimed at working both on the server and on the client, for example a small tool or an application for which part of the code can be shared amond server and client logics. To solve this, people derived universal module definition, but it makes a large boilerplate to maintain on each and every file.

(function (root, factory) {
  if (typeof define === 'function' &and;&and; define.amd) {
    define('increment', ['math'], factory);
  } else if (typeof exports === 'object') {
    module.exports = factory(require('math'));
  } else {
    root.returnExports = factory(root.b);
  }
}(this, function (math) {
  function increment(val) {
    return math.add(val, 1);
  }
  return { increment : increment };
}));

And this is where Browserify proves useful. It is aimed at reading CommonJS modules, analyzing dependencies with static analysis and bundling everything in a single javascript file, along with bootstrap code to link all of them. The resulting file thus works in the browser. For the use case of an application one wants to share logic, the whole application is authored in CommonJS modules.

// shared code
var math = require('math');

exports.increment = function(val) {
    return math.add(val, 1);
};

// server.js
var increment = require('increment');

/* server logic */
server.start();

// client.js
var increment = require('increment');

/* client side logic */
client.start(global);

Server is then ran using Node directly on server.js file and will natively use library code. At application build time, Browserify is used to compile client.js into a bundled asset including all dependencies. When a client loads the application, it simply downloads this bundle with a <script> tag and the application starts by itself.

The use case of libraries or tools is slightly different. My personal workflow is to author code in CommonJS modules

var math = require('math');

exports.increment = function(val) {
    return math.add(val, 1);
};

During build time, I use Browserify to compile library entry point into an asset from with a standalone build. A standalone build simply wraps the original module in a universal module definition so that the asset can be consumed both by CommonJS or AMD modules, but also using a browser global variable. The good news here is that boilerplate is not under version control and needs not be maintained. Once the asset is generated, I push both original modules and the browserified one to NPM. I still prefer to push original modules so that people looking at the code have a better time and using other packagers is still possible.

Building an asset with Browserify is really easy both in Gulp or Grunt.

// Gulp
var browserify = require('browserify');
var source = require('vinyl-source-stream');

gulp.task('browser', function() {
  return browserify('./src/increment.js')
    .bundle({ standalone : 'increment' })
    .pipe(source('browser.js'))
    .pipe(gulp.dest('browser'));
});

// Grunt
grunt.initConfig({
  browserify: {
    standalone: {
      src: [ 'increment.js' ],
      dest: './browser/dist/increment.js',
      options: {
        standalone: 'increment'
      }
    }
  }
});
grunt.loadNpmTasks('grunt-browserify');

I then had a small digression, because none of AMD or CommonJS modules are actual standards, and ES6 includes a module standard.

module 'Increment' {

  import add from 'Math';

  export default function increment(n) {
    return add(n, 1);
  }
}

Using that layout sounds more future-forward to me, and will hopefully eventually be supported by everyone. Detailing the standard would be worth one full talk! To use it currently, one can have a look to Traceur compiler, which is aimed at transpiling next javascript to something current platforms understand. This one at that day does not support modules but may be worth looking at.

## Other lightning talks

There were a lot other lightning talks. Bertrand followed and spoke about noder-js which has some similar goals to Browserify but offers more flexible packaging which may especially be useful for multi pages websites and for debugging. Noder-js supports some Node apis in the browser, especially the module-related ones. Browserify does this too and includes complex apis but I did not mention it during my talk as I am not convinced using require('http') in client side code makes that much sense. I thus prefer noder’s approach here.

Then Sebastien spoke about Firefox OS, Bertrand about debugging a Cordova application, Frédéric introduced the MEAN stack and Yacine closed with ionic.

Migrating from Octopress to raw Jekyll

Sat, 24 May 2014 03:17:00 -0500

Since I did not use many plugins provided by Octopress, the blog has been migrated to raw Jekyll. This was especially motivated by collections in Jekyll 2.0 to store readings and GPX traces.

The new page layout is simplistic but the plan is to make a more customized, personal one. GPX traces migration is not yet done but at least post and readings are. One of the side effect is that build time is greatly reduced.

On the overheads, post creation is a bit more tedious because I have to define the proper name including date manually while Octopress has a rake task for doing it. I also had to write my custom deploy script, because since I rely on latest versions of Jekyll I have to build locally.