In which we will visualise commute flows around the London Underground using Google Maps and Turf.js

Part 1

A “hairball” map of the London Underground

This week, we are going to have a play with a really large dataset of people’s movement around the London Underground, and do some flow mapping to represent where people go.

Let’s start with the basics: a simple flow map that displays all of the journeys that exist in a sample dataset of people swiping into and out of the London Underground using their Oyster Cards. Because we know which stations they entered and exited at, and we know where those stations are, it is relatively easy for us to draw a flow map showing them moving around the city!

Make a new web page using the usual template, name it something like week10.html

Set the view to London:

.setView( [51.48728233102447, -0.11360004378767831], 12)

Import turf.js:

<script src="https://cdn.jsdelivr.net/npm/@turf/turf@5/turf.min.js"></script>

Add a muted tile layer:

// this adds the basemap tiles to the map
L.tileLayer('https://server.arcgisonline.com/ArcGIS/rest/services/Canvas/World_Light_Gray_Base/MapServer/tile/{z}/{y}/{x}', {
  attribution: 'Tiles © Esri — Esri, DeLorme, NAVTEQ',
  maxZoom: 16
}).addTo(map);

The data that you need is available in JSON format at the following location:

http://geographicalinformation.science/maps/data/underground.js

Import the above javascript file into your web page in the usual way

underground.js contains a JSON Array of objects representing journeys.

console.log() the journeys variable in order to understand the structure of the dataset

Once you understand the structure of the data, create a new function called journeysToMap()

Now to draw each journey onto the map, all that is needed is to Loop through the data, adding a turf.lineString to the map stretching from the start of the journey to the end (remember that you do not need to flip the coordinates for turf, only for Leaflet).

Loop through the Add a turf.lineString for each journey to an array by completing the snippet below:

// loop through each journey (note that this declares the lines array as well as the loop control variable)
for (var i = 0, lines = [];	// YOU NEED TO FINISH THIS LINE

  // make a lineString and load into the array
  lines.push(
  	turf.lineString([	//note the requirement to use parseFloat() to convert the JSON coordinates from Strings to numbers
      [ parseFloat(journeys[i]['startX']), parseFloat() ], 	// YOU NEED TO FINISH THIS LINE	
      [ parseFloat() , parseFloat() ]	// YOU NEED TO FINISH THIS LINE
  	])
  );
}

NB: You will need to convert the String coordinate values from the JSON into Numbers using the function parseFloat() before you can use them to make your lines (e.g. parseFloat(journeys[i]['startX'])).

After the loop is complete, add the array to a turf.featureCollection (like we did in weeks 3, 4 and 9)

Add the featureCollection to the map as a L.geoJson() object (like we did in weeks 3-9).

Remember you have done all of this before! You are simply taking those things and applying them to a new context. If all goes to plan, you should end up with something like this:

Not great, is it? Welcome to your first hairball map!

Part 2

An ‘OD Curve’ Map of the London Underground

By now you should have a pretty ugly looking flow map. As we saw in the lecture, this is not necessarily the best way to visualise this kind of data. For example, you can’t tell which way people are moving, and there is no way to tell the difference between one person and 1000!

A more appropriate approach might be to use the Origin-Destination (OD) curves that we looked at in the lecture. That way we will get a visualisation that both looks less cluttered and accounts for magnitude (number of people) and directionality in the dataset. We are therefore now going to make a new version of this map using OD Curves instead of plain lineStrings.

How OD Curves Work

Computationally, OD Curves are made using Bézier Curves, which are commonly used in computer graphics to make nice looking curves inexpensively. The advantage of Bézier curves is that they are quite simple, being generally defined with 4 points (a start point, an end point, and two ‘control points’) that influence the path of the curve. Here is an illustration (from Wikipedia) of a Bézier curve with the associated 4 points:

As you already know, computers can’t understand a ‘curve’, but simply a set of very small, straight lines that look like a curve when they are assembled. Here is an animation (also from Wikipedia) of one being constructed, see how the curve is made up of lots of individual points (counted with the t variable at the bottom):

The algorithm for a Bézier curve, therefore, takes in 4 points and returns a set of points that can be connected by lines to look like a curve. We could now dig into how the algorithm works and write the code out by hand (have a look at it in the Wikipedia article if you haven’t already), but this is not the most productive use of our time today, so we are going to use our old friend turf.js to do it for us.

Unlike the Bézier curves above, however, which takes 4 points turns them into a curve, the Bézier function in turf.js takes in a line and returns a curve that follows the same path, essentially smoothing it out. You can see an example of this here, in a map of the sea boundary between Jamaica, Cuba and Haiti:

(In this case, we have passed the red dotted line to bezierSpline function, which returned the blue curve.)

To make an OD curve, therefore, our goal is to make an ‘L’ shaped line from the origin to the destination, which we can then convert to a Bézier curve like this illustration from my office to the museum, with a straight line shown in red, the ‘L’-shaped prototype in green and the resulting Bézier curve in blue:

Let’s make some curves…

So our job here is to create a function that can use the red line above to make the green line, and then use that to make the blue line.

Starting with the green line, this is simply an L-shaped lineString that goes from point a (from the red line) to point b (also from the red line) via point c, which is, as yet, unknown. Point c is simply a point that is located 1/6 of the distance between a and b away from point b at a bearing of 45 degrees. There is nothing scientific about this definition, I simply know that these values give a good OD Curve from a simple process of trial and error! Once we have the green line (a-c-b) we simply ask turf to make it into a Bézier curve and voila - we have our OD curve!

Let’s make a function to handle converting a and b into an OD Curve.

Define a new function makeCurve(a, b)

The first thing that we need to do in this function is calculate the distance between the (known) point b and the (unknown) point, and then work out 1/6 of that distance. This is simple, because turf has a turf.distance function built right in!

  // get the distance by which to offset point c from point b
  // this should be one sixth of the distance between a and b
  var oDist = turf.distance(a, b, {units: "meters"}) * 1/6;

Add the above snippet to your makeCurve() function

Next, we need to get a direction 45 degrees away from the direction between a and b. Whilst turf does have a turf.bearing function as well, it is unfortunately not quite as simple as turf.distance was…

Unfortunately, turf.bearing returns the direction relative to east (not north) and on a scale of -180 - 180, not 0 - 360. This is (for some reason) the mathematical convention for how to describe direction, but it is not much use to us geographers!

Resolving this issue is, much like last week, something that leads to a LOT of errors amongst GIS professionals. Also like last week, however, it can be resolved with a simple mathemetical function that converts it, and just like last week you do not need to memorise this function - simply understand that if you are measuring directions then you need to make sure to use it!!

/**
 * Get a turf bearing, corrected for north and wrapped 0-360
 */
function getBearing(a, b) {
  return (90 - turf.bearing(a, b) + 360) % 360;
}	

This simply takes the turf bearing away from 90 (to make it relative to north) and then users the modulus operator (%) to ‘wrap’ the resulting number to a scale of 0-360.

Add the above function to your code (remember to put it in the global scope!)

While you’re at it, add this function as well, which handles the ‘wrapping’ from 0-360 section on its own

/**
 * Ensure and angle is always between 0-360
 */
function wrapAngle(angle) {
  return (angle + 360) % 360;
}

The reason that this is necessary is that we need to re-wrap the coordinates after we have subtracted 45 degrees from it (because if our direction was 10°, for example, then we would end up with -35°, when clearly we would want 325°)!

Because this direction stuff is all a bit complicated, we should make a test function here (a bit like we did last week). The function below calculates the bearings sets of points of known direction (N, E, S and W). If the function gets all four bearings correct (0, 90, 180 and 270 degrees respectively), then we can be pretty confident that it works in all directions.

Add the below to your code and call it from initMap()

/**
 * Run tests for bearing calculations
 * Success looks like this:
 *		North: 0   true
 *		East:  90  true
 *		South: 180 true
 *		West:  270 true
 */			
function TEST_bearings(){

  // never
  var a = turf.point([53, 2]);
  var b = turf.point([53.000000000001, 2]);
  var c = getBearing(a, b);
  console.log("North: ", c, (c === 0));

  // eat
  a = turf.point([53, 2]);
  b = turf.point([53, 2.000000000001]);
  c = getBearing(a, b);
  console.log("East: ", c, (c === 90));

  // shredded
  a = turf.point([53.000000000001, 2]);
  b = turf.point([53, 2]);
  c = getBearing(a, b);
  console.log("South: ", c, (c === 180));

  // wheat
  a = turf.point([53, 2.000000000001]);
  b = turf.point([53, 2]);
  c = getBearing(a, b);
  console.log("West: ", c, (c === 270));
}

As you can see from the comment, if your functions are working well then the output from this function should look something like this (where true indicates that the expected value was returned):

North: 0   true
East:  90  true
South: 180 true
West:  270 true

Make sure that all four tests returned true

If so, then you are ready to carry on with creating the makeCurve() function: use getBearing() to calculate the bearing between our two locations and minus 45° from it (remember to use wrapAngle() to wrap the result!) and store it in a variable called oDirection

Okay, so we now know both the distance and the direction between point b and point c - meaning that we can now work out the coordinates for point c! Once again, turf has us covered - this time with turf.destination(), which calculates the ‘destination’ point at a given distance and direction from a given location - exactly what we want to know!

Implement the below code to get the destination point and then extract the coordinates from the result

// calculate point c (as an offset from point b) and extract coordinates
var c = turf.destination(b, oDist, oDirection, {units: "meters"}).geometry.coordinates;

Now we have locations a, b and c we can finally make our ‘L’ shaped line!

Construct a turf.lineString() that goes a → c → b and store it in a variable called line

Finally, we give our ‘L’ shaped line (line) to turf.bezierSpline(), which will return the corresponding OD Curve

Implement the below snippet to complete the makeODCurve() function

// calculate the bezier curve and return
return turf.bezierSpline(line, { resolution: 5000, sharpness: 0.9 });

Great - now we can make curves, we just need to update our journeysToMap() function to replace the lines with the curves:

Tweak journeysToMap() to make it like this:

/**
 * Load the journey lines onto the map, styling intelligently into 5 equal classes
 */
function journeysToMap() {

  // loop through each journey
  for (var i = 0, curves = [], bezier; i < journeys.length; i++) {

    // construct a Bezier (OD) Curve
    bezier = makeODCurve(
      [ parseFloat(journeys[i]['startX']), parseFloat(journeys[i]['startY']) ], 
      [ parseFloat(journeys[i]['endX']),   parseFloat(journeys[i]['endY'])   ]
    );

    // make the curve into a lineString including the count property and push into array
    curves.push(turf.lineString( bezier.geometry.coordinates, {journeyCount: parseInt(journeys[i]['cnt'])} ));
  }

  // compile curves into a feature collection and add to the map
  geojson = L.geoJson(turf.featureCollection(curves), { }).addTo(map);
}

Note how we have added the journeyCount property to each of our curves - this is important for later!

If that all worked, you should now have something like this:

Okay, not much better I’ll admit - but we’re getting closer, I promise!!

Classifying your Symbology

Some of you might be looking at the above and thinking that this visualisation is actually LESS clear than the one where you just had straight lines, and you’d be right! However, that will all change once we classify our symbology:

As you can see, we now get a much clearer idea of which journeys are the most popular, and we also get that all important directionality!

Let’s get to it then, we have classified a symbology before (in week 3 and week 4), so this should be relatively straightforward. Unlike those previous weeks, however, this time we are going to automatically classify the curves using an Equal Interval classification, meaning that each class contains the same range of values (e.g. 2, 4, 6, 8, 10 - each class has a range of 2).

To do this, we need a function that can loop through the dataset and work it out.

Create a new function called getStyleInterval() that takes in two arguments: curveCollection (a turf.featureCollection containing our dataset) and classCount (the number of classes to divide them into)

This function is quite straightforward - we will simply loop through our featyureCollection and find the largest (max) and the smallest (min) values stored in the journeyCount attribute of the curves. We then simply subtract min - max in order to get the range of values, and divide the result by the number of classes that we want in order to get the class interval.

To achieve this we need to go through a number of steps:

1. Create two variables, one called maxcontaining a value that is definitely smaller than the smallest journeyCount in the dataset, and one called min containing a value that is definitely larger than the largest journeyCount in the dataset. This can be achieved using a special value in JavaScript: Infinity.
2. Loop through every journey the dataset and test if the current journeyCount value is larger than max. If so, overwrite max with that value.
3. If the number is not larger than max, test if it smaller than min. If so, overwrite min with that value.

The result of this is that, by the end of the loop, max contains the biggest value in the dataset, and min contains the smallest - clever eh!.

Populate your getStyleInterval() function with the below code snippet - you will need to work out how it works and fill in the blanks:

/**
 * Automatically calculate the interval for a given number of classes
 */
function getStyleInterval(curveCollection, classCount) {

  // init variables for the largest and smallest variables encountered
  // Infinity is a special value in JavaScript that is larger than any other number
  var max = -Infinity;	
  var min =	// YOU NEED TO FINISH THIS LINE

  // get the range of the passenger count values
  for (var i = 0, journeyCount; i < curveCollection.features.length; i++) {

    // get the journeyCount for the current feature
    journeyCount = curveCollection.features[i].properties.journeyCount;

    // if it's bigger than the previous max, save it as the new max
    if (journeyCount > max) {
      
      	// YOU NEED TO ADD A LINE

      // otherwise, if it's smaller than the previous min, save it as the new min
    } else if (	// YOU NEED TO FINISH THIS LINE
      
      	// YOU NEED TO ADD A LINE
    }
  }

  // get the interval between each desired classification
  return Math.round((max - min) / classCount);
}

We can then call this function to determin our interval for 5 symbology classes:

Declare interval as a global variable then add the following line to the correct location in journeysToMap in order to

// determine an interval to use in styling the curves
interval = getStyleInterval(curveCollection, 5);

Now we know our interval we can make a style() function to style our L.geoJson() layer according to our classes. The first class (thin line, light colour, more transparent) is for the least frequently used routes, and then they increase (interval*2, interval*3, interval*4 etc.) in thickness, darkness and opacity with each subsequent class. As ever, I have chosen my colour scheme using Color Brewer, and the result looks like this (feel free to replace this with your own colour scheme!):

Let’s make a style function as usual:

/**
 * Set the style of the curve
 */
function style(feature) {

  // assign the appropriate colour based upon the journeyCount value
  if (feature.properties.journeyCount <= interval) {
    return {
      color: "#ffffb2",
      opacity: 0.3,
      weight: 2
    }
  } else if (feature.properties.journeyCount <= interval * 2) {
    return {
      color: "#fecc5c",
      opacity: 0.4,
      weight: 3
    }
  } else if (feature.properties.journeyCount <= interval * 3) {
    return {
      color: "#fd8d3c",
      opacity: 0.5,
      weight: 4
    }
  } else if  (feature.properties.journeyCount <= interval * 4) {
    return {
      color: "#f03b20",
      opacity: 0.8,
      weight: 6
    }
  } else {	// greater than interval *4
    return {
      color: "#bd0026",
      opacity: 0.9,
      weight: 7
    }
  }
}

Implement the above function and apply it to the options of your L.geoJson() constructor

If it worked then you should have something like this:

All that we are missing now is, of course, a legend! Previously we have placed these elsewhere on the page, but sometimes it is nice to add them to the map itself. Fortunately, we have added panels to our map several times before, and this is no different!

Add an infoBox to the map, and populate it with the following HTML:

<div class="legend">
  <i style="background:#bd0026"></i> <span id="interval5"></span> journeys<br/>
  <i style="background:#f03b20"></i> <span id="interval4"></span> journeys<br/>
  <i style="background:#fd8d3c"></i> <span id="interval3"></span> journeys<br/>
  <i style="background:#fecc5c"></i> <span id="interval2"></span> journeys<br/>
  <i style="background:#ffffb2"></i> <span id="interval1"></span> journeys<br/>
</div>

Remember that this infoBox will not need an .update() function, so you can remove that and just set the this._div.innerHTML in the .onAdd() event listener instead.

And while we’re at it, let’s update the CSS:

/* Style the legend */
.legend { 
  font-family: Arial, sans-serif;
  font-size: small;
  padding-top: 10px;
  text-align: left; 
  line-height: 18px; 
} 

/* Style the i tags within the legend - we use these to make the coloured squares */
.legend i { 
  width: 18px; 
  height: 18px; 
  float: left; 
  margin-right: 8px; 
}

/* Style the info box so that it looks like a Leaflet control */
.info { 
  font-family: Arial, sans-serif;
  padding: 6px 8px; 
  background: white; 
  background: rgba(255,255,255,0.8); 
  box-shadow: 0 0 15px rgba(0,0,0,0.2); 
  border-radius: 5px; 
} 

As you can see, the HTML snippet that I gave you is missing any values:

Instead, it has a series of (<span>)[https://www.w3schools.com/tags/tag_span.asp] tags that have the IDs interval1, interval2, and so on. (<span>)[https://www.w3schools.com/tags/tag_span.asp] is like (<div>)[https://www.w3schools.com/tags/tag_div.asp] except that it is generally used to identify smaller features (e.g. you might use a div to identify a paragraph or section of a word, whereas you might use a span top identify a single word or sentence). In this case, we are using the span as a placeholder that we will later fill in with our legend values (which we don’t know until we have worked out the interval). We can then fill them in with JavaScript as soon as we know what the interval will be

Add the below snippet to the correct location in your code so that the legend is populated with the interval value

// now we know the interval, fill in the legend
for (var i = 1; i <= 5; i++) {
  document.getElementById( 'interval' + i.toString() ).innerHTML = interval * i;
}

How does that above loop work? If you don’t know, ask!

If all has gone to plan, you should now be looking at something like this:

If so, congratulations, you’re all done!