Using D3.js to create dynamic maps and visuals that show competing climate change scenarios for the 21st century | by Zach Alexander | Jan, 2021

[ad_1]


After writing this data story, I realized that it may be helpful to write a piece outlining the power of creating web visualizations for audiences. As a data science graduate student (and data engineer by profession), I’m often disappointed by the lack of follow-through (me included) on how we communicate conclusions of our analyses and data that we have worked so tirelessly to perfect. In the end, sometimes the algorithms or models we create can fall flat if they aren’t given context or communicated to audiences in a compelling way. Although there are a lot of data visualization platforms (PowerBI, Tableau, etc.) out there that can spin up quick charts and graphs, building out data stories on the web can take analyses to the next level.

In this write-up, I’ll take a piece of one of the visualizations I created for my data story and hope to provide a foundation of how to use Angular and D3.js to build an animated map that simulates a climate change model for the 21st century.

If you would like to follow along with the full code, you can pull from this repository. It’ll contain all of the code outlined below, including:

  • Merging and transforming csv data into jsons (python)
  • Running a local Angular application (Angular-cli)
  • Installing D3.js and loading json files into an Angular component (Angular)
  • Using D3.js to create a world map and animate it over 4 time intervals (Angular and Javascript)

If you find yourself wanting to build on the animation that we start in this article, you can pull additional code from my comprehensive data story on this topic.

Before diving into any of the website work, a crucial first step in developing any data visualization is to think about what you ultimately want to portray to your audience.

My goal: I wanted to visually show the change in land temperature (in degrees F) by country over the next 80 years based on an IPCC climate change scenario.

With this goal, I then set out to find a compelling dataset.

Fortunately, there are countless climate change datasets available for use, and this World Bank dataset containing ensemble data that projects changes in land temperature over the next century (based on global circulation models and IPCC climate change scenarios) was exactly what I was looking for.

Great! I have a dataset, now how do I translate this to a web page?

Well, before we can tackle putting this data onto a webpage, we do have to think critically about what aspect of our data we’d like to present to our audience. Additionally, since D3.js works well with data formatted in JavaScript Object Notation (JSON) format, we’ll have to do some data transformation prior to building our visualization.

Since I would like to create a world map, we also are dealing with geographic coordinates, which take on an even more specific structure of json called “geojson”.

What is a geojson?

In short, geojson structure is an extension of the json format that is used specifically for encoding a variety of geographic data structures. When working with maps in D3.js, there are many built-in functions and processes that rely on your data being oriented in either geojson or topojson format. The difference between these two can be outlined in detail in this stackoverflow post. The format that we’ll be using, geojson, takes the structure below:

{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [38.00,−97.00]
},
"properties": {
"name": "United States"
}
}

For this simple example above, we can see that this data when read into D3.js would draw a point (or dot) in the very center of the contiguous United States. The “geometry” mentioned is of type “point”, and the “coordinates” attached to the point are where the dot would be drawn on the web page. You can also attach specific properties to a point, such as a “name”, which can also be referenced by D3.js (we’ll see this come into play with our color palette later).

Although this is a simple example, you can imagine the “coordinates” getting much longer when you draw specific polygons (or shapes). Essentially, when working with polygons, you are taking a list of points that connect to one another. Each polygon has a set number of coordinates based on a particular projection, and these coordinates are then rendered onto the page. As an example of this in action, if we were to take a look at the polygon coordinates for the state of Nevada, you could find it below:

{
"type": "Feature",
"properties": {"name": "Nevada"},
"geometry": {
"type": "Polygon",
"coordinates": [[[-117.027882,42.000709], [-114.04295,41.995232],[-114.048427,37.000263],[-114.048427,36.195153],[-114.152489,36.025367],[-114.251074,36.01989],[-114.371566,36.140383],[-114.738521,36.102045],[-114.678275,35.516012],[-114.596121,35.324319],[-114.574213,35.138103],[-114.634459,35.00118],[-115.85034,35.970598],[-116.540435,36.501861],[-117.498899,37.21934],[-118.71478,38.101128],[-120.001861,38.999346],[-119.996384,40.264519],[-120.001861,41.995232],[-118.698349,41.989755],[-117.027882,42.000709]]]
}
}

We can see that the “geometry” is much longer than our first example, and the “type” is now a polygon instead of “point”. The “coordinates” are listed based on the projection utilized, which would then render on the page.

With this new knowledge of geojsons, I devised a plan to take the csv files that have the climate change projections from Kaggle and ultimately transform them into json files for D3. Although a lot of data manipulation can be done right in JavaScript through many of D3s built-in functions (you can read csv data directly into JavaScript), I thought it would be more efficient to manipulate the data prior to loading it on the web page since the dataset is static (and won’t rely on access/connections to a third-party API). The upside of this is that it should make the code more efficient, our visualization more snappy, and cut down on the time needed to render maps and charts on page load.

To do this, I utilized python. You can find the entire jupyter notebook that documents my data transformation on Github.

In summary, after reading in the csv file from Kaggle, I first had to work through a fair amount of data aggregation for the various date ranges and model outputs. Then, I essentially took a public geojson file that had the polygons for all countries in the world, and merged the temperature data and other properties from the Kaggle csv files (utilizing the common identifier in the properties information) into each country’s properties. In the final chunk of code, I created a custom function to output the dataframes as json files:

# array of pandas dataframes with our various model data across the four time intervalsdataframes = [first20_med_a2, first20_med_b1, second20_med_a2, second20_med_b1, third20_med_a2, third20_med_b1, fourth20_med_a2, fourth20_med_b1]# empty array of json files that eventually will load in datajson_files = ['first20_med_a2.json', 'first20_med_b1.json', 'second20_med_a2.json', 'second20_med_b1.json', 'third20_med_a2.json', 'third20_med_b1.json', 'fourth20_med_a2.json', 'fourth20_med_b1.json']# custom function to take the pandas dataframes and store them in separate json filesdef createjsons():
for idx, d in enumerate(dataframes):
for i in features:
for index, row in d.iterrows():
if(row['Country'] == i['id']):
i['Change_c'] = row['Change_c']
i['Change_f'] = row['Change_f']
i['Date_range'] = row['Date_range']
i['Percentile'] = row['Pctile']
i['Scenario'] = row['Type']
else:
pass
with open(json_files[idx], 'w') as outfile:
json.dump(gj, outfile)
createjsons()

The ultimate goal of this step was to create separate json files that stored the country name, change in temperature (in degrees C), change in temperature (in degrees F), the date range, the model calculation percentile, and the scenario type.

As we can see below, a country property in one of my final json files looks like this after our data tidying:

# I abbreviated the length of the coordinates to cut down on size of code block{
"type": "Feature",
"id": "USA",
"geometry": {
"type": "MultiPolygon",
"coordinates": [[[[-155.54211, 19.08348], [-155.68817, 18.91619], [-155.93665, 19.05939], [-155.90806, 19.33888], [-156.07347, 19.70294], [-156.02368, 19.81422], [-155.85008, 19.97729], [-155.91907, 20.17395], [-155.86108, 20.26721], [-155.78505, 20.2487], [-155.40214, 20.07975], [-155.22452, 19.99302], ...]]]
},
"properties": {
"name": "United States of America",
"Change_c": 1.5040905413108334,
"Change_f": 2.7073629743595,
"Date_range": "2020-2040",
"Percentile": "median",
"Scenario": "a2"
}
}

Similar to our quick geojson examples above, we can see that we are utilizing many points to create our United States polygon, however I used python to merge in additional “properties” to be used later for my visualization. As we’ll see shortly, the value will be used for our choropleth coloration.

With the json files ready to go, we can then start to imagine what our visualization will look like. From a user’s standpoint, we’ll want to show the change in land temperature over time, which we can do by reading in our json files over specific intervals of time — all of which are animated via D3.js.

Okay! Now that we have our data files ready to go, we can spin up a local single page application.

Depending on the JavaScript framework you’d like to use, you’ll need to download the command-line interface (cli) to launch a local server for development and a package manager.

For this example, the package manager we’ll use is npm. One of the easiest ways to install npm is to download node.js, which will also include npm in your installation. The instructions for this can be found here.

As our Javascript framework, we’ll use Angular. To make our operations simple, you can download the Angular cli to start the development server. The instructions can be found here, and consist of four lines.

Once you’ve downloaded node.js and Angular cli, you can open a VSCode terminal window (or any terminal window) and run the following:

// make sure npm is installed on your computer before running this:npm install -g @angular/cli

After this install, find a place on your computer where you want to store your application files, then run (changing “my-dream-app” to whatever you’d like your directory to be called):

ng new my-dream-app

This will take a minute or two to fully run, but in the end it will create a bunch of files that consist of your new Angular application and provide the directory structure needed for compilation. When the command is complete, you can then navigate to the new application folder and run “ng serve” to start the development server.

// change directories into your new angular applicationcd my-dream-app// start the development serverng serve

If all runs correctly, you should be able to compile your newly created Angular application and see this in your terminal:

You should see this in your terminal window if your Angular application compiles successfully | Image by author

I won’t go into more detail about how Angular works in order to keep this part brief, but if you are completely new to Angular, please read this excellent post to learn about basic app development. Additionally, the Angular documentation is very helpful for those that prefer to get more in the weeds.

With our Angular development server running, we can now start to build out our visualization. First, we’ll need to install the D3.js package from npm in order to be able to use the library in our application. For this visual, I decided to install version 4 of D3 (for various reasons). To do this, stop your development server in your terminal (ctrl + c) and run:

npm install d3v4

When this is complete, you can then restart the development server by running:

ng serve

You should then see the same “Compiled successfully” message as earlier. Next, we can add our json files to the “assets” directory inside of our application (folder structure below):

|- application-name
|- src
|- assets
|- first20_med_a2.json
|- second20_med_a2.json
|- third20_med_a2.json
|- fourth20_med_a2.json

You can find these json files already pre-made here. However, you you’d like, you can also utilize the python script from above.

If you are using Angular, because we are working with Typescript, you’ll likely also have to do a quick configuration to load the json files directly into the app.component.ts file. This article is a great summary of how to do this. Likely, you’ll just have to add the following line of bolded code to your tsconfig.json file:

{
"compileOnSave": false,
"compilerOptions": {
.
.
.
"resolveJsonModule": true,
.
.
.
}
}

Rendering our first map

With our json data stored inside of the application directory, the D3 library imported into our node_modules, and Angular ready to accept json files, we can start to see the power of data visualizations on the web!

To render one of our json files, we first need to ensure we are referencing it in our app.component.ts file (folder structure below):

|- application-name
|- src
|- app
|- app.component.ts

Navigate to the app.component.ts file and import the json data into the component by adding the bolded code to your existing code code:

import { Component} from '@angular/core';
import * as d3 from 'd3v4';
// load the json file from the assets folder, give it a name
import * as firstModel from '../assets/first20_med_a2.json';
@Component({
selector: 'app-root',
templateUrl: './app.component.html',
styleUrls: ['./app.component.scss']
})
export class AppComponent { // make sure to reference the correct structure
firstModelData = firstModel['default'];
ngOnInit() { // print the json data in the browser console
console.log(this.firstModelData)
}
}

Save this file, return to your browser, and if all goes well, you should be able to see the json data printed to the console in development tools after the page refreshes:

Geojson data printed to the Google Chrome console | Image by author

We then need to remove the default html syntax in the app.component.html file and create a quick “div”. Navigate to the app.component.html file (folder structure below):

|- application-name
|- src
|- app
|- app.component.html

Then, delete out all of the html code currently present, and add the following div:

<div class = "world-map"></div>

Now, with the json data reading into the component, we can start to write our D3.js functions. Navigate back to the app.component.ts file, and write the following bolded code inside of the AppComponent class:

export class AppComponent { firstModelData = firstModel['default']; ngOnInit() {
console.log(this.firstModelData)
} setMap(width, height, dataset) { const margin = {top: 10, right: 30, bottom: 10, left: 30};
width = width - margin.left - margin.right;
height = height - margin.top - margin.bottom;
const projection = d3.geoMercator()
.rotate([-11, 0])
.scale(1)
.translate([0, 0]);
const path = d3.geoPath().projection(projection); const svg = d3.select('.world-map')
.append('svg')
.attr('viewBox', '0 0 1000 600')
.attr('preserveAspectRatio', 'xMidYMid')
.style('max-width', 1200)
.style('margin', 'auto')
.style('display', 'flex');
const b = path.bounds(datapull), s = .95 / Math.max((b[1][0] - b[0][0]) / width, (b[1][1] - b[0] [1]) / height), t = [(width - s * (b[1][0] + b[0][0])) / 2, (height - s * (b[1][1] + b[0][1])) / 2]; projection.scale(s).translate(t); svg.selectAll('path')
.data(datapull.features)
.enter()
.append('path')
.attr('d', path)
}
}

In short, this code declares a function accepting three parameters of width, height, and our dataset. It then defines a margin and subtracts the margin pixels from all four sides of the canvas. Next, since we are using map coordinates, we have to define a map projection, rotating the map slightly so the easternmost part of Russia doesn’t make its way onto the left-side of our canvas, and setting our scale and translation. We then take this projection and map it onto our path, which will eventually draw our polygons based on our projection inputs and our defined dimensions of our canvas.

This then creates our first svg by selecting our blank div using the syntax, “d3.select(“.world-map”). By appending the svg to our existing div, we then set the dimensions and make it responsive by utilizing viewBox and preserveAspectRatio, and eventually set a max width of 1200px and centering the map on the page (if the viewport is larger than 1200px).

We then want to ensure that the scale and translation map on correctly, and the defined bounds of our canvas are correct, so variables and help define the bounds of our Mercator projection.

Finally, we select our newly appended svg, read in our geojson features, and draw the path on the page! In order for this to render on the page, we need to take our instantiated function and invoke it in . Add this line of code into your app.component.ts file inside of the ngOnInit() function.

ngOnInit() {
console.log(this.firstModelData)
// add this line of code below to render your map: setMap(1000, 600, this.firstModelData)}

If all goes well, you should see something like this on your page:

First rendering of world map from the setMap() function | Image by author

Alright! Now we are getting somewhere! Unfortunately, though, our polygon fills are coming in black, and we’d like to create a choropleth coloration of each country based on the temperature change in degrees F. Remember when we merged that data into each country property using python? Well, now we can use the value for each country to differentiate our fill colors!

To do this, we can add in the following code inside of our function (still in our app.component.ts file), and then add an attribute to our svg:

setMap(width, height, dataset) {...const color_domain = [2.5, 4, 7, 9, 10];const color_legend = d3.scaleThreshold<string>().range(['#fee5d9', '#fcbba1', '#fc9272',  '#fb6a4a', '#de2d26', '#a50f15']).domain(color_domain);...svg.selectAll('path')
.data(datapull.features)
.enter()
.append('path')
.attr('d', path)
.style('fill', function(d) {
const value = d['Change_f'];
if (value) {
return color_legend(d['Change_f']);
} else {
return '#ccc';
}})
.style('stroke', '#fff')
.style('stroke-width', '0.5')
}

In short, we can use the d3.scaleThreshold() function in D3.js to help us create a choropleth color palette. I won’t go into a lot of detail about this for the sake of time, but you can read more about these scales here, if interested. Additionally, after adding the attribute to our path, I also made the polygon (country) outlines a bit thicker and gray.

If these additions are working correctly, you should see this rendered on your page:

Map rendering after color scale has been added | Image by author

Let’s animate!

If you’ve made it this far, you can definitely pat yourself on the back! The final step for this visualization is to animate it across different time intervals. If you remember from the beginning, our goal was to show the change in temperature for each country across four different time intervals:

  • 2020 to 2040
  • 2040 to 2060
  • 2060 to 2080
  • and 2080 to 2100

As you can now see, we were able to create a static choropleth map with just one json file showing temperature changes between 2020 and 2040. However, to animate it across our other three time intervals, we want to load other json files into this function over a set time. D3.js makes this relatively easy.

First, we’ll have to load the rest of our json files into our app.component.ts file similar to the first one we did above. To do this, you can add the following lines of bolded code:

import { Component, OnInit } from '@angular/core';
import * as d3 from 'd3';
# load the rest of the json files from the assets folder, give it a name
import * as firstModel from '../assets/first20_med_a2.json';
import * as secondModel from '../assets/second20_med_a2.json';
import * as thirdModel from '../assets/third20_med_a2.json';
import * as fourthModel from '../assets/fourth20_med_a2.json';
@Component({
selector: 'app-root',
templateUrl: './app.component.html',
styleUrls: ['./app.component.scss']
})
export class AppComponent {# make sure to reference the correct structure
firstModelData = firstModel['default'];
secondModelData = secondModel['default'];
thirdModelData = thirdModel['default'];
fourthModelData = fourthModel['default'];
jsons;...ngOnInit() {...}

Then, we need to store the json files in an array on page load. Therefore, we can add this into our function:

ngOnInit() {...this.jsons = [this.firstModelData, this.secondModelData, this.thirdModelData, this.fourthModelData]...}

Now, in order for our animation to work, we have to create a new function below our function, called :

transitionMap(json, i) {  const svg = d3.select('.world-map');  

const color_domain = [2.5, 4, 7, 9, 10];

const color_legend = d3.scaleThreshold<string>().range(['#fee5d9', '#fcbba1', '#fc9272', '#fb6a4a', '#de2d26', '#a50f15']).domain(color_domain);
svg.selectAll('path')
.data(json[i].features)
.transition()
.delay(100)
.duration(1000)
.style('fill', function(d) {
const value = d['Change_f'];
if (value) {
return color_legend(d['Change_f']);
} else {
return '#ccc';
}
})
}

This function will take two arguments, our json and the iterator value that will serve as our index number for our json array. When the function is invoked, it’ll select our map svg, refresh the color pallete, and then redraw our map with the data referenced in the corresponding json. Because our .data() function is referencing json[i], we can cycle through the json array we just created and load each of json files in succession after a delay and a specific duration.

Now, we can set a time interval to define our “i” value by using the setInterval() function in Javascript. By adding the following code to the bottom of our ngOnInit() function, this process will start on page load:

ngOnInit() {...  let time = 1;  let interval = setInterval(() => {
if (time <= 3) {
this.transitionMap(this.jsons, time)
time++;
} else {
clearInterval(interval);
}
}, 2000);
}

If this works correctly, you should have a successfully animated choropleth map render on your page:

Final D3.js map animation | Image by author

Congratulations! What next?

If you’ve made it all the way to the end, I’m sure you can start to imagine some additional items you can add to this visualization — including the four json files for the second climate change scenario.

For code to add buttons, sliders, legend, and additional data, you can go to my Github and pull more of my code!

In the end, the goal of this article was to walk through the process of building out a fairly complex map animation for the web. We tackled a few different items, from data tidying in python, to basic web development principles, and various visualization techniques. I hope that you enjoyed working with this dataset, and I encourage those with questions to reach out at any point.

Additionally, if you enjoyed this post, feel free to check out more of my stories where I discuss other topics I’m passionate about — i.e. data engineering, machine learning, and more!

You can also visit my website at zach-alexander.com to get in touch with questions! Thanks for reading!

Read More …

[ad_2]


Write a comment