Getting started with D3.js
First, download the latest version of D3 from the official website http://d3js.org/.
Or, to link directly to the latest release, copy this snippet:
<script src="http://d3js.org/d3.v3.min.js"></script>
In the basic examples, we can just open our HTML document in a web browser to view it. But when we need to load external data sources, we need to publish the folder on a web server such as Apache, nginx, or IIS. Python provides us with an easy way to run a web server with http.server; we just need to open the folder where our D3 files are present and execute the following command in the terminal.
$ python3 –m http.server 8000
In Windows, you can use the same command by removing the number 3 from python.
> python –m http.server 8000
The following examples are based on Mike Bostock's reference gallery, which can be found at https://github.com/mbostock/d3/wiki/Gallery.
All the codes and datasets of this chapter can be found in the author's GitHub repository at https://github.com/hmcuesta/PDA_Book/tree/master/Chapter3.
Bar chart
Probably, the most common visualization tool is the bar chart. As we can see in the following figure, the horizontal axis (X) represents the category data and the vertical axis (Y) represents a discrete value. We can see the count of pokemon by type with a random sort.
Tip
Discrete data are values that can only take certain values; in this case it is the number of pokemon by type.
In the following example, we process the pokemon list in JSON format (see Chapter 2, Working with Data) and we get the sum of pokemon by type, sorted by number in ascending order and then save the result in a CSV format. After the data processing, we visualize the result in a bar chart.
The first three records of the JSON file (pokemon.json) look like the following records:
[
{
"id": " 001",
"typeTwo": " Poison",
"name": " Bulbasaur",
"type": " Grass"
},
{
"id": " 002",
"typeTwo": " Poison",
"name": " Ivysaur",
"type": " Grass"
},
{
"id": " 003",
"typeTwo": " Poison",
"name": " Venusaur",
"type": " Grass"
},
. . . ]
In this preprocessing stage, we will use Python to turn the JSON file into a CSV format. We will perform an aggregation to get the number of each category of pokemon sorted in ascending order. After we get the resultant CSV file, we will start with the visualization in D3.js. The code for the preprocessing is shown as follows:
# We need to import the necessary modules.
import json
import csv
from pprint import pprint
#Now, we define a dictionary to store the result
typePokemon = {}
#Open and load the JSON file.
with open("pokemon.json") as f:
data = json.loads(f.read())
#Fill the typePokemon dictionary with sum of pokemon by type
for line in data:
if line["type"] not in typePokemon:
typePokemon[line["type"]] = 1
else:
typePokemon[line["type"]]=typePokemon.get(line["type"])+1
#Open in a write mode the sumPokemon.csv file
with open("sumPokemon.csv", "w") as a:
w = csv.writer(a)
#Sort the dictionary by number of pokemon
#writes the result (type and amount) into the csv file
for key, value in sorted(typePokemon.items(),
key=lambda x: x[1]):
w.writerow([key,str(value)])
#finally, we use "pretty print" to print the dictionary
pprint(typePokemon)
The result of the preprocessing can be seen in the following table. Each row has two values, the type and the amount of pokemon of a particular type.
To start working on D3, we need to create a new HTML file with the basic structure (head, style, and body). Next, we will include the styles and the script section as shown in the following steps:
In the CSS, we specified the style for the axis line, the font family, and size for the body and the bar color.
<style>
body {
font: 14px sans-serif;
}
.axis path,
.axis line {
fill: none;
stroke: #000;
shape-rendering: crispEdges;
}
.x.axis path {
display: none;
}
.bar {
fill: #0489B1;
}
</style>
Tip
We can define the colors in CSS using a hexadecimal code such as #0489B1 instead of the literal name "blue"; in the following link, we can find a color selector http://www.w3schools.com/tags/ref_colorpicker.asp.
Inside the body tag, we need to refer to the library,
<body> <script src="http://d3js.org/d3.v3.min.js"></script>
The first thing we need to do is define a new SVG canvas (<svg>) with a width and height of 1000 x 500 pixels, inside the body section of our HTML document.
var svg = d3.select("body").append("svg")
.attr("width", 1000)
.attr("height", 500)
.append("g")
.attr("transform", "translate(50,20)");
The transform attribute will help us to translate, rotate, and scale a group of elements (g). In this case we want to translate (move) the position of the margins (left and top) on the canvas with translate("left", "top"). We need to do this because we will need space for the labels in the axis X and Y of our visualization.
Now, we need to open the file sumPokemon.csv and read the values from it. Then, we will create the variable data with two attributes type and amount according to the structure of the CSV file.
The d3.csv method will perform an asynchronous request. When the data is available, a callback function will be invoked. In this case we will iterate the list data and we will convert the amount column to number (d.amount = +d.amount).
d3.csv("sumPokemon.csv", function(error, data) {
data.forEach(function(d) {
d.amount = +d.amount;
});
Now, we will set a labeled X axis (x.domain) using the map function to get all the type names of the pokemons. Next, we will use the d3.max function to return the maximum value of each type of pokemon for the Y axis (y.domain).
x.domain(data.map(function(d) { return d.type; }));
y.domain([0, d3.max(data, function(d) { return d.amount; })]);
Now, we will create an SVG Group Element which is used to group SVG elements together with the tag <g>. Then we use the transform function to define a new coordinate system for a set of SVG elements by applying a transformation to each coordinate specified in this set of SVG elements.
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0,550)")
.call(xAxis);
svg.append("g")
.attr("class", "y axis")
.call(yAxis)
.append("text")
.attr("transform", "rotate(-90)")
.attr("y", 6)
.attr("dy", ".71em")
.style("text-anchor", "end")
.text("Amount");
Finally, we need to generate .bar elements and add them to svg, then with the data(data) function, for each value in data, we will call the .enter() function and add a rect element. D3 allows selecting groups of elements for manipulation through the selectAll function.
Tip
In the following link, we can find more information about selections https://github.com/mbostock/d3/wiki/Selections.
svg.selectAll(".bar")
.data(data)
.enter().append("rect")
.attr("class", "bar")
.attr("x", function(d) { return x(d.type); })
.attr("width", x.rangeBand())
.attr("y", function(d) { return y(d.amount); })
.attr("height", function(d) { return height - y(d.amount); });
}); // close the block d3.csv
In order to see the result of our visualization, we need to visit the URL http://localhost:8000/bar-char.html, the result is shown in the following screenshot:
Pie chart
The purpose of the pie chart is to communicate proportions. The sum of all wedges represents a whole (100 percent). Pie charts help us to understand the distribution of proportions in an easy way. In this example, we will use the unordered list of pokemons by type sumPokemon.csv, which can be found at https://github.com/hmcuesta/PDA_Book/tree/master/Chapter3.
We need to define the font family and size for the labels.
<style>
body {
font: 16px arial;
}
</style>
Inside the body tag we need to refer to the library,
<body> <script src="http://d3js.org/d3.v3.min.js"></script>
First, we define the size (width, height, and radius) of the work area.
var w = 1160,
h = 700,
radius = Math.min(w, h) / 2;
Now, we will set a range of color that will be used in the chart.
var color = d3.scale.ordinal()
.range(["#04B486", "#F2F2F2", "#F5F6CE", "#00BFFF"]);
The function d3.svg.arc() creates a circle with an outer radius and an inner radius. See pie charts given later.
var arc = d3.svg.arc()
.outerRadius(radius - 10)
.innerRadius(0);
The function d3.layout.pie() specifies how to extract a value from the associated data.
var pie = d3.layout.pie()
.sort(null)
.value(function(d) { return d.amount; });
Now, we select the element body and create a new element <svg>.
var svg = d3.select("body").append("svg")
.attr("width", w)
.attr("height", h)
.append("g")
.attr("transform", "translate(" + w / 2 + "," + h / 2 + ")");
Next, we need to open the file sumPokemon.csv and read the values from the file and create the variable data with two attributes type and amount.
d3.csv("sumPokemon.csv", function(error, data) {
data.forEach(function(d) {
d.amount = +d.amount;
});
Finally, we need to generate the .arc elements and add them to svg, then with the data(pie(data)) function, for each value in data we will call the .enter() function and add a g element.
var g = svg.selectAll(".arc")
.data(pie(data))
.enter().append("g")
.attr("class", "arc");
Now, we need to apply the style, color, and labels to the group g.
g.append("path")
.attr("d", arc)
.style("fill", function(d) { return color(d.data.type); });
g.append("text")
.attr("transform", function(d) { return "translate(" + arc.centroid(d) + ")"; })
.attr("dy", ".60em")
.style("text-anchor", "middle")
.text(function(d) { return d.data.type; });
}); // close the block d3.csv
In order to see the result of our visualization we need to visit the URL http://localhost:8000/pie-char.html. The result is shown in the following screenshot:
In the following figure, we can see the pie chart with variation in the attribute inner Radius of 200 pixels in the function arc.
var arc = d3.svg.arc()
.outerRadius(radius - 10)
.innerRadius(200);
Scatter plot
Scatter plot is a visualization tool based in cartesian space with coordinates of axis X, Y between two different variables, in this case it can be value, categorical, or time represented in data points. Scatter plot allows us to see relationships between the two variables.
In the following figure, we can see a scatter plot, where each data point has two coordinates X and Y. The horizontal axis can take category or time values and in the vertical axis we represent a value.
In this example, we generate 20 random points (constrained in a range of 700 x 500) in a bi-dimensional array in JavaScript using the function Math.random() and store the result in the variable data.
var data = [];
for(var i=0; i < 20; i++ ){
var axisX = Math.round(Math.random() * 700);
var axisY = Math.round(Math.random() * 500);
data.push([axisX,axisY]);
}
Now, we select the element body and create a new element <svg> and define its size.
var svg = d3.select("body")
.append("svg")
.attr("width", 700)
.attr("height", 500);
We use the selector to create a circle for each data point in the variable data, defining the coordinate X as cx and the coordinate Y as cy, define the radius r to 10 pixels and pick a color fill.
svg.selectAll("circle")
.data(data)
.enter()
.append("circle")
.attr("cx", function(d) { return d[0]; })
.attr("cy", function(d) { return d[1]; })
.attr("r", function(d) { return 10; })
.attr("fill", "#0489B1");
Finally, we create the label of each point including the value of a coordinate X, Y in text format. We select a font family, color, and size.
svg.selectAll("text")
.data(data)
.enter()
.append("text")
.text(function(d) {return d[0] + "," + d[1]; })
.attr("x", function(d) {return d[0]; })
.attr("y", function(d) {return d[1]; })
.attr("font-family", "arial")
.attr("font-size", "11px")
.attr("fill", "#000000");
In the following screenshot, we see the scatter plot that will be seen in our web browser.
Single line chart
A line chart is a visualization tool that displays continuous data as a series of points connected by a straight line. It is similar to a scatter plot but in this case the points have a logical order and the points are connected—often used for time series' visualization. A time series is a sequence of observations of the physical world in a regular time span. Time series help us to understand trends and correlations. As we can see in the following figure, the vertical axis represents the value data and the horizontal axis represents the time.
For this example we will use the log of USA/CAD historical exchange rates from March 2008 to March 2013 with 260 records.
Tip
In the following link we can find the Historical Exchange Rates log to download http://www.oanda.com/currency/historical-rates/.
The first seven records of the CSV file (line.csv) look like the following records:
date,usd 3/10/2013,1.0284 3/3/2013,1.0254 2/24/2013,1.014 2/17/2013,1.0035 2/10/2013,0.9979 2/3/2013,1.0023 1/27/2013,0.9973 . . .
We need to define the font family and size for the labels and the style for the axis line.
<style>
body {
font: 14px sans-serif;
}
.axis path,
.axis line {
fill: gray;
stroke: #000;
}
.line {
fill: none;
stroke: red;
stroke-width: 3px;
}
</style>
Inside the body tag we need to refer to the library,
<body> <script src="http://d3js.org/d3.v3.js"></script>
We will define a format parser for the date value with d3.time.format. In this example, we have the data as: Month/Day/Year—%m/%d/%Y (for example, 1/27/2013). Where, %m represents the month as a decimal number from 01 to 12, %d represents the day of the month as a decimal number from 01 to 31, and %Y represents the year with century as a decimal number.
var formatDate = d3.time.format("%m/%d/%Y").parse;
Tip
To find out more about time formatting, please refer to the link https://github.com/mbostock/d3/wiki/Time-Formatting/.
Now, we define the X and Y axis with a width of 1000 pixels and height 550 pixels.
var x = d3.time.scale()
.range([0, 1000]);
var y = d3.scale.linear()
.range([550, 0]);
var xAxis = d3.svg.axis()
.scale(x)
.orient("bottom");
var yAxis = d3.svg.axis()
.scale(y)
.orient("left");
The line element defines a line segment that starts at one point and ends at another.
Tip
In the following link we can find the reference of SVG shapes https://github.com/mbostock/d3/wiki/SVG-Shapes/.
var line = d3.svg.line()
.x(function(d) { return x(d.date); })
.y(function(d) { return y(d.usd); });
Now, we select the element body and create a new element <svg> and define its size.
var svg = d3.select("body")
.append("svg")
.attr("width", 1000)
.attr("height", 550)
.append("g")
.attr("transform", "translate("50,20")");
Then, we need to open the file line.csv and read the values from the file and create the variable data with two attributes date and usd.
d3.csv("line.csv", function(error, data) {
data.forEach(function(d) {
d.date = formatDate(d.date);
d.usd = +d.usd;
});
We define the date in horizontal axis (x.domain) and in the vertical axis (y.domain) and set our value axis with the exchange rates value usd.
x.domain(d3.extent(data, function(d) { return d.date; }));
y.domain(d3.extent(data, function(d) { return d.usd; }));
Finally, we add the groups of points and the labels in the axis.
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0,550)")
.call(xAxis);
svg.append("g")
.attr("class", "y axis")
.call(yAxis)
svg.append("path")
.datum(data)
.attr("class", "line")
.attr("d", line);
}); // close the block d3.csv
In the following screenshot we can see the result of the visualization.
Multi-line chart
In a single variable we can see trends but often we need to compare multiple variables and even find correlations or cluster trends. In this example, we will evolve the last example to work with multi-line chart. In this case we will use data from historical exchange rates from USA, EUR, and GBP.
Tip
In the following link we can find the Historical Exchange Rates log to download http://www.oanda.com/currency/historical-rates/.
The first five records of the CSV file (multiline.csv) look like the following records:
date,USD/CAD,USD/EUR,USD/GBP 03/10/2013,1.0284,0.7675,0.6651 03/03/2013,1.0254,0.763,0.6609 2/24/2013,1.014,0.7521,0.6512 2/17/2013,1.0035,0.7468,0.6402 02/10/2013,0.9979,0.7402,0.6361 . . .
We need to define the font family and size for the labels and the style for the axis line.
<style>
body {
font: 18px sans-serif;
}
.axis path,
.axis line {
fill: none;
stroke: #000;
}
.line {
fill: none;
stroke-width: 3.5px;
}
</style>
Inside the body tag we need to refer to the library,
<body> <script src="http://d3js.org/d3.v3.js"></script>
We will define a format parser for the date value with d3.time.format. In this example we have the data as follows Month/Day/Year—%m/%d/%Y (for example, 1/27/2013).
var formatDate = d3.time.format("%m/%d/%Y").parse;
Now, we define the X and Y axis with a width of 1000 pixels and height 550 pixels.
var x = d3.time.scale()
.range([0, 1000]);
var y = d3.scale.linear()
.range([550, 0]);
We define an array of color for each line.
var color = d3.scale.ordinal()
.range(["#04B486", "#0033CC", "#CC3300"]);
var xAxis = d3.svg.axis()
.scale(x)
.orient("bottom");
var yAxis = d3.svg.axis()
.scale(y)
.orient("left");
var line = d3.svg.line()
.interpolate("basis")
.x(function(d) { return x(d.date); })
.y(function(d) { return y(d.currency); });
Now, we select the element body and create a new element <svg> and define its size.
var svg = d3.select("body")
.append("svg")
.attr("width", 1100)
.attr("height", 550)
.append("g")
.attr("transform", "translate("50,20)");
Then, we need to open the file multiLine.csv and read the values from the file and create the variable data with two attributes date and color.domain.
d3.csv("multiLine.csv", function(error, data) {
color.domain(d3.keys(data[0]).filter(function(key)
{return key !== "date"; }));
Now, we apply the format function to all the column date.
data.forEach(function(d) {
d.date = formatDate(d.date);
});
Then, we define currencies as separated array for each color line.
var currencies = color.domain().map(function(name) {
return {
name: name,
values: data.map(function(d) {
return {date: d.date, currency: +d[name]};
})
};
});
x.domain(d3.extent(data, function(d) { return d.date; }));
y.domain
([d3.min(currencies, function(c) { return d3.min(c.values, function(v) { return v.currency; }); }),
d3.max(currencies, function(c) { return d3.max(c.values, function(v) { return v.currency; }); })
]);
Now, we add the groups of points as well as the color and labels for each line.
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0,550)")
.call(xAxis);
svg.append("g")
.attr("class", "y axis")
.call(yAxis)
var country = svg.selectAll(".country")
.data(currencies)
.enter().append("g")
.style("fill", function(d) { return color(d.name); })
.attr("class", "country");
Finally, we add legend to multi-line series chart.
country.append("path")
.attr("class", "line")
.attr("d", function(d) { return line(d.values); })
.style("stroke", function(d) { return color(d.name); });
country.append("text").datum(function(d)
{ return {name: d.name, value: d.values[d.values.length - 1]}; })
.attr("transform", function(d) {
return "translate("+ x(d.value.date)+","+ y(d.value.currency)+")";
})
.attr("x", 10)
.attr("y", 20)
.attr("dy", ".50em")
.text(function(d) { return d.name; });
}); // close the block d3.csv
In the following screenshot we can see the result of the visualization.
