D3 notes
What it is
Data-Driven Documents (D3) can be seen as a JS library geared towards for data visualisation.
It lives in the HTML5 world, is JS-driven, and goes to HTML or SVG, and CSS.
Has a declarative model (telling it what should be done, rather than all details how)
eases updating a visualisation with new data, altering the handlers that make it interactive,
makes (smooth) transitions easier, etc.
The declarative style implies you separate data from what to do with that data. In some ways it is more of a Domain Specific Language.
For moderately complex things it saves you a lot of micromanaging, though may feel indirect and a bit odd at first, and stays a bit odd when building unusually complex things.
To get an idea of whether it fits your uses:
- look at things other people have done: [1]
- look at community extensions (often convenience functions or specific visualisations): [2]
- Note that D3's DOM-centricity makes interactivity easier,
- but also means that for huge amounts of data, canvas/webGL solutions may be faster.
- (D3 itself could be rendered to them but you lose part of the functionality) [3] [4] [5]
Functional introduction
Syntaxwise it's a chained-call, alter-the-current-selection thing, like jQuery and various other JS libraries.
As such, you can do things like
d3.selectAll("div").style("background-color", "#afa");
...but that's not really the point.
Being declarative, D3 itself is responsible for all DOM alterations. (and yes, the basic model is that each value relates to a DOM node).
That implies:
- you need to tell it about new data
- you need to tell it how it should react...
- when values changes (DOM element needs to change)
- when entries need to be added (DOM elements need to be created)
- when entries need to be removed (DOM elements can be removed)
There are a few different introductions. This is one. I personally needed a few before all the details clicked. If you want to know low levels, some may work better than others.
More in terms of code:
- you do a selection
- to tell D3 where data should go
- both under which DOM parent to create new DOM nodes
- and joining the data to that
- yes, for one-shot charts, you create is an empty selection. Also for the first iteration in animated ones.
- data()
- it also ends up making three lists
- update
- enter
- exit
- operations directly on the selection object you get implies the update set
- and puts placeholders in the selection, for what needs to be created(verify)
- the enter and exit sets are also stored in there separately, so you can switch to them later{{verify}
- enter() switches to the selection of what needs to be created in the DOM
- exit() changes selection to the things that should leave the DOM
data() and enter()'s append()(verify) also set the actual data you handed in in DOM properties (note: not attributes), but you usually don't need to care about that. And at first it can be confusing because you're sorta setting on things that don't exist quite yet (because declarative)
This visualizes said selections.
This goes into implementation details - in more detail than most care about.
(Note: When creating a one-shot static diagram/table/whatnot, you can leave some details out, particularly the removing)
Example - create once
Assuming we already have an svg element in the DOM, because we put it in the html:
<svg id="stuff" width="300" height="300"></svg>
Let's look at...
var svg = d3.select("svg#stuff");
var circles = svg.selectAll("circle");
circles.data([10,20,34]).enter().append("circle")
.attr("r", function(d) {return d;} )
.attr("cx", function(d) { return 2*d; })
.attr("cy", function(d) { return 3*d; });
Before that enter(), the selection would be empty.
(Also, without that enter, that append would be an immediate request to append a single node, as in e.g. circles.append("circle").attr("r", "20").attr("cx", "30").attr("cy", "40");)
But let's use enter(), because that's how D3 really works. It changes the selection, to the placeholders we that should be added (within a selection that was settled by data())
- changing the meaning of append() to a declaratively implied "create one node for each placeholder in that selection"
The first loop this does seems a little funny. For example,
- selectAll's node selection will be empty - but it still settles the parent the later functions will be working under (the d3 selection object is more state than a list)
- the first data() doesn't immediately set node attributes - but does settle the state we need for enter() and exit()
...so once you know what happens behind the scenes it's not really a special case.
Example - with changes
function randint(mi, ma) { //inclusive
return mi + Math.floor(Math.random() * Math.floor(1+ma-mi));
}
function rand_xys() {
var ret=[];
for (i=0;i<randint(3,8);i++) {
ret.push([randint(0,300),randint(0,300),randint(10,20)]);
}
return ret;
}
function loop() {
var data = rand_xys();
var svg = d3.select('svg#stuff');
var circles = svg.selectAll("circle");
var selection = circles.data(data);
selection // data()'s leaves the current selection focused update, which is the effect of the next:
// (this is often chained directly to the .data() call, because it's shorter)
.attr("r", function(d) { return d[2]; })
.attr("cx", function(d) { return d[0]; })
.attr("cy", function(d) { return d[1]; });
selection.enter().append("circle") // data() it also left state letting us focusing on what should be added
.attr("r", function(d) { return d[2]; })
.attr("cx", function(d) { return d[0]; })
.attr("cy", function(d) { return d[1]; });
selection.exit().remove(); // ...and now on what should be removed
}
And start with
setInterval(loop, 1000);
Notes:
- people often use indenting for readability, suggesting which parts are related.
- The update-enter-exit style works on the new selection state that data() returns
- You can't chain all of data-enter-exit(verify) (and there's no reason to other than to make code look brief)
- Simple examples get away with chaining everything when they don't use exit, so combine only update and enter, which happens to be possible (for practical reasons, see the next point)
- the above avoids doing that chain, just to point out it's the selection object you're really working on. You can be a few lines shorter than this in practice.
- There is often duplication between what needs to happen to the 'just added' and 'to be updated' set
- Since D3 v4 you can use merge(), which creates a new selection from both the update and enter sets. This is convenient (but optional) because there's often things you want to equally do to both.
- Earlier, in D3 v2 and v3, there was a trick: append()ing to enter would also alter the update selection, meaning that if you did the DOM work for update afterwards it would do both. This is hidden semantics though, so not great.
Transitions
Transitions make visual sense when such and updates move things around.
They are finite sequences, often from current state to new state.
To continue the above example, replace loop() with:
function loop() {
var data = rand_xys();
var svg = d3.select('svg#stuff');
var circles = svg.selectAll("circle");
// a separate object for reusability, allows staggering
var t = d3.transition().duration(500);
var s = circles.data(data);
s .transition(t)
.attr("r", function(d) { return d[2]; })
.attr("cx", function(d) { return d[0]; })
.attr("cy", function(d) { return d[1]; });
s.enter().append("circle")
.attr("cx", function(d) { return d[0]; })
.attr("cy", function(d) { return d[1]; })
.style("fill-opacity", 1e-6)
.transition(t)
.style("fill-opacity", 1)
.attr("r", function(d) { return d[2];});
s.exit().transition(t)
.style("fill-opacity", 1e-6)
.style("fill", 'red')
.attr("r", 0)
.remove();
}
And start with
setInterval(loop, 1000);
Notes:
- In general,
- things you want to happen immediately (e.g. initial stuff, class changes) go before the .transition,
- animated things after
- (consider you e.g. don't want new things to animate from position 0,0)
- these transitions can differ for enter, update, and exit sets.
- interrupting an ongoing transition is possible
- (here e.g. by making the transition time longer than the timer interval)
- but keep in mind that aborts things (e.g. opacity, position, radius) in half-transitioned state, which may well be (and stay) incorrect
- when an attribute is set with a function, D3 calls it
- first argument is the joined data, the second is the index within the selection
What happens:
- update set: radius and position for the update set are animated, so they move around while they grow/shrink
- enter set: position is set, radius is animated, so they grow at their intended spot, and fade in (which you can't really see until it's much slower)
- exit set: shrink to no radius, fade to red and to transparency (which you can't really see); then remove from the DOM
For a more interesting example, see
https://bl.ocks.org/mbostock/3808234
Scales and sizes
Responsive D3
Going a little deeper
On node order
By default, entering elements are added to the end, other value assignments will shift location.
When there is no special meaning to where data sits, you don't need to care.
In some cases there is good reason to keep the same content in the same node while you update.
As an analogy
- the first case is just bringing in empty chairs and removing unnecessary ones.
You're telling people to scoot over to a new chair whenever someone leaves.
- in the second case you never tell someone who stays to shift over, you just assign people to the new chairs and removing the ones for people that left.
...the analogy isn't perfect, in part because node order isn't necessarily draw order.
When doing animations, it's easier to visually keep track, which is about the draw order. (And for certain complex structures you really don't want to do everything from scratch for each minor change)
This is what data()'s second argument, key, is for.
Check out this series:
- https://bl.ocks.org/mbostock/3808218 - basics of adding and removing
- https://bl.ocks.org/mbostock/3808221 - uses key to keep things in the same nodes
- https://bl.ocks.org/mbostock/3808234 - adds animated transitions to show entering/leaving nodes
On nested data and nested selections
Going a little wider
Shapes
Layouts
A layout is basically a function that takes your data as input, and adds things like position and size.
Other stuff
Gallery of things to steal from
https://github.com/d3/d3/wiki/Gallery
Technical notes
parent size, and/or responding to resizes
https://stackoverflow.com/questions/16265123/resize-svg-when-window-is-resized-in-d3-js