Application building with D3

John Firebaugh

MapBox
jfire
jfirebaugh

11,728	sloc
6,379	sloc tests
36,010	users
1,728	closed issues
41	translations
3	lead developers
1	designer

Typical Web Application

︷

Hipster Web Application

iD

︷

⃠

D3

	jQuery	D3
Ajax	✓	✓
Attributes	✓	✓
CSS	✓	✓
Data	✓	✓
Dimensions	✓	✗
Effects	✓	✓
Events	✓	✓
Manipulation	✓	✓
Selectors	✓	✓
Traversal	✓	✓
Data Binding	✗	✓

Declarative > Imperative

Promote the important; demote the unimportant

jQuery


$('.layers li').each(function() {
    var li = $(this);
    if (layerVisible(li.data('layer-id'))) {
        li.addClass('visible');
    } else {
        li.removeClass('visible');
    }
});

Here's an example of what I mean. iD has a list of background layers and indicates the currently visible layers via a visible class. Suppose you need to update the DOM to match the current state. In jQuery, you have to iterate over each list item, and call addClass or removeClass as appropriate. Highly imperative approach.

D3


items.classed('visible', function(layer) { return layer.visible; });

Here's the equivalent with D3. It's much shorter and -- once you understand the basics of D3 -- the intent is obvious. You're directly declaring the relationship between the visible class and the state of the underlying layer.

Imperative Declarative

If you start reading about declarative languages, you'll -- not surprisingly -- find a lot of nerds arguing about definitions. What makes one language or framework "imperative" and another one "declarative"? There's no hard and fast answer. It's more productive to look at it as a spectrum. So plotting tools for building and maintaining a relationship between data and a representation of that data on a web page on this spectrum, we'd say that Raw DOM APIs are about the most imperative way we could do it; jQuery is a bit more declarative, in that it introduces a concept of a collection of DOM elements that can be manipulated as a group; and a tool with data-binding like D3 is more declarative still. It's interesting to think about what's to the right of D3 on this continuum. I asked on Twitter. Some of the suggestions were XSL, frameworks like Angular and Ember that support two-way data binding, which I think is a really good answer, and Genghis Khan.

Model-View-*Whatever

Frameworks encourage separation of business logic from presentation

Ok, frameworks. Virtually all client-side frameworks these days are built around a set of patterns that separate the concerns of the application into parts that model the domain and business logic, parts that represent the user interface and controls with which users view and interact with the data, and some third thing that connects or coordinates or mediates between those two. There are various names with various shades of meaning for this third thing, like "Controller" or "Presenter" or "ViewModel". The differences are subtle and ultimately not very important, so folks have taken to just calling the third part "Whatever". The Model and View that are the important part, because they encourage you to separate your business logic from your presentation logic.

Model ⇒ View Change

Backbone: up to you
Ember/Angular: smart templates

I do want to look at one particular aspect of how views and models are coordinated, and that is how they deal with change. Both change in a model that needs to produce a change in a view, and the reverse, user interaction in the view that needs to be pushed back to the model. Start with the forward direction. Different frameworks take different approaches to solving this problem. Backbone leaves it up to you to rerender your view in response to relevant change events from the model. In practice, this means either rerendering the entire template, or introducing what is effectively a secondary parallel rendering path, where specific portions of the DOM are updated in response to fine-grained change events. Ember and angular recognized this as an antipattern, and provide a template implementation that's smart enough to automatically update in response to changes in the model -- providing you work within the constraints of the model and observer framework that is provided.

Master the Trinity

Enter
Update
Exit

Work-minimization Techniques

Declare static properties on the enter selection
Declare dynamic properties on the update selection
Batch updates: no individual "instance changed" events
Filter the selection to the entities which actually changed

Declare static structure on the enter selection: If something never changes, don't waste time setting it on every render. This is usually things like constant CSS class values, input type attributes. It's also often the overall structure of the DOM. Set it up once by chaining off the enter selection.
Declare dynamic properties on the update selection: Things that change go in the update selection. Take advantage of the fact that the enter selection is automatically merged into the update selection when you append.
Batch updates: no individual "instance changed" events: As a consequence of the iD data model, where mutations are expressed as action functions that take as input a graph and return a new, modified graph, it doesn't have change events on the level of individual model instances. There's one change event, indicating that an action took place and the current graph has changed. This is actually a good thing, because it means updates are batched. For instance, when you move the map feature that represents a rectangular building, you produce new versions of at least five entities: the four corner nodes, and the way containing those nodes that represents the building. It would be incredibly inefficient to trigger five different change events and update the SVG in five individual steps.
Filter the selection to the entities which actually changed: So, instead, iD has a single "graph changed" event. However, the history representation is sophisticated enough that it's possible to calculate the difference between two versions of the graph. We use that to update the minimal set of DOM elements by filtering the selection down to the subset that actually changed.

Model ⇐ View Change

Templates

︷

Discontinuity of Expressiveness

Continuity of Expressiveness

Problem: D3 is too big!

Solution: SMASH!

mbostock/smash


D3_FILES = \
	node_modules/d3/src/start.js \
	node_modules/d3/src/arrays/index.js \
	node_modules/d3/src/behavior/behavior.js \
	node_modules/d3/src/behavior/zoom.js \
	node_modules/d3/src/core/index.js \
	[...]
	node_modules/d3/src/end.js

js/lib/d3.v3.js: $(D3_FILES)
	node_modules/.bin/smash $(D3_FILES) > $@

Problem: D3 is too small!

	jQuery	D3
Dimensions	✓	✗

Solution: Write Plugins!


d3.selection.prototype.dimensions = function (dimensions) {
    if (!arguments.length) {
        var node = this.node();
        return [node.offsetWidth,
                node.offsetHeight];
    }
    return this.attr({width: dimensions[0], height: dimensions[1]});
};

More Plugins!

d3/d3-plugins
shawnbot/d3-bootstrap

Pick technology that fits your problem domain.

In the end, D3 was just right for the problem space of a rich editor for geographic data that required a custom data model -- in particular an immutable graph -- and tight control over how to respond to partial changes. This is not to say that jQuery, Backbone, Ember et al aren't the right choice for many other applications -- my advice is to pick the technology that best fits the requirements of your problem domain, rather than what's popular, commonly recommended as a best practice, ticks the most checkboxes in comparison matrixes, or is rightmost on the imperative-declarative continuum. That said, I think using D3 as the primary technology is a great fit for many more problem domains than is commonly realized. I encourage you to see if it fits yours.