Creating, caching, and sending a PNG representation
To start, install ImageMagick: http://www.imagemagick.org/script/download.php. We will spawn a Node process to interface with the installed binary, below.
Before we build the graph dynamically, assume that there already exists an SVG definition stored on variable svg, which will contain a string similar to this:
<svg width="200" height="200">
<g transform="translate(100,100)">
<defs>
<radialgradient id="grad-0" gradientUnits="userSpaceOnUse" cx="0" cy="0" r="100">
<stop offset="0" stop-color="#7db9e8"></stop>
...
To convert that SVG to a PNG we would spawn a child process running the ImageMagick convert program, and stream our SVG data to the stdin of that process, which will output a PNG. In the example that follows we continue this idea to stream the generated PNG to the client.
We'll skip the server boilerplate -- suffice it to say that the server will be running on 8080 and will a client calling with some data to graph. What's important is how we generate and stream the pie chart back.
The client will send some querystring arguments indicating the values for this graph (such as 4,5,8, the relative size of the slices). What the server will do is generate a "virtual DOM" using the jsdom module, into which the D3 graphics library is inserted, as well as some javascript (pie.js in your code bundle) to take the values we have received and draw an SVG pie chart using D3, all within this server-side virtual DOM. We then grab that generated SVG code and convert it to a PNG using ImageMagick. In order to allow caching we store this PNG using a string filename formed from the cache values as a cacheKey, and while writing we pipe the streaming PNG back to the client:
jsdom.env({
...
html : `<!DOCTYPE html><p id="pie" style="width:${width}px;height:${height}px;"></p>`,
scripts : ['d3.min.js','d3.layout.min.js','pie.js'],
done : (err, window) => {
let svg = window.insertPie("#pie", width, height, values).innerHTML;
let svgToPng = spawn("convert", ["svg:", "png:-"]);
let filewriter = fs.createWriteStream(cacheKey);
filewriter.on("open", err => {
let streamer = new stream.Transform();
streamer._transform = function(data, enc, cb) {
filewriter.write(data);
this.push(data);
cb();
};
svgToPng.stdout.pipe(streamer).pipe(response);
svgToPng.stdout.on('finish', () => response.end());
// jsdom's domToHTML will lowercase element names
svg = svg.replace(/radialgradient/g,'radialGradient');
svgToPng.stdin.write(svg);
svgToPng.stdin.end();
window.close();
});
}
});
Recalling our discussion on streams, what is happening here should be clear. We generate a DOM (window) with jsdom, run the insertPie function to generate the SVG, and then spawn two streams: one to write the cache file, and one to the ImageMagick process. Using a TransformStream (both readable and writable) we implement its abstract _transform method to expect input from stdout of our ImageMagick stream, write that data to the local filesystem, and then re-push the data back into the stream, which is piped forward onto the response stream. We can now achieve the desired stream chaining:
svgToPng.stdout.pipe(streamer).pipe(response);
The client receives a pie graph, and a copy is written on the local file cache. In cases where the requested pie chart has already been rendered it can be directly streamed from a filesystem:
fs.exists(cacheKey, exists => {
response.writeHead(200, {
'Content-Type': 'image/png'
});
if (exists) {
fs.createReadStream(cacheKey).pipe(response);
return;
}
...
If you start the server and paste the following into your browser:
http://localhost:8080/?values=3,3,3,3,3
You should see a pie chart displayed:
While somewhat artificial, hopefully this shows how chains of different processes can be connected via streams, avoiding any intermediate storage in memory, which can be especially useful when passing data through and out of a highly trafficked network server.