Synchronous asynchronous JavaScript with ES6 Generators

Jan 27th 2015

In the one of my last articles I was looking at a few ways we could call asynchronous code in a synchronous flow. Each approach was trying to tackle the same problem of making 3 XHR requests (each to pull in a dummy tweet) and then displaying a status bar of total tweets, photos and favourites. Obviously we’d need the tweets before we could tally up any totals so we had to somewhat control these async requests in a synchronous fashion.

First we looked at nesting our requests into a ‘callback hell’ - a pattern familiar and avoided by Node.js developers. We then looked at 2 ways of breaking separate requests up into their own functions (creating slightly more code) and calling each consecutive function once a previous call was completed. And finally we looked at Promises and how they reduced lines of code and made things more legible.

I was pretty much sold on Promises as the answer to everything. That was, until I heard about _generator functions and iterator objects_, a new way to define functions that paused and continued and available in ECMAScript 6.

Now I’m not going to get into the nitty gritty of these ES6 iterators and generators, as far more intelligent people than me have already documented them in an much more elegant way than I ever could.

But what is the skinny?

A generator is defined by an * character
```
function *foo () { ... }
```
A generator function can be paused and resumed at any time in your app.
A generator function is paused by executing a yield keyword in the body of the function
```
function *foo () { yield 'bar'; }
```
When a generator function is called, it returns a Generator Iterator object
```
var iterator = foo(); // object
```
The Generator Iterator includes a next() method for stepping through the generator function
The Generator Iterator’s next() method returns an object with ‘value’ and ‘done’ properties.
And at the risk of delving too deep - generator functions can yield to other generator functions.

How about an example?

Ok, browser support for ES6 generator functions are few and far between. Firefox is ahead of the pack when it comes to ES6 implementation - however chrome is adding more and more support with every iteration. So with Firefox open, lets get into it.

Drop the following script into the Firefox console:

// generator function - declared with *
function *generator () {
  yield 'wow';
  yield 'this';
  yield 'is';
  yield 'sweet';
}

// create and iterator object
let iterator = generator();

Then call the iterator object’s next() method:

iterator.next(); // { value: "wow",  done: false }

Our iterator object has paused and returned the first value of the generator along with done: false. With done meaning:

“_Have I run to completion yet?_”

…and the result is false.

Now call the next() method 3 more times.

iterator.next(); // { value: "this",  done: false }
iterator.next(); // { value: "is",  done: false }
iterator.next(); // { value: "sweet",  done: false }

We’ve now yielded our 4 yield statements the body of our function, yet out last object is still saying done: false.

This is because we’re still paused at the very last yield statement.

Lets call next() one more time.

iterator.next(); // { value: undefined,  done: true }

OK, value: undefined. Well thats not super exciting. But we now know we’ve run to completion as done is now true.

If we wanted to, we could’ve returned something a bit more meaningful at the end of the function.

// generator function
function *generator () {
  yield 'wow';
  yield 'this';
  yield 'is';
  yield 'sweet';
  return 'dewd!'
}

// iterator object
let iterator = generator();

iterator.next(); // { value: "wow",  done: false }
iterator.next(); // { value: "this",  done: false }
iterator.next(); // { value: "is",  done: false }
iterator.next(); // { value: "sweet",  done: false }

// once more time to get the value from the return statement
iterator.next(); // { value: "dewd!",  done: true }

Like normal functions, Generator functions can take parameters. A simple example of this would be:

// generator function
function *generator (name) {
  return name + ', says hello';
}

// iterator object - passing 'Barry'
let iterator = generator('Barry');

iterator.next(); // { value: "Barry, says hello", done: true }

But you know what’s super cool? An Iterator’s next() method can also take a paramater.

// generator function
function *generator () {
  var name = yield 'Barry';
  return name + ', says hello';
}

// iterator object
let iterator = generator();

iterator.next(); // { value: "Barry",  done: false }
iterator.next('Sue'); // { value: "Sue, says hello", done: true }

One confusing step you’ll need to remember is when you call yield you’re pausing the generator. When you call iterator.next() again (with or without params) it will execute anything on the right-hand side of the current yield statement and continue through until it’s paused again.

You’ll notice in the example above, the first iterator.next(); is not passing through a value, as we’re simply pausing at yield. We then call iterator.next('Sue'); assigning the param Sue to name and we continue on to the return statement;

And that my friends is the most high level look at ES6 generator functions you’ll probably ever find.

Go ahead and visit some of the links mentioned above for more information, sanity and clarity.

Get on with it!

So now that we know what generator functions are and how they can be used to execute code in a controlled flow. Let’s now apply them to the same demo project as per my previous examples.

First I’m going to create a generator function called getTweets(). This function is going to yield 3 different XHR requests and jam the reponses (tweets) into an array.

let getTweets = function* () {
  let totalTweets = [];
  let data;

  // pause. On `iterator.next()` get the 1st tweet and carry on.
  data = yield get('https://api.myjson.com/bins/2qjdn');
  totalTweets.push(data);

  // pause. On `iterator.next()` get the 2nd tweet and carry on.
  data = yield get('https://api.myjson.com/bins/3zjqz');
  totalTweets.push(data);

  // pause. On `iterator.next()` get the 3rd tweet and carry on.
  data = yield get('https://api.myjson.com/bins/29e3f');
  totalTweets.push(data);

  // log the tweets
  console.log(totalTweets);
};

Wow, That’s it?

This is so much cleaner that any of my other approaches. It doesn’t even look like asynchronous code.

Before we can test this out - we’re going to need the a function named get(url) that will make the XHR request for us.

We have to make one small change to the existing get(url) function that I’ve been using for the previous examples.

Currently, the get(url) doesn’t actually return anything.
It makes a request and passes it to a callback.

We need get(url) to return something back to the yield statement so we can store this value in an array.

We need to turn get(url) into a Thunk.

At it’s most basic form, a thunk is a function that returns a function.

// Thunk
let get = function (url) {

  // return a function, passing in our callback
  return function (callback) {
    let xhr = new XMLHttpRequest();
    xhr.open('GET', url);
    xhr.onreadystatechange = function() {
      let response = xhr.responseText;
      if(xhr.readyState != 4) return;
      if (xhr.status === 200) {
        callback(null, response);
      }
      else {
        callback(response, null);
      }
    };
    xhr.send();
  };
};

So now we have our getTweets() generator function and our get(url) function.

Let’s test it out.


// create the iterator
var iterator = getTweets();

// call the first yield (pauses)
let result = iterator.next();

// our value the return function from get(url), so call it and pass in a callback
result.value(function(err, res){
  if (err) console.log('do something with this error', err);

  // get the response
  // We need to call next again and pass in the response to assign it back to a variable
  let result = iterator.next(res);
  result.value(function(err, res){
    if (err) console.log('do something with this error', err);

    // get the response
    // We need to call next again and pass in the response to assign it back to a variable
    let result = iterator.next(res);
    result.value(function(err, res){
      if (err) console.log('do something with this error', err);

      // get the response
      // We need to call next again and pass in the response to assign it back to a variable
      let result = iterator.next(res);
    })
  });
});

Here’s the gist for you to paste into the Firefox console.

Yikes!. It works, but we introduced a generator function to streamline our code, yet when we ran it - we’ve basically created another callback hell.

Also, having to explicitly call iterator.next() a specific amount of times is a real pain. What happens when I want to get another tweet or 10. Surely there’s got to be a cleverer way?!

Well, there is! We could create our own recursive function that calls the iterator’s next() method repeatedly, passing in any params until the iterator’s done property is true.

// A generator function runner
let runGenerator = function (generatorFunction) {

  // recursive next()
  let next = function (err, arg) {

    // if error - throw and error
    if (err) return it.throw(err);

    // cache it.next(arg) as result
    var result = it.next(arg);

    // are we done?
    if (result.done) return;

    // result.value should be our callback() function from the XHR request
    if (typeof result.value == 'function') {
      // call next() as the callback()
      result.value(next);
    }
    else {
      // if the response isn't a function
      // pass it to next()
      next(null, result.value);
    }
  }

  // create the iterator
  let it = generatorFunction();
  return next();
}

// intiliase and pass in a generator function
runGenerator(callSomeGeneratorFunction);

Drop the code from this gist into the Firefox console to try it out.

If you’re a Node.js developer and ‘rolling your own’ generator function runner isn’t your style, you can always rely on a tiny library named co.

But how can I use generators on the front-end, today?

Seeing as ES6 generators aren’t fully supported yet, you’re going to need a tool to convert your ES6 script into something most browsers can easily digest.

Regenerator is a simple CLI tool that makes it easy to convert your ES6 generator functions into usable ES5 goodness.

$ npm install regenerator
$
$ regenerator --include-runtime input.js > output.js

Example code

You can download the full code example (inc regenerator) here.

To build this example, CD into the project and run:

npm install