Site menu Javascript: Promises, async and await
e-mail icon
Site menu

Javascript: Promises, async and await

e-mail icon

There are many texts already about this subject, but I haven't found a "one-stop" article, so I ended up writing something like it. All code examples can be found in this Zip.

Callback hell

The code below implements two artificially asynchronous operations, op1 and op2. The delay is simulated with setTimeout() and failures are randomly simulated as well. They can be used the same way as any other callback-based API.

// Source inside Zip: cb.js

function cb_op1(callback)
{
	setTimeout(() => {
		let n = Math.random();
		if (n >= 0.75) {
			callback("Failed");
		} else {
			callback(null, n);
		}
	}, 1000);
}

function cb_op2(x, callback)
{
	setTimeout(() => {
		let n = Math.random();
		if (n <= 0.25) {
			callback("Failed");
		} else {
			callback(null, Math.floor(x / 0.75 * 1000));
		}
	}, 1000);
}

console.log("Start");

cb_op1((err, res) => {
	if (err) {
		console.log("Error op1: " + err);
		return;
	}
	cb_op2(res, (err, res) => {
		if (err) {
			console.log("Error op2: " + err);
			return;
		}
		console.log("Success: " + res);
	});
});

Callbacks are easy to understand on the first moment, but the code gets messy very fast when there is a chain of dependencies. In the case above, op2 depends on op1 and the callbacks were nested. If the chain had fifteen operations, it would be impossible to follow the execution path.

Many techniques and frameworks have been developed to reduce this clutter. The standard the emerged from these efforts is the promise. There are many implementations of promise, and current browsers and Node.js bundle a native Promise class.

The same code rewritten to use promises:

// Source inside Zip: promise.js

function op1()
{
	return new Promise((fulfill, reject) => {
		setTimeout(() => {
			let n = Math.random();
			if (n >= 0.75) {
				reject("op1 failed");
			} else {
				fulfill(n);
			}
		}, 1000);
	});
}

function op2(x)
{
	return new Promise((fulfill, reject) => {
		setTimeout(() => {
			let n = Math.random();
			if (n <= 0.25) {
				reject("op2 failed");
			} else {
				fulfill(Math.floor(x / 0.75 * 1000));
			}
		}, 1000);
	});
}

console.log("Start");

op1()
.then(op2)
.then((res) => {
	console.log("Success: " + res);
})
.catch((err) => {
	console.log("Error: " + err);
});

The idea of promises is to make the API client's life easier, in particular when there is a long chain of dependencies. Just add links to the chain: op1.then(op2).then(op3).then(op4)... In the code above, the last operation is just printing the result, so we pass a regular anonymous function.

Promises do not make things automatically easier for the API developer. For example, in op1 and op2 there is an "impedance mismatch" between setTimeout() and Promise because the former is an "old-style" API that takes a callback instead of return a promise. (BTW there are many libraries that bundle timeouts as promises, so in practice you would not have to write the adaptation code yourself.)

Finally, there is the .catch() method that handles an unfulfilled (rejected) promise. There may be more than one catch() along the chain, up to on per promise, should the error handling be specialized for each step. In thee example code, there is just one catch-all handler at the end of the chain.

Unless you are absolutely sure that all promises will be fulfilled, there should be at least one catch() at the end of the chain. Recent versions of Node.js throw an exception when a rejected promise is not handled (the old way was to ignore this condition).

Making promises

It is worthwhile to look in detail how one makes a promise:

function op1()
{
	return new Promise((fulfill, reject) => {
		... async operation ...
		... if operation succeeds ...
		fulfill(result);
		... if operation fails ...
		reject("fail because of XYZ");
	});
}

The promise constructor expects a callback that takes two function arguments, which I called fulfill and reject. The names don't matter and different coding styles use different names.

These functions come from the innards of the Promise class, but this doesn't matter either. All that matters is the asynchronous operation calls fulfill(result) or reject(error). If neither is called, the promise hangs forever.

Once the promise is created, the callback is called right away. That's why the promise chain is like op1().then(op2)... The promise op1 starts immediately, but op2 can only be created if op1() is fulfilled.

Since I like to use tabs for code indentation, and op1() does nothing but return a promise, my particular style ist o declare the promise-function like this:

function op1() { return new Promise((fulfill, reject) => {
	... code ...
});}

Matching impedances

There are many Javascript/Node.js libraries based on "old-style" callbacks. The trend is to migrate to promises, but there will still be many situations where adaptation code is required.

The example below illustrates how to repackage a callback-based operation (cb_op1) as a promise (op1).

// Source inside Zip: cbpromise.js

function cb_op1(callback)
{
	setTimeout(() => {
		let n = Math.random();
		if (n >= 0.75) {
			callback("op1 failed");
		} else {
			callback(null, n);
		}
	}, 1000);
}

function op1()
{
	return new Promise((fulfill, reject) => {
		cb_op1((err, res) => {
			if (err) {
				reject(err);
			} else {
				fulfill(res);
			}
		});
	});
}

But it is not necessary to do this manually; at least Node.js bundles the method util.promisify() that does exactly the same thing:

// Source inside Zip: cbpromisify.js

let op1 = util.promisify(cb_op1);

You still need to write adaptation code manually when the adapted API deviates from the callback standard e.g. the callback does not have the signature (err, res). But always look for a ready-made adaptation that can just be npm install'ed.

You may well need the inverse adaptation: given a promise-based API, make it compatible with a callback:

// Source inside Zip: promisecbadapt.js

function op1()
{
	return new Promise((fulfill, reject) => {
		setTimeout(() => {
			let n = Math.random();
			if (n >= 0.75) {
				reject("op1 failed");
			} else {
				fulfill(n);
			}
		}, 1000);
	});
}

function cb_op1(cb)
{
	op1()
		.then((res) => { cb(null, res) })
		.catch((err) => { cb(err) });
}

Again, it is generally not necessary to write manual adaptation code since the method util.callbackify() does all the work:

// Source inside Zip: promisecbfy.js

let cb_op1 = util.callbackify(op1);

Another adaptation trick is based on the fact that a Promise is just an object that happens to implement .then(). It does not need to be an instance of the native Promise class. The code below is a primitive example of "artificial promise" that can be mixed with "true" promises and async/await keywords:

// Source inside Zip: async2.js

function op1()
{
	let p = {};

	setTimeout(() => {
		let n = Math.random() * 0.75;
		p.handler(n);
	}, Math.random() * 1000);

	p.then = (handler) => {
		console.log("p.then called");
		p.handler = handler;
	};

	return p;
}

Before you start using the new keywords async and await, make sure you understand promises inside out, since the basis of the new syntax is the promise, and from time to time you will need to declare explicit promises due to limitations of the current syntax.

Async and await

Like it or not, asynchronous programming has been earning adepts and native support in many languages, mostly in the form of co-routines. The new keywords async and await allow to create and use promises with a cleaner syntax.

The code below reimplements our old example, lightly modified and using async/await instead of promises. Also, op2 invokes op1 directly so we can show off the await keyboard.

// Source inside Zip: async.js

function op1()
{
	return new Promise((fulfill, reject) => {
		setTimeout(() => {
			let n = Math.random();
			if (n >= 0.75) {
				reject("op1 failed");
			} else {
				fulfill(n);
			}
		}, 1000);
	});
}

async function op2()
{
	let x = await op1();

	let n = Math.random();
	if (Math.random()  <= 0.25) {
		throw "op2 failed";
	}

	return Math.floor(x / 0.75 * 1000);
}

console.log("Start");

op2()
	.then((res) => {
		console.log("Sucess: " + res)
	})
	.catch((err) => {
		console.log("Error: " + err)
	});

Some points about the example code:

The program's main scope is never async, so we could not call await op2() as it seems natural to do. But we can declare and invoke an anonymous async function to remedy this:

// Source inside Zip: async3.js

console.log("Start");

(async () => {
	try {
		let res = await op2();
		console.log("Sucess: " + res)
	} catch (err) {
		console.log("Error: " + err)
	}
})();

In the next example, we modify op2 so the dependency chain op1-op2 is again established by the main scope:

// Source inside Zip: async4.js

async function op2(x)
{
	let n = Math.random();
	if (Math.random()  <= 0.25) {
		throw "op2 failed";
	}

	return Math.floor(x / 0.75 * 1000);
}

console.log("Start");

(async () => {
	try {
		let x = await op1();
		let res = await op2(x);
		console.log("Sucess: " + res)
	} catch (err) {
		console.log("Error: " + err)
	}
})();

In the code above, op2 is a synchronous function, but it must be handled as a promise since it was declared async. We await'ed it and case is solved.

Even though async always returns an instance of the native Promise class, await accepts any implementation of promise, that is: any object that implements the method then().

The function passed as argument to a new Promise can be async, for example:

function op() { return new Promise(async (fulfill, reject) => {
	... code ...
});}

This allows the inner code to use await. But it is clear that a convoluted promise definition like this is in dire need of refactoring e.g. make op() an async function and do away with the explicit new Promise.

The following code tries to be a refactored version of former examples, by restricting adaptation code and explicit promises to an auxiliary function, and using async/await throughout the code.

// Source inside Zip: async5.js

function Timeout(to)
{
	return new Promise((fulfill, reject) => {
		setTimeout(() => { fulfill(); }, to);
	});
}

async function op1()
{
	await Timeout(1000);
	let n = Math.random();
	if (n >= 0.75) {
		throw("op1 failed");
	}
	return n;
}

async function op2()
{
	let x = await op1();
	if (Math.random() <= 0.25) {
		throw "op2 failed";
	}
	return Math.floor(x / 0.75 * 1000);
}

console.log("Start");

(async () => {
	try {
		let res = await op2();
		console.log("Sucess: " + res)
	} catch (err) {
		console.log("Err: " + err)
	}
})();

The async/await syntax is not perfect yet. For example, it does not allow to wait for many promises in parallel. To do that, it is still necessary to create a super-promise (Promise.all) explicitly. The proposed syntax await* for this case was not accepted. The huge number of callback-based APIs is also a hurdle for adoption of the new syntax.

Bonus: how to implement a promise

One way to better understand how a promise works, is to implement the Promise class from scratch. The following code reimplements the same old boring example, but this time using a "homebrew" promise class.

// Source inside Zip: brew.js

let IDLE = 0;
let RUNNING = 1;
let FULFILLED = 2;
let REJECTED = 3;

class HomebrewPromise {
	constructor(cmd, hold) {
		this.state = IDLE;
		this.result = null;
		this.err = null;
		this.cmd = cmd;
		this.then_promise = null;
		this.catch_cmd = null;
		if (! hold) {
			this.__run();
		}
	}

	__run() {
		this.state = RUNNING;
		let self = this;

		let fulfilled = (result) => {
			self.state = FULFILLED;
			self.result = result;
			self.__do_then();
		}

		let rejected = (err) => {
			self.state = REJECTED;
			self.err = err;
			self.__do_catch();
		}

		setTimeout(() => {
			this.cmd(fulfilled, rejected);
		}, 0);
	}

	then(cmd) {
		let self = this;

		this.then_promise = new HomebrewPromise(
				(fulfill, reject) => {
			let runcmd = cmd(self.result);
			if (! runcmd || ! runcmd.then) {
				// cmd not a Promise
				// pass result forward
				fulfill(self.result);
			} else {
				// cmd is a Promise
				runcmd.then(fulfill).catch(reject);
			}
		}, true);

		return this.then_promise;
	}

	catch(cmd) {
		this.catch_cmd = cmd;
		return this;
	}

	__do_then() {
		if (this.then_promise) {
			this.then_promise.__run();
		}
	}

	__do_catch() {
		if (this.catch_cmd) {
			this.catch_cmd(this.err);
		} else if (this.then_promise) {
			this.then_promise.__forwarded_error(this.err);
		} else {
			throw "Unhandled rejection: " + this.err;
		}
	}

	__forwarded_error(err) {
		this.state = REJECTED;
		this.err = err;
		this.__do_catch();
	}
}

function op1()
{
	return new HomebrewPromise((fulfill, reject) => {
		console.log("op1 started");
		setTimeout(() => {
			let n = Math.random();
			if (n >= 0.75) {
				reject("op1 failed");
			} else {
				fulfill(n);
			}
		}, 1000);
	});
}

function op2(x)
{
	return new HomebrewPromise((fulfill, reject) => {
		console.log("op2 started");
		setTimeout(() => {
			let n = Math.random();
			if (n <= 0.25) {
				reject("op2. failed");
			} else {
				fulfill(Math.floor(x / 0.75 * 1000));
			}
		}, 1000);
	});
}

console.log("Start");

op1()
.then(op2)
.then((res) => {
	console.log("Success: " + res);
})
.catch((err) => {
	console.log("Error: " + err);
});

The implementation of HomebrewPromise is straightforward except for a couple key points.

The first and big break is to return a new instance of our own class at then(f). The call to f() is deferred until the previous promise is fulfilled. So, our class needs to support two kinds of promise: immediate run and deferred run; the latter just for internal affairs.

The second trick is to distinguish whether f() — the function we received via .then(f) — is a promise or an ordinary function. But we can only find out after calling f(). Due to this, even if f() does return a promise, we learn it too late to use it as return value for .then(f).

The third trick is to forward an error to the next chain link, in case the client did not call catch() for the given promise. Since then(f) always returns an instance of our own class, we know that the next link of the chain is of our own kin, and we can call a "private" method (__forwarded_error()).

e-mail icon