Enforce types at runtime

APIs all the way down

Frontend and backend developers these days have to deal with many external APIs, usually REST APIs. Fetching data is easy, but making sure the data is valid and conforms to an expected schema can get messy.

Foreign APIs can change over time or simply deliver rubbish, especially on Friday afternoons or at midnight during a lunar eclipse.

It is therefore important to detect issues quickly and reliably to prevent error propagation to the deeper parts of our software architecture. (Imagine your trading bot relies on a stock-market API, e.g. yahoo-finance. You want to make sure all API responses are thoroughly checked before making any trading decisions.)

High expectations

Since I’m a spoiled Typescript developer, I expect my API calls to be type-safe and my IDE to indulge me with autocompletion. I may use a code generator here and there, if I get the proper description of the API. The code generator can wrap data fetching into nicely typed functions and implement type checking to some degree. But, this is not always possible.

What if I want to implement my own code generator? What if I’m hacking together a client for an API that wasn’t intended to be an API? 😉

Let’s start from scratch

const result = await fetch("https://jsonplaceholder.typicode.com/users/1")
  .then(resp => resp.json());
console.log(result.email);           // no autocompletion because result:any
console.log((result as User).email); // with autocompletion but no validation

Here, for autocompletion to work, the response must be somehow casted to User. Sadly, every time the type any is blindly recasted to User a baby unicorn dies a slow death.

What we actually want, is to ~~save the unicorns~~ ensure the data conforms to a given schema before using it.

// better, but don't do this at home
const user: User = isUser(data); // throws if not user 
console.log(user.email);         // with autocompletion

Somewhat better, apart from the ugly exception-throwing, but I’m sure we can do even more! Especially in Typescript.

Type guards

Typescript gives us a handy feature called type guards. We can implement our “isUser” function with a special return type data is User.

Our fancy “type predicate” function accepts any data and checks if it is a User.

const isUser = (data: any): data is User =>
  (data as User).email !== undefined; // just for demonstration purposes

Now comes the magic. Typescript compiler and our IDE are clever enough to treat the user variable, from now on, as User inside the if-branch:

if(isUser(user)) {
  console.log(user.email); // autocompletion WORKS because `user:User`
} else {
  console.error("This is no User!");
}

This is a 3-fold better solution since we (1) achieved schema validation, (2) got autocompletion, and (3) we didn’t break the control-flow with exceptions at the same time.

Well, unicorns are saved, but what about the poor lazy programmers? They now have to write all these fancy type guards. If only we could automate this somehow…

Parse don’t validate

There is an interesting world-view that instead of validating data we should rather be parsing them.

I like this quotation from the article [Parse don’t validate 1]1:

Consider: what is a parser? Really, a parser is just a function that consumes less-structured input and produces more-structured output. Parsers are an incredibly powerful tool: they allow discharging checks on input up-front, right on the boundary between a program and the outside world. Once those checks have been performed, they never need to be checked again! Ad-hoc validation leads to a phenomenon that the language-theoretic security field calls shotgun parsing [2]2.

For Typescript and Javascript, there are multiple packages suitable for this task. To name a few: superstruct, runtypes, io-ts, joi, zod.

My favourite is a relatively new package - zod. For those who like to combine typescript with functional programming approach from the fp-ts package, the io-ts might also be a natural fit.

Zod

In zod, we describe our types in a composable way and get a parser that validates all input data at runtime and if valid, returns a statically typed object.

Without showing how the User-parser (here ZUser) is actually defined, our example from above would look like this:

const result = ZUser.parse(input); // throw if not `User`

// result is User and autocompletion works 
console.log(result.email);

…or without throwing exceptions:

const result = ZUser.safeParse(input);
if (!result.success) {
  // handle error then return
  return;
}

// result is User and autocompletion works 
console.log(result.email);

Summary

I won’t go into details about zod here. Let’s keep it for another blogpost. For now, the takeaway is:

don’t blindly accept data from endpoints
parse, don’t validate
learn some library like zod to do it

References

[1]: Parse, don’t validate, Alexis King

[2]: The Seven Turrets of Babel: A Taxonomy of LangSec Errors and How to Expunge Them, F. Momot, S. Bratus, S. M. Hallberg and M. L. Patterson