Schema first design

API schema

JSON Schema is a popular way of defining data structures. The format can be easily parsed and used to automate the generation of validation logic, test cases, documentation and so on; you will never be short of tools no matter what technology you use.

Taking the example of the previous post, let’s see what our API’s schema could be:

import {JSONSchema} from 'json-schema-to-ts';

const schema = {
    $id: 'bank-accounts.schema.json',
    title: 'bank accounts service',
    description: 'The bank accounts service definition',
    type: 'object',
    properties: {
        commands: {
            type: 'object',
            properties: {
                transferMoney: {
                    type: 'object',
                    properties: {
                        input: {
                            type: 'object',
                            properties: {
                                from: {
                                    $ref: 'bank-accounts.schema.json#/definitions/bankAccountId'
                                },
                                to: {
                                    $ref: 'bank-accounts.schema.json#/definitions/bankAccountId'
                                },
                                amount: {
                                    $ref: 'bank-accounts.schema.json#/definitions/amount'
                                }
                            },
                            additionalProperties: false,
                            required: ['from', 'to', 'amount']
                        }
                    },
                    required: ['input'],
                    additionalProperties: false
                }
            },
            required: ['transferMoney'],
            additionalProperties: false
        }
    },
    required: ['commands'],
    additionalProperties: false,
    definitions: {
        bankAccountId: {type: 'string', description: 'The unique identifier of a bank account'},
        amount: {type: 'integer', description: 'A monetary amount, in cents'}
    }
} as const satisfies JSONSchema;

I invite you to go through the JSON schema documentation for more details on the syntax. But, even without that knowledge, you can understand the overall schema definition with some fields being informative(description, title, etc.) and others being declarative of the data contracts.

The syntax seems a bit verbose, but it brings a lot of flexibility and power as we will soon see. You could anyway avoid some boilerplate by generating the schema on the fly based on a record of command name/command input schema, or use any of your favourite IDE extensions that help with JSON schemas.

Runtime behaviour

Now that we have a definition of the contract, we can easily implement data validation. In the Node.js ecosystem, there are several libraries for building validation on top of JSON schemas, one of the most popular being ajv. Since we want our module contract to be fully defined by the schema, we can modify the defineModule of the framework built in the previous article to enforce this new policy. The new implementation could be

import _ from 'lodash';
import {createProvider} from 'dismoi';
import assert from 'node:assert'
import Ajv from 'ajv';

const ajv = new Ajv();

export const defineModule = ({commands, schema, injectables}) => {
    ajv.addSchema(schema);
    const commandListFromSchema = Object.keys(schema.properties.commands.properties);
    const commandListFromImplementation = Object.keys(commands);
    const symmetricDifference = new Set(commandListFromImplementation).symmetricDifference(new Set(commandListFromSchema));
    assert(symmetricDifference.size === 0, `discrepancy between schema and implementation: [${[...symmetricDifference]}]`)

    const commandWithValidation = _.mapValues(commands, (commandFactory, commandName) => {
        const inputSchema = schema.properties.commands.properties[commandName].properties.input;
        return withValidationDecorator(commandFactory, inputSchema)
    });

    const _injectables = {
        ...injectables,
        ..._.mapValues(commandWithValidation, withinTransactionDecorator)
    };

    return createProvider({injectables: _injectables, api: Object.keys(commands)});
};

Now, the function defineModule takes a schema as a parameter. We start by adding the schema to the ajv singleton so that we can reference the main schema in the $ref clauses (in practice, this, should be done slightly differently if you want to be able to reference completely different schemas,e.g. coming from different modules). We then use the new symmetricDifference of javascript Set to ensure that the schema is consistent with the provided implementation. This is not ideal in the sense that the error is thrown at runtime, but it is not necessarily a problem if you add a simple test to your suite that checks that you can define the module. We will soon see how you can have this signal directly when you code, using Typescript.
What’s left is to map the command input schema and the command factory together with the new withValidationDecorator decorator.

function withValidationDecorator(commandFactory, schema) {
    const validate = ajv.compile(schema);
    return (deps) => {
        const command = commandFactory(deps);
        return (input) => {
            const isValid = validate(input);
            if(!isValid){
                throw new Error('Invalid command input', {cause: validate.errors});
            }
            return command(input);
        }

    }
}

Nothing fancy here. We can then compose with the former withinTransactionDecorator decorator and we now have:

1. Our API fully defined by a json schema
2. The API can’t change without the schema being updated and vice versa, which somehow enforces keeping the documentation inline with the behaviour.
3. All our commands run inside a transaction (see previous post)

An important point to note is that just as the withinTransactionDecorator allowed the developer not to worry about the transactional behaviour while developing their functionality (i.e. the command), the withValidationDecorator gives them data validation at no cost(no need to modify the functional code), as long as they define the data contract inside the schema!

That’s great, but let’s see how we can improve the developer experience even further by using Typescript

Developer experience

In the Javascript ecosystem, Typescript, which brings static typing to Javascript, has also become a popular tool for describing data structures and enforcing their consistency throughout the software stack. However, types disappear at runtime and are less useful when it comes to building runtime logic (i.e. actually validating a data input passed to a function, for example). For these reasons, developers sometimes tend to duplicate the data structure definition to accommodate different use cases, or rely on libraries that are more specialised and less versatile than a dumb serialisable format like json.

The good news, is that you can use libraries to infer types from a JSON schema. If you paid attention to the schema definition in the introduction, you should have noticed that it satisfies a JSON schema. This gives you auto-completion when you write your schema, and makes sure you don’t have any syntax errors. We can also use the FromSchema from the same json-schema-to-ts library to infer the definition of the commands.

import {JSONSchema, FromSchema} from 'json-schema-to-ts';

type CommandsDef<Schema extends JSONSchema> = FromSchema<Schema> extends {
    commands: infer Commands
} ? Commands : never;

In the same way, we can go further and express the command signature (assuming for the moment, that the output is always void):

type CommandInput<Schema extends JSONSchema, Name extends keyof CommandsDef<Schema>> = CommandsDef<Schema>[Name] extends {
    input: infer Input
} ? Input : never;

type CommandFn<Schema extends JSONSchema, Name extends keyof CommandsDef<Schema>> = 
    (input: CommandInput<Schema, Name>) => Promise<void>

As you can see in the animation below, you can now explicitly say that a function is an implementation of the command defined by the schema, and you will get a compilation error if you change either the function signature, or the schema without reflecting the change in a compatible way.

Again, we can incorporate these new views on how to properly develop an API, directly into the framework.

import {ProviderFn} from 'dismoi';

type CommandInjectables<Schema extends JSONSchema> = {
    [CommandName in keyof CommandsDef<Schema>]: (deps: any) => CommandFn<Schema, CommandName>
}

declare function defineModule<
    Schema extends JSONSchema,
    Injectables extends Record<string, unknown>
>(input: {
    schema: Schema;
    commands: CommandInjectables<Schema>;
    injectables: Injectables
}): ProviderFn<Injectables & CommandInjectables<Schema>, (keyof CommandInjectables<Schema>)[]>

There are now even fewer moving parts:

• you must include an implementation for each command defined on the schema
• you cannot include a command implementation if there is no matching definition on the schema
• implementation interfaces must be compatible with the defined API

Conclusion

The schema has become the centerpiece of our module bringing a strong consistency between the contract and the implementation. Even better, it tailors the dev experience, forcing the developers to follow team conventions and quality standards (data validation is mandatory, for example). Yet, we have barely scratched the surface: we could use the schema to automatically generate documentation,a client library, or even test cases. Similarly, we have only used the notion of commands, but we could also add the API definition output format, error that can be thrown, events that can be raised, etc.