A backwards compatible, type safe system for JSON fields in Prisma

One of the great features of Postgres is its ability to store JSON values in a column. This enables a lot of powerful, and flexible features without needing to constantly manage migrations. One use case where this is comes in handy is managing user preferences.

User preferences are a constantly evolving feature in most apps and as I cover in this article I wrote earlier this week, being able to use JSON objects in both your database and in cookie storage allows you to have an isomorphic system for both authenticated and anonymous visitors.

Why should we care about having type safe JSON fields?

Weeeellll, you might not. You may not need to care about it.

When you shouldn’t care

In the case of this website I use TipTap as the content editor for my articles. It outputs a JSON schema that represents the content of every article and that is what I store in the database under a. JSON field. I do not care what the shape of that data is. TipTap does it for me. They take in the JSON process it for my editing needs and send it back out already formatted how they need it. I trust them. I do not need to check them. Then I would be on a hamster wheel of constantly updating my validations to keep up with their schema changes.

When you should care

Let’s go back to the original reason I am exploring this concept in the first place.

I need to manage user preferences for my app. These preferences effect what data and styles are rendered in my app. What happens to my rendering process when it get’s user preferences that do not match the shape they are expecting? I need to create a contract within my system that guarantees that the data I am pulling from the database and sending to the frontend to render my app is exactly what it is expecting so that I do not deliver buggy interfaces to my end user.

How to create type safety when using JSON fields in Prisma

First let’s show an abbreviated schema for our User model that shows how to add a JSON field. Buckle up because you’re not going to believe how hard this is.

model User {
  id          String @id @default(cuid()) @map("user_id")
  preferences Json

  @@map("user")
}

Okay, so it is in fact incredibly easy to add a new JSON field. So, let’s talk about how we can validate our JSON schema.

Creating validations with Zod

The answer to validating our data before sending it over to our frontend is Zod. If you are not familiar with Zod then you can read more here, but the TL;DR is that Zod allows you to create schema in your Typescript codebases that validate your types during development and your data at runtime as well.

So, we need to create a Zod schema for our user preferences and for our example we will be making a user schema as well:

const UserPreferencesValidator = z.object({
	theme: z.literal("dark").or(z.literal("light")),
	docs: z.object({
		authentication: z.object({
			kind: z.literal("team").or(z.literal("personal")),
			emailValidation: z.boolean(),
			twoFactor: z.boolean(),
		}),
	}),
})

const UserValidator = z.object({
	id: z.string(),
	preferences: UserPreferencesValidator,
})

Using our schema to validate data coming from Prisma

Now that we have our schema we need to use them to validate our data coming in and out of Prisma.

// Zod lets us make types directly from the schema we defined
type UserPreferences = z.infer<typeof UserPreferencesValidator>
type User = z.infer<typeof UserValidator>

function getUser(userId: string) {
	const payload = prisma.user.findUnique({
		where: { id: userId },
		select: {
			id: true,
			preferences: true,
		},
	})

	return UserValidator.parse(payload)
}

// By setting the preferences param to the inferred type
// we force all preference data to be validated before it is passed in.
function saveUserPreferences(userId: string, rawPreferences: UserPreferences) {
    const preferences = UserPreferencesValidator.parse(rawPreferences)
	const payload = prisma.user.update({
		where: { id: userId },
		data: { preferences },
	})

	return UserValidator.parse(payload)
}

Pretty straightforward. Now you can pass your preferences around your app knowing your not going to be working with anything other than the certified real deal.

What happens when your schema needs to change?

At some point in your products lifetime you are going to add some new preferences, or you are going to remove some, or you're going to rename and merge a few different preferences to make rendering a little easier. When that happens what do you do?

What you can't do

You cannot just change your schema and move on.

Your users preference data is stored in a database. That means when Steve, that hasn't logged in since you made the change, tries to load their account your system is going to fail perpetually because the preference JSON that lives in his account doesn't match what you changed it to.

What you can do

Just like a database schema we are going to create migrations. These migration functions will allow us to still parse legacy preferences and upgrade them to match the new requirements.

So, let's modify the previous example to be backwards compatible.

Step One: Add a version field to your schema

This will allow us to identify and funnel schema through the different migration functions based on what the users version currently is and what we need to upgrade it to.

Below we have added a version of 1 to our original schema and added and v2 schema that adds some new fields.

const UserPreferencesValidatorV1 = z.object({
	version: z.literal(1),
	theme: z.literal("dark").or(z.literal("light")),
	docs: z.object({
		authentication: z.object({
			kind: z.literal("team").or(z.literal("personal")),
			emailValidation: z.boolean(),
			twoFactor: z.boolean(),
		}),
	}),
})

const UserPreferencesValidatorV2 = z.object({
	version: z.literal(2),
	theme: z.literal("dark").or(z.literal("light")),
	docs: z.object({
		authentication: z.object({
			kind: z.literal("team").or(z.literal("personal")),
			emailValidation: z.boolean(),
			twoFactor: z.boolean(),
		}),
		billing: z.object({
			kind: z.literal("subscription").or(z.literal("oneTime")),
		}),
	}),
})

Step Two: Create migration helpers

We need to make the inferred types for not only our current user preference object, but the legacy one as well. Then we will need to make sure that the UserValidator accounts for both when parsing the preferences from the database, and finally we will need a function that will upgrade the v1 preferences to the v2 preferences.

type UserPreferencesV1 = z.infer<typeof UserPreferencesValidatorV1>
type UserPreferences = z.infer<typeof UserPreferencesValidatorV2>

// We use the .or method to allow the user validator
// to parse both current and legacy versions
const UserValidator = z.object({
	id: z.string(),
	preferences: UserPreferencesValidatorV2.or(UserPreferencesValidatorV1),
})

// This migrate function modifies the v1 schema to be 
// compliant with the v2 schema.
function migrateV1(v1: UserPreferencesV1): UserPreferences {
	return UserPreferencesValidatorV2.parse({
		...v1,
		version: 2,
		billing: { kind: "subscription" },
	})
}

Step Three: Use them when fetching and saving to Prisma

The only time we need to care about legacy versions of the schema is when we are querying the DB. Below we validate the returned user object and then migrate the preferences if they are legacy.

Since we have done the work on the entry point of the data into the app we do not have to worry about it on the back into the database. We can assume it will always be the latest schema version.

function getUser(userId: string) {
	const payload = prisma.user.findUnique({
		where: { id: userId },
		select: {
			id: true,
			preferences: true,
		},
	})

	const { id, preferences } = UserValidator.parse(payload)
	return {
		id,
		preferences:
			preferences.version === 1 ? migrateV1(preferences) : preferences,
	}
}

function saveUserPreferences(userId: string, rawPreferences: UserPreferences) {
    const preferences = UserPreferencesValidatorV2.parse(rawPreferences)
	const payload = prisma.user.update({
		where: { id: userId },
		data: { preferences },
	})

	return UserValidator.parse(payload)
}

That’s all there is to it. If you like to live on the edge, Prisma has introduced client extensions. Client extensions allow us to add new functionality on every call for a schema similar to the concept of middleware in express and other frameworks.

They have written a great example of how you can do that here:

Prisma Client Just Became a Lot More Flexible

Just substitute the schema in their examples with our schema above.