json:api, json-rpc and API-first patterns for Drupal

Meet Kinksters is built on top of a wide range of open source technologies, Drupal being chief among them.

This blog post is an attempt at covering the current state-of-the-art and best practice for “doing API-first” with Drupal, with a json:api focus.

I say “json:api focus” because there is solid support for GraphQL in Drupal, and I’m not here to discount it. That said, Drupal core explicitly prioritizes json:api support over GraphQL and as you’ll see here, the growing contributed module ecosystem has followed suit.

Understanding the value proposition of json:api

When I first started building decoupled Drupal sites, my instinct was to assume that the API specification (that is, the json:api spec) covered all aspects of my API, from request and response shapes to schema to URL structure. In truth, the json:api spec speaks only to a subset of these concerns. I found clarity in this statement on the project front page:

If you’ve ever argued with your team about the way your JSON responses should be formatted, JSON:API can be your anti-bikeshedding tool.

By following shared conventions, you can increase productivity, take advantage of generalized tooling, and focus on what matters: your application.

…

JSON:API covers creating and updating resources as well, not just responses.

Bikeshedding is a fun topic worth exploring if you haven’t run into it yet, but the point here is that json:api is really only concerned with the structure of your payloads. Not the code that generates them, nor the ORM managing the data, nor the URLs from which you receive or to which you post those payloads.

Once I understood this important point, the concepts discussed below came into sharper view.

Best practices for json:api and json-rpc in Drupal

Drupal site owners may enable a feature-rich json:api server by simply enabling the jsonapi core module. This simple act might even be sufficient to integrate with data consumers such as kiosks, aggregators or headless front-ends which access unauthenticated resources. This is a huge win for very little effort.

As applications grow more complex and require features such as authentication, write operations and procedure calls, so too must you adapt your site architecture.

The temptation of custom controllers

A common first tutorial for novice Drupal developers is to create a custom controller; this is a quick route to “Hello World” satisfaction. It is unsurprising then that many developers choose to implement APIs using custom controllers, especially as Drupal 8+ makes it easy to return JSON response objects instead of HTML.

This pattern quickly grows out of control, and in my experience also encourages a dangerous blurring of lines between controller logic and Drupal’s access control APIs. When every “endpoint” is written effectively from scratch, the result is an unwieldy hodgepodge of unmaintainable code. I can say this with confidence because I did it for years!

Additionally, what is the shape of the request and response payloads handled by these custom controllers? By definition, the answer is “whatever you want.” Considering how easy it is to shortcut Drupal’s built-in APIs for entity CRUD and access, this approach invites all manner of information disclosure, data corruption and access bypass.

Let’s return then to json:api’s “anti-bikeshedding” value proposition. Drupal’s implementation comes with a spec-compliant server with sensible defaults for URL structure; use it. (And extend as needed.)

When the defaults don’t fit

Meet Kinksters is a dating app, and as such we have a data model which maps rather well to json:api resources. Things like dating profiles, user blocks, and even payment methods all have automaticaly-generated json:api resources. But what about a requirement like our “introductions stack”?

Custom Resources

Meet Kinksters users may send “introductions,” which are basically unilateral matches. Users on the free tier may only see one received introduction at a time in their stack, while paid users can see all their suitors at a glance. This kind of advanced sorting and access control is beyond Drupal’s out of the box resource collections. But not by much.

We still want to leverage json:api’s request and response structure, because the returned data should be parsed according to the spec. In this case, we implement a custom resource using the JSON:API Resources module. Precisely because json:api defines no restriction on the URLs at which your API operates, custom resources can return any manner of primary and included data based on your site’s business rules.

Thus, our “introductions stack” custom resource returns documents (Drupal entities) in the context of the active user and their paid membership status.

Note that json:api doesn’t require that your request even contain a body at all! Consider a custom endpoint at /jsonapi/user/{uuid}/block which might perform some sort of operation to block the other user. (So sad!) The response could contain a json:api document (say, a block resource, if such a thing exists) or merely reply with a meta member with some status information. The specific details are up to you.

Notably, json:api Search API exposes search indexes using custom resources!

Compound responses and sparse fieldsets

Along a similar vein as custom resources, advanced use cases might prompt your server to include related (but unsolicited) json:api documents in responses. In our case, this means sending user profile and photo data alongside matches and introductions, because we know the client will require this information to fully render data in the UI. (What use is a dating profile without a photo?) Such functionality can be accomplished by setting default include parameters in the jsonapi_defaults module.

In this case, the json:api spec is rather permissive, but only if the client did not send an include query parameter in the request. Similar considerations apply to server-initiated sparse fieldsets.

JSON-RPC for non-resource operations

For operations that do not directly map to a Drupal entity, consider json-rpc. Using the contributed module, new methods are exceedingly easy to create, and similarly easy to implement on the client side.

The role of json-rpc

Just like my “a-ha” revelation about the value proposition for json:api, so too I needed experience and comparison to understand the value of json-rpc. Underlying the value of json-rpc is, well, RPC. There are many remote procedure call implementations in the wild, including Google’s gRPC. A dive into the RPC pool will quickly lead you to ask, do I need HTTP/2? Protobuf? For the web architect needing to choose a technology stack, this complexity can quickly become overwhelming.

For me, I only ever learned that I needed an RPC protocol because json:api’s resource focused model no longer matched my needs. In other words, json:api is great at modeling an API for a universe of discourse, or my application’s entities which might be stored in a relational database.

Actions such as “log in,” “log out” or “clear cache” are examples of procedure calls which may be undertaken over an API but do not map to CRUD entity operations.

The json-rpc specification is appealing due to its reduced complexity as compared to gRPC. Method names, parameter values and responses are all serialized as JSON, and lightweight clients are available in common languages such as PHP and JavaScript. No intermediate compilation is required.

Conclusion: Stick to the spec

When building an application that does not provide a public API (that is, you are not providing your API as-a-service) it can be easy to sidestep spec compliance for convenience. I believe this is a mistake.

Specifications such as json:api and json-rpc grow out of long-term, consensus-based work by those with a global view. By using the spec (and sticking to it!) you benefit from this hard work and consideration of trade-offs you may never have thought of or not yet encountered. In addition, you ensure your API may be consumed by spec-compliant clients, instead of tripping over your customizations.

Sticking to the spec might seem like “extra work” (don’t get me started on how this plays out at penny-pinching agencies!) but I assure you it’s worth doing. If you find yourself building an implementation that is antipattern to the spec, ask why. In my personal experience, this introspection has always led me to a better, more graceful and maintainable solution (and often with far less code.)