Migrating from MobX (State-Tree) to React Query

TL;DR: You should use React Query if:

• You fetch data from your backend frequently with minor or no changes (views or data transformations).
• You re-fetch data from your backend after mutable operations (post, put, delete, patch).
• You need polling or pagination.

Now, for the longer version.

Why change?

For the past six months, I have been working on a new project. Having already confirmed the usefulness of a TypeScript based stack, we decided to go for it again:

• Backend. Nest.js, TypeORM, PostgreSQL.
• Frontend. React with MobX State Tree for state management.

So far so good. I felt especially at ease since I already developed within the MobX State Tree framework for over a year and, despite my initial reluctance to shift from a more classical Redux environment, I ultimately found the MobX paradigm to appease my personal taste a lot more than its competitor. Nevertheless, as time passed, I couldn’t help but to notice several code smells; particularly in dealing with fetching and pushing data from and to the backend. Let me explain.

The MobX State Tree approach

MobX describes itself as follows:

MobX is a battle tested library that makes state management simple and scalable by transparently applying functional reactive programming (TFRP).

It provides a set of utilities through which object changes can be observed and therefore reacted to. In other words, it allows a state to be observed and managed. However, it does not actually provide a facility for manging a client-side application state. That’s where MobX State Tree comes into play.

Technically speaking, MobX-State-Tree (also known as MST) is a state container system built on MobX, a functional reactive state library.

Put bluntly, MST equips developers with a big JavaScript object used as a container for the application state, which can be reached from any point of the application, no matter how deep. Components can then affect siblings, descendants or anscenstors without the need for endless props forwarding. On top of that, it also enforces zealous type checks, so that whenever incompatible data is placed into the state against its previously defined model (e.g. a number into a string field, or a null into a non-nullable field), an error is thrown.

MST is a nice library. Models define what can be stored in the state, views derive computed data from the existing state and provide explicit memoization points, while actions can be used to immutably affect the state through familiar mutable-like operations. We used the model/view/action paradigm to describe all our client-side state, including the structure of our asynchronous operations.

Describing asynchronous operations

All products I have worked on in the past three and a half years are client-side rendered single page applications. As any developer knows with this kind of application, interaction with the backend happens asynchronously. Before a request is served by the backend, time will pass. Additionally, not all requests will be successful. When fetching a resource, then, we need to represent several different scenarios.

• Empty. When the resource is not present and the frontend didn’t yet issue a request for that resource.
• Loading. When the resource is not present, but the frontend already issued a request for it and it is waiting for a response. We usually like to display some kind of indicator during this phase, like a spinner, so that users know that the application is working.
• Error. When the resource is not present and the request has received a response, but the response only contained an error (code or message, or any other indication of an unexpected operation). In this case, we cannot display what we originally intended and must warn the user that something went wrong, possibly giving them an explanation and a way to amend the incorrect state.
• Success. When the request has received a successful response, which contains the intended resource. Now rendering may continue on its golden path.

I am using the term “resource” to indicate a piece of data that is stored in the backend but is needed in the frontend. The latter will need to issue a request in order to obtain the most recent version of the resource for display.

Now, this sounds easy enough. After all, MST allows for an observable state. Changing the state based on the outcome of a promise (our asynchronous vector) will cause the UI to re-render and apply the most appropriate visualization. There’s nothing inherently wrong with that. In fact, I even wrote a handy factory function that we used across the whole codebase.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117

/* eslint-disable no-param-reassign,@typescript-eslint/explicit-module-boundary-types */
import { applySnapshot, cast, types } from "mobx-state-tree";
import { asErrorModalBody, ErrorModalBody } from "@app/app/ErrorModalModel";
import { Nullable } from "@shared/dtos";

const randomId = () => Math.random().toString(16).substring(2);

export const makeAsyncModel = <T>() => types
    .model({
        requestId: types.maybeNull(types.string),
        pollingId: types.maybeNull(types.string),
        pending: types.boolean,
        completed: types.boolean,
        error: types.maybeNull(types.frozen<ErrorModalBody>()),
        payload: types.maybeNull(types.frozen<T>()),
    })
    .actions(self => ({
        reset(): void {
            applySnapshot(self, emptyAsync);
        },

        start(preservePreviousState?: boolean): string {
            const requestId = randomId();
            self.requestId = requestId;
            self.pending = true;

            if (!preservePreviousState) {
                self.completed = false;
                self.error = null;
                self.payload = null;
            }

            return requestId;
        },

        succeed(payload: T): void {
            self.pending = false;
            self.completed = true;
            self.error = null;
            self.payload = cast(payload);
        },

        fail(error: Error): void {
            self.pending = false;
            self.completed = true;
            self.error = asErrorModalBody(error);
            self.payload = null;
        },

        cancel(): void {
            self.pending = false;
            self.requestId = null;
        },
    }))
    .views(self => ({
        get key(): string {
            return makeAsyncKey(self);
        },

        /**
         * Empty means that no request was issued and there's no data in this model.
         * Effectively, the model was never touched.
         */
        get isEmpty(): boolean {
            return self.pending === false && self.completed === false && self.payload === null;
        },

        map<K>(mapper: (payload: T) => K): K | undefined {
            return self.payload === null ? undefined : mapper(self.payload);
        },
    }))
    .actions(self => ({
        async hookPromise(promise: Promise<T>, preservePreviousState?: boolean): Promise<void> {
            const requestId = self.start(preservePreviousState);
            try {
                const payload = await promise;
                if (requestId === self.requestId) {
                    self.succeed(payload);
                }
            } catch (e) {
                if (requestId === self.requestId) {
                    self.fail(e);
                }
            }
        },

        mutate(mutator: (payload: T) => T): void {
            if (self.completed === true && self.payload !== null) {
                self.payload = cast(mutator(self.payload));
            }
        },
    }))
    .actions(self => ({
        startPolling(apiMethod: () => Promise<T>, pollIntervalMs: number): void {
            const pollingId = randomId();
            self.pollingId = pollingId;

            const callApi = () => {
                if (pollingId === self.pollingId) {
                    self.hookPromise(apiMethod(), true)
                        .then(() => {
                            setTimeout(callApi, pollIntervalMs);
                        })
                        .catch(() => {
                            // ignore (handled by self.hookPromise)
                        });
                }
            };

            callApi();
        },

        stopPolling(): void {
            self.pollingId = null;
            self.cancel();
        },
    }));

We’d use the code above to quickly create MST state fragments for remote resources. Then, we’d have a React component shaped as follows:

1
2
3
4
5
6
7
8
9
10

export const SomeEntityView = observer((): React.ReactElement => {
    const { stateFragment: { sortedEntities, fetchEntities, clearEntities } } = useMst();

    useEffect(() => {
        fetchEntities();
        return clearEntities;
    }, [fetchEntities, clearEntities]);

    return <Async key={sortedEntities.key} model={sortedEntities} render={renderSortedEntities} />;
}

Async is a helper component to take care of the loading, success and error scenarios. As you can see, the asynchronous request is initiated as a side effect through useEffect and MobX observes stateFragment to notify the component of any update in its fields.

The streamlined framework we built to manage asynchronous operation works and thanks to the factory method we have very low duplication. However, it got more complicated when we introduced polling as a capability. As you can see, we are managing low level details like request ids in order to disambiguate between overlapping requests that are received out of order. Additionally, creating views requires going through the map helper, in order to avoid a plethora of undefined checks.

The need, albeit diluted over time, for handling low level concerns evoked a feeling of skepticism in me. “This cannot be the only way”, I thought. Enter React Query.

React Query

We didn’t inherently have a problem in our codebase… yet. However, the thought of having to implement pagination and infinite loading for virtualized lists on top of MST spooked me. Too many details to mind, which didn’t really bring value to the application. Not directly, at least, and not fast enough. Hence the choice to delegate all such details to a library that is built for that precise purpose, a library that is concerned with server-state first. We used a client-state library to represent server-state, hence the slight but noticeable disconnect between MST capabilities and our needs.

React Query is often described as the missing data-fetching library for React, but in more technical terms, it makes fetching, caching, synchronizing and updating server state in your React applications a breeze.

I only recently learned about the existance of React Query. Very often, not knowing what is out there is also the cause of sub-optimal tech choices. Thankfully, many people at Mind Foundry like to share articles, documentation and news about the tech panorama, so that we can stay updated on a constantly evolving state of the art.

I can identify one specific event that made it click for me. While developing a feature in the project I am currently working on, I incurred the need to call several endpoints from different components. The components themselves are dynamically generated at runtime, client-side, by interpreting a declarative piece of JSON. Each component specification may also contain the name of a well-defined endpoint (an “action”) that needs to be called when the component is mounted, so as to populate itself with correct data for visualization. With this scenario in mind, I had to write a piece of code that imperatively collected all actions from the JSON specification, removed all duplicates, and made the result available to all components. This is exactly what React Query does. I re-implemented the query cache from React Query on top of MST. After reading briefly about React Query, I realized at this very moment that I was breaking one of the dogmas of engineering: don’t reinvent the wheel! Riding the tide of such a crashing realization and with an eye to the future, I decided to spend some of my personal development time migrating a small chunk of code to React Query. After a first exploratory phase and after conferring with the team about it, we decided to proceed with the shift. Our codebase is still small and manageable enough that it only took a couple of days.

What’s so special about React Query anyway?

React Query applies a paradigm shift to data fetching, making it declarative instead of imperative. To me, using such a library makes a lot of sense, considering that React is a declarative framework. It feels right, because it embraces the same philosophy as the technology it’s catering to. The need for invoking side effects at render-time is now replaced with a more comfortable custom hook, which can be called from any component, anywhere in the application stack, without the need for observables. Read the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

const queryKeys = {
    inbox: ["inbox"],
    // ...
};

export const useInbox = (enabled: boolean, sortTasks: TaskSorter): UseQueryResult<InboxResponse, Error> => (
    useQuery(
        queryKeys.inbox,
        fetchInboxTasks,
        {
            enabled,
            refetchInterval: 5000,
            select: inboxResponse => sortTasksWithinInboxResponse(inboxResponse, sortTasks),
        },
    )
);

It defines a custom hook whose main job is to fetch a bunch of tasks to populate the user’s inbox. Quite easy and uncomplicated.

Its first parameter, queryKeys.inbox, defines a unique string array that identifies this query among all other queries, so that, regardless of how many times the hook is invoked within a page, its associated fetching operation is only performed once. The library is smart enough to remove duplication through a well-thought-out caching system, which supports time-to-live and invalidation, like any other cache. It takes care of problems I was going to address myself by design. One less thing to worry about, yay!

Its second parameter implements the actual fetching (or posting, patching, putting, deleting). You can plug in any asynchronous request library you prefer and it unlocks a flexibility in handling the actual network operations, which may not even involve the network at all: any promise-based routine can be handled this way!

Then, the final parameter, options, unlocks a wide range of capabilities. enabled is a boolean flag which turns the query on or off and can be used for an array of use cases. In the example above, we don’t want the inbox to refersh when a modal is visible. This option allows us to achieve that. Perhaps more useful in other circumstances, enabled allows queries to depend on each other, so that a query doesn’t fire until data required to issue it is available!

refetchInterval causes a re-fetch to happen on a timer. That’s polling. One line of code: much easier than implementing a polling mechanism over MST.

Finally, select applies transformations to the received data. It can be used to re-enact some of the functionality of MST views, or you can rely on the usual useMemo hook. This is probably the weaker side of React Query, but we need to consider that the library only manages server-side state, hence the lack of flexibility here.

The result of the custom hook above - UseQueryResult<InboxResponse, Error> - is an object containing a plethora of useful properties, among which I have to recount the most useful ones: isLoading, isSuccess, isError, isIdle, error and data. The latter subset of properties contributes in completely specifying every scenario I listed in my introduction, so that we may render appropriate UI fragments depending on the state of the query: whether it is loading, it has failed because of an error, or it finished successfully.

Pushing data upstream: mutations

Fetching only highlights half of the problem. The other half includes mutating operations like creating, updating or deleting resources. React Query provides a way to close the loop with mutations:

1
2
3
4
5
6
7
8
9
10

export const useInboxTaskAssignmentMutation = (): UseMutationResult<SuccessResponse, Error, InboxTask> => {
    const invalidateOnSuccess = useFireAndForgetInvalidation(queryKeys.inbox);
    return (
        useMutation(
            queryKeys.assignTaskToMe,
            (task: InboxTask) => assignTaskToMe(task.id),
            invalidateOnSuccess,
        )
    );
};

The custom hook shown above represents a POST request through which an user can assign a task to themselves from the inbox. The interface contract of useMutation is very similar to useQuery , yet its semantic is different: mutations are never retried by default, because they are not idempotent, in the general case. Mutations are therefore still imperative and need to be explicitly called as a reaction to user input. Apart from that, the result of the custom hook above is quite similar to the one from useQuery: it still contains fields like isLoading, isSuccess, etc… Moreover, it also contains a function called mutate that can be used to initiate the mutation.

As the documentation explains, mutations can be very useful to invalidate existing queries, effectively evicting their associated query entry from the query cache and forcing a refresh. In the example above, I am using another custom hook that fires off an invalidation without waiting for all the affected queries to be re-fetched.

1
2
3
4
5
6
7
8
9
10

export const useFireAndForgetInvalidation = (...keys: Array<QueryKey>) => {
    const queryClient = useQueryClient();
    return {
        onSuccess: () => {
            keys.forEach(key => {
                queryClient.invalidateQueries(key).catch(reason => console.error(`Failed to invalidate query key ${String(key)}: ${String(reason)}`));
            });
        },
    };
};

One of the strongest selling points of query invalidation consists in its hierarchical nature. If we define queries with the following keys:

• tasks
• tasks, 8
• tasks, 5, assignee

Then invalidating the key ["tasks"] will also perform a pattern matching search on all existing query keys and invalidate any whose prefix matches, effectively causing a re-fetch for all of the above.

In conclusion

To recap, React Query seemed a good choice for our project. It handles a lot of the concerns we used to code ourselves, lifting the burden of maintenance from our shoulders so that we can focus on domain logic. Beware, we didn’t get rid of MST, though. There are still some fragments of the application state that are purely client side and they need to be handled separately. I still think MST is a great tool and we’ll keep on using it - just not to keep server-state in sync! As with everything in engineering, no solution fits all problems.

Ultimately, this post focuses on the key points that motivated our shift and how React Query removes complexity from our side. It is not meant to sell it as a technology in a vacuum. Surely six months from now, I’ll be fighting with some unintuitive defaults or some overly smart behaviour that conflicts with our business logic. Yet, for now, all is well at Mind Foundry.