v0.11 Queries, Queryable, and useQuery

April 8, 2024 · 9 min read

Creator of Reactive Data Client

Besides the performance and data integrity benefits of normalizing the state, we get the added benefit of being able to safely access and query the store directly.

In this release we tune and simplify this functionality by unifying around the concepts of Querable Schemas. These include Entity, All, Collection, Query, and Union

The biggest impact of this change is the introduction of a new hook useQuery(), which allows direct store lookups using the Querable Schemas.

class User extends Entity {
  username = '';
  id = '';
  groupId = '';
  pk() {
    return this.id;
  }
  static index = ['username' as const];
}

const bob = useQuery(User, { username: 'bob' });
const bob = useQuery(User, { id: '5' });

Similarly, we can lookup Querables with controller and snapshot using the controller.get

const bob = snapshot.get(User, { username: 'bob' });

Additionally, we have invested in further performance improvements, resulting in around a 2x performance increase for most operations and Queries being 16x faster.

Migration guide

Breaking Changes:

useCache(new Index(MyEntity)) -> useQuery(MyEntity)
new Query -> new schema.Query
- useCache(myQuery) -> useQuery(myQuery)
new AbortOptimistic() -> snapshot.abort
Entity.useIncoming → Entity.shouldUpdate

Other Highlights:

useCache() accepts Endpoints with sideEffects (like Resource.update)
Allow Entity.pk() to return numbers.

Querable

Queryable schemas require an queryKey() method that returns something. These include Entity, All, Collection, Query, and Union.

interface Queryable {
  queryKey(
    args: readonly any[],
    queryKey: (...args: any) => any,
    getEntity: GetEntity,
    getIndex: GetIndex,
    // `{}` means non-void
  ): {};
}

schema.Query

Query was previously implemented as an Endpoint. Since we now have methods of retrieving Queryable schemas directly, we can simply make Query another Queryable schema itself. It should be basically the same, allowing arbitrary computation on the value from a store. #2921

const getUserCount = new schema.Query(
  new schema.All(User),
  (entries, { isAdmin } = {}) => {
    if (isAdmin !== undefined)
      return entries.filter(user => user.isAdmin === isAdmin).length;
    return entries.length;
  },
);

const userCount = useQuery(getUserCount);
const adminCount = useQuery(getUserCount, { isAdmin: true });

schema.Query disallows non-Queryable schemas now.

const getBob = new schema.Query(
  // Array is not Queryable
  new schema.Array({ bob: '' }),
  thing => {
    return thing;
  },
);

controller.get / snapshot.get

We can now access the store with just a Queryable Schema - we no longer need an entire Endpoint. #2921

Before
export const PostResource = createResource({
  path: '/posts/:id',
  schema: Post,
}).extend(Base => ({
  vote: new RestEndpoint({
    path: '/posts/:id/vote',
    method: 'POST',
    body: undefined,
    schema: Post,
    getOptimisticResponse(snapshot, { id }) {
      const { data } = snapshot.getResponse(Base.get, { id });
      if (!data) throw new AbortOptimistic();
      return {
        id,
        votes: data.votes + 1,
      };
    },
  }),
}));

After
export const PostResource = createResource({
  path: '/posts/:id',
  schema: Post,
}).extend('vote', {
  path: '/posts/:id/vote',
  method: 'POST',
  body: undefined,
  schema: Post,
  getOptimisticResponse(snapshot, { id }) {
    const post = snapshot.get(Post, { id });
    if (!post) throw snapshot.abort;
    return {
      id,
      votes: post.votes + 1,
    };
  },
});

useQuery

Renders any Queryable Schema from the store. Queries are a great companion to efficiently render aggregate computations like those that use groupBy, map, reduce, and filter. #2921

class User extends Entity {
  username = '';
  id = '';
  groupId = '';
  pk() {
    return this.id;
  }
  static index = ['username' as const];
}

const bob = useQuery(User, { username: 'bob' });

const getUserCount = new schema.Query(
  new schema.All(User),
  (entries, { isAdmin } = {}) => {
    if (isAdmin !== undefined)
      return entries.filter(user => user.isAdmin === isAdmin).length;
    return entries.length;
  },
);

const userCount = useQuery(getUserCount);
const adminCount = useQuery(getUserCount, { isAdmin: true });

const UserCollection = new schema.Collection([User], {
  argsKey: (urlParams: { groupId?: string }) => ({
    ...urlParams,
  }),
});

const usersInGroup = useQuery(UserCollection, { groupId: '5' });

Performance

Further optimizations of both the caching system, and all critical code paths has given a decent boost of 2x to the most common operations. With the addition of query caching capability, we see an even more impressive 16x improvement to our Query sorting benchmark.

These adds to the existing (up to 23x) lead over the legacy normalizr library.

Benchmarks over time | View benchmark

Other improvements

useCache() accepts Endpoints with sideEffects (like Resource.update) #2963
```
const lastCreated = useCache(MyResource.getList.push);
```

Allow pk to return numbers #2961

Before
class MyEntity extends Entity {
  id = 0;
  pk() {
    return `${this.id}`;
  }
}

After
class MyEntity extends Entity {
  id = 0;
  pk() {
    return this.id;
  }
}

Typing

Improve .extend() typing when using loose null checks and no body parameter #2962

Parameters<typeof new RestEndpoint({ path: '/test' }).extend({ path: '/test2' })> === []

Default Collection Args type is: 8377e

| []
| [urlParams: Record<string, any>]
| [urlParams: Record<string, any>, body: any]

Fixes
- schema.All denormalize INVALID case should also work when class name mangling is performed in production builds #2954
  - unvisit() always returns undefined with undefined as input.
  - All returns INVALID from queryKey() to invalidate what was previously a special case in unvisit() (when there is no table entry for the given entity)
- Missing nested entities should appear once they are present #2956, #2961

Migration guide

This upgrade requires updating all package versions simultaneously.

NPM
Yarn
esm.sh

yarn add @data-client/react@^0.11.0 @data-client/rest@^0.11.0 @data-client/test@^0.11.0 @data-client/img@^0.11.0 @data-client/hooks@^0.11.0

npm install --save @data-client/react@^0.11.0 @data-client/rest@^0.11.0 @data-client/test@^0.11.0 @data-client/img@^0.11.0 @data-client/hooks@^0.11.0

<script type="module">
  import * from 'https://esm.sh/@data-client/react@^0.11.0';
  import * from 'https://esm.sh/@data-client/rest@^0.11.0';
  import * from 'https://esm.sh/@data-client/test@^0.11.0';
  import * from 'https://esm.sh/@data-client/img@^0.11.0';
  import * from 'https://esm.sh/@data-client/hooks@^0.11.0';
</script>

Removed Index

Just use Entities directly in useQuery

Before
const UserIndex = new Index(User);

const bob = useCache(UserIndex, { username: 'bob' });

After
const bob = useQuery(User, { username: 'bob' });

new Query -> new schema.Query

Query is now a schema; used with useQuery

Before
const getUserCount = new Query(
  new schema.All(User),
  (entries, { isAdmin } = {}) => {
    if (isAdmin !== undefined)
      return entries.filter(user => user.isAdmin === isAdmin).length;
    return entries.length;
  },
);

const userCount = useCache(getUserCount);
const adminCount = useCache(getUserCount, { isAdmin: true });

After
const getUserCount = new schema.Query(
  new schema.All(User),
  (entries, { isAdmin } = {}) => {
    if (isAdmin !== undefined)
      return entries.filter(user => user.isAdmin === isAdmin).length;
    return entries.length;
  },
);

const userCount = useQuery(getUserCount);
const adminCount = useQuery(getUserCount, { isAdmin: true });

new AbortOptimistic() -> snapshot.abort

snapshot.abort removes the need to import AbortOptimistic #2957

Before
getOptimisticResponse(snapshot, { id }) {
  const { data } = snapshot.getResponse(Base.get, { id });
  if (!data) throw new AbortOptimistic();
  return {
    id,
    votes: data.votes + 1,
  };
}

After
getOptimisticResponse(snapshot, { id }) {
  const { data } = snapshot.getResponse(Base.get, { id });
  if (!data) throw snapshot.abort;
  return {
    id,
    votes: data.votes + 1,
  };
}

Entity.useIncoming → Entity.shouldUpdate

Make Entity.shouldUpdate name consistent with Entity.shouldReorder #2972

Before
class MyEntity extends Entity {
  static useIncoming(
    existingMeta: { date: number },
    incomingMeta: { date: number },
    existing: any,
    incoming: any,
  ) {
    return !deepEquals(existing, incoming);
  }
}

After
class MyEntity extends Entity {
  static shouldUpdate(
    existingMeta: { date: number },
    incomingMeta: { date: number },
    existing: any,
    incoming: any,
  ) {
    return !deepEquals(existing, incoming);
  }
}

useDLE() reactive native focus handling

Like useSuspense() and useFetch(); useDLE() will now potentially revalidate data on focus events.

When using React Navigation, useDLE() will trigger fetches on focus if the data is considered stale.

This could result in more fetches than occured previously.

Potential, but unlikely breaking changes

Schema.infer() -> Schema.queryKey() #2977

Most people have not overridden the infer() method. In this case you need to rename to queryKey(), as well as update the parameter list.

Before
class MyEntity extends Entity {
  static infer(
    args: readonly any[],
    indexes: NormalizedIndex,
    recurse: any,
    entities: any,
  ): any {
    if (SILLYCONDITION) return undefined;
    if (entities[this.key]?.[someId]) return someId;
    return super.infer(args, indexes, recurse, entities);
  }
}

After
class MyEntity extends Entity {
  static queryKey(
    args: readonly any[],
    queryKey: (...args: any) => any,
    getEntity: GetEntity,
    getIndex: GetIndex,
  ): any {
    if (SILLYCONDITION) return undefined;
    if (getEntity(this.key, someId)) return someId;
    return super.queryKey(args, indexes, recurse, entities);
  }
}

getEntity(key, pk?)

Gets all entities of a type with one argument, or a single entity with two

One argument
const entitiesEntry = getEntity(this.schema.key);
if (entitiesEntry === undefined) return INVALID;
return Object.values(entitiesEntry).map(
  entity => entity && this.schema.pk(entity),
);

Two arguments
if (getEntity(this.key, id)) return id;

getIndex(key, indexName, value)

Returns the index entry (value->pk map)

const value = args[0][indexName];
return getIndex(schema.key, indexName, value)[value];

Always normalize pk to string type #2961

Subtle change into what will be in the store. This will likely only matter when using @data-client/normalizr directly.

Before
{
  "Article": {
    "123": {
      "author": 8472,
      "id": 123,
      "title": "A Great Article"
    }
  },
  "User": {
    "8472": {
      "id": 8472,
      "name": "Paul"
    }
  }
}

After
{
  "Article": {
    "123": {
      "author": "8472",
      "id": 123,
      "title": "A Great Article"
    }
  },
  "User": {
    "8472": {
      "id": 8472,
      "name": "Paul"
    }
  }
}

state.results -> state.endpoints #2971

This will only likely matter when consuming @data-client/core directly as internal state is structure is opaque to @data-client/react.

{
  "entities": {
    "CoolerArticle": {
      "5": {
        "content": "more things here",
        "id": 5,
        "title": "hi"
      }
    }
  },
  "results": {
    "GET http://test.com/article-cooler/5": "5"
  }
}

{
  "entities": {
    "CoolerArticle": {
      "5": {
        "content": "more things here",
        "id": 5,
        "title": "hi"
      }
    }
  },
  "endpoints": {
    "GET http://test.com/article-cooler/5": "5"
  }
}

inferResults() -> memo.buildQueryKey() #2977, #2978

These methods are exported in @data-client/core and @data-client/normalizr.

denormalizeCached() -> memo.denormalize() #2978

Exported in @data-client/normalizr

Before
const endpointCache = new WeakEntityMap();
const entityCache = {};
denormalizeCached(
  input,
  schema,
  state.entities,
  entityCache,
  endpointCache,
  args,
);

After
const memo = new MemoCached();
memo.denormalize(input, schema, state.entities, args);

WeakEntityMap -> WeakDependencyMap #2978

Exported in @data-client/normalizr

We generalize this data type so it can be used with other dependencies.

Before
new WeakEntityMap();

After
new WeakDependencyMap<EntityPath>();

Upgrade support

As usual, if you have any troubles or questions, feel free to join our or file a bug

Querable​

schema.Query​

controller.get / snapshot.get​

useQuery​

Performance​

Other improvements​

Migration guide​

Removed Index​

new Query -> new schema.Query​

new AbortOptimistic() -> snapshot.abort​

Entity.useIncoming → Entity.shouldUpdate​

useDLE() reactive native focus handling​

Potential, but unlikely breaking changes​

Schema.infer() -> Schema.queryKey() #2977​

getEntity(key, pk?)​

getIndex(key, indexName, value)​

Always normalize pk to string type #2961​

state.results -> state.endpoints #2971​

inferResults() -> memo.buildQueryKey() #2977, #2978​

denormalizeCached() -> memo.denormalize() #2978​

WeakEntityMap -> WeakDependencyMap #2978​

Upgrade support​

Querable

schema.Query

controller.get / snapshot.get

useQuery

Performance

Other improvements

Migration guide

Removed Index

new Query -> new schema.Query

new AbortOptimistic() -> snapshot.abort

Entity.useIncoming → Entity.shouldUpdate

useDLE() reactive native focus handling

Potential, but unlikely breaking changes

Schema.infer() -> Schema.queryKey() #2977

getEntity(key, pk?)

getIndex(key, indexName, value)

Always normalize pk to string type #2961

state.results -> state.endpoints #2971

inferResults() -> memo.buildQueryKey() #2977, #2978

denormalizeCached() -> memo.denormalize() #2978

WeakEntityMap -> WeakDependencyMap #2978

Upgrade support