DataLoaders & Performance

DataLoaders & Performance

DataLoaders solve the N+1 problem in GraphQL. When you query a list of items with related data, naive resolvers make separate database queries for each item. DataLoaders batch these queries into a single database call.

The N+1 Problem

Here's the problem DataLoaders solve:

// Query
query {
  users {      // 1 query: SELECT * FROM users
    name
    posts {    // N queries: SELECT * FROM posts WHERE author_id = ?
      title
      comments { // N*M queries: SELECT * FROM comments WHERE post_id = ?
        text
      }
    }
  }
}

// For 10 users with 5 posts each:
// 1 query for users
// 10 queries for posts
// 50 queries for comments
// Total: 61 queries!

This is the N+1 problem. Each level of nesting multiplies the number of queries.

How DataLoaders Work

DataLoaders batch multiple queries into a single database call and cache results within a request.

const DataLoader = require('dataloader');

// Create a batch function
const postLoader = new DataLoader(async (userIds) => {
  // Single query to fetch all posts for all users
  const posts = await db.posts.find({
    authorId: { $in: userIds }
  });
  
  // Group posts by authorId
  const postsByUser = userIds.map(userId => 
    posts.filter(post => post.authorId === userId)
  );
  
  return postsByUser;
});

// Use in resolver
const resolvers = {
  User: {
    posts: (parent, args, context) => {
      return context.loaders.posts.load(parent.id);
    },
  },
};

The DataLoader batches all posts requests and executes a single database query.

Setting Up DataLoaders

Create DataLoaders per request in your context to ensure data isolation:

const DataLoader = require('dataloader');

function createLoaders(db) {
  return {
    user: new DataLoader(async (ids) => {
      const users = await db.users.find({ _id: { $in: ids } });
      const userMap = new Map(users.map(u => [u.id, u]));
      return ids.map(id => userMap.get(id));
    }),
    
    postsByUser: new DataLoader(async (userIds) => {
      const posts = await db.posts.find({ authorId: { $in: userIds } });
      return userIds.map(id => posts.filter(p => p.authorId === id));
    }),
    
    commentsByPost: new DataLoader(async (postIds) => {
      const comments = await db.comments.find({ postId: { $in: postIds } });
      return postIds.map(id => comments.filter(c => c.postId === id));
    }),
  };
}

// Create fresh loaders per request
const context = ({ req }) => ({
  user: authenticate(req),
  loaders: createLoaders(database),
});

New loaders for each request prevent data leaking between users.

Benefits of DataLoaders

Batching: Reduces N+1 queries to a constant number of queries.

Caching: Within a request, the same ID is only fetched once.

Deduplication: Multiple requests for the same data are combined.

Simplicity: Resolvers stay clean - just call loader.load(id).

// Without DataLoader: 3 queries
user.posts.load(1) // Query 1
user.posts.load(2) // Query 2
user.posts.load(3) // Query 3

// With DataLoader: 1 query
user.posts.load(1)
user.posts.load(2)
user.posts.load(3)
// DataLoader batches into: WHERE author_id IN (1, 2, 3)