MongoDB Query Optimization Dashboard
Home Blog MongoDB Query Optimization

Why Is My MongoDB Query Slow? How to Fix Slow Queries on Millions of Records

Real explain() output, compound index strategies, aggregation pipeline fixes, and when to bring in Elasticsearch — all from optimizing 40M+ records in production.

Akshay Bangar
Written By
Akshay Bangar
Backend Engineer & Database Optimization

I've built and optimized MongoDB-backed applications processing 40M+ records in production. This guide shares the exact diagnostic techniques, indexing strategies, and architectural patterns I use to turn 12-second queries into 80ms responses.

You've written a MongoDB query. It works perfectly in development with 500 documents. You push to production with 10 million documents and suddenly that same query takes 12 seconds. Your API times out. Your users see a loading spinner that never stops. Sound familiar?

This is the single most common problem backend developers face with MongoDB at scale, and I've seen it happen dozens of times — including in my own applications. When I built Mongo Explorer, a tool for processing and filtering 40 million+ business records, every single query optimization technique in this guide was learned the hard way: through production slowdowns, 3 AM debugging sessions, and reading thousands of explain() outputs.

This guide will walk you through exactly how to diagnose why your MongoDB query is slow, fix it with the right indexes, optimize your aggregation pipelines, and — when MongoDB indexes alone aren't enough — bring in Elasticsearch for lightning-fast full-text search and filtering.

Who is this guide for? Backend developers using MongoDB (with Node.js/Mongoose or any driver) who are experiencing slow queries in production. You should know basic MongoDB CRUD operations. No prior indexing or Elasticsearch experience needed — I'll explain everything from scratch with real examples.

1. "Why Is My MongoDB Query Slow?" — The Real Answer

Let me give you the answer upfront: your query is slow because MongoDB is scanning every single document in your collection to find the results. This is called a COLLSCAN (Collection Scan), and it's the #1 cause of slow queries.

Think of it like this: imagine you have a library with 10 million books and someone asks for all books by "John Grisham." Without an index (the card catalog), you'd have to walk down every aisle, pull out every book, and check the author. That's a COLLSCAN. With an index on the author field, you jump directly to "Grisham" — done in milliseconds.

Here are the most common reasons developers' MongoDB queries are slow:

ProblemWhat's HappeningHow Common
No index on query fieldsCOLLSCAN — scanning every document⭐⭐⭐⭐⭐ Most common
Wrong compound index orderIndex exists but MongoDB can't use it efficiently⭐⭐⭐⭐
$match after $lookup in aggregationFiltering happens after the expensive join⭐⭐⭐⭐
Using .skip() for paginationMongoDB still scans all skipped documents⭐⭐⭐
Unbounded arrays in documentsDocuments grow to megabytes, slow to read⭐⭐⭐
Missing .lean() in MongooseMongoose hydrates full model objects instead of plain JSON⭐⭐⭐
Regex queries without anchoring/keyword/ can't use indexes, /^keyword/ can⭐⭐

2. How to Use explain() to Find the Problem

Before you fix anything, you need to diagnose the actual bottleneck. MongoDB's explain() method is your most powerful debugging tool. Here's how to use it:

Running explain() on Your Query

// In the MongoDB Shell (mongosh)
db.contacts.find({
  company: "Google",
  status: "active"
}).sort({ createdAt: -1 }).explain("executionStats")
// In Node.js / Mongoose
const result = await Contact.find({
  company: "Google",
  status: "active"
}).sort({ createdAt: -1 }).explain("executionStats");

console.log(JSON.stringify(result, null, 2));

How to Read the explain() Output (What Actually Matters)

The explain() output is massive and intimidating. Ignore 90% of it. Here are the only 4 fields you need to check:

// ❌ SLOW QUERY — Before optimization (COLLSCAN)
{
  "executionStats": {
    "executionTimeMillis": 12430,          // 12.4 SECONDS — terrible
    "totalDocsExamined": 41283947,         // Scanned ALL 41M documents
    "totalKeysExamined": 0,                // No index keys used at all
    "nReturned": 2847                      // Only needed 2,847 results
  },
  "winningPlan": {
    "stage": "COLLSCAN"                    // 🔴 COLLECTION SCAN = problem
  }
}
Red Flag: If totalDocsExamined is vastly larger than nReturned, your query is doing way too much work. In the example above, MongoDB scanned 41 million documents to return just 2,847. That's a ratio of 14,500:1 — catastrophically inefficient.
// ✅ FAST QUERY — After adding a compound index (IXSCAN)
{
  "executionStats": {
    "executionTimeMillis": 83,             // 83ms — 150x FASTER
    "totalDocsExamined": 2847,             // Only examined what it returned
    "totalKeysExamined": 2847,             // Used index keys efficiently
    "nReturned": 2847                      // Perfect 1:1 ratio
  },
  "winningPlan": {
    "stage": "IXSCAN",                     // 🟢 INDEX SCAN = good
    "indexName": "company_1_status_1_createdAt_-1"
  }
}

The goal is simple: make totalDocsExamined as close to nReturned as possible. A 1:1 ratio means MongoDB is reading only the documents it actually needs.

3. How to Create the Right Indexes (The ESR Rule)

The most common question I see on StackOverflow: "I created an index but my query is still slow — why?" The answer is almost always: the fields in your index are in the wrong order.

MongoDB uses the ESR Rule (Equality → Sort → Range) for compound index field ordering:

PositionField TypeExampleWhy This Order?
1st — EqualityExact match filters (=)status: "active"Narrows the result set immediately
2nd — SortSort fields.sort({ createdAt: -1 })Allows in-memory sort to be skipped
3rd — RangeRange filters ($gte, $lt, $in)age: { $gte: 25 }Range breaks sort order, so it goes last

Real Example: The Query That Took 12 Seconds

Here's the actual scenario from my Mongo Explorer project. I needed to find all active contacts from a specific company, sorted by most recently added:

// The query
db.contacts.find({
  company: "Infosys",         // Equality
  status: "active",           // Equality
  employeeCount: { $gte: 50 } // Range
}).sort({ createdAt: -1 })     // Sort
// ❌ WRONG index (bad field order)
db.contacts.createIndex({ createdAt: -1, company: 1, status: 1 })
// Sort field is FIRST — this defeats the purpose. MongoDB can't efficiently 
// narrow results before sorting.

// ✅ RIGHT index (ESR rule)
db.contacts.createIndex({ company: 1, status: 1, createdAt: -1, employeeCount: 1 })
// Equality → Sort → Range. MongoDB narrows by company + status,
// walks the index in sorted order, and then filters by range.
Quick Tip: Check Your Existing Indexes
Before creating new indexes, see what you already have:
db.contacts.getIndexes()  // Lists all indexes on the collection
Every index consumes RAM and slows down writes. Don't create duplicates — remove unused indexes with db.contacts.dropIndex("index_name").

Covered Queries — The Fastest Query Possible

A covered query is one where MongoDB can answer the query entirely from the index, without ever reading the actual documents. It's the fastest possible query type.

// Create an index that "covers" all fields the query needs
db.contacts.createIndex({ company: 1, status: 1, email: 1 })

// This query is "covered" — MongoDB never touches the documents
db.contacts.find(
  { company: "Google", status: "active" },
  { email: 1, _id: 0 }  // Projection: only return 'email'
)

// explain() output for a covered query:
// totalDocsExamined: 0     ← Zero documents read!
// totalKeysExamined: 2847  ← All data came from the index

4. "Why Is My Aggregation Pipeline So Slow?"

Aggregation pipelines are the most powerful querying tool in MongoDB — and the easiest to accidentally make catastrophically slow. The order of your pipeline stages completely determines performance.

The #1 Mistake: Filtering After the Expensive Stage

// ❌ SLOW — $lookup runs on ALL 41M documents, THEN filters
db.contacts.aggregate([
  {
    $lookup: {
      from: "companies",
      localField: "companyId",
      foreignField: "_id",
      as: "companyDetails"
    }
  },
  { $unwind: "$companyDetails" },
  { $match: { status: "active", "companyDetails.industry": "Technology" } },
  { $sort: { createdAt: -1 } },
  { $limit: 50 }
])
// MongoDB joins ALL 41M contacts with companies, THEN filters.
// This took 45+ seconds in production.
// ✅ FAST — Filter FIRST, then join only the matching documents
db.contacts.aggregate([
  { $match: { status: "active" } },         // Step 1: Filter to 8M docs (uses index)
  { $sort: { createdAt: -1 } },             // Step 2: Sort (uses index)
  { $limit: 1000 },                         // Step 3: Only process 1000 docs from here
  {
    $lookup: {
      from: "companies",
      localField: "companyId",
      foreignField: "_id",
      as: "companyDetails"
    }
  },
  { $unwind: "$companyDetails" },
  { $match: { "companyDetails.industry": "Technology" } },
  { $limit: 50 },
  {
    $project: {                             // Step 4: Return only needed fields
      name: 1,
      email: 1,
      "companyDetails.name": 1,
      "companyDetails.industry": 1,
      createdAt: 1
    }
  }
])
// Now MongoDB filters 41M → 8M → 1000, THEN joins only 1000 docs.
// Execution time: 120ms.

The Golden Rules of Pipeline Performance

RuleWhy
$match FIRST — alwaysReduces documents flowing into every subsequent stage. Can use indexes.
$project EARLYDrop fields you don't need. Less data = faster pipeline.
$sort after $matchIf $match + $sort use the same compound index, sort is free (no in-memory sort).
$limit before $lookupPrevents joining millions of documents when you only need 50 results.
allowDiskUse: truePipelines have a 100MB RAM limit per stage. This flag uses disk as overflow.

5. "Why Does My Pagination Get Slower on Page 100?"

If you're using .skip() for pagination, your later pages will be progressively slower. Here's why and how to fix it:

// ❌ SLOW — .skip() still scans all skipped documents internally
// Page 1:  .skip(0).limit(20)    → fast (scans 20 docs)
// Page 50: .skip(980).limit(20)  → slower (scans 1000 docs)
// Page 500: .skip(9980).limit(20) → VERY slow (scans 10,000 docs)

db.contacts.find({ status: "active" })
  .sort({ createdAt: -1 })
  .skip(9980)
  .limit(20)
// MongoDB reads 10,000 documents, throws away 9,980, returns 20. Wasteful.
// ✅ FAST — Cursor-based pagination (keyset pagination)
// Instead of "skip N documents", say "give me documents AFTER this timestamp"

// Page 1 (no cursor needed)
const page1 = await Contact.find({ status: "active" })
  .sort({ createdAt: -1 })
  .limit(20);

// Page 2+ (use the last document's createdAt as the cursor)
const lastDoc = page1[page1.length - 1];
const page2 = await Contact.find({
  status: "active",
  createdAt: { $lt: lastDoc.createdAt }  // "after this timestamp"
}).sort({ createdAt: -1 }).limit(20);

// This is O(1) — equally fast on page 1 and page 10,000.
// Requires an index on { status: 1, createdAt: -1 }

6. Node.js / Mongoose-Specific Optimizations

If you're using Mongoose (which most Node.js developers do), there are Mongoose-specific performance traps you need to know about:

Use .lean() for Read-Heavy Queries

// ❌ SLOW — Mongoose creates full model instances with change tracking
const contacts = await Contact.find({ status: "active" });
// Each document is a full Mongoose Document with getters, setters,
// save(), validate(), and change tracking. Heavy on memory and CPU.

// ✅ FAST — .lean() returns plain JavaScript objects
const contacts = await Contact.find({ status: "active" }).lean();
// Returns raw JSON objects. 3-5x faster for read-only operations.
// Use this for API responses where you don't need to modify and save back.

Use .select() to Return Only Needed Fields

// ❌ Returns ALL 30+ fields for each document
const contacts = await Contact.find({ status: "active" });

// ✅ Returns only the 4 fields the API actually needs
const contacts = await Contact.find({ status: "active" })
  .select("name email company createdAt")
  .lean();

Avoid the N+1 Query Problem

// ❌ N+1 PROBLEM — 1 query + N additional queries
const contacts = await Contact.find({ status: "active" });
for (const contact of contacts) {
  const company = await Company.findById(contact.companyId);  // 1 query per contact!
  contact.company = company;
}
// If 1000 contacts are returned, this fires 1001 database queries.

// ✅ FIX — Use .populate() or $lookup (single query)
const contacts = await Contact.find({ status: "active" })
  .populate("companyId", "name industry")
  .lean();
// MongoDB resolves all company references in a single batch query.

7. When MongoDB Isn't Enough: Adding Elasticsearch

There comes a point where even perfectly indexed MongoDB queries aren't fast enough. If you need any of the following, it's time to add Elasticsearch as a dedicated search layer:

  • Full-text search across multiple fields (name + email + company + title)
  • Fuzzy matching — users misspelling "Gogle" and still finding "Google"
  • Autocomplete / search-as-you-type suggestions
  • Faceted filtering — filter by industry, location, company size simultaneously
  • Sub-100ms responses on queries across 40M+ documents with complex filters

The Architecture: MongoDB + Elasticsearch

flowchart LR App([Node.js Application]) -->|Writes & Updates| MongoDB[(MongoDB
System of Record)] MongoDB -->|Change Streams / CDC| Sync["Monstache
or Kafka Connector"] Sync -->|Real-time Sync| ES[(Elasticsearch
Search Layer)] App -->|Search Queries| ES style MongoDB fill:#023430,stroke:#00ED64,color:#fff style ES fill:#1a1a2e,stroke:#f59e0b,color:#fff style Sync fill:#111118,stroke:#818cf8,color:#f1f5f9

The architecture is clean:

  1. MongoDB remains your system of record. All writes (create, update, delete) go to MongoDB.
  2. Elasticsearch is your search engine. All search queries are routed to Elasticsearch.
  3. A sync layer keeps them in sync. Use MongoDB Change Streams with a tool like Monstache (open source) or a Kafka connector to pipe changes from MongoDB to Elasticsearch in real-time.

Setting Up Elasticsearch with Node.js

// Install the official Elasticsearch client
// npm install @elastic/elasticsearch

const { Client } = require("@elastic/elasticsearch");

const esClient = new Client({
  node: "http://localhost:9200",
  // For Elastic Cloud:
  // cloud: { id: "your-cloud-id" },
  // auth: { apiKey: "your-api-key" }
});

// Create an index with explicit mapping (don't rely on dynamic mapping)
await esClient.indices.create({
  index: "contacts",
  body: {
    mappings: {
      properties: {
        name:         { type: "text", analyzer: "standard" },
        email:        { type: "keyword" },   // keyword = exact match, no analysis
        company:      { type: "text", fields: { keyword: { type: "keyword" } } },
        industry:     { type: "keyword" },
        status:       { type: "keyword" },
        employeeCount: { type: "integer" },
        createdAt:    { type: "date" },
        location:     { type: "text", fields: { keyword: { type: "keyword" } } }
      }
    }
  }
});
// Multi-field search with fuzzy matching and filters
// User searches: "gogle" (typo) with filters: industry=Technology, status=active
const results = await esClient.search({
  index: "contacts",
  body: {
    query: {
      bool: {
        must: [
          {
            multi_match: {
              query: "gogle",              // User's misspelled search
              fields: ["name", "company", "email"],
              fuzziness: "AUTO"            // Automatically corrects typos
            }
          }
        ],
        filter: [                          // Filters use cache — very fast
          { term: { status: "active" } },
          { term: { industry: "Technology" } },
          { range: { employeeCount: { gte: 50 } } }
        ]
      }
    },
    sort: [{ createdAt: { order: "desc" } }],
    size: 50,
    _source: ["name", "email", "company", "industry", "createdAt"]
  }
});

// Response time: 23ms across 40M+ documents
// MongoDB equivalent (even with indexes): 800ms+
// Without indexes: 12,000ms+

Building Autocomplete / Search-as-You-Type

// Add a search_as_you_type field to your mapping
await esClient.indices.create({
  index: "contacts_autocomplete",
  body: {
    mappings: {
      properties: {
        name: { type: "search_as_you_type" },  // Built-in autocomplete support
        company: { type: "search_as_you_type" }
      }
    }
  }
});

// Query: user types "mic" → suggests "Microsoft", "Micron", "Michael Smith"
const suggestions = await esClient.search({
  index: "contacts_autocomplete",
  body: {
    query: {
      multi_match: {
        query: "mic",
        type: "bool_prefix",
        fields: ["name", "name._2gram", "name._3gram", "company", "company._2gram"]
      }
    },
    size: 10
  }
});
// Response time: 8ms — perfect for real-time search boxes

Syncing MongoDB → Elasticsearch with Change Streams

// Real-time sync using MongoDB Change Streams (Node.js)
const { MongoClient } = require("mongodb");
const { Client } = require("@elastic/elasticsearch");

const mongo = await MongoClient.connect("mongodb://localhost:27017");
const db = mongo.db("myapp");
const esClient = new Client({ node: "http://localhost:9200" });

// Watch for all changes on the contacts collection
const changeStream = db.collection("contacts").watch([], {
  fullDocument: "updateLookup"  // Include the full updated document
});

changeStream.on("change", async (change) => {
  try {
    switch (change.operationType) {
      case "insert":
      case "update":
      case "replace":
        await esClient.index({
          index: "contacts",
          id: change.documentKey._id.toString(),
          body: change.fullDocument
        });
        break;
      case "delete":
        await esClient.delete({
          index: "contacts",
          id: change.documentKey._id.toString()
        });
        break;
    }
  } catch (err) {
    console.error("ES sync error:", err.message);
    // In production: push to a dead-letter queue for retry
  }
});

console.log("Watching MongoDB changes and syncing to Elasticsearch...");
Production Tip: For production environments with millions of documents, use Monstache (a dedicated Go service) instead of writing your own Change Stream consumer. Monstache handles initial bulk sync, reconnection, error recovery, and transformation — things that are complex to build yourself. Install it with: go install github.com/rwynn/monstache/v6@latest

When to Use MongoDB vs. Elasticsearch

Use CaseUse MongoDBUse Elasticsearch
CRUD operations (create, read, update, delete)✅ Primary choice❌ Not designed for this
Transactions (multi-document atomicity)✅ Native ACID support❌ No transaction support
Exact match queries (find by ID, status)✅ Fast with indexes✅ Also fast
Full-text search (across multiple fields)⚠️ Basic $text index✅ Purpose-built, far superior
Fuzzy search (typo tolerance)❌ Not supported✅ Native fuzziness support
Autocomplete / suggestions❌ Regex is slowsearch_as_you_type field
Faceted filtering + aggregations⚠️ Slow on large datasets✅ Extremely fast
Log analytics / time-series⚠️ Not optimized✅ Designed for this

8. MongoDB Performance Checklist

Before you consider your MongoDB setup "production-optimized," verify every item:

  • Run explain() on every frequently-used query — verify IXSCAN, not COLLSCAN
  • Compound indexes follow ESR — Equality → Sort → Range field order
  • No unnecessary indexes — run db.collection.getIndexes() and drop unused ones
  • $match is FIRST in every aggregation pipeline
  • Cursor-based pagination — replaced .skip() with $lt / $gt on a sorted field
  • .lean() used on all read-only Mongoose queries
  • .select() used to return only needed fields
  • No N+1 queries — using .populate() or $lookup for joins
  • Document size under control — no unbounded arrays growing infinitely
  • Elasticsearch added for full-text search, fuzzy matching, and autocomplete
  • allowDiskUse: true set on aggregations that may exceed 100MB RAM limit
  • MongoDB Profiler enableddb.setProfilingLevel(1, { slowms: 100 }) to catch slow queries

Frequently Asked Questions

How many indexes should I have per collection?

There's no hard limit, but every index consumes RAM and slows down write operations (inserts, updates, deletes) because MongoDB must update all indexes. As a rule of thumb, 5-10 well-designed indexes per collection is typical. Use db.collection.stats() to see total index size, and ensure it fits in RAM.

Should I index every field that I query on?

No. Only index fields that appear in frequently executed queries, filters, and sorts. A field queried once a day doesn't need an index. Focus on the queries that power your API endpoints — those are hit thousands of times per hour.

Does MongoDB support full-text search without Elasticsearch?

Yes, MongoDB has a built-in $text index that supports basic full-text search. However, it's significantly limited compared to Elasticsearch: no fuzzy matching, no field boosting, no autocomplete, and poor performance on large datasets. For production search features, Elasticsearch is the industry standard.

How do I monitor slow queries in production?

Enable the MongoDB Database Profiler: db.setProfilingLevel(1, { slowms: 100 }). This logs all queries taking longer than 100ms to the system.profile collection. Query it with db.system.profile.find().sort({ ts: -1 }).limit(10) to see recent slow queries. On MongoDB Atlas, use the built-in Performance Advisor — it automatically recommends missing indexes.

Conclusion

Every MongoDB performance problem I've encountered in production comes back to the same fundamentals: missing indexes, wrong index field order, or doing too much work in the pipeline before filtering.

Start with explain(). Follow the ESR rule. Move $match first. Use cursor pagination. And when you outgrow what MongoDB indexes can handle — bring in Elasticsearch for the search layer.

These techniques took my Mongo Explorer queries from 12+ seconds to under 100ms across 40 million records. They'll do the same for yours.

Have a suggestion or found an issue?
I'm always looking to improve these guides. If you have any feedback, corrections, or just want to discuss MongoDB performance, please reach out to me here.