You've written a MongoDB query. It works perfectly in development with 500 documents. You push to production with 10 million documents and suddenly that same query takes 12 seconds. Your API times out. Your users see a loading spinner that never stops. Sound familiar?
This is the single most common problem backend developers face with MongoDB at scale, and I've seen it happen dozens of times — including in my own applications. When I built Mongo Explorer, a tool for processing and filtering 40 million+ business records, every single query optimization technique in this guide was learned the hard way: through production slowdowns, 3 AM debugging sessions, and reading thousands of explain() outputs.
This guide will walk you through exactly how to diagnose why your MongoDB query is slow, fix it with the right indexes, optimize your aggregation pipelines, and — when MongoDB indexes alone aren't enough — bring in Elasticsearch for lightning-fast full-text search and filtering.
1. "Why Is My MongoDB Query Slow?" — The Real Answer
Let me give you the answer upfront: your query is slow because MongoDB is scanning every single document in your collection to find the results. This is called a COLLSCAN (Collection Scan), and it's the #1 cause of slow queries.
Think of it like this: imagine you have a library with 10 million books and someone asks for all books by "John Grisham." Without an index (the card catalog), you'd have to walk down every aisle, pull out every book, and check the author. That's a COLLSCAN. With an index on the author field, you jump directly to "Grisham" — done in milliseconds.
Here are the most common reasons developers' MongoDB queries are slow:
| Problem | What's Happening | How Common |
|---|---|---|
| No index on query fields | COLLSCAN — scanning every document | ⭐⭐⭐⭐⭐ Most common |
| Wrong compound index order | Index exists but MongoDB can't use it efficiently | ⭐⭐⭐⭐ |
| $match after $lookup in aggregation | Filtering happens after the expensive join | ⭐⭐⭐⭐ |
| Using .skip() for pagination | MongoDB still scans all skipped documents | ⭐⭐⭐ |
| Unbounded arrays in documents | Documents grow to megabytes, slow to read | ⭐⭐⭐ |
| Missing .lean() in Mongoose | Mongoose hydrates full model objects instead of plain JSON | ⭐⭐⭐ |
| Regex queries without anchoring | /keyword/ can't use indexes, /^keyword/ can | ⭐⭐ |
2. How to Use explain() to Find the Problem
Before you fix anything, you need to diagnose the actual bottleneck. MongoDB's explain() method is your most powerful debugging tool. Here's how to use it:
Running explain() on Your Query
// In the MongoDB Shell (mongosh)
db.contacts.find({
company: "Google",
status: "active"
}).sort({ createdAt: -1 }).explain("executionStats")
// In Node.js / Mongoose
const result = await Contact.find({
company: "Google",
status: "active"
}).sort({ createdAt: -1 }).explain("executionStats");
console.log(JSON.stringify(result, null, 2));
How to Read the explain() Output (What Actually Matters)
The explain() output is massive and intimidating. Ignore 90% of it. Here are the only 4 fields you need to check:
// ❌ SLOW QUERY — Before optimization (COLLSCAN)
{
"executionStats": {
"executionTimeMillis": 12430, // 12.4 SECONDS — terrible
"totalDocsExamined": 41283947, // Scanned ALL 41M documents
"totalKeysExamined": 0, // No index keys used at all
"nReturned": 2847 // Only needed 2,847 results
},
"winningPlan": {
"stage": "COLLSCAN" // 🔴 COLLECTION SCAN = problem
}
}
totalDocsExamined is vastly larger than nReturned, your query is doing way too much work. In the example above, MongoDB scanned 41 million documents to return just 2,847. That's a ratio of 14,500:1 — catastrophically inefficient.
// ✅ FAST QUERY — After adding a compound index (IXSCAN)
{
"executionStats": {
"executionTimeMillis": 83, // 83ms — 150x FASTER
"totalDocsExamined": 2847, // Only examined what it returned
"totalKeysExamined": 2847, // Used index keys efficiently
"nReturned": 2847 // Perfect 1:1 ratio
},
"winningPlan": {
"stage": "IXSCAN", // 🟢 INDEX SCAN = good
"indexName": "company_1_status_1_createdAt_-1"
}
}
The goal is simple: make totalDocsExamined as close to nReturned as possible. A 1:1 ratio means MongoDB is reading only the documents it actually needs.
3. How to Create the Right Indexes (The ESR Rule)
The most common question I see on StackOverflow: "I created an index but my query is still slow — why?" The answer is almost always: the fields in your index are in the wrong order.
MongoDB uses the ESR Rule (Equality → Sort → Range) for compound index field ordering:
| Position | Field Type | Example | Why This Order? |
|---|---|---|---|
| 1st — Equality | Exact match filters (=) | status: "active" | Narrows the result set immediately |
| 2nd — Sort | Sort fields | .sort({ createdAt: -1 }) | Allows in-memory sort to be skipped |
| 3rd — Range | Range filters ($gte, $lt, $in) | age: { $gte: 25 } | Range breaks sort order, so it goes last |
Real Example: The Query That Took 12 Seconds
Here's the actual scenario from my Mongo Explorer project. I needed to find all active contacts from a specific company, sorted by most recently added:
// The query
db.contacts.find({
company: "Infosys", // Equality
status: "active", // Equality
employeeCount: { $gte: 50 } // Range
}).sort({ createdAt: -1 }) // Sort
// ❌ WRONG index (bad field order)
db.contacts.createIndex({ createdAt: -1, company: 1, status: 1 })
// Sort field is FIRST — this defeats the purpose. MongoDB can't efficiently
// narrow results before sorting.
// ✅ RIGHT index (ESR rule)
db.contacts.createIndex({ company: 1, status: 1, createdAt: -1, employeeCount: 1 })
// Equality → Sort → Range. MongoDB narrows by company + status,
// walks the index in sorted order, and then filters by range.
Before creating new indexes, see what you already have:
db.contacts.getIndexes() // Lists all indexes on the collection
Every index consumes RAM and slows down writes. Don't create duplicates — remove unused indexes with db.contacts.dropIndex("index_name").
Covered Queries — The Fastest Query Possible
A covered query is one where MongoDB can answer the query entirely from the index, without ever reading the actual documents. It's the fastest possible query type.
// Create an index that "covers" all fields the query needs
db.contacts.createIndex({ company: 1, status: 1, email: 1 })
// This query is "covered" — MongoDB never touches the documents
db.contacts.find(
{ company: "Google", status: "active" },
{ email: 1, _id: 0 } // Projection: only return 'email'
)
// explain() output for a covered query:
// totalDocsExamined: 0 ← Zero documents read!
// totalKeysExamined: 2847 ← All data came from the index
4. "Why Is My Aggregation Pipeline So Slow?"
Aggregation pipelines are the most powerful querying tool in MongoDB — and the easiest to accidentally make catastrophically slow. The order of your pipeline stages completely determines performance.
The #1 Mistake: Filtering After the Expensive Stage
// ❌ SLOW — $lookup runs on ALL 41M documents, THEN filters
db.contacts.aggregate([
{
$lookup: {
from: "companies",
localField: "companyId",
foreignField: "_id",
as: "companyDetails"
}
},
{ $unwind: "$companyDetails" },
{ $match: { status: "active", "companyDetails.industry": "Technology" } },
{ $sort: { createdAt: -1 } },
{ $limit: 50 }
])
// MongoDB joins ALL 41M contacts with companies, THEN filters.
// This took 45+ seconds in production.
// ✅ FAST — Filter FIRST, then join only the matching documents
db.contacts.aggregate([
{ $match: { status: "active" } }, // Step 1: Filter to 8M docs (uses index)
{ $sort: { createdAt: -1 } }, // Step 2: Sort (uses index)
{ $limit: 1000 }, // Step 3: Only process 1000 docs from here
{
$lookup: {
from: "companies",
localField: "companyId",
foreignField: "_id",
as: "companyDetails"
}
},
{ $unwind: "$companyDetails" },
{ $match: { "companyDetails.industry": "Technology" } },
{ $limit: 50 },
{
$project: { // Step 4: Return only needed fields
name: 1,
email: 1,
"companyDetails.name": 1,
"companyDetails.industry": 1,
createdAt: 1
}
}
])
// Now MongoDB filters 41M → 8M → 1000, THEN joins only 1000 docs.
// Execution time: 120ms.
The Golden Rules of Pipeline Performance
| Rule | Why |
|---|---|
$match FIRST — always | Reduces documents flowing into every subsequent stage. Can use indexes. |
$project EARLY | Drop fields you don't need. Less data = faster pipeline. |
$sort after $match | If $match + $sort use the same compound index, sort is free (no in-memory sort). |
$limit before $lookup | Prevents joining millions of documents when you only need 50 results. |
allowDiskUse: true | Pipelines have a 100MB RAM limit per stage. This flag uses disk as overflow. |
5. "Why Does My Pagination Get Slower on Page 100?"
If you're using .skip() for pagination, your later pages will be progressively slower. Here's why and how to fix it:
// ❌ SLOW — .skip() still scans all skipped documents internally
// Page 1: .skip(0).limit(20) → fast (scans 20 docs)
// Page 50: .skip(980).limit(20) → slower (scans 1000 docs)
// Page 500: .skip(9980).limit(20) → VERY slow (scans 10,000 docs)
db.contacts.find({ status: "active" })
.sort({ createdAt: -1 })
.skip(9980)
.limit(20)
// MongoDB reads 10,000 documents, throws away 9,980, returns 20. Wasteful.
// ✅ FAST — Cursor-based pagination (keyset pagination)
// Instead of "skip N documents", say "give me documents AFTER this timestamp"
// Page 1 (no cursor needed)
const page1 = await Contact.find({ status: "active" })
.sort({ createdAt: -1 })
.limit(20);
// Page 2+ (use the last document's createdAt as the cursor)
const lastDoc = page1[page1.length - 1];
const page2 = await Contact.find({
status: "active",
createdAt: { $lt: lastDoc.createdAt } // "after this timestamp"
}).sort({ createdAt: -1 }).limit(20);
// This is O(1) — equally fast on page 1 and page 10,000.
// Requires an index on { status: 1, createdAt: -1 }
6. Node.js / Mongoose-Specific Optimizations
If you're using Mongoose (which most Node.js developers do), there are Mongoose-specific performance traps you need to know about:
Use .lean() for Read-Heavy Queries
// ❌ SLOW — Mongoose creates full model instances with change tracking
const contacts = await Contact.find({ status: "active" });
// Each document is a full Mongoose Document with getters, setters,
// save(), validate(), and change tracking. Heavy on memory and CPU.
// ✅ FAST — .lean() returns plain JavaScript objects
const contacts = await Contact.find({ status: "active" }).lean();
// Returns raw JSON objects. 3-5x faster for read-only operations.
// Use this for API responses where you don't need to modify and save back.
Use .select() to Return Only Needed Fields
// ❌ Returns ALL 30+ fields for each document
const contacts = await Contact.find({ status: "active" });
// ✅ Returns only the 4 fields the API actually needs
const contacts = await Contact.find({ status: "active" })
.select("name email company createdAt")
.lean();
Avoid the N+1 Query Problem
// ❌ N+1 PROBLEM — 1 query + N additional queries
const contacts = await Contact.find({ status: "active" });
for (const contact of contacts) {
const company = await Company.findById(contact.companyId); // 1 query per contact!
contact.company = company;
}
// If 1000 contacts are returned, this fires 1001 database queries.
// ✅ FIX — Use .populate() or $lookup (single query)
const contacts = await Contact.find({ status: "active" })
.populate("companyId", "name industry")
.lean();
// MongoDB resolves all company references in a single batch query.
7. When MongoDB Isn't Enough: Adding Elasticsearch
There comes a point where even perfectly indexed MongoDB queries aren't fast enough. If you need any of the following, it's time to add Elasticsearch as a dedicated search layer:
- Full-text search across multiple fields (name + email + company + title)
- Fuzzy matching — users misspelling "Gogle" and still finding "Google"
- Autocomplete / search-as-you-type suggestions
- Faceted filtering — filter by industry, location, company size simultaneously
- Sub-100ms responses on queries across 40M+ documents with complex filters
The Architecture: MongoDB + Elasticsearch
System of Record)] MongoDB -->|Change Streams / CDC| Sync["Monstache
or Kafka Connector"] Sync -->|Real-time Sync| ES[(Elasticsearch
Search Layer)] App -->|Search Queries| ES style MongoDB fill:#023430,stroke:#00ED64,color:#fff style ES fill:#1a1a2e,stroke:#f59e0b,color:#fff style Sync fill:#111118,stroke:#818cf8,color:#f1f5f9
The architecture is clean:
- MongoDB remains your system of record. All writes (create, update, delete) go to MongoDB.
- Elasticsearch is your search engine. All search queries are routed to Elasticsearch.
- A sync layer keeps them in sync. Use MongoDB Change Streams with a tool like Monstache (open source) or a Kafka connector to pipe changes from MongoDB to Elasticsearch in real-time.
Setting Up Elasticsearch with Node.js
// Install the official Elasticsearch client
// npm install @elastic/elasticsearch
const { Client } = require("@elastic/elasticsearch");
const esClient = new Client({
node: "http://localhost:9200",
// For Elastic Cloud:
// cloud: { id: "your-cloud-id" },
// auth: { apiKey: "your-api-key" }
});
// Create an index with explicit mapping (don't rely on dynamic mapping)
await esClient.indices.create({
index: "contacts",
body: {
mappings: {
properties: {
name: { type: "text", analyzer: "standard" },
email: { type: "keyword" }, // keyword = exact match, no analysis
company: { type: "text", fields: { keyword: { type: "keyword" } } },
industry: { type: "keyword" },
status: { type: "keyword" },
employeeCount: { type: "integer" },
createdAt: { type: "date" },
location: { type: "text", fields: { keyword: { type: "keyword" } } }
}
}
}
});
Fast Search: What Took 12s in MongoDB Now Takes 23ms
// Multi-field search with fuzzy matching and filters
// User searches: "gogle" (typo) with filters: industry=Technology, status=active
const results = await esClient.search({
index: "contacts",
body: {
query: {
bool: {
must: [
{
multi_match: {
query: "gogle", // User's misspelled search
fields: ["name", "company", "email"],
fuzziness: "AUTO" // Automatically corrects typos
}
}
],
filter: [ // Filters use cache — very fast
{ term: { status: "active" } },
{ term: { industry: "Technology" } },
{ range: { employeeCount: { gte: 50 } } }
]
}
},
sort: [{ createdAt: { order: "desc" } }],
size: 50,
_source: ["name", "email", "company", "industry", "createdAt"]
}
});
// Response time: 23ms across 40M+ documents
// MongoDB equivalent (even with indexes): 800ms+
// Without indexes: 12,000ms+
Building Autocomplete / Search-as-You-Type
// Add a search_as_you_type field to your mapping
await esClient.indices.create({
index: "contacts_autocomplete",
body: {
mappings: {
properties: {
name: { type: "search_as_you_type" }, // Built-in autocomplete support
company: { type: "search_as_you_type" }
}
}
}
});
// Query: user types "mic" → suggests "Microsoft", "Micron", "Michael Smith"
const suggestions = await esClient.search({
index: "contacts_autocomplete",
body: {
query: {
multi_match: {
query: "mic",
type: "bool_prefix",
fields: ["name", "name._2gram", "name._3gram", "company", "company._2gram"]
}
},
size: 10
}
});
// Response time: 8ms — perfect for real-time search boxes
Syncing MongoDB → Elasticsearch with Change Streams
// Real-time sync using MongoDB Change Streams (Node.js)
const { MongoClient } = require("mongodb");
const { Client } = require("@elastic/elasticsearch");
const mongo = await MongoClient.connect("mongodb://localhost:27017");
const db = mongo.db("myapp");
const esClient = new Client({ node: "http://localhost:9200" });
// Watch for all changes on the contacts collection
const changeStream = db.collection("contacts").watch([], {
fullDocument: "updateLookup" // Include the full updated document
});
changeStream.on("change", async (change) => {
try {
switch (change.operationType) {
case "insert":
case "update":
case "replace":
await esClient.index({
index: "contacts",
id: change.documentKey._id.toString(),
body: change.fullDocument
});
break;
case "delete":
await esClient.delete({
index: "contacts",
id: change.documentKey._id.toString()
});
break;
}
} catch (err) {
console.error("ES sync error:", err.message);
// In production: push to a dead-letter queue for retry
}
});
console.log("Watching MongoDB changes and syncing to Elasticsearch...");
go install github.com/rwynn/monstache/v6@latest
When to Use MongoDB vs. Elasticsearch
| Use Case | Use MongoDB | Use Elasticsearch |
|---|---|---|
| CRUD operations (create, read, update, delete) | ✅ Primary choice | ❌ Not designed for this |
| Transactions (multi-document atomicity) | ✅ Native ACID support | ❌ No transaction support |
| Exact match queries (find by ID, status) | ✅ Fast with indexes | ✅ Also fast |
| Full-text search (across multiple fields) | ⚠️ Basic $text index | ✅ Purpose-built, far superior |
| Fuzzy search (typo tolerance) | ❌ Not supported | ✅ Native fuzziness support |
| Autocomplete / suggestions | ❌ Regex is slow | ✅ search_as_you_type field |
| Faceted filtering + aggregations | ⚠️ Slow on large datasets | ✅ Extremely fast |
| Log analytics / time-series | ⚠️ Not optimized | ✅ Designed for this |
8. MongoDB Performance Checklist
Before you consider your MongoDB setup "production-optimized," verify every item:
- ☐ Run
explain()on every frequently-used query — verify IXSCAN, not COLLSCAN - ☐ Compound indexes follow ESR — Equality → Sort → Range field order
- ☐ No unnecessary indexes — run
db.collection.getIndexes()and drop unused ones - ☐
$matchis FIRST in every aggregation pipeline - ☐ Cursor-based pagination — replaced
.skip()with$lt/$gton a sorted field - ☐
.lean()used on all read-only Mongoose queries - ☐
.select()used to return only needed fields - ☐ No N+1 queries — using
.populate()or$lookupfor joins - ☐ Document size under control — no unbounded arrays growing infinitely
- ☐ Elasticsearch added for full-text search, fuzzy matching, and autocomplete
- ☐
allowDiskUse: trueset on aggregations that may exceed 100MB RAM limit - ☐ MongoDB Profiler enabled —
db.setProfilingLevel(1, { slowms: 100 })to catch slow queries
Frequently Asked Questions
How many indexes should I have per collection?
There's no hard limit, but every index consumes RAM and slows down write operations (inserts, updates, deletes) because MongoDB must update all indexes. As a rule of thumb, 5-10 well-designed indexes per collection is typical. Use db.collection.stats() to see total index size, and ensure it fits in RAM.
Should I index every field that I query on?
No. Only index fields that appear in frequently executed queries, filters, and sorts. A field queried once a day doesn't need an index. Focus on the queries that power your API endpoints — those are hit thousands of times per hour.
Does MongoDB support full-text search without Elasticsearch?
Yes, MongoDB has a built-in $text index that supports basic full-text search. However, it's significantly limited compared to Elasticsearch: no fuzzy matching, no field boosting, no autocomplete, and poor performance on large datasets. For production search features, Elasticsearch is the industry standard.
How do I monitor slow queries in production?
Enable the MongoDB Database Profiler: db.setProfilingLevel(1, { slowms: 100 }). This logs all queries taking longer than 100ms to the system.profile collection. Query it with db.system.profile.find().sort({ ts: -1 }).limit(10) to see recent slow queries. On MongoDB Atlas, use the built-in Performance Advisor — it automatically recommends missing indexes.
Conclusion
Every MongoDB performance problem I've encountered in production comes back to the same fundamentals: missing indexes, wrong index field order, or doing too much work in the pipeline before filtering.
Start with explain(). Follow the ESR rule. Move $match first. Use cursor pagination. And when you outgrow what MongoDB indexes can handle — bring in Elasticsearch for the search layer.
These techniques took my Mongo Explorer queries from 12+ seconds to under 100ms across 40 million records. They'll do the same for yours.
I'm always looking to improve these guides. If you have any feedback, corrections, or just want to discuss MongoDB performance, please reach out to me here.