Implementing Pagination with MongoDB Skip and Limit

In the earliest versions, Postulate loaded all snippets in a project, displaying them in a continuous feed on a long, long page. This is obviously terrible for both user experience and performance. The solution to this is pagination: only showing 10 items at a time, and allowing users to click through pages to browse older snippets.

MongoDB has an extremely straightforward way to implement pagination: using skip and limit operations on a cursor. skip(n) skips n items in a query, while limit(m) returns only the next m items starting from the n-th one.

Previously I would have simply returned await SnippetModel.find(conditions). To implement pagination, I add another query param page (i.e. make a request to /api/snippet?page=1), then use this query param to set skip and limit operations to get the corresponding snippets:

const snippets = SnippetModel .find(conditions) .skip((+req.query.page - 1) * 10) .limit(10)

For the frontend to know how many pages there are, I also get the total count of snippets from MongoDB:

const count = await SnippetModel .find(conditions) .count();

On the frontend, I have a state variable snippetPage, and use the count returned from the API to render the relevant buttons and info text:

<p className="opacity-25 mt-8"> Showing snippets {(snippetPage - 1) * 10 + 1} -{(snippetPage < Math.floor(snippets.count / 10)) ? snippetPage * 10 : snippets.count} of {snippets.count} </p> {snippets.count > 10 && ( <div className="mt-4"> {Array.from({length: Math.ceil(snippets.count / 10)}, (x, i) => i + 1).map(d => ( <button className={"py-2 px-4 rounded-md mr-2 " + (d === snippetPage ? "opacity-50 cursor-not-allowed bg-gray-100" : "")} onClick={() => setSnippetPage(d)} disabled={+d === +snippetPage} >{d}</button> ))} </div> )}

snippets is fetched through useSWR:

const {data: snippets, error: snippetsError}: responseInterface<{snippets: DatedObj<SnippetObj>[], authors: DatedObj<UserObj>[], count: number }, any> = useSWR(`/api/snippet?projectId=${projectId}&search=${snippetSearchQuery}...&page=${snippetPage}&sort=${orderNew ? "-1" : "1"}`, fetcher);

useSWR re-fetches data every time the given URL changes, so both the contents and count of snippets are updated for every change in filter or page state variables.

You might notice that there's also a sort parameter in the URL. Sorting used to be handled on the frontend, i.e. all snippets would be received, then the frontend would sort them in either newest or oldest first order. With multiple pages of items, though, this approach doesn't work. Sorting needs to be done on the database before pagination.

The database query from earlier actually has an extra sort operator on it:

const snippets = SnippetModel .find(conditions) .sort({"createdAt": req.query.sort ? +req.query.sort : - 1}) .skip((+req.query.page - 1) * 10) .limit(10)

On the frontend, changing the sorting order also sets the page back to 1, so that users aren't jarred into an unexpected spot in a new query.

<button className="underline opacity-50 hover:opacity-100 transition ml-auto flex-shrink-0" onClick={() => { setOrderNew(!orderNew); setSnippetPage(1); }} >{orderNew ? "View oldest first" : "View newest first"}</button>

All of this comes together for a much more user-friendly and performant notetaking experience.