A WordPress content export misses posts after the first API page
Inspect WordPress REST pagination headers when a headless archive or sitemap omits posts, retrieve every intended page, and reconcile the published URL inventory.
By Crawlito · 6 min read
In short
- WordPress REST collections are paginated; one successful response is not the full inventory.
- Use pagination metadata and stable record identifiers to reconcile the intended public records.
- Verify generated routes, archive links, and sitemap entries after fixing the importer.
On this page
- Compare the expected inventory with one API response
- Inspect the pagination headers directly
- Trace how the build decides it has finished
- Account for content changing during the export
- Map the complete records to the public site correctly
- Verify older pages through their real discovery paths
- Make incomplete imports fail visibly
- Frequently asked questions
If a headless WordPress archive or generated sitemap includes only the newest posts, check whether its importer fetches just one REST API page. A successful collection response can be intentionally limited. Read the pagination headers, retrieve the complete intended public collection, and reconcile unique post identifiers before rebuilding the site.
This differs from a WordPress post being unpublished or excluded deliberately. First define which records belong in the public output so that fixing pagination does not expose drafts or restore content that editors intentionally removed.
Compare the expected inventory with one API response
Choose several known published posts, including an older article missing from the frontend. Record their WordPress identifiers and intended public URLs.
Fetch the same collection used by the build. Do not substitute an administrator-only query if the importer normally reads public content.
The WordPress REST pagination handbook explains that collection endpoints limit their results and expose pagination controls and total-count headers.
A response containing valid JSON and several posts can still be incomplete for an archive-generation task. Count the returned records and compare them with the collection's metadata.
Check filters such as category, post type, language, or publication status. A filtered public collection is different from the total number of records visible in the WordPress administration screen.
The content inventory guide helps define which content belongs in scope before comparing counts.
Keep the original query string in the investigation note. Pagination evidence is meaningful only for the exact filtered collection that produced it.
Inspect the pagination headers directly
For a small diagnostic sample, request two records at a time:
curl -sS -D page-one-headers.txt -o page-one.json \
'https://cms.example.com/wp-json/wp/v2/posts?per_page=2&page=1'
curl -sS -D page-two-headers.txt -o page-two.json \
'https://cms.example.com/wp-json/wp/v2/posts?per_page=2&page=2'Inspect X-WP-Total and X-WP-TotalPages in the responses, then compare the post IDs in the bodies.
The small page size makes the boundary visible without downloading an entire archive. It is a diagnostic choice, not a recommended production batch size for every site.
WordPress documents a per_page limit of 100. Increasing the value beyond the supported limit is not a substitute for implementing pagination.
If an intermediary strips the headers, inspect its configuration and the response seen by the importer. Do not conclude that the collection has no more records merely because one client cannot access the metadata.
Check error responses explicitly. An invalid page or malformed query should not be treated as an empty successful batch.
Trace how the build decides it has finished
Find the importer or build step that calls the collection endpoint. Look for a single request whose returned array is treated as the complete source.
Also inspect loops that stop after a fixed number of pages, stop when one record is filtered out, or overwrite previous results instead of accumulating them.
Write the intended algorithm in plain terms:
- Request a page with the approved filters.
- Read the pagination metadata.
- Retrieve the remaining intended pages.
- Combine records by stable identifier.
- Confirm the collection is complete before publishing.
The exact implementation depends on the build system, but its completion condition should be explicit.
Do not hide request failures by returning an empty array and continuing the release. That can produce a successful build with a silently reduced public archive.
Record whether the importer uses the same query parameters on every page. Dropping a filter mid-loop can combine unrelated collections and make a total-count comparison misleading.
Account for content changing during the export
A live collection can change while several pages are fetched. New posts, deletions, or ordering changes can shift records between pages.
Choose a documented stable ordering appropriate to the collection and keep it consistent throughout the run. For large or frequently changing sites, use a publishing process that can reconcile changes rather than assuming one sequence is a perfect snapshot.
Deduplicate by the content record's stable identifier, not by title. Two distinct posts can have similar titles, and one post can be renamed.
Compare the final identifier set with the intended source inventory. A matching total count alone cannot detect one missing record replaced by a duplicate.
If a bounded retry is necessary, repeat the affected collection deliberately and record the reason. Do not run an unbounded loop against the CMS when the totals keep changing.
Keep the export timestamp and query definition with the build evidence. They explain which source state the generated site was intended to represent.
Map the complete records to the public site correctly
Once pagination works, verify that every included record maps to the intended frontend route. The WordPress API's link field may refer to the CMS host rather than the public headless site.
The headless WordPress sitemap guide covers that separate hostname-mapping problem.
Preserve the frontend's publication rules. Retrieving more records does not mean every API record should automatically become a public route.
Check custom post types separately if they belong in the site. Fixing the standard posts collection does not establish that another endpoint is complete.
Rebuild the archive, route list, and sitemap from the reconciled inventory. If these outputs use different data paths, confirm that each consumes the complete intended set.
Avoid manually adding missing URLs to a sitemap while leaving the route generator incomplete. A sitemap entry that points to a page the build never created does not restore the article.
Verify older pages through their real discovery paths
Select an older previously omitted article and open its public URL directly. Confirm the expected heading and body, not merely a generic 200 fallback.
Then follow a normal archive or category link to it. A generated route can exist while remaining absent from the site's navigable content structure.
Check its sitemap entry and canonical URL. These should identify the public frontend address consistently.
The orphan-page guide explains why a reachable URL can still be difficult to discover when the surrounding link inventory is incomplete.
Use a sample from the beginning, middle, and end of the retrieved collection. This is more informative than checking only the newest page, which was already working before the repair.
For a manageable archive, reconcile every generated URL against the unique source identifiers. Report any intentional exclusions separately from missing records.
Make incomplete imports fail visibly
Add a completion check to the publishing process. It should distinguish a complete collection, an intentionally filtered collection, and a failed or partial retrieval.
Keep diagnostics concise: query definition, pages requested, unique records retrieved, expected metadata, and any failed request. Avoid logging private content or credentials.
Do not build a brittle assertion that all WordPress sites always have the same number of records. The useful invariant is consistency with the source and the approved publishing rules for that run.
For a client handoff, show the previously missing identifiers and the corresponding verified public URLs. That provides evidence of restored coverage without inventing an indexing or traffic outcome.
Retest the importer after API, caching, or CMS-plugin changes. Pagination behavior can be affected by the delivery path even when the article content itself has not changed.
Frequently asked questions
Does a 200 response mean the API returned every post?
No. It can be a valid page of a larger collection. Inspect the pagination metadata and the importer's completion logic.
Can I set per_page to a very large number instead?
Use the documented limits and fetch multiple pages. An unsupported request size is not a reliable full-inventory strategy.
Should the importer include drafts to match the admin count?
No. Reconcile against the intended public collection and preserve the site's publication and authorization rules.
After restoring the archive, request a site-wide Crawlito audit to review public routes and discovery signals across the published site.
Filed under Platform guides. Topics: wordpress, sitemap, internal-links