Bubble SEO settings: robots.txt, sitemap and crawler checks
Where Bubble's robots.txt, sitemap, canonical and page metadata settings live, what each one does, and how to test what crawlers get from a Bubble app.
By Crawlito · 9 min read
In short
Bubble's app-wide SEO controls sit under Settings > SEO / metatags: Customize robots.txt, Expose a sitemap file, the canonical URL option and the header and body script fields.
By default Bubble serves a robots.txt that keeps the development version of your app, under /version-test/, out of search indexes.
Bubble's sitemap includes dynamic page URLs from the database, filters them by privacy rules and, per its reference, holds at most 50,000 URLs per page.
Page titles, descriptions and social images are set in the page's element inspector and can be dynamic, using sources such as Current page thing.
Bubble's manual does not describe how page body text is delivered to crawlers, so fetch your live pages with curl and check the raw HTML yourself.
Bubble gives you most of the SEO controls a crawler cares about in one place: Settings > SEO / metatags, plus the page-level fields in each page's element inspector. There you can customize robots.txt, expose a sitemap that includes your dynamic pages, turn on a canonical URL tag for your primary domain, and set titles, descriptions and social images per page. What the settings cannot tell you is whether the body text of a page reaches a crawler that does not run JavaScript, so finish with a raw HTML test of your live pages.
This post walks through the settings as Bubble's manual documents them at the time of writing, then shows how to check the result. If you are comparing builders, the post on Lovable, Bolt and v0 apps covers the AI app builders.
Where Bubble's SEO settings live
Bubble splits SEO into app-wide settings and page settings. The SEO / metatags reference lists the app-wide ones:
Setting
Where
What it does
Facebook & Twitter title, site name, description
Settings > SEO / metatags
Default text shown when the app is shared on social networks
Thumbnail used for links
Settings > SEO / metatags
Default share image, with a "Test/refresh data" link for Facebook's cache
Expose the type of tags for text elements
Settings > SEO / metatags
Lets you give text elements h1, h2 and h3 tags
Enable canonical url
Settings > SEO / metatags
Adds a Bubble-defined canonical tag pointing at your primary domain
Customize robots.txt (advanced)
Settings > SEO / metatags
Replaces the default robots.txt with your own rules
Expose a sitemap file
Settings > SEO / metatags
Generates a sitemap for the pages you select
Script/meta tags in header, Script in the body
Settings > SEO / metatags
Adds code to every page
301 Redirections
Settings > SEO / metatags
Maps old URLs to new ones
Hosting files in the root directory
Settings > SEO / metatags
Serves uploaded files at the root of your domain
Redirect all requests to the domain
Settings > Domain / email
Older reference entry for sending appname.bubbleapps.io traffic to your custom domain
Bubble's help guide names the canonical option "Point URLs to primary domain for better SEO", while the reference calls it "Enable canonical url". Look for either label in your editor.
How Bubble handles robots.txt
Bubble places a robots.txt file at the root of every app. The reference says the default file makes sure search engine crawlers do not index the test version of your app, which lives under /version-test/. Your live pages are open to crawlers unless you add rules. On the uncustomized Bubble apps we fetched at the time of writing, the default file was just this:
User-agent: *
Disallow: /version-test/
To add rules, check Customize robots.txt (advanced). Bubble marks this as a setting for SEO experts, and the warning is fair: one wrong Disallow line can remove a whole section from search. Before you edit, open https://yourdomain.com/robots.txt and copy what Bubble serves now, so you keep its Disallow: /version-test/ rule.
Bubble's own example hides private pages by page name:
Because Bubble page URLs use the page name as the first path segment, a Disallow: /dashboard line covers every dynamic URL under that page too. Rules are matched against the start of the path, as RFC 9309 defines, so the same line also blocks any other page whose name starts with dashboard, such as /dashboard-demo.
You can also add groups for named AI crawlers in the same file. A group for one crawler replaces the * group for that crawler, so repeat any rules you still want it to follow:
Two limits to keep in mind. First, robots.txt is a request, not access control: RFC 9309 says so plainly, and Bubble's manual says the same. Protect private data with privacy rules and login pages, not with Disallow. Second, Bubble's reference warns that a savvy outsider can see the list of every page in an app, even pages left out of the sitemap, so do not rely on obscurity either.
How Bubble's sitemap works
Check Expose a sitemap file and Bubble shows your page names so you can tick the ones to include. For a page that has a Type of content, such as a Product page, Bubble also includes the matching database entries as separate URLs. That is useful: a product detail page that no other page links to can still be found through the sitemap.
The reference adds three details that change what ends up in the file:
Privacy rules filter the URLs. Bubble only lists entries that a logged-out visitor can find. Its own example of a logged-out visitor is Google's crawler. To keep a record out of the sitemap, make sure logged-out users do not have "Find in Searches" permission on it.
There is a 50,000-URL cap. The reference says Bubble restricts sitemaps to at most 50,000 URLs per page, as per the sitemap specification, and includes the first 50,000 if you go over.
You can host your own. Bubble's SEO guide says the most common SEO use of Hosting files in the root directory is uploading a custom sitemap .xml file. Files there are served at yourdomain.com/filename once you deploy to live, and at yourdomain.com/version-test/filename before that.
The manual pages we read do not state the path of the generated sitemap. After deploying, open /sitemap.xml on your domain to confirm where it is, then submit that URL in Google Search Console and Bing Webmaster Tools. Adding a Sitemap: line with the full sitemap URL to a customized robots.txt also helps crawlers find it. For what sitemaps do and do not do for AI search, see XML sitemaps and AI discovery.
Canonical URLs and your primary domain
Every Bubble app has a built-in address such as my-app.bubbleapps.io. Once you connect a custom domain, the same pages could be reached at two hosts. Bubble's custom domain guide says requests to yourapp.bubbleapps.io are redirected to your domain automatically once it is set up, and that Bubble also redirects between example.com and www.example.com, depending on which one you entered.
The older Domain / email reference instead describes a Redirect all requests to the domain checkbox for the bubbleapps.io redirect, so if your editor shows it, check it. Either way, test the result with curl -sI https://yourapp.bubbleapps.io/ and look for a redirect to your domain.
Turn on the canonical option as well. The reference says it adds a canonical tag for your primary domain and, with a custom domain, hides the built-in Bubble domain in favour of it. A redirect plus a matching canonical gives crawlers one consistent answer about which host is the real one.
Dynamic pages need one more check. Bubble builds their URLs from the thing's slug, then a backup field, then the unique ID. The page slugs guide notes two traps:
If a privacy rule hides the slug field, visitors who cannot see it get the unique ID in the URL instead. Unless the slug is visible to everyone, a logged-out crawler is such a visitor.
A backup field does not give a clean URL: Bubble appends the unique ID to a web-friendly version of that field, so set slugs if you want readable URLs.
If the index page has a thing assigned, its URLs take the form /index/slug, not /slug.
Page titles, descriptions and Open Graph tags
Page-level fields live in the page's element inspector, which you open by double-clicking the page. Bubble's page SEO guide describes them:
Field
Output
Bubble's guidance
Page Title
The <title> tag
Shown in search results and the browser tab
Title (for SEO / FB)
og:title
Under 60 characters, include the main keywords
Description (for SEO / FB)
og:description
Under 160 characters
Social Media Image
The share image
Shown when the page is shared on Facebook and X
Page HTML header
Extra tags in <head>
Meta tags, structured data, canonical and language tags
Each field can be dynamic. On a Product page, set the title to something like Current page thing's Name so every product URL gets its own title instead of one shared string. Bubble also accepts sources such as Do a search for, Get data from page URL and App text.
When you paste code into Script/meta tags in header, use only <script>, <meta> and <link>. Bubble's reference warns that browsers move anything else into the body, which can push your other head tags out of the head too. The post on invalid head metadata shows why that makes crawlers ignore tags you thought were set.
How to test what crawlers get from a Bubble page
Bubble's SEO pages do not describe how a page's body text is delivered: whether elements and database content are in the HTML the server sends, or drawn by JavaScript after load. That matters because Google renders JavaScript, while in the most recent public measurement the major AI crawlers did not, as client-side rendering and empty HTML explains. So test your own live pages rather than assume.
Use a deployed live URL, not /version-test/, and pick a dynamic page whose text comes from the database.
You want a title and description specific to this record and a canonical on your custom domain. A generic title on every URL means the dynamic fields are not reaching the server HTML.
Replace the phrase with a sentence from that database entry. A count of 0 means the text is not in the HTML the server sends.
Compare with the rendered page. Open the same URL in a browser, view the rendered DOM in developer tools, and compare the visible text with step 2. The post on raw vs rendered HTML explains how to read the difference.
Check robots.txt and the sitemap on the live host:
Confirm the rules are the ones you wrote and the sitemap returns 200.
If step 2 fails, the fixes in Bubble are limited, because you do not control the server rendering. What you can do: keep the head fields dynamic so titles and descriptions are right in the raw HTML, add structured data through the page HTML header or an HTML element, keep heavy searches off page load as Bubble's guide advises, and for content-heavy public pages such as a blog or docs, consider hosting them on a static or server-rendered site on a subdomain.
Frequently asked questions
Does Bubble create a robots.txt automatically?
Yes. Bubble places one at the root of every app, and by default it keeps the development version under /version-test/ out of search indexes. Check Customize robots.txt (advanced) under Settings > SEO / metatags to write your own rules.
Does the Bubble sitemap include dynamic pages?
Yes, for pages with a Type of content. Bubble adds one URL per matching database entry, leaves out entries a logged-out visitor cannot find under your privacy rules, and its reference sets a cap of 50,000 URLs per page.
Why does my Bubble page show a unique ID instead of a slug?
Either the thing has no slug (Bubble then falls back to a backup field with the ID appended, or to the bare unique ID), or a privacy rule hides the slug field from the visitor. Allow everyone to view the slug field, or set slugs with the "Set a thing's slug" action.
Can AI crawlers read my Bubble app?
Bubble's documentation does not say how page body text reaches crawlers that do not run JavaScript. Fetch a live page with curl and search the HTML for a sentence from the page to find out for your app.
Check one of your Bubble pages
Run the free check on a live Bubble page, ideally a dynamic one that pulls text from the database. It shows what the main AI crawlers receive in the raw HTML and whether robots.txt lets them in, with no account needed.
Client-side rendering can hand AI crawlers an empty HTML shell. See what they receive, the symptoms, the fixes ranked by effort and how to verify each one.
Raw HTML vs rendered HTML: what each one is, how to compare them with view-source, curl, DevTools and Search Console, and when the gap hides content from crawlers.