Website brief
A website AI can read
What AI engines need from your site, how to tell if yours is in the way, and when a move to Webflow makes sense.
The short version
- AI engines read your pages to decide what to say about you. A page they can’t reach or read can’t be quoted.
- Many AI crawlers read the page source and don’t run scripts. Text that only appears after scripts run can be invisible to them.
- AI search rewards current, specific pages. A site that needs a developer for every change falls behind.
01What AI engines need from a site
| Need | What it means | How to check |
|---|---|---|
| Access | Crawlers can reach the page. robots.txt and bot protection let them in. | Look for OAI-SearchBot (ChatGPT search), PerplexityBot and Googlebot in robots.txt and your firewall rules. |
| Text in the page source | What you sell, pricing and FAQs are in the HTML, not added later by scripts. | View the source of your pricing page and search for a sentence you can see on screen. |
| Speed | Pages load fast on phones, where many buyers first land. | Google’s “good” for the largest content on the page is under 2.5 seconds. |
| Clear structure | One H1 that says what the page is, headings that match buyers’ questions, the answer first. | Read only the headings. Do they tell the story? |
| Structured data | Markup that says what a page is: your organization, a product, a FAQ. | Run a key page through Google’s Rich Results Test. |
| Fresh, dated pages | Current facts, with dates, and a sitemap that lists the pages. | When did your key pages last change? |
Blocking GPTBot keeps your pages out of OpenAI’s model training; it doesn’t remove you from ChatGPT search, which uses OAI-SearchBot. Google-Extended works the same way for Gemini: it doesn’t affect Google Search or AI Overviews.
02Signs your site is in the way
- Key pages, often pricing and FAQ, are built by scripts, and their text isn’t in the page source.
- The largest content takes more than 2.5 seconds to load on a phone.
- Dozens of plugin scripts load on the homepage, each one an update to manage.
- A new page or a layout change needs a developer ticket.
- Someone on marketing spends part of most weeks on updates, security patches or fixing what broke.
- Your company is described one way on the homepage and another on LinkedIn.
Mobile load (largest content)5.8sGoogle’s “good” is under 2.5s
Homepage weight4.2 MBMostly scripts and images
Plugin scripts on the homepage23Each one an update to manage
Key pages AI crawlers can’t read2 of 6Pricing and FAQ
Runs onWordPress 6.6 · Elementor · Yoast SEO · WooCommerce (unused, still loading) · HubSpot forms · Calendly
Acme is a made-up company. Example data.
03When a move to Webflow makes sense
Move when
- Marketing waits on a developer for page changes.
- Keeping the site running takes your team’s time week after week.
- Fixing what AI crawlers can’t read means rebuilding templates anyway.
- The site is due for a redesign.
Stay when
- The site is fast and AI crawlers can read it.
- Your team changes pages without help.
- Then fix the pages, not the platform.
| The job | On WordPress | On Webflow |
|---|---|---|
| Hosting and uptime | Your host, and your team when it’s down | Webflow |
| Core, theme and plugin updates | Your team or agency | Webflow; no plugins to update |
| Security patches and SSL | Your team or agency | Webflow |
| Backups | A plugin, if someone set it up | Webflow, automatically |
| A new page or edit | Often a developer ticket | Your marketing team, in the editor |
A move with us
- Plan the moveYour content, the pages that matter, and every old address mapped to its new one with a permanent redirect, so you keep your search traffic.
- Rebuild for AI searchClean pages, structured data and answer-first layouts from the start.
- Set up the CMSCollections for what you publish, so your team adds pages without a developer.
- Train and hand overYour team runs it. Webflow handles hosting, updates and security.
Some tools we recommend are from partners. How we work with partners: gtmnext.com/disclosure