Four Kitchens
Insights

Does AI understand my website the way I hope it does?

8 Min. ReadDigital strategy

Remember a year or two ago, before AI was big? Back when our focus was on SEO? When it comes to SEO, there’s very little info that Google has revealed about its inner workings. But there are many people who stake their entire livelihood on being “SEO experts.” This means that there are a lot of people asserting things that will improve your SEO, but without rigorous demonstration. Today the focus is now on AEO/GEO (Answer Engine Optimization / Generative Engine Optimization) and this is even more true; Anthropic and OpenAI reveal even less than Google about how they crawl content.

But in both cases there is one truth that does seem reliable: There’s no single thing that we can do that would greatly harm, or greatly help our SEO/AEO/GEO rankings. Instead, it’s the culmination of hundreds (if not thousands) of small things; whether that’s:

  • the sentiment with which people mention your organization on Reddit
  • the semantic structure of your HTML
  • whether your content is written in a question-and-answer format
  • the accuracy of your Wikipedia entry
  • and so much more…

Can AI read and understand my web content?

Wait, can humans even understand it?

At Four Kitchens, we work with a lot of higher-ed institutions. While not the majority, there are too many universities whose websites cannot be deciphered by real people. If you send a person unfamiliar with your institution to a random page on your site, will they be able to understand how it relates to the rest of the website? Or why the page exists? Or how to navigate to a page that addresses their next logical question? Or will their response be, “Why are there seven navigation menus on the page? There’s more navigation than there is content!”

If humans can’t understand your website, then AI won’t either.

If you’re in this situation, then I truly empathize with you. You have a difficult task ahead to advocate your leadership for an entire overhaul. But you need to take the plunge. If you instead jump to anything else in this list of recommendations, you’ll just be rearranging the deck chairs. Meanwhile your peer universities will be flying ahead.

Can AI understand content in collapsed elements, like accordions or carousels?

The consensus of web professionals is that if the content is in the original HTML of the page, then AI will access it just fine, even if it would require a human to click something before it appears.

But there is no consensus on how that content is weighted. Some AI systems might assign lower relevance scores to content that requires user interaction to access, viewing it as less immediately valuable. However, no one has done any controlled testing in the last decade, and that means that no one has firm data for LLM-based crawlers.

But, in the real world, the solution is to have great training for content editors. If you’ve trained people for how to craft great content for the web, then no one is going to create a page solely of accordions. The most important content will already be in headings and intro paragraphs.

Can AI understand content loaded via JavaScript?

If the content is not in the original HTML of the page, and is only loaded after a click (or some other event), AI crawlers don’t easily find this content, if they find it at all (again, there is not any hard proof on this).

Here’s a typical scenario for a university: On many pages for the engineering department there are “request information” buttons linking to a page that has an RFI form (some other day I’ll write a rant about how 17-year-olds aren’t going to click on a button labelled “request information”). This form is the only thing on the page. It’s a Slate form that JavaScript generates after the page loads. But AI only sees this as an empty page.

Or JavaScript that generates a chart that highlights some stats about your college. AI won’t see those either.

Here again, the first solution is solid training for content editors. If your editors know that a chart needs the underlying data to be available for accessibility (maybe as a link to a spreadsheet), then AI will pick up on that too. If the editor knows to write an intro sentence for the form, then AI will also understand that.

But content editors sometimes forget, so we often set up tools to nudge them in the right direction, like Editoria11y.

Make things explicit for AI crawlers with structured content

I feel like I buried the lede here. You’re looking for the most actionable thing, and here it is: The biggest technical thing that you can do to improve how AI understands our content is to be more explicit with our structured content.

Think about this example: Every university website has a content type for “Person Profile” and it’s probably got 30+ fields. But for some of that data, the AI needs to infer what it means. See what I mean:

  • The page title is probably this person’s name, but is “Sarkar” their first name or last name?
  • The next line appears to be their professional titles, but are “Associate Professor” and “Post Doc Fellow” two separate titles or one?
  • Is “B.A. Engineering” a degree they hold, or is that the department they teach in?
  • etc.

SEO/AEO/GEO crawlers would really appreciate it if we’d just spell it out explicitly. We can include a chunk of JSON-LD at the top of the page that is visible to robots, but not humans. Example:

{
 "@context": "http://schema.org",
 "@type": "Person",
 "@id": "https://example.com/alex-doe",
 "name": "Alex Doe",
 "alternateName": "Allie Doe",
 "givenName": "Alex",
 "familyName": "Doe",
 "nationality": "Canada",
 "birthPlace": {
 "@type": "Place",
 "address": {
  "@type": "PostalAddress",
  "addressLocality": "Springfield",
  "addressRegion": "MA",
  "addressCountry": "USA"
 }
 },
 "alumniOf": {
 "@type": "CollegeOrUniversity",
 "name": "Aula do Comércio",
 "sameAs": "https://en.wikipedia.org/wiki/Business_school#:~:text=Aula%20do%20Com%C3%A9rcio"
 },
 "gender": "Female",
 "description": "Technologist",
 "disambiguatingDescription": "Co-founder of Elite Examples",
 "jobTitle": "Technical Director",
 "worksFor": {
 "@type": "Organization",
 "name": "Elite Examples",
 "sameAs": [
  "https://elite-strategies.com/",
  "https://x.com/elite-examples"
 ]
 },
 "url": "https://example.com/alex-doe",
 "image": "https://jsonld.com/img/patrickCoombeProfile.jpg",
 "address": {
 "@type": "PostalAddress",
 "addressLocality": "Salt Spring Island",
 "addressRegion": "BC",
 "addressCountry": "Canada"
 },
 "sameAs": [
 "https://github.com/elite-examples",
 "https://x.com/elite-examples",
 "https://www.facebook.com/elite-examples",
 "https://bsky.app/profile/elite-examples.com",
 "https://www.instagram.com/elite-examples/",
 "https://www.producthunt.com/@elite-examples",
 "https://wordpress.org/support/users/elite-examples/",
 "https://www.youtube.com/@elite-examples",
 "https://example.com/",
 "https://www.amazon.com/stores/elite-examples/author/B0101010101"
 ],
 "author": {
 "@type": "Book",
 "name": "Some great book",
 "url": "https://www.amazon.com/example-ebook/dp/B01010101",
 "datePublished": "2015-08-25",
 "numberOfPages": 142,
 "inLanguage": "English",
 "bookFormat": [
  "http://schema.org/EBook",
  "http://schema.org/Paperback"
 ],
 "isbn": [
  "0123456789",
  "987-6543210"
 ],
 "sameAs": [
  "https://www.goodreads.com/book/show/example-ebook"
 ],
 "author": {
  "@id": "https://example.com/alex-doe"
 },
 "aggregateRating": {
  "@type": "AggregateRating",
  "ratingValue": "4.75",
  "reviewCount": 12,
  "bestRating": "5",
  "worstRating": "1"
 }
 }
}

There are similar schemas for things like articles, events, publications, etc. The key thing is to not get too hung up on the structure. It’s very flexible, and almost all attributes are optional. There are two important points to remember:

  • It’s more important to think about how to use JSON-LD to disambiguate any current vagueness, rather than trying to encode 100% of your available data. But would it be even better to make those changes for humans? Do you just need to add some labels and split things into lists?
  • Only add the attributes that you’re committed to maintain over time. Just like with every feature you add to the website. If you’re going to mark in JSON-LD that this course starts in September 2026, be sure to keep that up to date.

Content is key: the real opportunity is in writing better content

I mentioned how JSON-LD is the biggest technical impact that you can make. However, the even bigger impact that you can have is almost certainly with the content itself.

  • Are content creators writing in a way that matches how people actually ask questions? AI tools mirror how people talk and ask things, so content built around real questions performs better in AI search. This means: department and program pages should directly answer things like “What can you do with a degree in X?” or “How is the comparative literature PhD structured?” rather than only using institutional/marketing prose.
  • Are there definition-first sentences, explicit headings (ideally phrased as questions), one clear claim per paragraph, and self-contained sections that don’t require reading the whole page to make sense? Models retrieve passages, not whole pages, so each passage needs to stand on its own. But there’s nothing new about this. This is the same advice that we’ve been giving about the web for the last 20 years.
  • Are departments taking full use of the available content types? Every time that someone writes a new book, or publishes a major paper, does the department add it as a ‘Publication’ linking to both the person’s faculty profile, and the paper on PubMed (or wherever it lives)? By tying these things together it gives AI authority and citation signals. Leverage the tools you have before you create something new.
  • Is your team keeping the most trafficked/cited pages fresh? Keeping dateModified accurate and updating program/admissions pages when they change genuinely helps for queries like “current requirements for X major.”
  • Are you managing how your organization is talked about on the rest of the internet? Off-site influence was always a part of SEO, and with AI that still holds true. Some of this is easy to address, like your Wikipedia page. Some of it is more nuanced, like how people talk about you on Reddit, or LinkedIn.

This is just a short list of examples. Sending these to your content editors might go a long way. But they are only informed guesses about what your real problem might be. You will gain a lot from doing a deeper dive.

For some of our clients we’ve been running what we’ve called an “AI Readiness Report”. We’re leveraging some SEO/AEO/GEO analysis tools, and then synthesizing it into a report written in a way that the school dean could read and understand where the opportunities are. We did one recently for a client that was eye-opening. The most memorable takeaway was, “Your most cited page by AI is instructions on how to cook beans over a campfire. Also lots about tanning animal hides. The university homepage is fifth.”

Is there a similar problem on your institution’s website?

The future of AI content scraping

Everything that I wrote here is about the past and present of SEO/AEO/GEO. But if we imagine a year or two in the future, I think that AI bots will be consuming the web the same way that people do; taking in the whole page including the colours, images, animation, brand, emphasis, and emotion. If this is true, it means that we shouldn’t focus too much on the limited abilities of today’s bots. Instead, most of our efforts should still be on how to make our sites better for real humans.