Getting a news website found, trusted, and read on Google

How Google Search actually works

Three separate systems, and none of them can be paid, tricked, or fast-tracked.

Google Search runs on three stages that happen mostly out of your control: crawling (automated programs called crawlers find pages), indexing (Google works out what a page is about and stores it), and ranking (Google's systems decide what to show for a given query, out of everything in the index). A newsroom's job is to make each of those three stages as easy as possible — nothing more.

Discovery happens mostly through links

Googlebot finds the overwhelming majority of pages by following links from pages it already knows about — your own internal links and links from other sites. A sitemap (covered in the next section) is a useful assist, especially for a fast-moving news archive, but it's a supplement to link discovery, not a replacement for it. Practically, that means your homepage, section fronts, and article pages all need real, crawlable <a href> links pointing at every new story — not JavaScript-only click handlers, and not links buried behind pagination that only loads on scroll.

The three "Search Essentials" pillars

Google groups everything that affects eligibility into three buckets, and this guide follows the same structure:

• Technical requirements — the bare minimum a page needs to even be considered (below).
• Spam policies — behavior that gets a page ranked lower or removed entirely (Section 07).
• Best practices — everything else that improves how well a compliant page performs (most of this guide).

Check what Google already has before changing anything

Before restructuring anything, search site:yourdomain.com on Google to see what's currently indexed. If your recent stories show up, most of the fundamentals are already working and the highest-leverage work is further down this guide (structured data, dates, Discover, E-E-A-T). If they don't, work through Section 01 first.

Technical foundations

The bare minimum Google needs, plus the structural choices that matter more at news-publishing scale.

Google's technical bar is genuinely low — most sites clear it without trying. A page is eligible to be indexed once three things are true:

• Googlebot isn't blocked from finding and fetching it (no login wall, no disallow rule, nothing private).
• The page works — it returns an HTTP 200 success status, not an error.
• The content is indexable — text in a supported file format, and not in violation of the spam policies in Section 07.

robots.txt: a traffic valve, not a lock

A robots.txt file tells well-behaved crawlers which parts of the site to leave alone. For a news site it's mainly useful for keeping crawlers out of search results pages, admin routes, or duplicate print/AMP variants you don't want crawled directly — not for hiding a story. A URL disallowed in robots.txt can still surface in Search with a bare title and no snippet if enough other pages link to it, because Google never had to fetch the page to learn the URL exists. If a page genuinely needs to stay out of Google, block it with a noindex meta tag (which requires the page to be crawlable) or put it behind a login — not robots.txt.

Google News' crawler, Googlebot-News, respects the same robots.txt file as regular Googlebot, so a disallow rule written for one blocks the other too; there's no separate news-only robots directive to maintain.

Descriptive, stable URLs

Google can turn words in a URL path into the breadcrumb line shown under a result, and a clean path is easier for readers to parse and share. For a site with more than a couple thousand URLs — true of almost any news archive within its first year — organizing stories into topical directories (/world/, /politics/, /sport/) also lets Google learn that some sections change hourly and others rarely, and crawl each accordingly. That's a free efficiency gain that costs nothing to set up beyond a sane routing scheme.

Duplicate content and canonicalization

When the same story is reachable at more than one URL — a print view, an AMP variant, tracking parameters, a staging mirror — Google will pick one canonical version to show and largely ignore the rest. That's not a penalty; it's just wasted crawl attention on URLs nobody asked for, and a confusing set of links for anyone trying to cite or share the piece. Fix it with a permanent redirect from the non-preferred URL, or a rel="canonical" link element if a redirect isn't possible. Left alone, Google will usually guess correctly on its own — but for a wire-syndication network specifically, see the syndication guidance in Section 07, because the choice of canonical there also determines who gets ranking credit for a shared story.

Mobile-first indexing and secure delivery

Google predominantly indexes the mobile version of a page, so parity matters: the mobile article page needs the same body text, images, structured data, and metadata as desktop — not a stripped-down version. Serve everything over HTTPS; it's both a baseline trust signal for readers and one of the page-experience factors covered in Section 09.

If the site renders with JavaScript

Google can execute JavaScript and index client-rendered content, but it adds a rendering step that can fail silently — a headline or byline injected after load can end up invisible to the crawler if something times out. This matters more for news than most verticals because Googlebot-News in particular has narrower JavaScript handling than general-purpose Googlebot (Section 11). Server-render or pre-render at least the headline, byline, dateline, and body text of article pages, and use the URL Inspection tool in Search Console to confirm the rendered HTML actually contains them.

Redirects, done straight

Use a permanent (301) redirect for anything that has moved for good — a domain migration, a CMS re-platform, a folded-in section. Keep the redirect chain to one hop, and make sure mobile and desktop redirect the same way; a mismatch (or a redirect that depends on which device or referrer is asking) is exactly the pattern Google's spam policies flag as a sneaky redirect, covered in Section 07.

Sitemaps, and the News Sitemap specifically

An ordinary sitemap tells Google what exists. A News Sitemap tells it what's new, and it works on a strict 48-hour clock.

A standard XML sitemap — the full list of URLs you want crawled — is optional but worth having on any site above a few thousand pages, since some CMSes generate one automatically. A News Sitemap is a different, additional tool aimed squarely at fast-moving publishers: either extend the main sitemap with news-specific tags, or (Google's own recommendation) keep a separate sitemap reserved just for news articles, which also makes performance easier to track in Search Console.

Required tags

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
    xmlns:news="http://www.google.com/schemas/sitemap-news/0.9">
  <url>
    <loc>https://www.thedailydispatch.example/world/port-strike-talks-resume.html</loc>
    <news:news>
      <news:publication>
        <news:name>The Daily Dispatch</news:name>
        <news:language>en</news:language>
      </news:publication>
      <news:publication_date>2026-09-21T14:32:00+00:00</news:publication_date>
      <news:title>Port Strike Talks Resume After Overnight Break</news:title>
    </news:news>
  </url>
</urlset>

Combine it with an image sitemap

News Sitemaps can carry image sitemap tags in the same file, which is worth doing given how much Discover and Top Stories traffic is image-driven (Section 08). Submit the sitemap URL through Search Console once, and it'll be re-crawled roughly as often as the rest of the site — there's no separate news-sitemap submission cadence to manage.

Structured data for articles

JSON-LD doesn't buy placement in Top Stories, but it removes ambiguity about title, author, and dates that Google would otherwise have to guess at.

There's no structured-data requirement to be eligible for Top Stories or any other Google News surface — eligibility runs on content and policy compliance, not markup. What Article markup buys you is precision: a machine-readable statement of the headline, lead image, author, and dates that removes the guesswork Google's systems would otherwise apply by scanning the rendered page. Use the schema.org type that matches the content — NewsArticle for reported news, BlogPosting for commentary/blog-format pieces, or the generic Article when neither fits precisely.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "NewsArticle",
  "headline": "Port Strike Talks Resume After Overnight Break",
  "image": [
    "https://www.thedailydispatch.example/img/port-strike-1x1.jpg",
    "https://www.thedailydispatch.example/img/port-strike-4x3.jpg",
    "https://www.thedailydispatch.example/img/port-strike-16x9.jpg"
  ],
  "datePublished": "2026-09-21T14:32:00+00:00",
  "dateModified": "2026-09-21T18:05:00+00:00",
  "author": [{
    "@type": "Person",
    "name": "Amara Osei",
    "jobTitle": "Trade Correspondent",
    "url": "https://www.thedailydispatch.example/staff/amara-osei"
  }],
  "publisher": {
    "@type": "Organization",
    "name": "The Daily Dispatch",
    "logo": "https://www.thedailydispatch.example/logo-square.png"
  }
}
</script>

The properties worth setting

Author markup, done right

• List every visible byline as a separate object — "author": [{"name":"Willow Lane"},{"name":"Regula Felix"}] — never merge two names into one string.
• Give each author a url or sameAs pointing at a staff bio page. This is also what E-E-A-T self-assessment (Section 05) and Google's transparency expectations (Section 06) are checking for.
• Keep author.name to just the name. Job title goes in jobTitle, the outlet's name goes in a separate publisher object, and honorifics have their own honorificPrefix/honorificSuffix fields — don't fold any of that into the name string itself.
• Match the type to the entity — Person for a byline, Organization for wire-service or staff-desk credits (e.g. "Dispatch Newsroom"). Never use the generic Thing type.

Multi-part stories and canonical tags

For a story split across several pages, point rel="canonical" at either the specific page or a single "view all" page — never at page 1 of the series by default, which just relocates the duplicate-content problem from Section 01 into your structured data.

Subscription and paywalled articles

If any content sits behind a paywall or registration wall and you still want it crawled and indexed, mark that up explicitly with isAccessibleForFree and a hasPart/cssSelector pointer at the gated section. This is what keeps a paywall from reading as cloaking to Google's systems — full context on that trade-off is in Section 07's Flexible Sampling note.

<body>
  <p>This opening paragraph is free to read.</p>
  <div class="paywall">The rest of the reporting sits behind a subscription.</div>
</body>

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "NewsArticle",
  "isAccessibleForFree": false,
  "hasPart": {
    "@type": "WebPageElement",
    "isAccessibleForFree": false,
    "cssSelector": ".paywall"
  }
}
</script>

Only .class selectors work for cssSelector, sections can't be nested, and — critically — every version of the page needs consistent bot access: if Googlebot and Googlebot-News aren't authorized to see the gated text the same way a paying reader does, Search Console will report a content mismatch. List more than one gated section as an array rather than nesting them.

Organization markup, once, on the homepage or About page

Add Organization structured data in one place — home page or About page, not every article — with whichever of name, url, logo, address, contactPoint, foundingDate, and sameAs (links to your social/press profiles) genuinely apply. It feeds knowledge-panel details and disambiguates your masthead from similarly-named outlets. If you use a logo, it must be at least 112×112px, on a plain background, and reachable by Googlebot — the same rule that makes news thumbnails work in Section 08.

Testing before you ship

Validate new markup with the Rich Results Test, then check a live URL with the URL Inspection tool in Search Console to confirm Google is actually reading what you intended — passing the test tool doesn't guarantee a rich result will render, only that the markup itself is well-formed.

Byline dates and freshness

Dates are the single most-scrutinized field on a news article page, and Google checks them from two directions at once.

Google estimates a "byline date" — the date shown next to a result — from several signals together, deliberately avoiding reliance on any single one, because any one source (a visible label, a CMS timestamp, structured data) can be wrong or missing on its own. In practice that means the visible date on the page and the structured-data date need to independently agree.

What to put on the page

• Show a genuinely visible date and time, clearly labeled — "Published," "Posted," or "Last updated" all work — ideally positioned between the headline and the body copy, which is also where Google's crawler looks first.
• The date is required; the time is optional but recommended, with a correct timezone (mind daylight saving).
• Keep the visible date and the datePublished/dateModified structured-data values in agreement.
• The date describes when the page was published or updated — never the date of the event being reported on. If a story needs to communicate an event's date, that's what Event structured data is for, kept separate from the article's own dates.
• Minimize other, unrelated dates competing for attention on the same page (an unrelated "related stories" timestamp, a footer copyright year) if you notice Google picking the wrong one.
• Never post-date an article into the future.

Keep the byline and dateline out of your snippet text

Google's crawler looks for body text starting right after the headline to build a search snippet. If the byline and date aren't clearly set apart from the first sentence in the markup — visually and structurally — a snippet can end up swallowing "By Amara Osei, Sept 21, 2026 —" instead of the actual lead sentence. Wrap the byline/date block in its own container, distinct from the article body's opening paragraph.

E-E-A-T and people-first content

Not a ranking factor you can toggle on — a description of what Google's ranking systems are already trying to reward.

E-E-A-T stands for Experience, Expertise, Authoritativeness, and Trust — a framework Google uses to describe the qualities its automated ranking systems look for, and the same framework its human search quality raters are trained on when sample-testing whether ranking changes are working as intended. Rater judgments never feed directly into rankings for any individual page — think of it as a restaurant reading comment cards, not the chef taking orders from them — but the rater guidelines are a genuinely useful mirror for self-assessment. Of the four letters, trust is the foundation; experience, expertise, and authoritativeness all exist to build it, and a page doesn't need to demonstrate all three to be considered trustworthy.

This bar rises specifically for YMYL topics — content that could meaningfully affect someone's health, finances, safety, or civic life — and a newsroom's daily output regularly lands squarely in that category: elections, public-health reporting, courts, personal finance, disasters. Treat those beats as the ones where sourcing, correction policies, and author credentials matter most.

Self-assessment, borrowed from Google's own checklist

Reasonable proxies for "is this piece actually good" —

• Does it add original reporting, on-the-record sourcing, or analysis, rather than rounding up what wire copy or competitors already said?
• Would another outlet, or a reader, want to cite it?
• Is authorship obvious, with a byline linking to real background on the reporter?
• Is it free of easily-checked factual errors and sloppy production?

The "people-first" test — and its warning signs

Content made primarily for an existing or intended audience — people who'd come to the site directly — tends to hold up. Content made primarily to attract search traffic tends not to, and Google spells out several tells of the latter that map uncomfortably well onto some newsroom growth tactics: publishing high volumes across unrelated topics hoping something ranks; chasing a trending term outside your actual beat purely because it's trending; heavy automation without a genuine reporting or editing step; and — worth calling out on its own — promising an answer that doesn't exist yet, such as implying a confirmed release date, verdict, or resignation before one is actually confirmed. The date-manipulation warning from Section 04 belongs on this same list.

Who, how, and why

Google frames content evaluation as three questions, and they translate directly into newsroom practice:

• Who made this? — a byline should be self-evident wherever a reader would expect one, linking to a real bio with background and beat.
• How was it made? — for data journalism, investigations, or anything AI-assisted, a visible methodology note or AI-use disclosure builds the same trust a "how we tested this" box does in a product review. Disclosure is expected wherever a reader would reasonably wonder how the piece came together.
• Why was it made? — this is the one that matters most. If the honest answer is "to be useful to our readers," everything else in this guide compounds. If the honest answer is "because it might rank," using automation toward that end specifically crosses into the scaled-content-abuse spam policy covered in Section 07.

Google News eligibility, as it actually works today

If you've read an older guide that describes "applying" to Google News, it's describing a process Google dismantled in 2024–2025.

What didn't change is the eligibility bar itself: content that follows Google's Search and Google News policies is automatically eligible for consideration across Top Stories, the News tab, and Discover, the moment it's crawled. Google's own systems rank what qualifies by relevance, prominence, authoritativeness, freshness, location, and language — the same kind of signal mix this whole guide is about, not a separate gate.

So what is Publisher Center still for?

Management, not submission: content labels (marking sections as opinion, satire, or press releases so Google can display them appropriately), and logo/branding assets for News Showcase and Reader Revenue Manager if you use those products. It is no longer where you get a site added to Google News — a properly built, policy-compliant site gets picked up without it.

The content policies that gate news surfaces specifically

Beyond the general Search spam policies (Section 07), Google News, Top Stories, and Discover enforce a news-specific policy set. In short, they exclude: dangerous or harmful instructions, deceptive impersonation or concealed ownership, harassment, hateful or discriminatory content, manipulated media presented as authentic, medical claims against scientific consensus, sexually explicit material, violent extremism, gratuitously graphic violence, and gratuitous profanity. Two more are worth stating in full because they're easy to misjudge from inside a revenue team:

• Ads and sponsored content — advertising shouldn't outweigh the editorial content on a page, and sponsored or branded content must never be dressed up as independent reporting. Any sponsorship — ownership stakes, payment, material support — needs clear, upfront disclosure to the reader.
• Misleading preview content — a headline, snippet, or thumbnail can't promise a detail the article itself doesn't deliver, the print-era "curiosity gap" problem in modern form.

Transparency: the single biggest lever

Google is explicit that readers (and its own systems) need to be able to tell who's behind the content:

• Clear, visible bylines and dates on every story (Section 04).
• Real information about individual authors, the publication, and its publisher.
• Disclosure of the company or network that owns the outlet.
• Working contact information.

A maintained About page covering the masthead, ownership, and how to reach an editor is one of the highest-leverage single pages a news site can build — it feeds this policy, the E-E-A-T self-assessment in Section 05, and the Organization markup in Section 03 simultaneously.

What must be blocked or avoided

The spam policies that apply to every site, plus the ones written with wire services, syndication deals, and advertorial sections specifically in mind.

Scraped and lightly-rewritten content

If any part of the site republishes other outlets' reporting — including through automated synonym-swapping, translation-and-back, or "spinning" — without adding real value or crediting the source, that content must be blocked from indexing and from Google News specifically. Google's scaled-content-abuse policy calls out AI-generation used to mass-produce pages with no added value as the same category of problem, whatever tool produced it.

Legitimate syndication, handled correctly

Wire services, press-release distribution, and syndication partnerships between news publishers are explicitly not spam — Google's own policy names this carve-out directly. The obligation runs the other way: if your outlet is the one distributing content for others to republish, ask your syndication partners to block or canonicalize their copies back to your original, so the original keeps the ranking credit instead of splitting it across a dozen mirrors.

The site reputation policy — advertorials, freelancers, and hosted sections

This is the policy most likely to snag a news site's revenue-side content, so it's worth walking through carefully. It applies when third-party content — freelance work, a white-label deals section, sponsored content — is hosted mainly to borrow a domain's already-earned authority, so that content ranks better than it could stand-alone. Hosting third-party content is not, by itself, the problem; the problem is when it's there because of the host's ranking power rather than being genuinely part of the publication.

Google reviews this on facts like: does the section's design and editorial quality match the rest of the site, is authorship and editorial responsibility clearly stated, and does identical content appear verbatim across many other unrelated sites. In practice:

• Fine: a branded deals/coupons section built with a commercial partner, clearly labeled, linked from normal navigation, editorially curated for your audience specifically.
• Fine: an original freelancer-written piece with clear byline and editorial sign-off — even if that freelancer writes similar pieces for other outlets too, as long as the piece itself is original to your site.
• A problem: an unbylined affiliate page, unreachable from normal site navigation, that turns out to duplicate a marketplace's own product copy verbatim — the classic sign of a hosted section that exists only to borrow domain authority.

For sites serving readers in the European Economic Area, a violation here is handled differently from the rest of the world: instead of a sitewide manual action, the offending section is ranked as its own separate entity, independent of the main domain's authority, rather than demoted outright.

Paid and unnatural links

Selling links that pass ranking credit, running large-scale "distribute our press release with a keyword-rich link" campaigns, or accepting advertiser links without qualification all violate the link-spam policy. Any commercial, sponsored, or affiliate link needs rel="sponsored" or rel="nofollow" — normal, expected practice for an ad-funded newsroom, not a penalty to avoid triggering.

The smaller, still-worth-knowing list

• Keyword stuffing — unnatural repetition of search terms, including lists of city names or phone numbers with no real content around them.
• Cloaking — showing crawlers different content than readers see. A properly-marked paywall (Section 03) is explicitly not cloaking; a paywall that hides content from Google entirely while still trying to rank for it is.
• Sneaky redirects — sending mobile readers somewhere different from what desktop readers (or Google) see at the same URL.
• Expired-domain abuse — buying a defunct outlet's old domain purely to inherit its authority for unrelated content. Acquiring a masthead and its archive for legitimate editorial reasons is a different thing entirely, but the line is exactly how much of the original publication's actual purpose survives the acquisition.

Discover, Top Stories, and Preferred Sources

Three surfaces, three different behaviors — and one brand-new feature most guides haven't caught up with yet.

Google Discover

Discover is the interest-feed surface (mobile app and Chrome) that needs no special markup or opt-in — any indexed page is automatically eligible, provided it clears Discover's content policies (the same set as Section 06). What actually moves the needle:

• Titles and previews that state plainly what the piece is about — no withheld details or exaggerated framing purely to drive a click.
• Avoiding sensationalism aimed at outrage or morbid curiosity rather than informing.
• Large, genuinely relevant lead images: at least 1,200px wide, over 300,000 total pixels, ideally a 16:9 crop. Enable large-image treatment with the max-image-preview:large robots meta tag (or via AMP). Specify your preferred image with schema.org image markup or an og:image tag — never a logo, and avoid text-heavy graphics, which crop poorly into Discover's feed format.
• An overall strong page experience (Section 09) — Discover draws on the same quality and page-experience signals as regular Search.

Discover traffic is intentionally more volatile than keyword-driven Search traffic — it shifts with reader interest, with Google's own adjustments to which content formats appear, and with the same core updates that move regular rankings. Treat it as a genuine but unpredictable bonus on top of Search traffic, not a channel to plan revenue around. Track it in Search Console's Discover performance report, which covers roughly 16 months and folds in Chrome-surface traffic.

Top Stories

Top Stories is the news-specific block that appears above regular results for news-oriented queries, populated automatically for any site meeting the content policies in Section 06 — there's no separate opt-in. The one point worth correcting explicitly: AMP has not been required for Top Stories since 2021, when Google's Page Experience update replaced the AMP requirement with Core Web Vitals thresholds that apply equally to AMP and standard pages. If an older guide, vendor, or internal wiki still describes AMP as a Top Stories requirement, that guidance is several years out of date.

Preferred Sources — new, and worth setting up

Preferred Sources lets a reader explicitly mark your outlet as a source they want to see more of; your content then carries a "preferred" badge and is more likely to surface in Top Stories, and — for readers who've made the same choice — in AI Mode and AI Overviews. It's available globally for Top Stories in every Search language; qualifying for the AI Mode/AI Overviews version additionally requires opting in to Search generative-AI features in Search Console. Only a full domain or subdomain qualifies — a subdirectory like example.com/news isn't independently eligible, so this favors outlets on their own domain over a section hosted inside a larger site.

Check whether your site already appears in the source preferences tool, then make it easy for readers to add you with a two-line embed:

<script async src="https://news.google.com/swg/js/v1/publisher.js"></script>
<div google-add-preferred-source-btn></div>

That renders a localized, Google-styled button wherever you place the <div>. If your CMS can't run the script, a plain deeplink works instead: https://www.google.com/preferences/source?q=yourdomain.com, usable as a text link, a styled button, or in a newsletter and social posts.

Page experience and Core Web Vitals

The one part of "page experience" that's a direct, measurable ranking input — and the part most likely to be quietly wrecked by your own ad stack.

There's no single "page experience score." Google's ranking systems weigh several aspects together — good Core Web Vitals, HTTPS delivery, mobile display, ads that don't overwhelm the content, freedom from intrusive interstitials, and a page where the main content is easy to tell apart from everything else on it. Of these, Core Web Vitals is the piece with a direct line into ranking; the rest align with what ranking already rewards without being separate levers of their own. A perfect Core Web Vitals score is not, by itself, a path to a first-place ranking — Google is explicit that chasing a flawless score purely for SEO's sake can be time better spent elsewhere; the goal is genuinely a better experience for readers.

Interstitials: where subscription and newsletter prompts cross a line

A newsletter signup box, subscription paywall, or app-install prompt isn't against policy on its own — but one that fully blocks the article the instant a reader arrives from a search result, especially on mobile, is exactly what Google's interstitials guidance flags as hurting page experience. A gate that appears after some scroll or reading time, or a dismissible banner that leaves the headline and lede visible underneath, keeps the same business goal without the same penalty risk.

Where to check the numbers

Search Console's Core Web Vitals report shows real-user data grouped by URL pattern; its HTTPS report flags anything still served insecurely. PageSpeed Insights and Chrome's Lighthouse panel are the go-to tools for diagnosing a specific slow template before it accumulates enough real-user data to show up in Search Console at all.

Optimizing for AI Overviews and generative search

How Google's Gemini-driven AI Overviews discover, extract, synthesize, and cite news reporting — and how to position your journalism as the primary ground-truth source.

Google's search results increasingly present an AI Overview at the top of query results, particularly for informational, breaking, and explanatory queries. These syntheses do not replace indexing; rather, they operate as a generative synthesis layer on top of Google's core retrieval and ranking systems. To be cited and linked as a source card in an AI Overview, an article must first clear the standard technical and quality bars of Search and Google News.

How generative search systems extract information

Large language models utilized in search rely on retrieval-augmented generation (RAG). When a user submits a query, Google retrieves top-ranking, authoritative documents from its index, passes relevant passages to the synthesis model, and grounds the generated summary in those passages. Articles with clear, unambiguous factual statements and well-defined semantic structures are substantially more likely to be selected as grounding sources.

Structuring news copy for machine comprehension

Writing for AI Overviews does not mean writing for robots; it means reviving the classic inverted pyramid with extreme clarity:

• Direct answers up front: Begin breaking news stories with the concrete outcome (who, what, when, where, why) in paragraph one. Avoid preamble, rhetorical questions, or narrative suspense that delays the core facts.
• Definitive statement headers: Use descriptive subheadings (<h2>, <h3>) that state the topic directly (e.g., "Timeline of the trade negotiations" instead of "What happened next").
• Structured data lists: When summarizing key findings, policy provisions, or timelines, use ordered (<ol>) or unordered (<ul>) HTML lists. Generative models extract list items cleanly for bulleted citations.
• Explicit entity naming: Avoid vague pronouns ("the leader", "the company") in lead sentences; use full proper nouns on first reference to anchor entity recognition.

AI crawler governance: Search vs training

Publishers must distinguish between crawlers that index content for Search and AI Overviews, and crawlers used exclusively to train commercial AI foundation models. Blocking Google's primary search crawlers removes your publication from Google Search, Google News, and AI Overviews entirely.

User-agent: *
Allow: /

# Google-Extended controls Gemini/Vertex model training without affecting Search or AI Overviews
User-agent: Google-Extended
Disallow: /

# Prevent third-party commercial AI scrapers while preserving search engines
User-agent: GPTBot
Disallow: /

User-agent: CCBot
Disallow: /

Googlebot and Googlebot-News specifics

Understanding the dual-crawler pipeline, rendering lifecycles, crawler verification, and caching headers.

Google employs multiple crawler variations across its systems. For news publishers, two crawlers determine content discovery and appearance: Googlebot (the primary web crawler) and Googlebot-News (the dedicated news surface crawler). Understanding their distinct operational profiles prevents costly indexation delays.

The JavaScript hydration hazard for breaking news

While standard Googlebot can render client-side JavaScript (Single Page Applications built with React, Vue, or Angular), the rendering queue is asynchronous. Googlebot-News operates on immediate cycles to populate real-time Top Stories carousels. If an article's headline, byline, dateline, and body text require client-side execution to appear in the DOM, Googlebot-News may crawl an empty shell. Always provide server-side rendered (SSR) or pre-rendered static HTML for article pages.

Verifying authentic Googlebot requests

Never rely solely on the User-Agent header to grant crawler access, as malicious scrapers frequently spoof Googlebot. Verify crawler authenticity using two-step reverse DNS lookup:

# Step 1: Run reverse DNS on the incoming client IP address
$ host 66.249.66.1
1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.

# Step 2: Run forward DNS on that hostname to verify IP match
$ host crawl-66-249-66-1.googlebot.com
crawl-66-249-66-1.googlebot.com has address 66.249.66.1

HTTP status codes and edge caching headers

During breaking news events, Googlebot-News may request updated article URLs every few seconds. Returning proper HTTP caching headers preserves origin server capacity while ensuring crawlers receive instant updates:

• Support conditional GET (304 Not Modified): Honor If-Modified-Since request headers and ETag tags. If content has not changed since Googlebot's last visit, respond with 304 Not Modified with zero response body.
• Stale-While-Revalidate: Use Cache-Control: public, max-age=60, stale-while-revalidate=300 at the CDN edge so readers and crawlers get sub-50ms responses while background updates populate seamlessly.
• Immediate 410 for permanently removed articles: If an article was retracted or removed permanently, return 410 Gone rather than 404 Not Found. Googlebot de-indexes 410 URLs significantly faster.

Monitoring, verification, and diagnostics

The essential telemetry stack for newsrooms: tracking indexation velocity, diagnosing silent drops, and verifying rich results before publication.

News SEO operates at minutes-level latency. Waiting for weekly ranking reports is ineffective when stories have a 48-hour lifespan. Newsrooms require continuous verification and real-time monitoring across three tiers: Search Console telemetry, automated schema validation, and server edge logs.

Search Console: The three distinct news performance reports

Google Search Console segments news reporting into three separate tabs. Monitoring each reveals specific surface health:

• Search Results (Web): Measures general query rankings and universal web snippets. Look at average position and query impressions.
• Discover: Measures push-interest feed impressions. High-velocity spikes with high CTR (8%–15%) driven by high-resolution imagery and timely topics.
• Google News: Tracks traffic from news.google.com and the Google News mobile apps. Indicates publisher authority within topic clusters.

Troubleshooting News SEO regressions

Pre-publish verification tools

Before launching site-wide template changes or publishing major investigative projects, validate against Google's official testing suites:

• Google Rich Results Test (search.google.com/test/rich-results): Verifies NewsArticle, BreadcrumbList, and author entity syntax.
• Search Console URL Inspection Tool: Use "Test Live URL" to view rendered DOM, HTTP response code, and indexability flags.
• Schema Markup Validator (validator.schema.org): Verifies full Schema.org compliance without search-engine specific restrictions.

Launch and newsroom audit checklist

A complete, battle-tested pre-publish and continuous deployment checklist for news engineers, product managers, and editors.

Use this unified checklist during code deployments, CMS updates, and editorial onboarding to guarantee strict compliance with Google Search Central standards.

1. Technical Infrastructure

• robots.txt accessible: Returns HTTP 200 at /robots.txt; explicitly lists standard and news sitemaps; does not disallow Googlebot or Googlebot-News.
• News Sitemap active: Dynamic /sitemap-news.xml lists only stories published $\le 48$ hours ago; includes <news:publication> and <image:image> tags.
• Canonical tags configured: Every article page features an absolute, self-referential <link rel="canonical">.
• Discover image preview enabled: <meta name="robots" content="max-image-preview:large"> included in document head.
• Mobile viewport & HTTPS: Responsive viewport meta tag present; HTTP redirects cleanly to HTTPS (301); SSL certificate valid.

2. Structured Data & Schema.org

• NewsArticle JSON-LD present: Embeds headline, images array, datePublished, and dateModified in valid ISO 8601 format.
• Author Person entity linked: Author is defined as a Person object linking to a dedicated staff profile URL with biographical credentials.
• Publisher Organization declared: References site name, URL, and logo image object ($\ge 512 ext{px}$).
• Paywall tagging: If subscription content is present, isAccessibleForFree: false and hasPart.cssSelector are explicitly configured.

3. Editorial Transparency & E-E-A-T

• Visible byline & dateline: Real human reporter bylines with visible publication and update dates positioned between headline and body text.
• Author Bio pages: Author links lead to staff bio pages detailing beat, professional background, and verified credentials.
• Editorial policies published: Accessible links in navigation or footer for Editorial Policy, Corrections Policy, and Contact Information.
• Ownership & masthead: About page transparently identifies publisher entity, ownership structure, and editorial leadership.

4. User Experience & Core Web Vitals

• Core Web Vitals passed: LCP < 2.5s, INP < 200ms, and CLS < 0.1 on mobile devices.
• Zero layout shift on media: Images and dynamic elements specify explicit width, height, or CSS aspect-ratio.
• Non-intrusive interstitials: Newsletter and subscription prompts trigger on scroll or intent; never obstruct primary article reading on initial page view.
• Google Preferred Sources button: Follow widget or Subscribe with Google publisher embed active for reader preference tracking.

Official documentation and authoritative sources

Direct citations and official guides from Google Search Central, Google News, and Schema.org standards.

This field guide is synthesized directly from Google's official engineering guidelines, policy updates, and W3C specifications. Key references include: