Loading...

Skip to content
MemonaIQ
ToolsLearnCompany
Sign in
ToolsLearnCompanyBrowse all tools
Knowledge Center
In this topic
Technical SEO
How to find broken links on your websiteTechnical SEO checklist: crawl, index and page basicsWebsite crawling explained: how crawlers find pagesXML sitemaps: format, limits and what to includerobots.txt rules: syntax, matching and common mistakes
A-Z glossary
HomeKnowledgeTechnical SEO
Technical SEO•4 min read•beginner

Technical SEO checklist: crawl, index and page basics

A working technical SEO checklist: robots.txt access, noindex and canonicals, status codes, redirects, sitemaps, titles, internal links, HTTPS and page weight.

What a technical SEO check covers

Technical SEO covers whether search engines can fetch, read and index your pages. Check that crawlers can reach every page you want found and that those pages allow indexing. Then confirm that URLs return the right status codes and that each page has unique metadata and internal links pointing to it.

It doesn't cover content quality or links from other sites, and passing every item doesn't decide rankings. Run the list before a launch and after major releases, because one template change can add noindex to every page.

Crawl access and indexability

These items decide whether a page can appear in search at all, so check them first.

  • robots.txt lives at /robots.txt on each host and doesn't block pages you want in search. Look for a leftover Disallow: / from staging.
  • robots.txt returns 200, or 404 if you have none. Under RFC 9309, a 5xx error on that file tells crawlers to treat the whole site as disallowed.
  • Pages meant for search carry no noindex, either in a robots meta tag or in an X-Robots-Tag response header.
  • Each indexable page has a canonical link that points to itself or to the preferred version, as an absolute URL that returns 200.
  • No page is both blocked in robots.txt and marked noindex. A crawler that can't fetch the page never sees the noindex.

Status codes and redirects

Status codes tell crawlers whether a URL exists, has moved or has failed.

  • Live pages return 200. Removed pages return 404 or 410, not 200 with an error message (a soft 404).
  • Permanent moves use 301 or 308, and temporary ones use 302 or 307.
  • Each redirect goes straight to the final URL, with no chains or loops.
  • Plain HTTP redirects to HTTPS, and one hostname (www or the bare domain) redirects to the other.
  • Internal links point to final URLs instead of URLs that redirect.

Sitemap, titles and descriptions

A sitemap helps crawlers find pages, and titles and descriptions shape how each page appears in results.

  • The XML sitemap lists only canonical URLs that return 200 and allow indexing.
  • Each sitemap file stays within 50,000 URLs and 50 MB uncompressed, and robots.txt declares it with a Sitemap line.
  • Every page has a unique title element that names its topic, with the important words near the front.
  • Every important page has a unique meta description that summarizes it. Search engines may write their own snippet, but a good description gives them an accurate option.
  • Each page has a clear main heading that matches its topic.

Internal links, HTTPS and structured data

Internal links decide how easily crawlers reach each page. HTTPS and structured data affect what visitors see in the browser and in search results.

  • Every important page is linked from at least one other page with an a element that has an href. Google documents that it follows links in that form, and links that work only through JavaScript click handlers may be missed.
  • Important pages sit a few clicks from the homepage. Pages many clicks deep are harder for crawlers and visitors to reach.
  • Every URL in the sitemap is also linked from somewhere on the site. A page with no internal links (an orphan page) depends on the sitemap alone to be found.
  • Every page loads over HTTPS with a valid certificate, and no page loads scripts, styles or images over plain HTTP (mixed content).
  • Structured data, usually JSON-LD, parses without syntax errors and describes content visible on the page. Passing a validator doesn't mean rich results will appear.

Page weight basics

Heavy pages load slowly, especially on mobile connections, and slow loading shows up in Core Web Vitals. Check these basics before deeper performance work.

  • Images are compressed, sized for the space they fill, and served as WebP or AVIF where browsers support them.
  • HTML, CSS and JavaScript are compressed in transit with gzip or Brotli.
  • Static files send caching headers so repeat visits reuse them.
  • Scripts that the first view doesn't need load with defer or async, or later.
  • Images below the fold use loading="lazy", while the main image near the top loads normally.
Previous guide
How to find broken links on your website
Next guide
Website crawling explained: how crawlers find pages
MemonaIQ tools

Try it on your own input

Check what this guide describes against your own URL, domain, code or file.

Website Launch Readiness Checker

Check one homepage before launch for noindex, robots.txt blocks, a reachable sitemap, metadata, HTTPS and redirects. Requires sign-in.

Open tool
URL Intelligence Platform

Crawl up to 100 pages to find broken pages, missing titles, descriptions and H1s, and duplicate titles. Requires sign-in.

Open tool

References

  • Google Search Central: Search Essentials
  • Google Search Central: Make your links crawlable
  • RFC 9309: Robots Exclusion Protocol

On this page

What a technical SEO check coversCrawl access and indexabilityStatus codes and redirectsSitemap, titles and descriptionsInternal links, HTTPS and structured dataPage weight basics
Updated:2026-10-25
Format:Technical Guide
Related tool
Website Launch Readiness Checker

Check one homepage before launch for noindex, robots.txt blocks, a reachable sitemap, metadata, HTTPS and redirects. Requires sign-in.

Open tool
MemonaIQ

Focused tools for everyday digital work.

  • 18 focused tools
  • 9 run in your browser
Browse all tools Get help

Featured tools

URL Intelligence PlatformImage CompressorInvoice StudioDNS LookupAI Token Counter

Explore

All toolsKnowledge CenterGuidesResourcesCompany

Support

HelpContactSecurity

Legal

PrivacyTermsCookies

© 2026 MemonaIQ. Made in Colombo.

Back to top

MemonaIQ