Loading...

Skip to content
MemonaIQ
ToolsLearnCompany
Sign in
ToolsLearnCompanyBrowse all tools
HomeKnowledge CenterTechnical SEO
Technical SEO

Technical SEO

The technical side of search: crawling, robots.txt, XML sitemaps, broken links and a technical SEO checklist.

5
Guides
19 min
Reading

Start here

First read4 min read

How to find broken links on your website

Find broken links by crawling your internal pages, checking external links in bulk, sorting failures by cause, re-checking the uncertain ones and fixing each.

Read guide

Beginner

3 guides

Technical SEO checklist: crawl, index and page basics

A working technical SEO checklist: robots.txt access, noindex and canonicals, status codes, redirects, sitemaps, titles, internal links, HTTPS and page weight.

4 min read

Website crawling explained: how crawlers find pages

How a web crawler works: seed URLs, fetching, link extraction, the frontier, depth and page limits, robots.txt, politeness, JavaScript and crawl reports.

4 min read

XML sitemaps: format, limits and what to include

What an XML sitemap does and doesn't do, its format and size limits, how sitemap index files work, where to declare it, and which URLs belong in it.

3 min read

Intermediate

1 guides

robots.txt rules: syntax, matching and common mistakes

How robots.txt works under RFC 9309: User-agent, Disallow, Allow and Sitemap lines, wildcard matching, why it can't hide pages from search, and where it lives.

4 min read

Tools for this topic

Try what these guides describe on your own URLs, domains, code or files.

URL Intelligence Platform

Crawl up to 100 pages for broken pages, or paste a list of external links to check status codes and redirects. Requires sign-in.

Open tool
Website Launch Readiness Checker

Check one homepage before launch for noindex, robots.txt blocks, a reachable sitemap, metadata, HTTPS and redirects. Requires sign-in.

Open tool
URL Intelligence Platform

Crawl up to 100 pages to find broken pages, missing titles, descriptions and H1s, and duplicate titles. Requires sign-in.

Open tool

Other topics

NetworkingDNS & DomainsHTTP & The WebTLS & Web Security
All Knowledge Center topics
MemonaIQ

Focused tools for everyday digital work.

  • 18 focused tools
  • 9 run in your browser
Browse all tools Get help

Featured tools

URL Intelligence PlatformImage CompressorInvoice StudioDNS LookupAI Token Counter

Explore

All toolsKnowledge CenterGuidesResourcesCompany

Support

HelpContactSecurity

Legal

PrivacyTermsCookies

© 2026 MemonaIQ. Made in Colombo.

Back to top

MemonaIQ