What is Crawl Budget?
Turkish: Tarama Bütçesi
Crawl budget is the crawl capacity and priority a search engine gives a site, affecting which URLs are discovered and refreshed.
What is Crawl Budget?
Crawl budget is the balance of capacity and demand that affects how often and how deeply search engine bots such as Googlebot crawl URLs on a site. It becomes important on sites with thousands of products, filters, pagination pages, or archive URLs.
What Affects It?
Crawl capacity is about how reliably the server responds to bot requests. Slow responses, 5xx errors, and long redirect chains can make bots crawl more cautiously. Crawl demand is related to popularity, freshness, internal links, and whether a URL appears worth indexing.
How Is It Optimized?
- Expose important URLs in a clean sitemap
- Control unnecessary filter and parameter combinations
- Reduce duplicate content, infinite calendars, and internal search result pages
- Fix broken links, 404/5xx rates, and redirect chains
- Use robots.txt to limit areas that do not need crawling
Note for Small Sites
For healthy sites with a few hundred pages, crawl budget is usually not the primary issue. Content quality, internal linking, and technical fixes may matter more for SEO.
Business Use
On large e-commerce, classifieds, news, and documentation sites, crawl budget affects how quickly new pages are discovered. If new products or campaign pages are indexed late, revenue can be affected, so log analysis, Search Console data, and URL inventory should be reviewed together.
Related Terms
robots.txt is a plain text file at a site's root that tells search engine crawlers which URL areas they may or may not crawl.
SEO (Search Engine Optimization)SEO improves how search engines crawl, understand, and rank a site so it can appear for relevant searches without paid ads.
Sitemap (XML)A sitemap is an XML roadmap that tells search engines about indexable URLs, last modification dates, and alternate language versions.