TrawlerHostBucketDataUrlList

TrawlerCrawling

GoogleApi.ContentWarehouse.V1.Model.TrawlerHostBucketDataUrlList

6
out of 10
Medium
SEO Impact
Per each list that wishes to schedule url we return one of these:

SEO Analysis

AI Generated

Part of Google's web crawling infrastructure (Trawler is Google's internal name for their web crawler). This model governs how Googlebot fetches and processes web pages, affecting crawl efficiency and frequency. Crawl management directly impacts how quickly new content is discovered and how often existing content is refreshed in the index.

Actionable Insights for SEOs

  • Optimize crawl budget by fixing broken links and reducing redirect chains
  • Use robots.txt and sitemap.xml effectively to guide crawling
  • Monitor Google Search Console for crawl errors and indexing issues

Attributes

9
Sort:|Filter:
ClientCanCrawlboolean(
Default: nil

Is this client/requestorid allowed to crawl now? (based on resource use)

IsDefaultNodeboolean(
Default: nil

Is this the 'default' user's list

IsListForUrlboolean(
Default: nil

Was this the list that a given request landed in?

NumCurrentFetchesinteger(
Default: nil

# of current active fetches

NumUrlsinteger(
Default: nil

# of urls currently in the queue

PartnerTotalCapacityQpsnumber(
Default: nil

Total qps for this partner dedicated hostload, for non partner it will be -1.0.

PartnerTotalUsedQpsnumber(
Default: nil

Currently used qps for this partner dedicated hostlaod, for non partner it will be -1.0.

RequestTypestring
Default: nilFull type: String.t

The type of the request (low latency vs. high throughput)

RequestorFpstring
Default: nilFull type: String.t

The fp64 of the requestor string