TrawlerHostBucketData

TrawlerCrawling

GoogleApi.ContentWarehouse.V1.Model.TrawlerHostBucketData

6
out of 10
Medium
SEO Impact
============================ Next Tag: 22 ============================ Data about the scheduling host bucket a URL was in (if the client wants to use this, e.g. for more intelligent scheduling, etc).

SEO Analysis

AI Generated

Part of Google's web crawling infrastructure (Trawler is Google's internal name for their web crawler). This model governs how Googlebot fetches and processes web pages, affecting crawl efficiency and frequency. Crawl management directly impacts how quickly new content is discovered and how often existing content is refreshed in the index.

Actionable Insights for SEOs

  • Optimize crawl budget by fixing broken links and reducing redirect chains
  • Use robots.txt and sitemap.xml effectively to guide crawling
  • Monitor Google Search Console for crawl errors and indexing issues

Attributes

12
Sort:|Filter:
ClientTrafficFractionnumber(
Default: nil

How much existing traffic

ClientWeightFractionnumber(
Default: nil

belong to the client How much weight the client

CurrentActiveConnectionsinteger(
Default: nil

How many connections are actively used for downloading ?

IsFullboolean(
Default: nil

Is this bucket currently full ?

LastScheduleIntervalMsstring
Default: nilFull type: String.t

How many ms ago we last scheduled a url

MaxActiveConnectionsnumber(
Default: nil

The current hostload value (# of connections) - if negative does not apply

MediumTermLoadnumber(
Default: nil

The load the recent times (the actual hostload that we apply is MaxActiveConnections / MediumTermLoad) - a hostload of 1.00 is normal, while over 1.0 is higher than normal load

MinInterRequestSecsnumber(
Default: nil

The min delay between requests (in secs) - if negative does not apply

NonFullIntervalMsstring
Default: nilFull type: String.t

If is full, when is becoming non-full (in ms)

TotalCapacityQpsnumber(
Default: nil

The following four fields attempt to make things simpler for clients to estimate available capacity. They are not populated yet as of 2013/08/21. Even after they are populated, they may change. So talk to trawler-dev@ before you use the fields. Total qps for this hostid

TotalUsedQpsnumber(
Default: nil

Currently used qps

urllistTrawlerHostBucketDataUrlList →
Default: nilFull type: list(GoogleApi.ContentWarehouse.V1.Model.TrawlerHostBucketDataUrlList.t