GoliathAwardBot
GoliathAwardBot collects airline award prices (the points or miles a seat costs) for Goliath, a free travel tool that AI assistants connect to. It crawls an airline’s site only after that airline has given permission.
How to recognize it
| User-Agent | GoliathAwardBot/0.6 (+https://goliath.travel/bot) |
|---|---|
| robots.txt name | GoliathAwardBot |
| Request signatures | Every request is signed with Web Bot Auth (HTTP Message Signatures, Ed25519). Signature-Agent: "https://goliath.travel" |
| Key directory | https://goliath.travel/.well-known/http-message-signatures-directory (key id mtwERfE26yAzkwubS49GEl2QC9F_n1Zr9ATRDRnYGaw) |
| Network | Requests leave from Cloudflare Workers. Trust the signature, not an address. |
What it collects
For each airline that has given permission, GoliathAwardBot asks the same award-price question that the airline’s own website asks, for its list of routes: a fixed starting set, plus routes Goliath users searched between airports already on it, up to 200 per airline. It covers the dates the airline sells, up to about 11 months ahead. It stores the lowest award price per date and cabin, with taxes, seats left at that price and the flights, and shows them to Goliath users with the time they were seen. It never logs in, books, holds seats or collects personal data.
How it behaves
- It reads robots.txt at least once a day and obeys Allow, Disallow and Crawl-delay for
GoliathAwardBot, or for*if the file does not name it. If robots.txt can’t be reached, or the server answers with an error, it doesn’t crawl that site. If the site has no robots.txt (a 4xx answer), it treats every path as allowed, as RFC 9309 says. - It sends one search at a time to each airline and waits at least 10 seconds between searches (longer if Crawl-delay says so). One search can take two requests in a row. It keeps to a small daily request budget for each airline.
- It never sends cookies set by bot-protection challenges, never solves CAPTCHAs, never changes its address and never pretends to be a browser.
Airlines
| Airline | Status | Why |
|---|---|---|
| RwandAir DreamMiles | Not crawled: waiting for the airline’s permission | Terms of use: no republishing without RwandAir’s prior written consent. |
| Bahamasair Flamingo Miles | Not crawled: waiting for the airline’s permission | No published permission for automated collection; ask Bahamasair. |
| Virgin Australia Velocity | Not crawled: waiting for the airline’s permission | Terms of use forbid automated access without Virgin Australia’s prior written consent. |
| Frontier Miles | Not crawled: waiting for the airline’s permission | Terms of use prohibit scraping and automated collection. |
| Alaska Atmos Rewards | Not crawled | alaskaair.com robots.txt (updated 2026-11-10) disallows /search/results? for every crawler, AI agents included. |
| Aerolíneas Plus | Not crawled | aerolineas.com.ar robots.txt disallows /flights-offers, /booking and the flexible-dates calendar. |
Block it or allow it
To block it, add this to robots.txt:
User-agent: GoliathAwardBot
Disallow: /
To allow it through bot protection, allow requests whose Web Bot Auth signature uses the key directory above.
Contact
A contact address will be published here before GoliathAwardBot contacts any site.