On today's web, HTTP results depend on the HTTP User-Agent you use

Websites now routinely serve different content to different HTTP User-Agents, with some getting errors, redirections, or challenge pages while others receive actual content. This makes it increasingly risky to assume results from one User-Agent apply to another, especially with browser-like User-Agents that are frequently forged by crawlers and malware.