ShopflixBot

Shopflix uses a web crawler, a type of Internet bot, to download the XML product feed. The crawler uses the HTTP header "User-Agent: ShopflixBot" (after communication with the seller), as well as simple headers like: "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/37.0.2062.94 Chrome/37.0.2062.94 Safari/537.36".

ShopflixBot only crawls the following URLs:

• The XML product feed link as registered in the Merchant Portal

• The image links in the XML: Shopflix needs the images for creating the products.

• The product links in the XML: Shopflix may need to check the product pages.

  ShopflixBot uses the following IPs:

• IPv4: 18.193.166.252

ShopflixBot requires access to sellers XML URLs without any rate limits.

Note

To minimize the possibility of access issues for ShopflixBot to sellers XML URLs, it is recommended to exclude the IPs used by ShopflixBot from rate-limiting tools like BitNinja, Cloudflare, etc. ShopflixBot always maintains a reasonable request rate. ShopflixBot does not perform any form of authentication. Therefore, it is recommended to exclude the specific IPs from any ACL mechanism and add them to the whitelist IP list.

An easy way to simulate ShopflixBot is to use the command:

curl -v -A "ShopflixBot accept: application/xml" <seller_xml_url>