Companies are waging an invisible data war online. And your phone might be an unwitting soldier.
Retailers from Amazon and Walmart to tiny startups want to know what their competitors charge. Brick and mortar retailers can send people, sometimes called “mystery shoppers,” to their competitors’ stores to make notes on prices.
Online, there’s no need to send people anywhere. But big retailers can sell millions of products, so it’s not feasible to have workers browse each item and manually adjust prices. Instead, the companies employ software to scan rival websites and collect prices, a process called “scraping.” From there, the companies can adjust their own prices.
Companies like Amazon and Walmart have internal teams dedicated to scraping, says Alexandr Galkin, CEO of the retail price optimization company Competera. Others turn to companies like his. Competera scrapes pricing data from across the web, for companies ranging from footwear retailer Nine West to industrial outfitter Deelat, and uses machine-learning algorithms to help its customers decide how much to charge for different products.