Retailers and other businesses have long kept track of what consumers buy and scoured public records, social media and other resources for insight on how to sell consumers more products.
But many consumers likely have no idea how much a retailer can divine about them. Industry practices came to light last month in a New York Times article on Target, which studies customers' shopping patterns and tries to identify which patrons are pregnant.
Identifying expectant customers is just one task of the 50-some Target employees who analyze customer purchases and other data, looking for ways to increase sales. The team's researchers help determine the coupons customers receive in the check-out line and what they see when one visits the retailer's website.
The team is led by Andrew Pole, Target's manager of guest marketing analytics. The company declined to make Pole available for an interview. But he told the New York Times in detail how Target had learned to identify pregnant customers, well before they might volunteer that information to the company through a baby registry.
That gives Target an early start on promoting and selling maternity and baby products to customers. Pregnant women are particularly valuable because they have to buy a variety of new supplies and equipment, from crib sheets to baby monitors.
The data can help Target turn new mothers into lifelong customers.
The Times' article generated a lot of attention and controversy, far more than when Pole boasted about the retailer's power to predict pregnancies during an analytics industry conference in 2010.
"We ideally want to acquire and convert prenatal mothers before they have their babies," he told attendees of Predictive Analytics World. "How the heck do we do that? We develop a model to predict if a woman is likely pregnant with child. We've done this."
Pole said at the conference that the model is based on detecting early nesting patterns. What a woman buys and when — things like vitamins and cribs — can indicate if she is pregnant. Pole also said the analysis reveals many more pregnant customers than Target would otherwise be able to identify. (Click here to watch his presentation.)
"Now all of sudden, I got 30 percent more guests that I'm pretty darn sure are pregnant" Pole said. "In fact, I know this one is in the second trimester. This one is in the third trimester. I even know what to offer them and what are the relevant times to offer those things. To find 30 percent more guests was a big win for us."
Maybe at the check-out. But not with customers like Ann Keeler, who shops at the Target in downtown Minneapolis. She is not surprised that Target and other retailers track her purchases. But knowing that a retailer tries to divine if women are pregnant makes her uncomfortable.
"That's ... Orwellian," she said. "When things get too personal, it's not good."
Target's research even caught the attention of Comedy Central's Stephen Colbert.
"Target knows when you've missed your period. Period," he joked recently.
Target officials declined an interview request. The company issued a statement saying that, like many companies, Target uses research tools to understand customer preferences and provide offers that are relevant to them.
The company's website includes privacy policies that detail what information it collects and how it is used.
Still, Target is venturing further than other retailers, according to Eric Siegel, a consultant who has taught data mining at Columbia University.
"What Target is doing there is unusual, because typically what retailers and other companies do is they predict what will you buy," he said. "Obviously, whether somebody is pregnant is definitely a private concept. That's certainly sensitive information."
Siegel expects there will be more controversy in the future about how companies analyze and use customer data. He hopes that will lead to a serious public discussion about balancing the benefits of data mining with the privacy expectations of individuals.
University of Minnesota information and decision sciences professor Ravi Bapna said corporations and other organizations should already be doing that. But from what he has observed, they're not.
"These issues are not being discussed at the appropriate level in most companies right now," he said. "Data science is giving us incredible insights but we have to be careful in how we use those insights."
Bapna said that if organizations reflect what customers want and consider fair and proper, then data mining efforts can benefit both sides.