Amazon gets an edge with its secret squad of PhD economists

Estimating inflation is a tricky and complex task. In the United States, the government's Bureau of Labor Statistics sends testers to stores to record the price of everything from cheese to tires, and surveys consumers over the phone about what they spent on gas and funeral services.

Amazon thinks it could do it better.

With help from outside researchers, the company's economists are working on a way to measure inflation using thousands of transactions across its own platform. Automatically analyzing product descriptions allows them to better assess the quality of a dress or a juicer or a bathmat, theoretically creating a more accurate, up-to-date index of how much things cost.

That's just one way Amazon is using the squad of economists it has recruited in recent years. The company has turned so many businesses, from retailing to cloud computing, inside out. Now Amazon is upending the traditional role of economists within companies, as well as the field of economics.

Amazon is now a large draw from the relatively small talent pool of PhD economists, which in the United States grows by about only 1,000 new graduates every year. Although the definition of "economist" is fuzzy, the discipline is generally understood as the study of how people use resources and respond to incentives.

In the past few years, Amazon has hired more than 150 PhD economists, making it probably the largest employer in the field behind institutions like the Federal Reserve, which has hundreds of economists on staff. It was the only company with a recruiting booth at the American Economics Association's annual conference in January, handing out free pens and logoed stress balls.

Unlike economists in academia or government, the work of Amazon's economists is almost entirely secret, and staff are required to sign non-disclosure agreements to keep it that way. But according to background interviews and Amazon itself, integrating economists has been critical to the company's phenomenal growth in e-commerce.

Amazon's economists game out real estate decisions, set the lowest prices that will deliver a profit, precisely determine what customers care about and whether advertisements are working — all using machine-learning algorithms that automate decisionmaking on a massive scale. It's the kind of asset that smaller companies can't always pay for, allowing Amazon to pull further and further away from the competition.

Other tech companies that also make effective use of economists — like Uber, which has a 30-person team — speak with frank admiration of the apparatus Amazon has built. "Amazon is the only firm that has hired a lot of economists, let alone done it successfully," Uber's chief economist, Jonathan Hall, said in a speech to the American Economics Association in January. While many companies have hired economists as public-facing spokespeople or to guide general corporate strategy, Hall explained, both he and Amazon try to integrate them as key advisers on nearly every business decision, using enormous amounts of data to replace intuition with science.

And in the meantime, Amazon is reshaping the economics field in its own image.

"The folks at Amazon and Uber are not doing the same sort of traditional chief economist's role," said Tom Beers, executive director of the National Association of Business Economics. "They're doing something completely different."

The rise of tech economists

Economists are not new to private companies, where they've long helped forecast macroeconomic conditions to guide strategic decisions about what to produce, which markets to enter and where to source raw materials.

They're not even new to the tech industry: Companies like IBM, Intel and Microsoft have had them for decades. Google's chief economist, Hal Varian, helped design the novel auction model for the company's IPO and also helped make the case that Google wasn't violating antitrust laws. Google now says it has about 300 economists and statisticians on staff, though it wouldn't give a more detailed breakdown. It is believed in the field that Amazon employs more PhD economists than any other tech company.

In the past several years, a smattering of smaller consumer-facing tech companies — like the home-search platform Homeaway and the job website Indeed — have hired economists to crunch their user data and generate insights that can be packaged for journalists as a form of earned media. They also have a government relations function, generating analyses that quantify their company's positive impact on a particular community (and thereby arguing against regulation).

Two years ago, the National Association of Business Economics started an annual conference just for tech economists, featuring remarks by four different Amazon economists and a reception sponsored by the company.

The number of tech companies advertising for economists in the American Economics Association's job listings database has risen from 15 in the 2014 to 2015 academic year to 21 in 2017 to 2018, according to a recent paper from Harvard Business School professor Michael Luca and Stanford Graduate Business School professor Susan Athey — a former economist at Microsoft. So far, the current year is on track to beat that number. And it's probably an undercount, given that not all tech companies advertise their economist postings there.

In all of those places, the transformative element has been data, and the ability to analyze it using the tools of software engineers combined with the economist's expertise in asking questions about what causes what.

"Imagine if you're a very large retailer that has a large number of locations," said Martin Fleming, chief economist at IBM. "We could typically predict with about 30% to 40% accuracy what the likely performance is of that retail location. With the current capabilities with machine learning and artificial intelligence, we can get close to 90% accuracy in forecasting business at a particular location."

That capability is perhaps most important to a company like Amazon, which has possibly more data than any other company — as well as a variety of lines of business that need data to operate, from lending to web services to last-mile delivery.

"You're basically trying to clean up waste"

The architect of Amazon's massive team of data crunchers is Pat Bajari, and he's as respected an academic as they come.

Since getting his PhD at the University of Minnesota in 1993, he has churned out papers on subjects like home prices, government procurement, auctions and collusion. After working at Harvard, Stanford, and Duke, Bajari joined Amazon as a vice president and chief economist in 2010, and two years later took an appointment at the University of Washington.

Amazon declined to make any of its economists available for interviews, or to answer questions on the record about their work. But the economists occasionally speak publicly, and Bajari appeared at the American Economics Association's conference to present a paper about how big data affects firm performance, using Amazon's enormous trove of data and experiments on forecasting retail demand. After explaining the results, he shared some thoughts about how economics has shaped tech companies.

"What I've seen change in the industry, starting about eight years ago, is firms got more serious about using the scientific method and removing chunks of guesswork within companies," Bajari said, with a characteristic nervous laugh in between sentences. "You're basically trying to clean up waste."

At other companies, economists are often clustered in a small team, but at Amazon, they are integrated into many teams across the company. In a glossy recruiting brochure, Amazon describes how its economists help build risk models for lending to third-party sellers, advise on product design and engagement tracking for devices like Alexa and Kindle, help target customers for its booming cloud services business, and forecast server capacity needs for the consumer website.

Among the 46 jobs and internships currently posted for economists on Amazon's hiring website, there are positions open for economists to help fine-tune seller pricing, figure out the best way to route trucks through Amazon's vast distribution network and identify the characteristics of top-performing talent in order to make the best hires.

For example, Amazon runs a program called Connections, which sends out small questionnaires to employees: Does your work provide you with opportunities to learn new things? Does your team always put the customer first? How often does bureaucracy get in the way of your ability to deliver results?

To improve that feedback, Amazon tries out interventions like training managers to interact better with their subordinates. Originally, the company brought on a team of psychologists, other scientists and product managers, but before long, it became apparent that they weren't well suited to achieving what Amazon was ultimately after: Better performance. Economists, by contrast, were able to analyze which interventions led to higher worker productivity.

"The psychologists had a really hard time at Amazon, because they weren't trained in what economists are trained in, which is how this relates to profitability," said one former Amazon economist who spoke on the condition of anonymity. "Amazon is a very data-driven place, and if you can't prove that your program is beneficial to customers, you're at risk of having your program defunded."

A the root of figuring out whether an HR pilot program worked, and many other things economists do at Amazon, is something called "causal inference": Determining what caused what, and therefore what to do more of and what to cut off. Take a Prime subscription, for example. Do consumers sign up mostly for the free shipping? The streaming video? Or the discounts at Whole Foods? Economists can come close to an answer, enabling the company to make better decisions about which benefits to include and which to scrap. Other decisions — like how to target ads, where to put bookstores and warehouses, and how much an Echo device should cost — are also vetted by economists.

In most companies, that kind of work might be done by any data scientist. But according to another former Amazon economist, Bajari has advocated for hiring people with formal economics training to advise on as many business decisions as possible.

"They kind of make the point that economists have more specific skill sets that are better suited for a lot of business problems," the former Amazon staffer said. "Because if you're writing an economics paper, you're studying things like the impact of the minimum wage, often without the luxury of running experiments."

Raiding the ivory tower

Building a team of 150 PhD economists is no easy task — not least because in graduate school, students learn that the private sector is a fallback option for people who couldn't land a job at a university. Corporate economists rarely publish in academic journals, which are the coin of the realm in academia. And unlike many fields in the humanities, economists can be paid well, especially at business schools.

Amazon, of course, can pay better. Even rank-and-file economists make up to $160,000, which is where all Amazon employees top out in base salary, with stock options that make their total compensation a lot higher.

However, that's not the main pitch Amazon makes to newly-minted PhDs. Its division leaders, many of whom came from tenured positions at top schools and have long lists of publications on their CVs, often write "I'm hiring!" in their LinkedIn profiles. Primarily, they advertise exclusive access to the mountains of data Amazon has at its disposal, which can be painstaking and time-consuming for researchers at universities to obtain.

"You should be agonizingly jealous of the sorts of problems we work on and the almost anxiety-provoking magnitude of data with which we get to work," wrote Wilko Schulz-Mahlendorf, a senior manager and economist who joined Amazon in 2013, in his LinkedIn profile.

Now, it's the universities that are integrating machine learning and artificial intelligence into their curricula, knowing that they're essential skills for working with the kind of information now available and the biggest questions facing the economy today.

For many academics, there's a downside to that: Private companies now create and control much of the raw material necessary to understand the modern world.

"Before, economists used to work on public data," said Beatrice Cherrier, an historian of economics at the University of Cergy Pontoise in France. "And now, if they want to study behavior, the tech companies have it, and it's proprietary."

Not only does that allow the Amazons of the world to not publish anything that might make them look bad, Cherrier explains — it also makes it difficult to replicate any studies that they might release, since the source data is under wraps.

But for those on the inside, Amazon is a playground full of delights.

Daryl Fairweather joined the company soon after receiving her PhD in economics from the University of Chicago in 2014. She was working for a consulting firm in San Diego, stressed out by deadlines on projects that she didn't find meaningful, when the Amazon option was suddenly everywhere.

"It wasn't like you had to go in and interview for just one job. You could interview for lots of jobs at the same time," said Fairweather, who left Amazon in 2018 to become the chief economist for the real estate website Redfin. Once hired, research questions were everywhere. "If I felt there was something really worth digging into, I could go dig into it. I wasn't told ahead of time what was important."

The other big draw: Even though the work Amazon's economists do may never see the light of day, within the company, it influences decisions that affect millions of people. The most junior economists can present their research before senior vice presidents, and they're evaluated on how their recommendations impact the company's bottom line.

"That was personally really exciting. The work that I do has clear value that you can see right away — it's not just hypothetical," Fairweather said. "When you publish in an academic journal, it's hard to know that lawmakers are making different choices based on what your results are. When you're in the company, it's very clear that you're changing peoples' minds."