- 09 Aug 2022
- Cold Call Podcast
A Lesson from Google: Can AI Bias be Monitored Internally?
Dr. Timnit Gebru was the co-lead of Google’s Ethical AI research team—until she raised concerns about bias in the company’s large language models and was forced out in 2020. Her departure sent shockwaves through the AI and tech community and raised fundamental questions about how companies safeguard against bias in their own AI. Should in-house ethics research continue to be led by researchers who best understand the technology, or must ethics and bias be monitored by more objective researchers who aren’t employed by companies? Professor Tsedal Neeley discusses how companies can approach the problem of AI bias in her case, “Timnit Gebru: 'SILENCED No More' on AI Bias and The Harms of Large Language Models.”
- 15 Sep 2020
- Working Paper Summaries
Time and the Value of Data
This paper studies the impact of time-dependency and data perishability on a dataset's effectiveness in creating value for a business, and shows the value of data in the search engine and advertisement businesses perishes quickly.
- 23 Mar 2020
- Working Paper Summaries
The Effects of Hierarchy on Learning and Performance in Business Experimentation
Do senior managers help or hurt business experiments? Analyzing a dataset of more than 6,300 experiments on the A/B/n testing platform Optimizely, this study suggests that involving senior executives in experimentation teams can have surprising consequences.
- 28 Nov 2018
- HBS Case
On Target: Rethinking the Retail Website
Target is one big-brand retailer that seems to have survived and even thrived in the apocalyptic retail landscape. What's its secret? Srikant Datar discusses the company's relentless focus on online data. Open for comment; 0 Comments.
- 13 Feb 2017
- Research & Ideas
Paid Search Ads Pay Off for Lesser-Known Restaurants
Researchers Michael Luca and Weijia Dai wanted to know if paid search ads pay off for small businesses such as restaurants. The answer: Yes, but not for long. Open for comment; 0 Comments.
- 28 Feb 2013
- Working Paper Summaries
Do Display Ads Influence Search? Attribution and Dynamics in Online Advertising
The introduction of online metrics such as click through rate (CTR) and cost per acquisition (CPA) by Google and other online advertisers has made it easy for marketing managers to justify their online ad spending in comparison to the budgets used for television and other media. However, these metrics suffer from two fundamental problems: (a) they do not account for attribution, since they give credit to the last click and ignore the impact of other ad formats that may have helped a consumer move down the conversion funnel, and (b) they ignore the dynamics, since they only account for the immediate impact of ads. As firms spend more of their ad dollars on online search and display, managers and researchers alike recognize a need for more careful attribution adjustment that takes into account the journey consumers follow before conversion as well as account for the impact of ads over time. In this paper, the authors use time series models to infer the interaction between search and display ads and also capture their impact over time. Examining data from a bank that used online advertising to acquire new customers for its checking account, the authors found that display ads have a significant impact on search applications, as well as clicks. The majority of this spillover was not instant, but took effect only after two weeks. On the other hand, search advertising did not lead to an increase in display applications. However, search ads showed significant dynamic effects on search applications that made them very cost effective in the long run. Key concepts include: Classic metrics used in practice are highly biased since they do not account for the effects documented in this study. As a result, firms may be making suboptimal budget allocation decisions. Managers should carefully consider the interaction and dynamic effects of search and display advertising. In the study, revised measures of ad effectiveness lead to a very different budget allocation than the one used currently by the firm. Even though the proposed allocation gives credit to display due to its effect on search applications, the search ad budget should be increased by 36% from its current level due to its strong dynamic effects. The display ad budget should be decreased by 31%. Closed for comment; 0 Comments.
Why Technology Alone Can't Solve AI's Bias Problem
Engineers designed "fair-ranking algorithms" to prevent artificial intelligence from marginalizing certain groups. While these tools help, research by Himabindu Lakkaraju finds that they can't completely override the most stubborn source of bias: people.