Personalizing the listening experience (slides)

Bandit algorithms have gained increased attention in recommender systems, as they provide effective and scalable recommendations. These algorithms use reward functions, usually based on a numeric variable such as click-through rates, as the basis for optimization. Detecting and understanding implicit measures of user satisfaction are essential for enhancing recommendation quality. When users interact with a recommendation system, they leave behind fine grained traces of interaction signals, which contain valuable information that could help gauging user satisfaction. Quantifying such a notion of satisfaction from implicit signals involves understanding the diverse needs of users and their expectations of what is a successful streaming session. Such needs often include how users feel, and the expectations that music recommended to them align with their mood or their intent of the moment. To fulfill these, Spotify provide users with curated playlists, ranging from ''sleep" to ''run". In addition, and to account for the diverse user interests and plethora of musics, genre playlists have been made available to users, ranging from rap, pop, jazz, to niche ones. As a result, millions of playlists are available to users to listen to based on their intent and needs. Given such a heterogeneity in user needs, and the different intents of content (playlists), it becomes important to consider both user and content behavior to formalize the notion of satisfaction, and in turn design the appropriate reward models to capture these. This talk will describe methods explored to provide a more informed reward function, that account for the type of user and the type of content.  

Mounia Lalmas is a Director of Research at Spotify, and the Head of Tech Research in the Personalization mission of Spotify, where she leads an interdisciplinary team of research scientists working on personalization and discovery. Mounia also holds an honorary professorship at University College London. Before that, she was a Director of Research at Yahoo, where she led a team of researchers working on advertising quality for Gemini, Yahoo native advertising platform. She also worked with various teams at Yahoo on topics related to user engagement in the context of news, search, and user generated content. Prior to this, she held a Microsoft Research/RAEng Research Chair at the School of Computing Science, University of Glasgow. Before that, she was Professor of Information Retrieval at the Department of Computer Science at Queen Mary, University of London. Her work focuses on studying user engagement in areas such as native advertising, digital media, social media, search, and now music. She has given numerous talks and tutorials on these and related topics. She is regularly a senior programme committee member at conferences such as WSDM, WWW and SIGIR. She was co-programme chair for SIGIR 2015 and WWW 2018. She is also the co-author of a book written as the outcome of her WWW 2013 tutorial on "measuring user engagement".

Optimizing for longer-term outcomes (slides, paper)

Many personalization algorithms today are optimizing for short-term rewards like clicks or plays, but what we ultimately want to optimize is often a longer-term outcome, like long-term user satisfaction. In this talk we will talk about some attempts to align personalization algorithms to longer term rewards. We will also show an old and computationally efficient method to estimate a long term reward in a discrete decision setting. We will show that this method leads to richer rankings than possible when considering only a single decision point, as well as demonstrate its effectiveness on two different datasets.

 David Hubbard has been a Senior Research Scientist at Netflix since 2010 in the Data Science and Engineering group. He has worked on a range of problems from estimating the popularity of shows Netflix may want to acquire, to predicting who will watch those new shows, to estimating the tenure of our subscribers. David got his Masters in Applied Math from UCLA and his main areas of interest are large scale machine learning and interpretability.

Benoit Rostykus is a Senior Machine Learning Researcher. At Netflix, he has worked on problems related to causality, large-scale learning in a sparse settings, combinatorial optimization, and high-performance data processing. Prior to Netflix, Benoit worked in various adtech and biotech startups. Benoit holds Masters Degrees From Ecole Centrale Paris and Ecole Normale Supérieure de Cachan.

Horizon: Deep Reinforcement Learning at Scale (slides)

To build a recommender system, we must provide answers to two sets of questions: (1) "What will happen if I recommend item X?" and (2) "How should I pick which item to recommend?". Typically, the first set of questions are answered with supervised learning: we build models to forecast whether someone will click on an ad, or visit a post. The second set of questions are more open-ended. In this talk we will dive into how we can answer "how" questions, starting with heuristics and search. This will lead us to bandits, reinforcement learning, and Horizon: an open-source platform for training and deploying reinforcement learning models at massive scale. At Facebook, we are using Horizon in a variety of control tasks, spanning recommender systems, marketing & promotion distribution, and bandwidth optimization. The talk will cover the key components of Horizon and the lessons we learned along the way that influenced the development of the platform.

Jason Gauci leads the Applied Reinforcement Learning team @ Facebook AI. Jason has 13 years of experience building machine learning systems at Apple, Google Research, and Lockheed Martin Applied Research, and has a PhD in computer science from UCF with a focus on Neuroevolution.

NLP for detecting political narratives: Applying big data insights to small domains (slides)

Marvelous.ai is developing natural language tools to analyze political discourse in news and social media. We are particularly interested in how messaging spreads, both negatively (propaganda) and positively (pro-democracy campaigns). In this talk, I will describe our approach to detecting political narratives focusing on 2018 and 2020 US elections. Narrative detection and political discourse are rich textual domains, but they lack labelled datasets of any appreciable size. Instead, we’ve used insights learned from large datasets and relied on human-in-the-loop active learning techniques, allowing us to quickly create domain-specific models with high precision. I will also speak about our efforts to make NLP tools and datasets accessible to the general, non-technically inclined public.

Olya Gurevich holds a PhD in Linguistics from UC Berkeley. She has been a computational linguist, data scientist, and engineering manager at Powerset, Microsoft Bing, Topsy, and Apple’s Siri. In 2018, she joined forces with Christopher Walker and Danielle Deibler to found Marvelous.ai.

Estimating Personalized Preferences from Consumer Shopping Data

In a series of papers, we develop models for consumer preferences based on their choices from a set of alternatives. Our model combines standard modeling approaches from economics and marketing with recent developments in machine learning, in particular matrix factorization techniques. We evaluate performance of the models for counterfactuals using held-out data where items are out of stock or where prices change. We show that when personalized prices are based on model estimates, firm profits can increase substantially.

Susan Athey is the Economics of Technology Professor at Stanford Graduate School of Business. She previously taught at the Economics departments at MIT, Stanford and Harvard. Her current research focuses on the economics of digitization, marketplace design, and the intersection of econometrics and machine learning. She has worked on several application areas, including timber auctions, internet search, online advertising, the news media, and the application of digital technology to social impact applications. As one of the first “tech economists”, she served as consulting chief economist for Microsoft Corporation for six years, and now serves on the boards of Expedia, Lending Club, Rover, Turo, and Ripple, as well as non-profit Innovations for Poverty Action.

Machine Learning-Powered Search Ranking of Airbnb Experiences

Airbnb Experiences are handcrafted activities designed and led by expert hosts that offer a unique taste of local scene and culture. Since the launch of Airbnb Experiences in November 2016 we managed to bring Experiences to more than 1,000 destinations worldwide, including unique places like Easter Island, Tasmania, and Iceland. As the marketplace grew, Search & Personalization became very important factors for the continued rapid growth and success of the marketplace. In this talk, I will describe the steps we took to develop a Machine Learning powered Search Ranking framework at different growth stages of the marketplace, from small to mid-size and large. Over the course of one and a half years, we ran more than 15 experiments iterating on the algorithm and were able to collectively improve bookings by more than 20%. In addition to driving overall bookings we focused on secondary objectives as well, such as high-quality experiences and new promising experiences. I will present details about the features we built out over time, such as user personalization, real-time model scoring, personalized navigation, ranking dashboards for explaining the predictions and optimization of secondary objectives.

Mihajlo Grbovic is a Principal Machine Learning Scientist at Airbnb. He holds a Ph.D in Machine Learning from Temple University in Philadelphia. He has more than 10 years of technical experience in applied Machine Learning, acting as a Science Lead in a portfolio of projects at Yahoo and now at Airbnb. During his time at Yahoo, he worked on integrating Machine Learning in various Yahoo Products, such as Yahoo Mail, Search, Tumblr & Ads. Some of his biggest accomplishments include building Machine Learning-powered Ad Targeting for Tumblr, being one of the key developers of Email Classification for Yahoo Mail and introducing the next generation of query-ad matching algorithms to Yahoo Search Ads. Dr. Grbovic joined Airbnb in 2016. His work focuses mostly on Search & Recommendation problems for Airbnb Homes, Experiences and Locations. Most recently, he worked on building out a Machine Learning-powered Search for Airbnb Experiences. Dr. Grbovic published more than 50 peer-reviewed publications at top Machine Learning and Web Science Conferences, and co-authored more than 10 patents. He was awarded the Best Paper Award at KDD 2018 Conference. His work was featured in Wall Street Journal, Scientific American, MIT Technology Review, Popular Science and Market Watch.

Reinforcement Learning for Recommender Systems: A Case Study on Youtube

While reinforcement learning (RL) has achieved impressive advances in games and robotics, it has not been widely adopted in recommender systems. Framing recommendation as an RL problem offers new perspectives, but also faces significant challenges in practice. Industrial recommender systems deal with extremely large action spaces – many billions of items to recommend and complex and dynamic user states -- billions of users, who are unique at any point in time. Furthermore, establishing causal connection between past recommendation and long-term user behavior is challenging because of high variance/noise in user feedback. The talk will discuss our work on scaling up policy-based reinforcement learning algorithms, e.g., REINFORCE to a production recommender system at Youtube. We proposed algorithms to address data biases when deriving policy updates from logged implicit feedback. I will also discuss some follow up work on exploration and outstanding research questions in applying RL in recommender systems.

Minmin Chen is a Research Scientist at Google Brain. Her work uniquely focuses on sequence modeling and reinforcement learning in recommender systems to optimize long term user experience. She did her PhD study at Washington University in St. Louis on representation learning and domain adaptation and B.S. at Tsinghua University. She has over 30 publications at top ML and recommender system conferences.

Graph Neural Networks for Recommender Systems (slides)

Deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains a challenge. In this talk we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a data efficient Graph Convolutional Network (GCN) algorithm PinSage, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model. We deploy PinSage at Pinterest and train it on 7.5 billion examples on a graph with 3 billion nodes representing pins and boards, and 18 billion edges. According to offline metrics, user studies and A/B tests, PinSage generates higher-quality recommendations than comparable deep learning and graph-based alternatives. To our knowledge, this is the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.

Jure Leskovec is Associate Professor of Computer Science at Stanford University, Chief Scientist at Pinterest, and investigator at Chan Zuckerberg Biohub. His research focuses on machine learning and data mining large social and information networks, their evolution, and the diffusion of information over them. Computation over massive data is at the heart of his research and has applications in computer science, social sciences, marketing, and healthcare. This research has won several awards including a Lagrange Prize, Microsoft Research Faculty Fellowship, the Alfred P. Sloan Fellowship, and numerous best paper awards. Leskovec received his bachelor's degree in computer science from University of Ljubljana, Slovenia, and his PhD in in machine learning from the Carnegie Mellon University and postdoctoral training at Cornell University.

How is this show similar to that? (slides)

Distance metric learning or similarity learning is an active research topic that is applicable to many contexts. In similarity learning, examples are typically projected into a feature vector space, in which the distance in the projected space preserves the semantic similarity in the input space. In this talk, we will survey recent methods in deep metric learning, and how they relate to Netflix’s recommendation algorithms. We will present case studies of similarity learning and illustrate tentative approaches.

Selen Uguroglu is a Research Scientist at Netflix working on problems related to personalization and recommendations. She completed her PhD in Computer Science from Carnegie Mellon University where she received Presidential Fellowship. Prior to Netflix, she was at Apple Siri where she worked on query and natural language understanding.