Efficient UCB-type learning algorithms for lost-sales inventory system with lead times and censored demand

Abstract: In this paper, we focus on the periodic-review, single-product inventory system with lost-sales and positive lead times, which is a notoriously challenging system with a wide range of real-world applications. We consider the joint learning and optimization problem where the decision-maker does not know the demand distribution a priori and can only use the past sales (censored demand) data to optimize the problem. Different from the existing literature on this problem (Huh et al. (2009), Zhang et al. (2019), Agrawal and Jia (2019)) that requires the convexity property of the underlining system. In this paper, we develop a UCB-typed learning framework that efficiently utilizes the properties of the system. We show that this learning framework not only can be applied to converge to the best base-stock policy, similar to the existing literature, but can also be applied to converge to the best capped base-stock policy, where the convexity property does not hold. We emphasize that the learning algorithms developed in this paper are not Naïve adoptions of the UCB algorithm, as 1) there is an exponentially large number of inventory states in the system, and 2) due to the lead-time, each order has long-term effects on future periods. We show that the regret rate of the learning algorithm is tight, up to a logarithmic term. We also conducted extensive numerical experiments to demonstrate the efficiency of the proposed algorithms. Advisor: Huanan Zhang, Harold and Inge Marcus Department of Industrial and Manufacturing Engineering, Pennsylvania State University, University Park, PA 16802, huz157@psu.edu. Coauthor: Linwei Xin Booth School of Business, University of Chicago, Chicago, IL, Linwei.Xin@chicagobooth.edu