Prediction in the Presence of Response-Dependent Missing Labels

In various settings, limitations of sensing technologies or other sampling mechanisms result in missing labels, where the likelihood of a missing label is an unknown function of the data. For example, satellites used to detect forest fires cannot sense fires below a certain size threshold. In such cases, training datasets consist of positive and pseudo-negative observations (true negatives or undetected positives with small magnitudes). We develop a new methodology and non-convex algorithm which jointly estimates the magnitude and occurrence of events, utilizing prior knowledge of the detection mechanism. We provide conditions under which our model is identifiable. We prove that even though our approach leads to a non-convex objective, any local minimizer has an optimal statistical error (up to a log term) and the projected gradient descent algorithm has geometric convergence rates. We demonstrate on both synthetic data and a California wildfire dataset that our method outperforms existing state-of-the-art approaches.

© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.”

Files

Metadata

Work Title Prediction in the Presence of Response-Dependent Missing Labels
Access
Open Access
Creators
  1. Hyebin Song
  2. Garvesh Raskutti
  3. Rebecca Willett
Keyword
  1. Training
  2. Satellites
  3. Conferences
  4. Signal processing algorithms
  5. Fires
  6. Forestry
  7. Signal processing
License In Copyright (Rights Reserved)
Work Type Article
Publisher
  1. IEEE
Publication Date July 11, 2021
Publisher Identifier (DOI)
  1. 10.1109/ssp49050.2021.9513750
Source
  1. 2021 IEEE Statistical Signal Processing Workshop (SSP)
Deposited July 19, 2022

Versions

Analytics

Collections

This resource is currently not in any collection.

Work History

Version 1
published

  • Created
  • Added Prediction_in_the_presence_of_response_dependent_missing_labels-1.pdf
  • Added Creator Hyebin Song
  • Added Creator Garvesh Raskutti
  • Added Creator Rebecca Willett
  • Published
  • Updated Work Title, Keyword Show Changes
    Work Title
    • Prediction in the Presence of Response-Dependent Missing Labels
    • ! Prediction in the Presence of Response-Dependent Missing Labels
    Keyword
    • Training, Satellites, Conferences , Signal processing algorithms, Fires, Forestry, Signal processing
  • Updated Work Title Show Changes
    Work Title
    • ! Prediction in the Presence of Response-Dependent Missing Labels
    • Prediction in the Presence of Response-Dependent Missing Labels
  • Updated