Interactive Anomaly Detection in Dynamic Communication Networks

Network flows are the basic components of the Internet. Considering the serious consequences of abnormal flows, it is crucial to provide timely anomaly detection in dynamic communication networks. To obtain accurate anomaly detection results in dynamic networks, supervision from experts is highly demanded. However, to obtain high-quality ground truth of abnormal flows, we suffer from two major problems: (1) limited labor resources: Experts with the latest domain knowledge are much fewer than the large number of flows; and (2) dynamic environment: Considering the new abnormal patterns (i.e., new attacks) and continuously changing network structures, it requires timely supervision to adaptively update the parameters. To tackle these problems, we propose HADDN, a novel bandit framework for periodic-updated anomaly detection in dynamic communication networks. We formulate the task as a bandit problem, where by interactions, supervision is offered by human experts to provide the ground truth to a fraction of flows. We construct semi-parametric expected rewards to optimize the estimation of flows' abnormality in limited interactions. Also, we utilize feature-based clusters and structural correlations to make connections between historical flows and new flows to improve both efficiency and accuracy of abnormality estimation. What's more, we provide two implementations for the semi-parametric expected reward of the proposed HADDN with theoretical proof. Experimental evaluations on public datasets demonstrate the substantial improvement of our proposed approaches compared to state-of-art anomaly detection methods.

Files

Metadata

Work Title Interactive Anomaly Detection in Dynamic Communication Networks
Access
Open Access
Creators
  1. Xuying Meng
  2. Yequan Wang
  3. Suhang Wang
  4. Di Yao
  5. Yujun Zhang
Keyword
  1. Anomaly detection
  2. Interactive learning
  3. Dynamic networks
  4. Communication networks
  5. Semi-parametric bandits
License In Copyright (Rights Reserved)
Work Type Article
Publisher
  1. IEEE/ACM Transactions on Networking
Publication Date July 26, 2021
Publisher Identifier (DOI)
  1. https://doi.org/10.1109/TNET.2021.3097137
Deposited July 25, 2022

Versions

Analytics

Collections

This resource is currently not in any collection.

Work History

Version 1
published

  • Created
  • Added Interactive_Anomaly_Detection_in_Dynamic_Communication_Networks.pdf
  • Added Creator Xuying Meng
  • Added Creator Yequan Wang
  • Added Creator Suhang Wang
  • Added Creator Di Yao
  • Added Creator Yujun Zhang
  • Published
  • Updated Keyword, Publication Date Show Changes
    Keyword
    • Anomaly detection, Interactive learning, Dynamic networks, Communication networks, Semi-parametric bandits
    Publication Date
    • 2021-12-01
    • 2021-07-26
  • Updated