Backdoor / Trojan

A backdoor or Trojan in machine learning refers to a hidden behavior embedded within a model that is activated by specific triggers, leading to targeted mispredictions or undesired outputs. This phenomenon can be mathematically characterized through the manipulation of the loss function during training, where certain inputs are intentionally misclassified. The architecture of the model may include additional layers or neurons that are designed to respond to these triggers, often without affecting the model's performance on benign inputs. The implications of backdoor attacks are significant, as they can compromise the integrity of machine learning systems, particularly in security-sensitive applications. The study of backdoor attacks is closely related to adversarial machine learning, where the focus is on understanding how models can be deceived through carefully crafted inputs. Key algorithms for detecting backdoors include anomaly detection techniques and model interpretability methods that analyze the decision boundaries of the model.