The Risk of Over-Reliance: Are AI-Driven Safety Systems Too Fragile?