AI security has been a high-activity domain for over a decade, with a multitude of demonstrated threats to integrity, availability, and confidentiality across the entire AI system lifecycle. Nevertheless, the attack surface remains particularly complex; new vectors are now emerging that exploit vulnerabilities within both the software and hardware implementations of AI models. Among these threats, physical attacks — specifically side-channel analysis and fault injection — are receiving increasing attention. In this presentation, we will primarily focus on attacks targeting model parameters during inference, as well as recent threats emerging during training. Finally, we will address the fundamental challenges of practicality and evaluation, which are critical in the current context of upcoming certifications related to the AI Act.