There is a strong momentum towards data-driven services at all layers of society and industry. This started from large scale web-based applications such as Web search engines (e.g., Google, Bing), social networks (e.g., Facebook, TikTok, Twitter, Instagram) and recommender systems (e.g., Amazon, Netflix) and is becoming increasingly pervasive thanks to the adoption of handheld devices and the advent of the Internet of Things. Recent initiatives such as Web 3.0 are coming with the promise of decentralising such services for empowering users with the ability to gain back control over their personal data, and prevent a few economic actors from over concentrating decision power. However, decentralising online services calls for decentralising the data and the machine learning algorithms on which they heavily rely. While Federated Learning allows training machine learning models over decentralised data, it still relies on the centralised computation of model aggregations. In this presentation, I will present recent research works targeting the decentralisation of machine learning beyond the well know Federated Learning concept. A particular focus will be given on recent advances and open challenges for enforcing security and privacy in decentralised machine learning.