Overview
DeTox-Fed is a federated graph-learning framework designed for detecting toxic conversations within decentralized social networks (DSNs), specifically targeting platforms within the Fediverse such as Pleroma, Mastodon, and Lemygrad. This framework addresses content moderation challenges inherent to DSNs, where independent instances manage their own data, adhere to diverse moderation policies, and often possess only partial views of ongoing conversations.
Research Context
The proliferation of decentralized social networks has introduced complexities in content moderation. A primary challenge stems from the architecture of these networks: individual instances operate independently, maintaining separate data stores. Consequently, these instances often implement distinct moderation policies and typically have access to incomplete snapshots of broader conversations. The design of DeTox-Fed aims to mitigate these challenges by enabling collaborative toxicity detection without mandating the sharing of raw conversational data or moderation labels between instances.
Approach
DeTox-Fed operates on a federated learning paradigm. The methodology involves several steps:
- Local Graph Construction: Each independent instance within the DSN constructs a local conversation graph. In these graphs, individual nodes represent complete conversation trees. Edges between nodes are established based on shared user participation across different conversations.
- Federated Graph Neural Network (GNN) Training: A Graph Neural Network (GNN) is employed within a federated learning setup. This architecture allows multiple instances to collaboratively train a toxicity classifier. Crucially, this training occurs while maintaining the locality of data, meaning raw conversation data remains on its original instance and is not directly shared with other instances or a central server.
- Multi-Modal Signal Integration: Unlike moderation methods that rely solely on textual analysis, DeTox-Fed integrates multiple types of signals to enhance detection accuracy. These signals include the structural characteristics of conversations, patterns of user interactions, conversation-level statistics, and aggregate sentiment indicators.
Findings
The DeTox-Fed framework underwent evaluation using a large dataset sourced from Pleroma conversations. The evaluation demonstrated the following operational characteristics and effectiveness:
- The framework achieved stable detection of toxic conversations.
- This stability was observed even under conditions of limited availability of local moderation labels.
- Performance remained consistent despite partial participation from client instances within the federated learning network.
- The system exhibited stable detection across varying moderation thresholds.
Why This Matters
The results indicate that a federated graph-based approach to moderation offers a promising direction for developing semi-automated moderation tools in decentralized social networks. This methodology addresses key privacy and data sovereignty concerns by allowing instances to contribute to a shared moderation model without compromising the confidentiality of their raw data holdings.