Cross-Silo Federated Learning: An Empirical Study on Learning Gains

Abstract

Federated Learning (FL) is an essential approach for the collaborative training of Machine Learning (ML) models on distributed and sensitive data, being crucial for privacy protection in cross-silo scenarios. However, its adoption is widely limited by the uncertainty surrounding the learning gain, i.e., whether the global model offers better performance and generalization than a model exclusively trained locally. This uncertainty represents an incentive factor often overlooked in literature focused on security and heterogeneity challenges. To improve participant performance and motivation, this work proposes a novel two-stage learning scheme: FedFil (Filtered Federated Learning). FedFil operates through:

1. local data filtering, which employs Anomaly Detection to remove influential datapoints before FL training;
2. post-FL Fine-Tuning, to personalize the global model to the specific characteristics of the client.

This project, ”Cross-Silo Federated Learning: An Empirical Study on Learning Gains”, conducts a thorough empirical investigation to quantify the learning gain achieved with FedFil, identify which clients benefit most from collaboration, and evaluate whether local adaptation improves global accuracy gain, on census data. Our results indicate that FedFil, by filtering influential local datapoints before the learning rounds and personalizing the global model afterwards, achieves an average learning gain of around 8% on local evaluations and 6% on generalization evaluations.

Project Structure

dissertation_project/
│
├── LICENSE
├── README.md (this file)
├── requirements.txt
│
└── federated-learning/
    ├── data/
    │   └── README.md
    ├── example/
    │   ├── tests/
    │   │   ├── filtering_logs/
    │   │   │   └── ...
    │   │   ├── intermediate_results/
    │   │   │   └── ...
    │   │   ├── results/
    │   │   │   ├── adult/
    │   │   │   │   └── ...
    │   │   │   └── sent140/
    │   │   │       └── ...
    │   │   └── test_run_fl.py
    │   ├── config_exp.py
    │   └── model.py
    ├── federated/
    │   ├── client/
    │   │   └── fed_client.py
    │   ├── server/
    │   │   ├── fed_agg.py
    │   │   ├── fed_avg.py
    │   │   ├── fed_differencial_privacy.py
    │   │   └── fed_per.py
    │   ├── feddata/
    │   │   ├── acs_data_states.py
    │   │   ├── adult_fl_loader.py
    │   │   └── sent140_loader_silo_exp.py
    │   ├── config.py
    │   └── fed_learning_exp.py