Training, Evaluation, and Model Management

7.1 Task configuration

Each training task should be defined in a configuration file rather than embedded directly in ad-hoc scripts. This supports repeatability and helps reviewers understand what was trained.

Configuration item	Example
Task name	smoke, sepsis, fraud, readmission.
Model type	Logistic regression, neural network, CNN, transformer adapter.
Federation type	Horizontal FL for the main path; vertical FL only where entity alignment is required.
Aggregation strategy	FedAvg for a baseline; FedProx or robust aggregation where heterogeneity or malicious updates are concerns.
Round count and local epochs	Start small for pilots and increase only where justified.
Privacy controls	Secure aggregation enabled or disabled; DP preset and projected privacy budget.
Evaluation metrics	Accuracy, AUC, F1, calibration, fairness or subgroup metrics, per-participant metrics.

Task configuration should also capture the round configuration and aggregation choices needed for repeatability, including round count, local epochs, client selection, participant weighting, dropout handling, and whether heterogeneity-mitigating strategies such as FedProx or robust aggregation are enabled.

7.2 Training-round workflow

1. The coordinator selects eligible participants for the round.

2. The coordinator sends the current model and round configuration to clients.

3. Each client trains locally on its own data.

4. Each client applies configured privacy controls, such as DP-SGD and secure aggregation masking.

5. Clients send model updates to the coordinator.

6. The coordinator aggregates updates and produces a new global model.

7. The model is evaluated using agreed metrics and the run is logged.

8. Training continues until the round budget, convergence criterion, or privacy budget is reached.

7.3 Privacy and leakage testing

Before a model is released or used operationally, the team should test whether the training process or model output introduces unacceptable leakage risk. The specific tests depend on the model, data, and threat model.

Test	Purpose
Membership-inference test	Assesses whether an attacker can infer whether a record was part of training.
Gradient or update leakage test	Assesses whether individual updates could reveal sensitive input information where secure aggregation is not used.
Small-group metric review	Assesses whether reported metrics reveal information about very small groups.
Per-client performance review	Checks whether the global model harms or under-serves particular participants.
Robustness or poisoning test	Checks whether abnormal client updates can distort model behaviour.

7.4 Fairness and per-participant evaluation

Aggregate global performance can hide poor performance for specific participants or groups. Pilot and production reports should include per-participant metrics and subgroup metrics where permitted and appropriate.

Metric view	Purpose
Global metrics	Shows overall performance of the federated model.
Per-participant metrics	Shows whether performance is acceptable at each site.
Subgroup metrics, where permitted	Shows whether performance is uneven across relevant groups.
Calibration and threshold analysis	Shows whether model scores can be interpreted consistently across participants.
Drift metrics	Shows whether performance changes over time or after retraining.

7.5 Model registry and release

A trained global model should not be treated as a loose file. It should be registered with enough metadata to support review, audit, rollback, and deletion where required.

• Record the model version, code version, training configuration, aggregation strategy, privacy settings, and participating clients.

• Record evaluation results globally and per participant.

• Record privacy-test results and residual-risk decisions.

• Define who can access, export, deploy, or redistribute the model.

• Maintain rollback capability for corrupted, degraded, or unapproved models.

The model registry should also record release gates, export controls, deletion rules, model-card and dataset-summary references, and rollback targets. These artefacts help reviewers understand whether the trained model is approved for the intended use and whether it can be withdrawn if needed.