Voting on Features: Federated Feature Selection Without Sharing Data

def feature_election(masks, scores, sample_counts):
    # 实现细节
    pass

import math
import numpy as np
def feature_election(masks, scores, n_samples, freedom_degree):
masks = np.asarray(masks, dtype=bool)
scores = np.abs(np.asarray(scores, dtype=float))
n = np.asarray(n_samples, dtype=float)
intersection = masks.all(axis=0)
union = masks.any(axis=0)
disputed = np.flatnonzero(union & ~intersection)
# min-max normalise each client's scores so different selectors are comparable
lo = scores.min(axis=1, keepdims=True)
span = scores.max(axis=1, keepdims=True) - lo
norm = np.divide(scores - lo, span, out=np.zeros_like(scores), where=span > 0)
weights = n / n.sum()
votes = weights @ norm
k = math.ceil(freedom_degree * len(disputed))
winners = disputed[np.argsort(-votes[disputed], kind="stable")[:k]]
global_mask = intersection.copy()
global_mask[winners] = True
stats = {
"intersection": int(intersection.sum()),
"union": int(union.sum()),
"disputed": len(disputed),
"kept": int(global_mask.sum()),
}
return global_mask, stats

# site_executor.py
import os
import pandas as pd
from sklearn.model_selection import train_test_split
from nvflare.app_opt.feature_election import FeatureElectionExecutor
class SiteExecutor(FeatureElectionExecutor):
"""在站点上运行：加载该站点自身的CSV文件，然后由特征选举完成后续处理。"""
def __init__(self, data_dir, fs_method="lasso", task_name="feature_election"):
super().__init__(fs_method=fs_method, eval_metric="f1", task_name=task_name)
self.data_dir = data_dir
def execute(self, task_name, shareable, fl_ctx, abort_signal):
if self.X_train is None:
site = fl_ctx.get_identity_name()  # "site-1", "site-2", ...
df = pd.read_csv(os.path.join(self.data_dir, f"{site}.csv"))
X, y = df.drop(columns="target"), df["target"]
X_tr, X_val, y_tr, y_val = train_test_split(
X, y, test_size=0.2, stratify=y, random_state=0)
self.set_data(X_tr, y_tr, X_val, y_val, feature_names=list(X.columns))
return super().execute(task_name, shareable, fl_ctx, abort_signal)

# job.py
import argparse
import glob
import json
import os
from site_executor import SiteExecutor
from nvflare.app_opt.feature_election import FeatureElectionController
from nvflare.job_config.api import FedJob
# 每个站点保留其已使用的选择器
SELECTORS = {"site-1": "lasso", "site-2": "random_forest", "site-3": "mutual_info"}
parser = argparse.ArgumentParser()
parser.add_argument("--freedom-degree", type=float, default=0.5)
parser.add_argument("--auto-tune", action="store_true")
parser.add_argument("--workspace", default="/tmp/nvflare/feature_election_demo")
args = parser.parse_args()
job = FedJob(name="feature_election_demo")
job.to_server(FeatureElectionController(
freedom_degree=args.freedom_degree, aggregation_mode="weighted",
min_clients=3, num_rounds=5, auto_tune=args.auto_tune, tuning_rounds=5,
wait_time_after_min_received=0))
for site, method in SELECTORS.items():
job.to(SiteExecutor(data_dir=os.path.abspath("data"), fs_method=method), site)
job.simulator_run(args.workspace)
pattern = os.path.join(args.workspace, "**", "feature_election_results.json")
results = json.load(open(glob.glob(pattern, recursive=True)[0]))
mask = results["global_mask"]
print(f"freedom_degree={results['freedom_degree']:.2f}  kept={sum(mask)}  "
f"signal={sum(mask[:20])}/20  noise={sum(mask[20:])}")

$ python job.py --freedom-degree 0.0
freedom_degree=0.00 kept=15 signal=10/20 noise=5
$ python job.py --freedom-degree 0.25
freedom_degree=0.25 kept=32 signal=18/20 noise=14
$ python job.py --freedom-degree 0.5
freedom_degree=0.50 kept=48 signal=20/20 noise=28
$ python job.py --freedom-degree 0.75
freedom_degree=0.75 kept=65 signal=20/20 noise=45
$ python job.py --freedom-degree 1.0
freedom_degree=1.00 kept=81 signal=20/20 noise=61
$ python job.py --auto-tune
freedom_degree=0.60 kept=55 signal=20/20 noise=35

Both failures from the previous section show up. On their own data the Lasso site selected 35 features with 12 of the 20 signal features among them, the random forest site 50 with all 20, and the mutual information site 50 with 15. At a freedom degree of 0 the strictest selector holds a veto, and the mask keeps only 10 signal features. At 1 every site's noise gets in, and the mask carries 61 noise features. At 0.5 the vote recovers all 20 signal features with 28 noise features, less than half the noise of the union. I repeated the runs at 0, 0.5 and 1 with two other seeds for the table and the pattern held: 8 and 9 signal features at 0, 19 at 0.5, and 61 and 62 noise features at 1.

Auto-tuning has one limitation here. It tried freedom degrees between 0.5 and 0.7, saw validation F1 between 0.776 and 0.781, and settled on 0.6 with 55 features. The hill climb optimizes F1 alone, so when F1 is this flat it has no reason to prefer the smaller mask, and 0.5 gives the same signal with 7 fewer noise features. If model size matters to you, check the neighbouring values yourself.

## What it offers

In the FLASH paper (Christofilogiannis et al., 2025) we evaluated Feature Election with three clients on 50/30/20 stratified splits, trained with FedAvg. We used three models (Gaussian Naive Bayes, a linear SGD classifier and an MLP), four datasets (Mushroom, Adult Income, Heart Disease, Wisconsin Breast Cancer) and three local selectors: Lasso (Tibshirani, 1996); Sequential Attention (Yasuda et al., 2023), which selects features greedily using attention weights inside a network; and PyImpetus, an implementation of PPFS (Hassan et al., 2025), a wrapper that searches for the target's Markov blanket, the smallest feature set that makes the target independent of the rest, and needs no tuning.

The consistent effect was on model size. Parameter count, and with it the traffic sent every round, fell by 49% on average. The largest cut was 88.8% (SGD classifier, Adult Income, Sequential Attention), Gaussian Naive Bayes on Breast Cancer with Lasso shrank by 81.0%, and the smallest cut was 6.7% (SGD, Income, PyImpetus).

The effect on F1, measured without hyperparameter tuning, was large in some cells and absent or negative in others.

Model | Dataset | Selector | F1, no selection | F1, Feature Election
--- | --- | --- | --- | ---
MLP | Breast Cancer | PyImpetus | 0.41 | 0.94
SGD | Adult Income | - | 0.02 | 0.35
Gaussian NB | Mushroom | Lasso | 0.81 | 0.88
- | - | - | - | -
Lasso | - | - | 0.93 | 0.85

In the first two rows the untuned model on all features barely worked and the elected features made it usable, although 0.35 is still weak. In the third, selection added seven points to a model that already worked, and in the last it cost eight. On Heart Disease the SGD and MLP scores did not move (0.88 to 0.89 with or without selection), while Gaussian Naive Bayes gained up to five points (0.83 to 0.88), and the parameter count dropped in every case. Against a plain Flower (Beutel et al., 2020) FedAvg baseline with no selection and no tuning, the best FLASH configuration (which in several cells includes hyperparameter tuning) matched or beat it in 9 of 12 model and dataset pairs and lost slightly in the other 3, for example Gaussian Naive Bayes on Breast Cancer at 0.96 against 0.98. The dependable gain is fewer parameters shipped every round; accuracy was usually preserved and sometimes much improved, depending on dataset, model and selector.

## Tuning the dial automatically

该论文手动设置自由度（freedom degree），并将自动化设置列为未来工作。后续实现增加了服务器端的爬山算法。服务器提出一个掩码（mask），每个站点在其验证数据上对该掩码进行评分并返回一个数值，服务器根据评分调整方向和步长，当评分趋于平稳时停止，并保留其见过的最佳掩码。

在NVIDIA FLARE对一个包含100列、3个站点的合成表进行扫描的实验中，联合选择保留了69列。自动调优选择了fd=0.5，保留了43列（其中6列是噪声），F1值达到0.838。当自由度高于约0.3时，F1曲线在0.004范围内保持平稳，因此调优器在相同F1值下找到了比联合选择少38%列数的掩码，仅用了5轮调优。

*在包含3个站点的100列合成表上，不同自由度下的验证F1（顶部）和保留列数（底部）。自动调优在fd=0.5时停止。图片由作者提供。*

对于跨3个站点的一个包含64个单元隐藏层的MLP模型，该掩码将每轮传输的数据量从153 KB降低到68 KB（减少56%）。包括选举和调优在内的总成本为3.4 KB，在不到一轮训练后即可收回成本，经过20轮后总成本为2.99 MB（对比集中式模型的1.32 MB）。这些数据是根据该运行的参数数量估算得出的，而非在生产网络上的实际测量值，实际网络中的协议开销、压缩和加密会改变这些数值。

## 独立基准测试的发现

到目前为止的所有内容均来自我自己的实验。VTT和芬兰东部大学的Sharma等人（2026）发表在MICCAI 2026会议论文集上的论文提供了外部视角。该论文介绍了他们自己的联邦LASSO方法LAFFS，并将其与现有的联邦特征选择方法Feature Election进行基准测试。他们运行了公开的NVIDIA FLARE示例，使用Lasso作为本地选择器，并通过五轮爬山算法自动调整自由度。其他基准方法包括在NVIDIA FLARE中运行的scikit-learn的L1逻辑回归、每个站点单独训练的模型，以及在合并数据上的集中式模型。

与FLASH论文（Christofilogiannis等，2025）相比，这些基准测试的条件要困难得多。四个基准数据集包含5,966至19,993个特征，仅有72至187个样本，分布在5至18个站点之间。表格显示了他们在外部交叉验证折叠上的平均AUROC值。对于Feature Election，我展示了每个数据集上其三个下游分类器中的最佳表现，而研究论文报告了所有三个分类器的结果。

方法

ALLAML（5个站点）

Prostate_GE（10个站点）

SMK_CAN_187（18个站点）

leukemia（5个站点）

每个站点单独训练

0.747

0.582

0.529

0.663

FLARE中的L1逻辑回归

0.968

0.925

0.669

0.983

Feature Election（Lasso，自动调优）

0.985

0.922

0.710

0.934

LAFFS

0.987

0.947

0.702

0.996

在合并数据上的集中式模型

0.964

0.711

0.974

LAFFS在四个数据集中的三个上取得了最高的联邦得分。Feature Election在SMK_CAN_187数据集（18个站点，接近20,000个特征）上表现最佳，其得分0.710与合并训练的0.711几乎持平。在ALLAML数据集上，Feature Election与LAFFS的差距仅为0.002，在Prostate_GE数据集上与L1基线持平，在leukemia数据集上明显落后（0.934 vs 0.996）。各折叠间的差异较大，标准差在0.01到0.2之间，因此联邦方法之间的大部分差距都在一个标准差范围内。所有联邦方法都远超单独训练的站点。

论文还指出了一项弱点。在包含200个特征中20个有效特征的合成数据表中，LAFFS能够准确筛选出所有有效特征，而基于Lasso的特征选举方法却让部分无效特征通过（其筛选数量少于其他选择器）。这正是作者主张联合优化稀疏模型的论据，而他们在选择纯度方面的实验结果也支持这一观点。

我的观察是，由独立团队运行的通用投票方案，在六倍于我测试站点数量的场景中，其表现与专用方法和联合训练的性能保持接近。不同方法的权衡在于各自假设前提的差异。LAFFS基于LASSO逻辑回归构建，其作者已公开该算法的专利申请。特征选举方法兼容任意本地选择器和模型，并在Apache 2.0协议下开源。

## 立即运行

相同的选举规则有两个框架实现版本。Flower版本是Flower Hub应用，@christofilojohn/feature-election [2]；可通过命令 `flwr new @christofilojohn/feature-election` 创建，并使用 `flwr run .` 在本地模拟运行。NVIDIA FLARE版本是模块 `nvflare.app_opt.feature_election`，从2.8.0版本起NVFlare仓库包含可运行示例 [9]。两者均采用Apache 2.0协议开源，README文件中列出了支持的选择器、配置参数和部署选项。

## 局限性

我发表的评估仅使用了三个客户端且仅采用FedAvg训练。上述独立基准测试扩展到了18个站点，但尚未验证更大规模和强非独立同分布数据的表现。在该基准测试中，联合优化方法选出的噪声特征数量甚至少于选举方法。目前尚无该算法的正式证明，例如无法确定选举出的掩码与联合表选择器结果的接近程度。论文未报告运行结果的方差或收敛速度，因此上表中出现的微小F1差异应视为单一配置下的结果。在隐私保护方面，掩码和评分信息相比原始数据泄露更少，但仍可能暴露各站点数据的部分信息，且与差分隐私不同，这不是正式的隐私保障机制。

## 历史与未来

FLASH论文荣获IEEE FLTA 2025（杜布罗夫尼克）最佳学生论文奖。我在2026年伦敦Flower AI峰会（2026年4月15-16日）以海报形式展示了混合选择器结果，并于2026年9月24日NVIDIA FLARE日（虚拟会议）发表了主题演讲"NVFlare中的特征选举：面向表格数据的隐私保护特征选择"。接下来是2026年10月19-20日波士顿联邦人工智能峰会，我将发表关于特征选举及后续研究方向的简短演讲。如果你需要处理跨站点的表格问题，或有一个可拆分模拟的表格，建议运行特征选举并对比选举掩码与常规选择器在联合表上的选择结果。

## 引用本工作

如果你在研究中使用或基于特征选举方法，请引用FLASH论文（Christofilogiannis等，2025），该论文首次提出了这一算法。

## 参考文献

1. I. Christofilogiannis, G. Valavanis, A. Shevtsov, I. Lamprou, S. Ioannidis, "FLASH: A Framework for Federated Learning with Attribute Selection and Hyperparameter Optimization," 2025 3rd International Conference on Federated Learning Technologies and Applications (FLTA), Dubrovnik, Croatia, 2025, pp. 93–100. https://doi.org/10.1109/FLTA67013.2025.11336571

2. 在 Flower Hub 上的 Feature Election 应用。https://flower.ai/apps/christofilojohn/feature-election

3. H. B. McMahan, E. Moore, D. Ramage, S. Hampson, B. Agüera y Arcas, "从去中心化数据中高效学习深度网络的通信优化方法," AISTATS, PMLR, 2017, 第 1273–1282 页。

4. R. Tibshirani, "通过 Lasso 进行回归收缩与变量选择," 《皇家统计学会杂志：B辑（方法论）》, 第 58 卷, 第 1 期, 第 267–288 页, 1996。

5. T. Yasuda, M. Bateni, L. Chen, M. Fahrbach, G. Fu, V. Mirrokni, "用于特征选择的序列注意力机制," ICLR, 2023. https://arxiv.org/abs/2209.14881

6. A. Hassan, J. H. Paik, S. R. Khare, S. A. Hassan, "基于马尔可夫毯的包装式特征选择方法," 《模式识别》, 第 158 卷, 111069, 2025。

7. D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y. Gao, L. Sani, K. H. Li, T. Parcollet, P. P. B. de Gusmão, N. D. Lane, "Flower: 一个友好的联邦学习研究框架," arXiv:2007.14390, 2020. https://arxiv.org/abs/2007.14390

8. H. R. Roth, Y. Cheng, Y. Wen, I. Yang, Z. Xu, Y.-T. Hsieh 等, "NVIDIA FLARE: 从仿真到现实世界的联邦学习," arXiv:2210.13291, 2022. https://arxiv.org/abs/2210.13291

9. NVIDIA FLARE 特征选举示例（NVFlare 2.8.0 及以上版本）。https://github.com/NVIDIA/NVFlare/tree/main/examples/advanced/feature_election

10. G. Sharma, E. Moradi, J. Pajula, M. Hilvo, J. Tohka, "LAFFS：基于 LASSO 的联邦特征选择," 《医学图像计算与计算机辅助干预——MICCAI 2026 工作坊与挑战》, LNCS 17252, Springer Nature Switzerland, 2026（即将出版，https://papers.miccai.org/miccai-2026-sat/DeCaF_015.html）。

11. S. Banerjee, E. Elmroth, M. Bhuyan, "Fed-FiS：一种新型信息论联邦特征选择方法用于学习稳定性," 神经信息处理国际会议（ICONIP）, 2021。

12. J. Hermo, V. Bolón-Canedo, S. Ladra, "Fed-mRMR：一种无损联邦特征选择方法," 《信息科学》, 第 669 卷, 120609, 2024. https://doi.org/10.1016/j.ins.2024.120609

13. A. Li, H. Peng, L. Zhang, J. Huang, Q. Guo, H. Yu, Y. Liu, "FedSDG-FS：面向垂直联邦学习的高效且安全的特征选择方法," IEEE INFOCOM, 2023。

14. S. Zhang, H. Jin, Z. Ye, J. Yang, J. Zhang, D. Wu, X. Zheng, D. Song, "通过双层混合育种协同粒子群优化算法进行联邦多标签特征选择，结合流形和稀疏性正则化," 《计算机、材料与连续体》, 第 86 卷, 第 1 期, 2026. https://doi.org/10.32604/cmc.2025.068044

披露：FLASH 的研究工作由 Horizon Europe 项目 SYNAPSE（协议编号 101120853）资助。我是 FLASH 的第一作者，也是本文中所述 Flower Hub 应用和 NVIDIA FLARE 贡献的作者。所有图片均由作者创作。表格由 Python 代码生成，部分表格使用 GenAI（Claude）进行了增强处理。