The SpectrumFM framework comprises three key stages. First, in the data collection and processing stage, diverse spectrum data from multiple sources are gathered and preprocessed to ensure consistency and compatibility across datasets. Second, during the pre-training stage, the model learns fundamental spectrum representations through self-supervised learning tasks, namely, masked reconstruction and next-slot signal prediction. Finally, in the fine-tuning stage, the pre-trained model is adapted to specific downstream tasks, including AMC, WTC, SS, and AD.

The hyperparameters for SpectrumFM are as follows. The mask ratio r is set to 15%, the number of signal symbols is set to 128, the number of attention heads H is set to 4, the latent dimension d is set to 256, the feedforward dimension dfeed is set to 512, and the number of SpectrumFM encoder layers L is set to 16. The pre-training phase consists of 10 epochs with a batch size of 256 and a learning rate of 0.001. The AdamW optimizer is employed for optimization, and early stopping is utilized to prevent overfitting. During the fine-tuning stage, the same learning rate of 0.001 and the AdamW optimizer are used to further adapt the model to specific downstream tasks.

A checkpoint pretrained on RML2018 is available at https://pan.nuaa.edu.cn/share/e2b1cf13f330187efbc9a918e6 The pretraining code is located in the pretrain.py file, while the fine-tuning code can be found in the amc.py file.

@ARTICLE{11301740,
  author={Zhou, Fuhui and Liu, Chunyu and Zhang, Hao and Wu, Wei and Wu, Qihui and Quek, Tony Q. S. and Chae, Chan-Byoung},
  journal={IEEE Journal on Selected Areas in Communications}, 
  title={SpectrumFM: A Foundation Model for Intelligent Spectrum Management}, 
  year={2025},
  volume={},
  number={},
  pages={1-1},
  keywords={Feature extraction;Radio spectrum management;Accuracy;Wireless communication;Foundation models;Signal to noise ratio;Wireless sensor networks;Robustness;Communication system security;Training;Spectrum foundation model;automatic modulation classification;wireless technology classification;spectrum sensing;anomaly detection},
  doi={10.1109/JSAC.2025.3644783}}