Skip to content

QZGao/spectral_fit

Repository files navigation

Bayesian Fitting of the Pulsar Spectra

Q. Gao, et al. (2026). Bayesian statistical analysis of 897 pulsar flux density spectra. Physical Review D 113, 063007. arXiv:2511.18348. doi.org/10.1103/1y28-hzk7.

This repository contains code to fit pulsar radio spectra using Bayesian methods.

The dataset is based on a compilation of literature data, including the latest pulsar-spectra catalogue (version 2.1.0) and a set of additional recent measurements.

The fitting methods include various spectral models and options for handling outliers and systematic uncertainties. Although most of the options are not used in the paper, they present our attempts to improve the fitting process and account for data issues.

Installation

  1. Prepare a Python environment between versions 3.9 and 3.10. A virtual environment is recommended.

  2. Install the required packages (pandas has to be downgraded to version < 2.0 due to compatibility issues with pulsar-spectra):

pip install -r requirements.txt
  1. Then, upgrade pandas back to the latest version:
pip install --upgrade pandas

Usage

Data compilation and fitting

Default Bayesian fitting with Cauchy likelihood (method used in the thesis):

python fit.py

Bayesian fitting with Gaussian likelihood, and a dynamic $e_{\text{fac}}$ that is calculated based on a 50% $e_{\text{quad}}$ (method used in the paper):

python fit.py --gaussian --efac_qbound 0.5

Catalogue options:

  • --jname <name>: Specific pulsar name (string). Can be a single name or multiple names separated by ;.
  • --lit_set <file>: Customize literature list (string, default: None).
  • --atnf: Include The ATNF Pulsar Catalogue (flag, default: False).
  • --atnf_ver <version>: The ATNF Pulsar Catalogue version (string, default: 1.54).
  • --jan_set: Use Jankowski et al. (2018)'s (reproduced) dataset (flag, default: False). This is an incomplete dataset reproduced from their literature list, to our best effort.
  • --refresh: Refresh the catalogue (flag, default: False).

Fitting behaviour and priors:

  • --model <models>: Specific model name(s) (string). Default: simple_power_law;broken_power_law;log_parabolic_spectrum;high_frequency_cut_off_power_law;low_frequency_turn_over_power_law;double_turn_over_spectrum (multiple models can be given separated by ;).
  • --no_requirements: Do not require the dataset to have at least 4 points and a frequency range of at least 2 (flag, default: False).
  • --fixed_freq_prior: Use fixed frequency prior (flag, default: False).

Outlier handling / likelihood options:

  • --gaussian: Use Gaussian likelihood (default is Cauchy likelihood when omitted) (flag, default: False).
  • --gaussian_patch: Use simplified Gaussian distribution that works better for a small dataset (flag, default: False).
  • --outliers_rm: Remove outliers from the dataset (flag, default: False).
  • --outliers_min <value>: Set minimum YERR / Y (when --outliers_rm is not set) (float).
  • --outliers_min_plus <value>: Add a systematic error to the uncertainty instead of replacing it (float).
  • --outliers_all <value>: Ignore reported YERRs and set all YERR/Y to this value (float).

Systematic error / extra uncertainty parameters:

  • --equad <value>: Add an additional systematic error (float).
  • --efac <value>: Multiply reported uncertainties by this systematic error factor (float).
  • --efac_qbound <value>: Determine $e_{\text{fac}}$ dynamically based on an $e_{\text{quad}}$ bound (float).

AIC / Jankowski et al. (2018) method:

  • --aic: Use Jankowski et al. (2018)'s AIC-based method instead of the Bayesian fit (flag, default: False). Code is adapted from pulsar-spectra.
  • --aic_no_corr: Do not apply the small-sample correction term in the AIC calculation (flag, default: False).
  • Note: none of the following CLI flags will work if the AIC method is used: --fixed_freq_prior, --gaussian, --gaussian_patch, --outliers_rm, --outliers_min, --outliers_min_plus, --outliers_all, --equad, --efac, --efac_qbound, --no_checkpoint, --corner.

Output, multiprocessing and plotting:

  • --label <label>: Output directory label (used when --outdir is not set).
  • --outdir <dir>: Output directory (if not set, a timestamped output/outdir_YYYY-MM-DD_HH-MM-SS is created).
  • --override: Override finished jobs (flag, default: False).
  • --nproc <int>: Number of parallel processes to use (int). Default: cpu_count() - 1 or 1.
  • --no_checkpoint: Do not save intermediate pickle dump files (flag, default: False).
  • --no_plot: Do not generate plots (flag, default: False).
  • --corner: Generate a corner plot of the posterior distribution of parameters (flag, default: False).
  • --plot_hide_model_name: Hide model name on the plot (flag, default: False).
  • --plot_hide_legend: Hide the legend on the plot (flag, default: False).
  • --plot_legend_top: Place the legend at the top of the plot instead of the default position (flag, default: False).
  • --plot_bigger_font: Use a bigger font size on the plots (flag, default: False).
  • --pdf: Save plots as PDF instead of PNG (flag, default: False).
  • --print_lit: Print literature list and save it to the output directory (flag, default: False). Adding this CLI flag will exit the program after printing without performing any fitting.

Processing fitted results

Extract variables from the output files:

python processing.py <output_dir>              # default variable is 'log_evidence'
python processing.py <output_dir> --var "aic"  # extract AIC values (if AIC method was used)

Extract estimated parameter values (including errors) from the output files:

python processing.py <output_dir> --var "param_estimates"

Extract $\chi^2$-like good-fit statistics from the output files:

python processing.py <output_dir> --var "good_fit"

Extract frequency-flux plots from the output files (<filter> can be a list of pulsar names separated by ;, or an expression regarding the number of measurements in a pulsar e.g. >=15):

python processing.py <output_dir> --plot <filter>           # default format to be extracted is "png"
python processing.py <output_dir> --plot <filter> --plot_format "pdf" # specify format

Dataset

The literature dataset used in this work is compiled from various sources. The main component is the pulsar-spectra catalogue version 2.1.0. In addition, we have added several recent publications and some older literature that were not included in the catalogue. We also removed some literature that did not include calibration in their measurements.

Cite key Citation Pulsar count Frequency range (MHz) Note
McLean_1973 McLean (1973) 18 408-408 1
Sieber_1973 Sieber (1973) 27 38-10690 23
Bartel_1978 Bartel et al. (1978) 18 14800-22700 13
Manchester_1978a Manchester et al. (1978) 224 40-408 1
Izvekova_1981 Izvekova et al. (1981) 73 39-102 13
Dewey_1985 Dewey et al. (1985) 34 390-390 1
Stokes_1985 Stokes et al. (1985) 20 390-390 1
Slee_1986 Slee et al. (1986) 44 80-160 4
Stokes_1986 Stokes et al. (1986) 5 430-430 1
Fruchter_1988 Fruchter et al. (1988) 1 430-430 4
Fruchter_1990 Fruchter et al. (1990) 1 1490-1490 4
Wolszczan_1992 Wolszczan & Frail (1992) 1 430-1400 1
Wielebinski_1993 Wielebinski et al. (1993) 4 33900-34800 4
Malofeev_1993 Malofeev (1993) 33 61-102 1
Bailes_1994 Bailes et al. (1994) 3 436-436 4
Navarro_1995 Navarro et al. (1995) 1 411-1404 4
Seiradakis_1995 Seiradakis et al. (1995) 188 1315-10550 1
Lorimer_1995b Lorimer et al. (1995) 278 408-1606 13
Manchester_1995 Manchester & Johnston (1995) 2 1400-8300 1
Camilo_1995 Camilo & Nice (1995) 29 430-430 1
Qiao_1995 Qiao et al. (1995) 61 660-1440 1
Manchester_1996 Manchester et al. (1996) 55 436-436 1
Biggs_1996 Biggs & Lyne (1996) 4 408-408 1
Camilo_1996 Camilo et al. (1996) 19 430-800 4
Hoensbroech_1997 von Hoensbroech & Xilouris (1997) 27 1410-10450 1
Kramer_1997 Kramer et al. (1997) 4 14600-43000 1
van_Ommen_1997 van Ommen et al. (1997) 82 800-960 13
Kijak_1997 Kijak et al. (1997) 4 4850-4850 4
Kijak_1998 Kijak et al. (1998) 83 4850-4850 1
Shrauner_1998 Shrauner et al. (1998) 20 82-82 1
Kramer_1998 Kramer et al. (1998) 23 1410-1579 1
Toscano_1998 Toscano et al. (1998) 19 436-1660 1
Stairs_1999 Stairs et al. (1999) 19 410-1414 1
Weisberg_1999 Weisberg et al. (1999) 98 1418-1418 1
Kramer_1999 Kramer et al. (1999) 15 2695-4850 1
Han_1999 Han & Tian (1999) 106 1435-1435 1
Maron_2000 Maron et al. (2000) 281 40-87000 23
Malofeev_2000 Malofeev et al. (2000) 211 102-102 13
Kouwenhoven_2000 Kouwenhoven (2000) 68 325-325 15
Lommen_2000 Lommen et al. (2000) 3 430-1400 1
McGary_2001 McGary et al. (2001) 3 1452-1452 15
Giacani_2001 Giacani et al. (2001) 2 1420-8460 1
Kuzmin_2001 Kuzmin & Losovsky (2001) 30 102-111 1
Manchester_2001 Manchester et al. (2001) 100 1374-1374 1
Morris_2002 Morris et al. (2002) 120 1374-1374 1
Maron_2004 Maron et al. (2004) 3 8350-8350 4
Esamdin_2004 Esamdin et al. (2004) 2 327-327 1
Hobbs_2004a Hobbs et al. (2004) 453 1400-1400 1
Karastergiou_2005 Karastergiou et al. (2005) 48 3100-3100 13
Champion_2005b Champion (2005) 1 327-430 1
Champion_2005a Champion et al. (2005) 17 430-430 1
Lorimer_2005 Lorimer et al. (2005) 38 400-430 1
Johnston_2006 Johnston et al. (2006) 31 8356-8356 13
Crawford_2007 Crawford & Tiffany (2007) 2 1384-3100 1
Kijak_2007 Kijak et al. (2007) 11 325-1060 13
Champion_2008 Champion et al. (2008) 1 1400-5000 4
Deller_2009 Deller et al. (2009) 9 1650-1650 1
Levin_2010 Levin et al. (2010) 1 1400-9000 2
Joshi_2009 Joshi et al. (2009) 3 626-1400 1
Bates_2011 Bates et al. (2011) 18 6591-6591 13
Kijak_2011 Kijak et al. (2011) 15 610-4850 13
Keith_2011 Keith et al. (2011) 9 17000-24000 13
Hessels_2011 Hessels et al. (2011) 12 350-350 4
Anderson_2012 Anderson et al. (2012) 1 4800-8767 2
Lynch_2012 Lynch et al. (2012) 12 2000-2000 1
Kowalinska_2012 Kowalińska et al. (2012) 5 8350-8350 4
Demorest_2013 Demorest et al. (2013) 17 327-2300 1
Dowell_2013 Dowell et al. (2013) 1 41-81 1
Zakharenko_2013 Zakharenko et al. (2013) 40 20-25 13
Manchester_2013 Manchester et al. (2013) 20 700-3100 1
Boyles_2013 Boyles et al. (2013) 13 820-820 1
Stovall_2014 Stovall et al. (2014) 67 350-820 1
Dembska_2014 Dembska et al. (2014) 19 610-8350 1
Ng_2015 Ng et al. (2015) 57 325-1352 1
Dembska_2015 Dembska et al. (2015) 6 610-610 1
Dai_2015 Dai et al. (2015) 24 730-3100 13
Lazarus_2015 Lazarus et al. (2015) 127 1375-1375 1
Han_2016 Han et al. (2016) 228 1274-1523 13
Bhattacharyya_2016 Bhattacharyya et al. (2016) 12 322-322 1
Basu_2016 Basu et al. (2016) 1 325-1280 13
Bilous_2016 Bilous et al. (2016) 158 149-149 13
Bell_2016 Bell et al. (2016) 17 154-154 13
Kondratiev_2016 Kondratiev et al. (2016) 48 149-149 15
Frail_2016 Frail et al. (2016) 200 148-148 15
Levin_2016 Levin et al. (2016) 37 1500-1500 4
Kijak_2017 Kijak et al. (2017) 12 325-610 13
Xue_2017 Xue et al. (2017) 48 185-185 1
Mignani_2017 Mignani et al. (2017) 1 97500-343500 1
Zhao_2017 Zhao et al. (2017) 26 8600-8600 1
Murphy_2017 Murphy et al. (2017) 60 76-227 153
Basu_2018 Basu et al. (2018) 6 325-1280 1
Jankowski_2018 Jankowski et al. (2018) 418 728-3100 13
Brinkman_2018 Brinkman et al. (2018) 12 327-1400 1
Johnston_2018 Johnston & Kerr (2018) 585 1360-1360 1
RoZko_2018 Rożko et al. (2018) 2 325-5900 1
Gentile_2018 Brook et al. (2018) 28 430-2100 1
Surnis_2019 Surnis et al. (2019) 3 325-1170 1
Sanidas_2019 Sanidas et al. (2019) 288 135-135 1
Zhang_2019 Zhang et al. (2019) 3 768-3968 1
Jankowski_2019 Jankowski et al. (2019) 205 843-843 1
Kaur_2019 Kaur et al. (2019) 1 81-220 1
Xie_2019 Xie et al. (2019) 32 1369-1369 1
Zhao_2019 Zhao et al. (2019) 71 4820-5124 1
Bilous_2020 Bilous et al. (2020) 43 54-64 1
Crowter_2020 Crowter et al. (2020) 1 350-1500 1
Michilli_2020 Michilli et al. (2020) 19 129-1532 1
Curylo_2020 Curyło et al. (2020) 1 150-150 1
Tan_2020 Tan et al. (2020) 20 119-1532 1
Bondonneau_2020 Bondonneau et al. (2020) 64 53-65 1
Alam_2021 Alam et al. (2021) 47 430-2100 1
Gordon_2021 Gordon et al. (2021) 44 3000-3000 25
Han_2021 Han et al. (2021) 201 1250-1250 1
Bondonneau_2021 Bondonneau et al. (2021) 12 50-50 1
Johnston_2021 Johnston et al. (2021) 44 1369-1369 1
Shapiro_Albert_2021 Shapiro-Albert et al. (2021) 3 430-1500 1
Spiewak_2022 Spiewak et al. (2022) 189 945-1623 1
Lee_2022 Lee et al. (2022) 22 70-352 1
Bhat_2023 Bhat et al. (2023) 120 154-154 1
Anumarlapudi_2023 Anumarlapudi et al. (2023) 150 888-888 25
Posselt_2023 Posselt et al. (2023) 1237 941-1640 2
Gitika_2023 Gitika et al. (2023) 89 944-1625 1
Keith_2024 Keith et al. (2024) 597 1284-1284 4
Wang_2024 Wang et al. (2024) 10 2250-8600 4
Kumar_2025 Kumar et al. (2025) 96 35-79 4

Footnotes

  1. Literature present in the pulsar-spectra catalogue version 2.0.4. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101

  2. Supplementary literature added to the catalogue as part of the effort of the work. 2 3 4 5 6 7

  3. Literature present in Jankowski et al. (2018)'s dataset. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

  4. New literature added in the pulsar-spectra catalogue version 2.1.0. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

  5. Literature with measurements obtained through imaging surveys. 2 3 4 5 6 7

About

Fit pulsar radio spectra using Bayesian methods.

Resources

License

Stars

Watchers

Forks

Contributors

Languages