The output directory of TrackRefiner contains the following files:

• TrackRefiner.{CellProfiler_Output_CSV_File_Name}_{Elongation_Rate_Method}_analysis.csv

This file contains the results of refined bacterial tracking along with additional features calculated for individual bacteria, their life history, or bacterial families.

Example: Trackrefiner.FilterObjects_Average_analysis.csv

To know more about the features reported in this CSV file, refer to Reported Feature Explanations.

• TrackRefiner.{CellProfiler_Output_CSV_File_Name}_{Elongation_Rate_Method}_neighbors.csv

This file provides bacterial neighbor information after noise objects have been removed.

Example: Trackrefiner.FilterObjects_Average_neighbors.csv

• TrackRefiner.{CellProfiler_Output_CSV_File_Name}_{Elongation_Rate_Method}_logs.csv

This file provides information on objects that have been identified as noise and removed by TrackRefiner, bacteria whose links have been modified or removed (flagged as unexpected beginning or unexpected end) by TrackRefiner, and bacteria that lacked links in the CellProfiler output and for which TrackRefiner was unable to create new links.

Example: Trackrefiner.FilterObjects_Average_logs.csv

Explanation of Flags in this File:

NoiseObject:

If True, the object has been detected as noise and removed by TrackRefiner.

UnexpectedEnd_by_CellProfiler:

If True, this indicates that the bacterium did not have a link to the next time step in the CellProfiler output, and TrackRefiner couldn’t create a link for it.

UnexpectedEnd_by_Trackrefiner:

If True, the link of this bacterium to the next time step has been dropped by TrackRefiner, and no new link was created.

UnexpectedBeginning_by_CellProfiler:

If True, this indicates that the bacterium did not have a link to the previous time step in the CellProfiler output, and TrackRefiner couldn’t create a link for it.

UnexpectedBeginning_by_Trackrefiner:

If True, the link of this bacterium to the previous time step has been dropped by TrackRefiner, and no new link was created.

TrackingLinkModified:

If True, the link of this bacterium to the previous time step has been modified, and a new link was created.

Note: Seeing a bacterium flagged as an unexpected beginning or unexpected end does not necessarily indicate an error. In scenarios such as microfluidic experiments or due to segmentation issues, bacteria may reasonably lack links to previous or next-time steps.

• log.txt

This file provides a detailed log of the processing steps during TrackRefiner's execution. Users can find the following information in this file:

1. Process Start and End Times:

When the TrackRefiner process started and completed.

2. User Command:

The exact command used to execute TrackRefiner, including all input parameters and settings.

3. Progress Updates:

The percentage completion of the process, along with timestamps for each stage.

4. Tracking Error Summary:

Counts of various events related to tracking, including: Noise objects identified and removed. Unexpected beginning and unexpected end bacteria generated by TrackRefiner and CellProfiler. Bacteria with modified links.

5. Output Directory:

The location where TrackRefiner output files were saved.

6. Execution Time:

The total duration of the process in the format day:hours:minutes:seconds.

7. Resource Usage Summary:

Maximum and average CPU and memory usage during the execution of TrackRefiner.

Example: log.txt

In the plot folder, the following visualizations have been generated:

• distribution of bacteria lengths across the dataset (Filename: AreaShape_MajorAxisLength.jpg)

• distribution of the average length of bacteria throughout their life histories. (Filename: Average_Length.jpg)

• distribution of the elongation rate of bacteria throughout their life histories. (Filename: Elongation_Rate.jpg)

• distribution of the velocity of bacteria throughout their life histories. (Filename: Velocity.jpg)

• distribution of bacteria length at the first time step of their life histories. (Filename: startVol.jpg)

• distribution of bacteria length at the last time step of their life histories. (Filename: targetVol.jpg)

• distribution of cell cycle duration (life spans) of bacteria. (Filename: LifeHistory.jpg)

The ML folder contains files related to the machine learning models trained by TrackRefiner to predict bacterial behaviors, including division, continuity, and distinguishing between division and continuity links.

TrackRefiner employs three machine learning models: the continuity model, which predicts whether a bacterium will continue its life history; the division model, which predicts whether a bacterium will divide and produce daughter cells; and the division_vs_continuity model, which distinguishes between divided and non-divided bacteria.

Here is a detailed description of the folder contents:

• Feature Weights Logs ({Model_Type}.{Classifier}.feature_weights_training_all_data.log)

These files highlight the importance (weights) of features used during the training of the machine learning models on the complete dataset.

Examples:

continuity.LogisticRegression.feature_weights_training_all_data.log.txt continuity.LogisticRegression.feature_weights_training_all_data.log.txt

division.LogisticRegression.feature_weights_training_all_data.log.txt division.LogisticRegression.feature_weights_training_all_data.log.txt

division_vs_continuity.LogisticRegression.feature_weights_training_all_data.log.txt division_vs_continuity.LogisticRegression.feature_weights_training_all_data.log.txt

• Performance Metrics ({Model_Type}.{Classifier}.performance.csv)

These files summarize the performance of the machine learning models, detailing metrics for different training and evaluation scenarios. For each model type (continuity, division, and division_vs_continuity), two models have been trained: one using the full dataset (Training on all data) and another using 80% of the data (Training on train split). The performance of the latter model has been also evaluated on the remaining 20% of the data (Evaluation on test split). Metrics include:

Total number of samples, number of samples for each class, True negatives (TN), false positives (FP), false negatives (FN), true positives (TP), Accuracy and specificity of the models.

File Examples:

continuity.LogisticRegression.performance.csv continuity.LogisticRegression.performance.csv

division.LogisticRegression.performance.csv division.LogisticRegression.performance.csv

division_vs_continuity.LogisticRegression.performance.csv division_vs_continuity.LogisticRegression.performance.csv

• Probability Distribution of Highly Likely Correct Links ({Model_Type}.{Classifier}_highly_likely_correct_links_distribution.jpg)

These files contain probability distributions of highly likely correct links for different training and evaluation scenarios, including:

Training on all available data.

Training on a train split (80% of the data).

Evaluation on a test split (20% of the data), based on the model trained using the train split.

File Examples: continuity_LogisticRegression_highly_likely_correct_links_distribution.jpg

division_LogisticRegression_highly_likely_correct_links_distribution.jpg

division_vs_continuity_LogisticRegression_highly_likely_continuity_links_distribution.jpg

The pickle_files directory is created by Trackrefiner if you enable the option to save results in .pickle format. This happens when you pass the -k or --save_pickle argument to the trackrefiner-cli or select the Save in .pickle format option in the trackrefiner-gui.

Example: pickle_files.zip

How to read this file:

The following script demonstrates how to read a .pickle file and access various measures and features of bacteria.

import pickle

if __name__ == '__main__':

    pickle_file_path = 'Trackrefiner.step-000070.pickle'

    # Load the data from the specified pickle file
    pickle_data = pickle.load(open(pickle_file_path, 'rb'))

    # Access the dictionary keys in the loaded data.
    dictionary_keys = pickle_data.keys()
    # ['stepNum', 'lineage', 'cellStates']
    print(dictionary_keys)

    # Extract the time step from the dictionary.
    timestep = pickle_data['stepNum']
    print('time step: ' + str(timestep))

    # Extract the lineage information from the dictionary.
    lineage = pickle_data['lineage']
    # Lineage contains a mapping of cell IDs to their parent IDs {id: parent_id}.
    print('lineage: ' + str(lineage))

    cs = pickle_data['cellStates']

    # indices of all bacteria
    print(cs.keys())

    # Retrieve the information of a specific bacterium with index 197.
    bacterium = cs[197]

    # Attributes of the bacterium object:
    # 'id', 'label', 'cellType', 'divideFlag', 'cellAge', 'growthRate', 'LifeHistory',
    # 'startVol', 'targetVol', 'pos', 'time', 'radius', 'length', 'dir', 'ends', 'strainRate', 'strainRate_rolling'

    # Access and store specific attributes of the bacterium.
    bacterium_id = bacterium.id  # ID of the bacterium.
    bacterium_growth_rate = bacterium.growthRate  # Growth rate (Elongation rate) of the bacterium.
    bacterium_life_history = bacterium.LifeHistory  # Life history of the bacterium.
    bacterium_strainRate_rolling = bacterium.strainRate_rolling  # Rolling strain rate of the bacterium.

    print('bacteria id: ' + str(bacterium_id))
    print('Growth rate: ' + str(bacterium_growth_rate))
    print('Life History: ' + str(bacterium_life_history))
    print('strainRate_rolling: ' + str(bacterium_strainRate_rolling))