The docs for 'run_flow_or_task' say:In case a flow is not yet published, it is published after executing the run.

I wonder why this is always done. I can imagine that people just want to run a set of flows on a task without uploading it later. Maybe they want to test very weird configurations that they will never upload runs for, and these will all live on the server as flows without runs.

Would it not make more sense to defer the uploading until a run is also uploaded? Hence, flow uploading would be done in run_publish if the flow is unknown to OpenML.