Conversation
build test
name
fixed path
workflow dispatch
test
push
other way of writing env
version parser
kek
20 concurrent jobs
test_paralel
set versions as argument
environment
join array
is env
py builder
java and wrapper validator
exit code
max parallel set to 20
show version list
publisher
workflow refactor
publish dir rename
fail-fast option
fail-fast
generate all the necessary publish files on successful build action
oopsi chdir
github token
write permissions
workflow token
token test
write action permission
temp clear
workflow token
publish test
template fix
workflow refactor
cleanup
matrix based build workflow
string versions
checkout v3
ubuntu runner
make gradle script executable from workflow
stop publish generator and add bash script
publish.py
Outdated
| stderr=subprocess.STDOUT, | ||
| shell=True, | ||
| env=env, | ||
| cwd=os.path.join(self._PUBLISH_TEMP_DIR, self._version) |
There was a problem hiding this comment.
What is the thought behind changing the working directory? I don't think it causes any issues, since Gradle will make sure any file access in the build script is relative to the project directory, but then this change has no effect at all?
There was a problem hiding this comment.
gradle doesn't allow doing multiple builds at the same time since it only uses one dir, so if we try to multithread it, it will break because "the same resource is being accessed from multiple processes", so i decided to just move the files gradle needs to survive to another directory named like the version and do the build there
It doesn't affect at all because there is only one process as you can see here https://github.com/OrnitheMC/feather-mappings/pull/134/files#diff-3252d091f3490b6e1103b1e47ed6e4d909cb8d2bb279bc54730383e0860ea566R184 but to keep the mutithread support for the future just in case we want to use it
this doesnt change anything, its more relevant for build than to publish, i can remove it from publish.py iyw, i just copy pasted :P
There was a problem hiding this comment.
gradle doesn't allow doing multiple builds at the same time since it only uses one, so if we try to multithread it, it will break because "the same resource is being accessed from multiple processes", so i decided to just move the files gradle needs to survive to another directory named like the version and do the build there
ahhh I see, so it's needed to make the build_all work.
this doesnt change anything, its more relevant for build than to publish, i can remove it from publish.py iyw
at that point might as well just remove the publish script and publish through the feather script tbh
There was a problem hiding this comment.
is not exactly the same, i'd need to change the feather.py behaviour for the publish option to add the retry logic
or we don't want that anymore and just use the job action retry manually?
There was a problem hiding this comment.
the manual retry is probably good enough tbh
run feather script, remove publish.py
|
done |
This refactor has 3 parts
.github/workflows/build.yml
.github/workflows/build_all.yml
This has to be run before the "Publish" workflow
steps
.github/workflows/scripts/matrix_gen.py, which takes all available unique versions frommappings/, split them into 20 different arrays and save the output..github/workflows/scripts/workflow_gen.py, this will split all versions into n publish workflows, every workflow will have up to 230 versions each in its matrix, and it will then commit and push those generated workflows.Requirements
For this build_all workflow to work you need to make 2 changes:
feather-mappings > settings > Actions > General > Workflow permissions > Read and write permissionsprofile > settings > Developer settings > Personal access tokens > Tokens (classic) > Generate new token (classic)This is so the build_all workflow can commit and push new github actions to the workflows directory
.github/workflows/build_all.yml
Once the build_all workflow has been run, there should be some new publish_{n}.yml that will trigger when the
.github/workflows/build_all.ymlcompletesOn Publish run, the generated ones will run as well running a 230 job max matrix each
The publish.py script has also been rewritten with the retry logic if fails