Merged
Conversation
Contributor
Author
|
Some things that still need fixing before this can be merged:
|
caaavik-msft
commented
Aug 4, 2023
cincuranet
reviewed
Aug 9, 2023
LoopedBard3
reviewed
Aug 9, 2023
LoopedBard3
reviewed
Aug 9, 2023
LoopedBard3
reviewed
Aug 9, 2023
DrewScoggins
previously approved these changes
Aug 10, 2023
Member
There was a problem hiding this comment.
LGTM. Obviously there is a lot here, and I would be lying if I said that I now fully understand all of the intricacies. Overall, the change looks good though and I am confident that this shouldn't cause any breakage in our existing data pipelines.
We should make sure that we are being vigilant as we roll this out to ensure that data is ending up in the correct place, both for existing runs and new ones that get queued through this process.
LoopedBard3
reviewed
Aug 10, 2023
cincuranet
approved these changes
Aug 11, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains initial work around making it so that devs inside Microsoft can use crank to schedule performance runs on Helix machines locally, rather than having to do them through CI pipelines only.
Currently, a lot of the logic around running performance tests are stored in a mix of Azure Pipelines yaml files, shell scripts, powershell scripts, python scripts, MSBuild project files across both the dotnet/runtime and dotnet/performance repositories. This PR attempts to move as much of that logic as possible into python scripts so that they can be run outside the context of a CI pipeline, and eventually we can update our CI pipelines to run these python scripts instead.
I also took this opportunity to fix up our python scripts to have proper type hints set, as well as configuring some settings so that we get good autocompletion and linting inside VSCode.
I have set up a crank agent that runs inside corpnet, which those inside Microsoft on a corpnet-connected machine will be able to use. For Microsoft employees working remotely, they can either use a DevBox (which has access to corpnet), or they can talk to a member of the .NET Performance Team to get an Azure Relay connection string which will allow communication outside corpnet.
An example of using crank to schedule a performance run on a remote machine is:
This will run all the Linq microbenchmarks on a win-x64 helix machine against a local pre-built corerun. The results will be returned back and displayed in the terminal, it may also be uploaded to the Azure Data Explorer to view there.