Hopefully this is could be a temporary change and Go can find an upstream solution. The performance improvement is significant enough to pull in though.

Hopefully this is could be a temporary change and Go can find an upstream solution. The performance improvement is significant enough to pull in though.

Looks like it was merged in Go master; golang/go#53084 - see golang/go@f1b1557 (not sure if more is needed after that>), which would mean this will be in go1.20?

If so, should we add a TODO comment to remove it after we switched to go1.20?

Looks like it was merged in Go master; golang/go#53084 - see golang/go@f1b1557 (not sure if more is needed after that>), which would mean this will be in go1.20?

crypto/sha256: add sha-ni implementation ( https://go-review.googlesource.com/c/go/+/408795/1 ) doesn't seem merged.
Not sure it will be merged before Go 1.20.

Added a code comment

func NewSHA256() hash.Hash {
        // An equivalent of sha256-simd is expected to be merged in Go 1.20 (1.21?): https://go-review.googlesource.com/c/go/+/408795/1
        return sha256simd.New()
}

Strange that github claims there are conflicts but doesn't list any file conflicts?

@AkihiroSuda Would it be relevant to have this new computation in BuildKit too?

@AkihiroSuda Would it be relevant to have this new computation in BuildKit too?

Yes, I guess we can safely have this in BuildKit (and Moby, etc.)

This is pretty cool, but I wonder if this should be opt-in given that the performance advantage is fairly negligible unless you are doing a lot digesting vs the breadth of code that relies on sha256.
This could even be a plugin that is disabled by default where the plugin init would register the hasher.

opt-in

Any drawback to enable SIMD by default?

This could even be a plugin that is disabled by default where the plugin init would register the hasher.

I'm not sure. The plugin will be NOP when https://go-review.googlesource.com/c/go/+/408795/1 is merged.

This is pretty cool, but I wonder if this should be opt-in given that the performance advantage is fairly negligible unless you are doing a lot digesting vs the breadth of code that relies on sha256.

Are you concerned mostly with bugs in this implementation or reliability of the CPU instructions or something else?

This could even be a plugin that is disabled by default where the plugin init would register the hasher.

Another alternative would be to make this a build tag. Then there wouldn't be an extra plugin hanging around if this gets folded into Go.

Does the Go upstream have aarch64 support already? If not, sha256-simd would cover more architectures than the upstream change.

Does the Go upstream have aarch64 support already?

Yes https://github.com/golang/go/blob/master/src/crypto/sha256/sha256block_arm64.s