Skip to content

setupEnv: Run some processes concurrently#2346

Merged
mgsloan merged 2 commits intocommercialhaskell:masterfrom
sjakobi:async-setupenv
Aug 6, 2016
Merged

setupEnv: Run some processes concurrently#2346
mgsloan merged 2 commits intocommercialhaskell:masterfrom
sjakobi:async-setupenv

Conversation

@sjakobi
Copy link
Member

@sjakobi sjakobi commented Jul 6, 2016

This should make most commands slightly faster.
On my machine, stack path takes ~20% less time than before, a no-op
stack build in a single-package project ~15%.

@sjakobi
Copy link
Member Author

sjakobi commented Jul 6, 2016

So, what I actually don't know, is whether this safe, especially querying ghc-pkg asynchronously.

Can someone with more experience with asynchronous programming take a look?

BTW here's some debug output from a stack version on top of my better-process-logging branch for #2327:

~ $ stack-bpl-async -v path --dist-dir --global-pkg-db
Version 1.1.3, Git revision 7934d6ce5eb9b841a6f248b427e2e8812fd69845 (dirty) (3753 commits) x86_64 hpack-0.14.0
2016-07-06 19:40:57.493231: [debug] Checking for project config at: /home/simon/stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:823:9)
2016-07-06 19:40:57.493629: [debug] Checking for project config at: /home/stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:823:9)
2016-07-06 19:40:57.493763: [debug] Checking for project config at: /stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:823:9)
2016-07-06 19:40:57.493909: [debug] No project config file found, using defaults. @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:845:13)
2016-07-06 19:40:57.496410: [debug] Checking whether stack was built with libgmp4 @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:327:5)
2016-07-06 19:40:57.496909: [debug] Run process: /usr/bin/ldd /home/simon/.local/bin/stack-bpl-async @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.512142: [debug] Process finished in 15 ms: /usr/bin/ldd /home/simon/.local/bin/stack-bpl-async @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.512561: [debug] Stack was not built with libgmp4 @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:331:14)
2016-07-06 19:40:57.512688: [info] Run from outside a project, using implicit global project config @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:462:13)
2016-07-06 19:40:57.514316: [info] Using resolver: nightly-2016-06-21 from implicit global project's config file: /home/simon/.stack/global-project/stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:476:32)
2016-07-06 19:40:57.514602: [debug] Trying to decode /home/simon/.stack/build-plan-cache/x86_64-linux/nightly-2016-06-21.cache @(Kkm9qBx54bb6fPiRXOOC4T:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:51:5)
2016-07-06 19:40:57.527888: [debug] Success decoding /home/simon/.stack/build-plan-cache/x86_64-linux/nightly-2016-06-21.cache @(Kkm9qBx54bb6fPiRXOOC4T:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:60:13)
2016-07-06 19:40:57.534286: [debug] Getting system compiler version @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Setup src/Stack/Setup.hs:345:17)
2016-07-06 19:40:57.534606: [debug] Run process: /usr/bin/ghc --info @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.560165: [debug] Process finished in 25 ms: /usr/bin/ghc --info @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.563992: [debug] Asking GHC for its version @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Setup.Installed src/Stack/Setup/Installed.hs:97:13)
2016-07-06 19:40:57.564233: [debug] Getting Cabal package version @(Kkm9qBx54bb6fPiRXOOC4T:Stack.GhcPkg src/Stack/GhcPkg.hs:165:5)
2016-07-06 19:40:57.564356: [debug] Getting global package database location @(Kkm9qBx54bb6fPiRXOOC4T:Stack.GhcPkg src/Stack/GhcPkg.hs:48:5)
2016-07-06 19:40:57.564437: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc --numeric-version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.565233: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.565852: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.602662: [debug] Process finished in 37 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.606331: [debug] Process finished in 34 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.607750: [debug] Process finished in 43 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc --numeric-version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.607943: [debug] Resolving package entries @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Setup src/Stack/Setup.hs:226:5)
2016-07-06 19:40:57.608657: [debug] Getting global package database location @(Kkm9qBx54bb6fPiRXOOC4T:Stack.GhcPkg src/Stack/GhcPkg.hs:48:5)
2016-07-06 19:40:57.608962: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.631108: [debug] Process finished in 21 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
global-pkg-db: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/lib/ghc-8.0.1/package.conf.d
dist-dir: .stack-work/dist/x86_64-linux/Cabal-1.24.0.0

@sjakobi sjakobi mentioned this pull request Jul 6, 2016
Closed
@sjakobi
setupEnv: Run some processes concurrently
875954c 
This should make most commands slightly faster.
On my machine, `stack path` takes ~20% less time than before, a no-op
`stack build` in a single-package project ~15%.
@sjakobi
Copy link
Member Author

sjakobi commented Jul 21, 2016

So, what I actually don't know, is whether this safe, especially querying ghc-pkg asynchronously.

Now, I figure there's no reason why ghc-pkg wouldn't be able to perform two reads concurrently.

@mgsloan
Copy link
Contributor

mgsloan commented Aug 6, 2016

Makes sense!

@mgsloan mgsloan merged commit 099ca78 into commercialhaskell:master Aug 6, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sjakobi @mgsloan @borsboom