Skip to content

Add cache for $LOAD_FEATURES and $LOAD_PATH#2353

Merged
dbussink merged 2 commits intorubinius:masterfrom
LTe:cache-for-require
May 21, 2013
Merged

Add cache for $LOAD_FEATURES and $LOAD_PATH#2353
dbussink merged 2 commits intorubinius:masterfrom
LTe:cache-for-require

Conversation

@LTe
Copy link
Copy Markdown
Contributor

@LTe LTe commented May 17, 2013

Rubinius will create cache for require short paths. When user
require file ie:

require 'path'
CodeLoader.loaded_features_index => { 'path' => '/real/path' }

When we try require 'path' once again Rubinius will not travel via
$LOAD_PATH because loaded_features_index include 'path'. But when user
will try require '/real/path' Rubinius will find this feature in
$LOADED_FEATURES. In the case of not finding the file anywhere Rubinius
will try find file in $LOAD_PATH and fill in cache.

When $LOADED_FEATURES change we need to invalidate cache!

Benchmark

require 'benchmark'

`mkdir f`

1000.times do |time|
  `mkdir -p f/dir#{time}`
end

`touch f/dir888/single_file.rb

Benchmark.bm do |b|
  1000.times do |time|
    $LOAD_PATH << "f/dir#{time}"
  end

  b.report { 1000.times { require "single_file.rb" }}
end

`rm -rf ./f`

Before patch:

      user     system      total        real
  1.940122   1.112070   3.052192 (  3.059322)

After patch:

      user     system      total        real
  0.100006   0.024002   0.124008 (  0.143300)

@dbussink
Copy link
Copy Markdown
Contributor

Looking at this, I think it's better to compose LoadPath with an array than to inherit from Array. Is there any reason you need to inherit from Array?

@dbussink
Copy link
Copy Markdown
Contributor

Also wondering, how does this affect performance of for example starting Rails? Does it help there as because of the cache?

@LTe
Copy link
Copy Markdown
Contributor Author

LTe commented May 17, 2013

Yes, rails app boot on ~13 sec and after patch ~11 :)

@dbussink
Copy link
Copy Markdown
Contributor

I'm still unsure about a custom LoadPath class, is that needed at all for this change? It doesn't seem to be needed for the LOADED_FEATURES changes right? Also, is there a reason for all the sychronization in it? Array isn't thread safe either, so is there a reason this adds it? Were there cases where this was a problem?

If the thread safety of LOADED_FEATURES / LOAD_PATH is an issue, I think we should address that separately so we don't mix different changes which makes discussing and reviewing them harder.

@LTe
Copy link
Copy Markdown
Contributor Author

LTe commented May 19, 2013

is that needed at all for this change?

No, this is feature from ruby 2.0. Each element in $LOAD_PATH is frozen and $LOAD_PATH is cached in exactly the same way. So I think we can remove that part from pull request.

Also, is there a reason for all the sychronization in it?

I'm not sure - I was based on the implementation of $LOAD_FEATURES

I think we should address that separately so we don't mix different changes which makes discussing and reviewing them harder.

Agree, I will update this pull request and I will remove all changes related to $LOAD_PATH

LTe added 2 commits May 19, 2013 20:49
Rubinius will create cache for require short paths. When user
require file ie:

  require 'path'
  CodeLoader.loaded_features_index => { 'path' => '/real/path' }

When we try require 'path' once again Rubinius will not travel via
$LOAD_PATH because loaded_features_index include 'path'. But when user
will try require '/real/path' Rubinius will find this feature in
$LOADED_FEATURES. In the case of not finding the file anywhere Rubinius
will try find file in $LOAD_PATH and fill in cache.

When $LOADED_FEATURES change we need to invalidate cache!
@LTe
Copy link
Copy Markdown
Contributor Author

LTe commented May 19, 2013

@dbussink I updated my pull request.

@LTe
Copy link
Copy Markdown
Contributor Author

LTe commented May 20, 2013

rails runner on empty rails app

Before patch

real    0m11.360s
user    0m20.165s
sys 0m0.736s

After patch

real    0m11.572s
user    0m19.749s
sys 0m0.608s

Runner on rails app with gems:

cat Gemfile | wc -l
64

Before patch

real    0m13.097s
user    0m18.877s
sys 0m1.808s

After patch

real    0m11.593s
user    0m17.857s
sys 0m0.792s

@LTe
Copy link
Copy Markdown
Contributor Author

LTe commented May 21, 2013

I based on @headius implementation jruby/jruby#452

dbussink added a commit that referenced this pull request May 21, 2013
Add cache for $LOAD_FEATURES and $LOAD_PATH
@dbussink dbussink merged commit 45a0ef7 into rubinius:master May 21, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants