Skip to content

Optimize integer zset scores in listpack (converting to string and back)#10486

Merged
oranagra merged 9 commits intoredis:unstablefrom
oranagra:zset_listpack_int
Apr 17, 2022
Merged

Optimize integer zset scores in listpack (converting to string and back)#10486
oranagra merged 9 commits intoredis:unstablefrom
oranagra:zset_listpack_int

Conversation

@oranagra
Copy link
Copy Markdown
Member

@oranagra oranagra commented Mar 28, 2022

When the score doesn't have fractional part, and can be stored as an integer,
we use the integer capabilities of listpack to store it, rather than convert it to string.
This already existed before this PR (lpInsert dose that conversion implicitly).

But to do that, we would have first converted the score from double to string (calling d2string),
then pass the string to lpAppend which identified it as being an integer and convert it back to an int.
Now, instead of converting it to a string, we store it using lpAppendInteger`.

Unrelated:

  • Fix the double2ll range check (negative and positive ranges, and also the comparison operands
    were slightly off. but also, the range could be made much larger, see comment).
  • Unify the double to string conversion code in rdb.c with the one in util.c
  • Small optimization in lpStringToInt64, don't attempt to convert strings that are obviously too long.

Benchmark;

Up to 20% improvement in certain tight loops doing zzlInsert with large integers.
(if listpack is pre-allocated to avoid realloc, and insertion is sorted from largest to smaller)

@oranagra oranagra requested review from sundb and yossigo March 28, 2022 12:18
Copy link
Copy Markdown
Collaborator

@sundb sundb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@filipecosta90 filipecosta90 added the action:run-benchmark Triggers the benchmark suite for this Pull Request label Mar 29, 2022
Copy link
Copy Markdown
Contributor

@zuiderkwast zuiderkwast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea.

Comment thread src/util.c Outdated
Comment on lines +566 to +567
double min = -4503599627370495; /* (2^52)-1 */
double max = 4503599627370496; /* -(2^52) */
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the comments for min and max are swapped.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the comments are wrong, but i'm also uncertain the values are correct.
in integer's 2's complement, the negative side is the larger one (unlike this code).
however, doubles have a distinct sign bit, so i think their range to the positive and negative is the same.
i think the numbers should be 4503599627370495 and -4503599627370495

but i also think this check doesn't necessarily have to be here, and can't cause much harm.
i.e. i suppose we can also safely keep a value of 4503599627370495*4 as long without precision loss (but not *3).
and also, even if the range check would have been too big (allowing some values that can't be stored in long), the later conversion check is what actually decides, so this range check is just an optimization to avoid casting overheads?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my last commit and the updated top comment.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/util.c Outdated
Since doubles have distict sign bit, the range should be +/- ((2^52)-1)
inclusive.
Note that this is just a speedup to avoid the casting check, which is
what actually matters.
Comment thread src/util.c Outdated
Comment thread src/util.c Outdated
Comment thread src/util.c Outdated
Comment thread src/util.c
@sundb
Copy link
Copy Markdown
Collaborator

sundb commented Apr 2, 2022

Using the following test code shows that performance is indeed improved.

Here are a few test optimizations to avoid additional performance consumption

  1. Pre-expansion listpack, avoid listpack realloc.
  2. Insert scores from largest to smallest, so that entries are always inserted at the head of the listpack, avoiding searching the whole listpack.
max score min score time consume (unstable) time consume (this PR) performance enhancement
4503599627370494 4503599627370494 - 128 37.98 s 32.73 +1.16%
-4503599627370494 + 128 -4503599627370494 38.39 s 33.61 s +1.14%

Test codes

long long start = ustime();
for (int j = 0; j < 1000000; j++) {
    unsigned char *lp = lpNew(2183);
    for (long long i = 4503599627370494; i > 4503599627370494 - 128; i--) {
        sds ele = sdscatfmt(sdsempty(), "key%I", i);
        lp = zzlInsert(lp, ele, i);
        sdsfree(ele);
    }
    zfree(lp);
}
printf("consume: %lld \n", ustime() - start);

When we have a double score and we know we cannot convert it to an
integer, we convert it to a string, but then listpack was again trying
to convert that string to an integer.

Now we have an lpInsertDouble API that avoids these overheads.

Note: i'm not happy with the result, specifically the interface of
lpInsert (i.e. the NOT_INT part)

Unrelated: improve lpStringToInt64 not to try converting long strings to
integers.
@oranagra
Copy link
Copy Markdown
Member Author

oranagra commented Apr 5, 2022

i pushed a commit that avoids the extra string to integer conversion attempt, didn't try to benchmark it yet.
also, i'm not happy with the result (specifically the interface of lpInsert), maybe someone has an idea...

p.s. the right solution to all of that is probably to add native double encoding for listpack (i.e. save the actual double bits), but that's complicated, and i'm not certain we have spare bits in the header to denote that type.

Comment thread src/listpack.c Outdated
@oranagra
Copy link
Copy Markdown
Member Author

oranagra commented Apr 6, 2022

benchmark using the code from #10486 (comment)

test lpInsertDouble lpInsertString / Integer unstable
time 27999859 28029440 33364418
improvement vs unstable 1.1915 (~20%) 1.1903 (~20%) 1.0

this is expected since the last commit attempts to resolve issues from when fractional parts are used, but for some reason when i try to measure the same with fractional numbers, i don't see any difference between the 3 options.

@sundb
Copy link
Copy Markdown
Collaborator

sundb commented Apr 6, 2022

benchmark using the code from #10486 (comment)

test lpInsertDouble lpInsertString / Integer unstable
time 27999859 28029440 33364418
improvement vs unstable 1.1915 (~20%) 1.1903 (~20%) 1.0
this is expected since the last commit attempts to resolve issues from when fractional parts are used, but for some reason when i try to measure the same with fractional numbers, i don't see any difference between the 3 options.

https://github.com/redis/redis/blob/3e09a8c09770dfbe5c8a1c3d2ebc4599448ba7c4/src/util.c#L591
When using fractional numbers, this line of code will consume most of the CPU,
resulting in other optimizations that are not as obvious.

@oranagra
Copy link
Copy Markdown
Member Author

oranagra commented Apr 6, 2022

so maybe i should revert my last commit.
the optimization it brings of avoiding an attempt to convert a string to integer, isn't relevant in that code flow, and it does mess up the code / interfaces a bit.

@sundb
Copy link
Copy Markdown
Collaborator

sundb commented Apr 7, 2022

@oranagra Perhaps we can reserve lp*double(), we can store the bits of the double directly into the listpack in the future.

@oranagra
Copy link
Copy Markdown
Member Author

oranagra commented Apr 7, 2022

i don't see a point in reserving the API if we don't have the code to back it. we can add that later again if needed.
currently it looks ugly. so unless i can find that it helps with performance, i'd rather drop it.

@sundb
Copy link
Copy Markdown
Collaborator

sundb commented Apr 14, 2022

I made an attempt to store double in listpack, it's really complicated, not in the storage, but in the interface of listpack (lpGet, lpGetValue, lpFind) will be changed dramatically, and eventually the caller (hash, zsortset, list, stream) will need to handle double encoding(a lot of changes), but it only improves zsortset.
It also has the side effect that when replying the score of zsortset, it will need to be converted to a string again using snprintf.

@zuiderkwast
Copy link
Copy Markdown
Contributor

Good job @sundb. So I guess the conclusion is that it's not worth it, storing IEEE doubles in listpack.

Maybe we can optimize double <---> string conversion instead? There are some libraries that are several times faster than snprinf and strtod.

For snprintf, there's the Grisu2 and Grisu3 algorithms. Paper: https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf, implementation (C++, 3-clause BSD): https://github.com/google/double-conversion. I haven't found a plain C implementation yet.

For strtod, there's https://github.com/fastfloat/fast_float and https://github.com/lemire/fast_double_parser and blog posts like https://lemire.me/blog/2020/03/10/fast-float-parsing-in-practice/. Daniel Lemire has a lot of papers and blog posts on these topics.

@sundb
Copy link
Copy Markdown
Collaborator

sundb commented Apr 14, 2022

@zuiderkwast You remind me of #8825

Comment thread src/util.c Outdated
@filipecosta90
Copy link
Copy Markdown
Contributor

Good job @sundb. So I guess the conclusion is that it's not worth it, storing IEEE doubles in listpack.

Maybe we can optimize double <---> string conversion instead? There are some libraries that are several times faster than snprinf and strtod.

For snprintf, there's the Grisu2 and Grisu3 algorithms. Paper: https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf, implementation (C++, 3-clause BSD): https://github.com/google/double-conversion. I haven't found a plain C implementation yet.

For strtod, there's https://github.com/fastfloat/fast_float and https://github.com/lemire/fast_double_parser and blog posts like https://lemire.me/blog/2020/03/10/fast-float-parsing-in-practice/. Daniel Lemire has a lot of papers and blog posts on these topics.

@zuiderkwast WRT to #8825 I've opened a PR that uses a plain C implementation of grisu2. (tests are still failing and I need to address it but it would be of interest to have your opinion on it ). PR link: #10587

@oranagra
Copy link
Copy Markdown
Member Author

@yossigo please review (see updated top comment). i'd like to merge this.

@oranagra oranagra changed the title zset store score as integer in listpack when possible Optimize integer zset scores in listpack (converting to string and back) Apr 17, 2022
@oranagra oranagra merged commit 0c4733c into redis:unstable Apr 17, 2022
@oranagra oranagra deleted the zset_listpack_int branch April 17, 2022 14:16
Comment thread src/util.c
* i.e. all double values in that range are representable as a long without precision loss,
* but not all long values in that range can be represented as a double.
* we only care about the first part here. */
if (d < -LLONG_MAX/2 || d > LLONG_MAX/2)
Copy link
Copy Markdown
Contributor

@enjoy-binbin enjoy-binbin Apr 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a warning, https://github.com/redis/redis/runs/6057010293?check_suite_focus=true#step:4:139
PR link: #10595

util.c:574:42: error: implicit conversion from 'long long' to 'double' changes value from 4611686018427387903 to 4611686018427387904 [-Werror,-Wimplicit-const-int-float-conversion]
    if (d < -LLONG_MAX/2 || d > LLONG_MAX/2)

enjoy-binbin added a commit to enjoy-binbin/redis that referenced this pull request Apr 18, 2022
There is a implicit conversion warning in clang:
```
util.c:574:23: error: implicit conversion from 'long long' to 'double'
changes value from -4611686018427387903 to -4611686018427387904
[-Werror,-Wimplicit-const-int-float-conversion]
    if (d < -LLONG_MAX/2 || d > LLONG_MAX/2)
```

introduced in redis#10486

Co-authored-by: sundb <[email protected]>
oranagra pushed a commit that referenced this pull request Apr 18, 2022
There is a implicit conversion warning in clang:
```
util.c:574:23: error: implicit conversion from 'long long' to 'double'
changes value from -4611686018427387903 to -4611686018427387904
[-Werror,-Wimplicit-const-int-float-conversion]
    if (d < -LLONG_MAX/2 || d > LLONG_MAX/2)
```

introduced in #10486

Co-authored-by: sundb <[email protected]>
@oranagra oranagra mentioned this pull request Apr 27, 2022
enjoy-binbin pushed a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
…ck) (redis#10486)

When the score doesn't have fractional part, and can be stored as an integer,
we use the integer capabilities of listpack to store it, rather than convert it to string.
This already existed before this PR (lpInsert dose that conversion implicitly).

But to do that, we would have first converted the score from double to string (calling `d2string`),
then pass the string to `lpAppend` which identified it as being an integer and convert it back to an int.
Now, instead of converting it to a string, we store it using lpAppendInteger`.

Unrelated:
---
* Fix the double2ll range check (negative and positive ranges, and also the comparison operands
  were slightly off. but also, the range could be made much larger, see comment).
* Unify the double to string conversion code in rdb.c with the one in util.c
* Small optimization in lpStringToInt64, don't attempt to convert strings that are obviously too long.

Benchmark;
---
Up to 20% improvement in certain tight loops doing zzlInsert with large integers.
(if listpack is pre-allocated to avoid realloc, and insertion is sorted from largest to smaller)
enjoy-binbin added a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
There is a implicit conversion warning in clang:
```
util.c:574:23: error: implicit conversion from 'long long' to 'double'
changes value from -4611686018427387903 to -4611686018427387904
[-Werror,-Wimplicit-const-int-float-conversion]
    if (d < -LLONG_MAX/2 || d > LLONG_MAX/2)
```

introduced in redis#10486

Co-authored-by: sundb <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

action:run-benchmark Triggers the benchmark suite for this Pull Request

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

7 participants