Skip to content
This repository was archived by the owner on Apr 1, 2026. It is now read-only.

fix: Extend row hash to 128 bits to guarantee unique row id#632

Merged
tswast merged 9 commits intomainfrom
fix_hash
Apr 26, 2024
Merged

fix: Extend row hash to 128 bits to guarantee unique row id#632
tswast merged 9 commits intomainfrom
fix_hash

Conversation

@TrevorBergeron
Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Apr 22, 2024
@TrevorBergeron TrevorBergeron marked this pull request as ready for review April 23, 2024 16:07
@TrevorBergeron TrevorBergeron requested review from a team and tswast April 23, 2024 16:07
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Apr 23, 2024
@tswast
Copy link
Copy Markdown
Contributor

tswast commented Apr 25, 2024

ERROR tests/system/small/ml/test_llm.py::test_create_embedding_generator_multilingual_model

Looks like a test flake. Will raise an issue with our on-call this week.

else:
array_value = self._create_total_ordering(table_expression)
array_value = self._create_total_ordering(
table_expression, table_size=table.num_rows
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please try this with a VIEW type table? I don't think num_rows is always populated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this won't be populated for views or potentially with streaming inserts. defaulting to assume table is large and need 128 bits.

def _create_total_ordering(
self,
table: ibis_types.Table,
table_size: int,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise, we might need to handle missing size here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added.

@tswast tswast merged commit 9005c6e into main Apr 26, 2024
@tswast tswast deleted the fix_hash branch April 26, 2024 16:12
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: m Pull request size is medium.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants