Skip to content

[DuckDB] feat: Query index information for DuckDB dynamically#1171

Closed
ahmed-kamal2004 wants to merge 4 commits intosqlancer:mainfrom
ahmed-kamal2004:duck
Closed

[DuckDB] feat: Query index information for DuckDB dynamically#1171
ahmed-kamal2004 wants to merge 4 commits intosqlancer:mainfrom
ahmed-kamal2004:duck

Conversation

@ahmed-kamal2004
Copy link
Contributor

This PR resolves issue #1163

Before change

sb.append(Randomly.fromOptions("i0", "i1", "i2", "i3", "i4")); // cannot query this information

The indexes names are hard-coded, there is no way AFAIK to store or retrieve them

After change

  • store the indexes information in the table schema
  • extend SQLancer ability to generate random indexes for DuckDB testing, this ensures unique indexes names up to 100 index_name, now indexes names generation strategy is limited by ‎AbstractSchema::getFreeIndexName only.

Snippet of the generated creation statements for Indexes
Screenshot from 2025-03-20 15-19-36
Screenshot from 2025-03-20 15-18-47

Files Changed

  • DuckDBIndexGenerator.java : [OLD-Remved] random selection of indexes names out of a defined set of names - [NEW-Added] following generation strategy implemented & used by other database engines implementations
  • DuckDBSchema.java : [NEW-Added] store indexes information inside DuckDBTable object for further using

Testing

  • Running for almost 2.5 million queries on DuckDB
    Screenshot from 2025-03-20 15-49-13
    command used java -jar sqlancer-*.jar --num-threads 4 duckdb --oracle NOREC

Copy link
Contributor

@mrigger mrigger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I have added some improvement suggestions.


}

public static final class DuckDBIndex extends TableIndex {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could actually just use the TableIndex class here instead.


import sqlancer.SQLGlobalState;

public class DuckGlobalState extends SQLGlobalState<DuckDBOptions, DuckDBSchema> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have a global state class for DuckDB.

@ahmed-kamal2004 ahmed-kamal2004 requested a review from mrigger March 23, 2025 16:06
@mrigger
Copy link
Contributor

mrigger commented Mar 27, 2025

Thanks a lot for the PR! It looks good to me, but it seems the tests are currently failing due to a test oracle report. Could you perhaps try reducing the bug-inducing test cases? It might be a bug in DuckDB.

@mrigger
Copy link
Contributor

mrigger commented Mar 27, 2025

Based on the current output, it seems that we are repeatedly generating indexes with the same name, so it seems like there might be another issue.

@ahmed-kamal2004
Copy link
Contributor Author

@mrigger actually yes this is another issue, but is it was already here before the change, what I thinking of as a solution for this to get indexes with diff names is to use counter and reset "whenever needed like on schema change".

@mrigger
Copy link
Contributor

mrigger commented Mar 29, 2025

This functionality is already present in SQLancer and also other DBMSs use it. In the DuckDBIndexGenerator, when instantiating the SQLQueryAdapter, you can see that an argument is set that indicates a schema change.

@mrigger mrigger closed this Apr 3, 2025
Elshaarawy-1 added a commit to Elshaarawy-1/sqlancer that referenced this pull request Feb 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants