-
-
Notifications
You must be signed in to change notification settings - Fork 4
Migrations, named parameters, security tests #48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
263c161
bd sync: 2025-12-30 17:37:01
ocean e7b8fe2
test: add comprehensive cross-connection security tests (el-5ef)
ocean 42c49af
feat: add STRICT table option support for migrations
ocean ebaf841
feat: Implement named parameter execution support
ocean 1810d61
docs: Update docs with latest changes
ocean 4c73365
chore: Update beads
ocean 29b9e46
chore: Correct beads config
ocean aa1e9cb
chore: Correct beads config
ocean 632c3ba
Merge sync branch 'beads-sync'
ocean 623d2b5
tests: Fix various issues in tests and formatting
ocean a54933b
fix: Address CodeRabbit PR review comments
ocean a7c62e6
fix: Add named parameter normalisation to prepared statement functions
ocean 32bdb94
fix: Make cross-connection transaction isolation test strict
ocean 1f2808f
fix: Remove double-disconnect of shared setup connection in security …
ocean a0ed2d4
fix: Improve security tests and fix credo warnings
ocean 9220914
fix: Prefix unused variable with underscore in get_map_value_flexible
ocean e5c8874
fix: Thread state through loop iterations in security test setup
ocean 4adec5e
fix: Remove empty map pattern and fix state threading in security tests
ocean 912554f
fix: Use idiomatic refute instead of assert with negation
ocean 4a900f7
fix: Standardise unused variable naming for Credo consistency
ocean f76b64e
chore: Update config and beads
ocean 4b59b25
fix: Correct error tuple handling and binary blob round-trip in fuzz …
ocean 276583f
fix: Replace unbounded persistent_term cache with bounded ETS LRU cache
ocean b626ebf
fix: Propagate parameter introspection errors instead of silently fal…
ocean 1810e6c
tests: Fix DB cleanup
ocean 58e1b38
fix: Fix credo nesting warnings
ocean f50799c
fix: Update cache docs
ocean 72241ad
fix: Return descriptive error for invalid argument types in normalise…
ocean 4e4d167
Merge branch 'main' into migrations-params-tests
ocean 5945889
tests: Fix suggestions
ocean 79648eb
fix: Resolve FilterTooNarrowError in savepoint injection fuzz test
ocean File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -485,7 +485,115 @@ end) | |
|
|
||
| ### Prepared Statements | ||
|
|
||
| Prepared statements offer significant performance improvements for repeated queries and prevent SQL injection. As of v0.7.0, statement caching is automatic and highly optimised. | ||
| Prepared statements offer significant performance improvements for repeated queries and prevent SQL injection. As of v0.7.0, statement caching is automatic and highly optimised. Named parameters provide flexible parameter binding with three SQLite syntaxes. | ||
|
|
||
| #### Named Parameters | ||
|
|
||
| SQLite supports three named parameter syntaxes for more readable and maintainable queries: | ||
|
|
||
| ```elixir | ||
| # Syntax 1: Colon prefix (:name) | ||
| "SELECT * FROM users WHERE email = :email AND status = :status" | ||
|
|
||
| # Syntax 2: At-sign prefix (@name) | ||
| "SELECT * FROM users WHERE email = @email AND status = @status" | ||
|
|
||
| # Syntax 3: Dollar sign prefix ($name) | ||
| "SELECT * FROM users WHERE email = $email AND status = $status" | ||
| ``` | ||
|
|
||
| Execute with map-based parameters: | ||
|
|
||
| ```elixir | ||
| # Prepared statement with named parameters | ||
| {:ok, stmt_id} = EctoLibSql.Native.prepare( | ||
| state, | ||
| "SELECT * FROM users WHERE email = :email AND status = :status" | ||
| ) | ||
|
|
||
| {:ok, result} = EctoLibSql.Native.query_stmt( | ||
| state, | ||
| stmt_id, | ||
| %{"email" => "[email protected]", "status" => "active"} | ||
| ) | ||
| ``` | ||
|
|
||
| Direct execution with named parameters: | ||
|
|
||
| ```elixir | ||
| # INSERT with named parameters | ||
| {:ok, _, _, state} = EctoLibSql.handle_execute( | ||
| "INSERT INTO users (name, email, age) VALUES (:name, :email, :age)", | ||
| %{"name" => "Alice", "email" => "[email protected]", "age" => 30}, | ||
| [], | ||
| state | ||
| ) | ||
|
|
||
| # UPDATE with named parameters | ||
| {:ok, _, _, state} = EctoLibSql.handle_execute( | ||
| "UPDATE users SET status = :status, updated_at = :now WHERE id = :user_id", | ||
| %{"status" => "inactive", "now" => DateTime.utc_now(), "user_id" => 123}, | ||
| [], | ||
| state | ||
| ) | ||
|
|
||
| # DELETE with named parameters | ||
| {:ok, _, _, state} = EctoLibSql.handle_execute( | ||
| "DELETE FROM users WHERE id = :user_id AND email = :email", | ||
| %{"user_id" => 123, "email" => "[email protected]"}, | ||
| [], | ||
| state | ||
| ) | ||
| ``` | ||
|
|
||
| Named parameters in transactions: | ||
|
|
||
| ```elixir | ||
| {:ok, :begin, state} = EctoLibSql.handle_begin([], state) | ||
|
|
||
| {:ok, _, _, state} = EctoLibSql.handle_execute( | ||
| """ | ||
| INSERT INTO users (name, email) VALUES (:name, :email) | ||
| """, | ||
| %{"name" => "Alice", "email" => "[email protected]"}, | ||
| [], | ||
| state | ||
| ) | ||
|
|
||
| {:ok, _, _, state} = EctoLibSql.handle_execute( | ||
| "UPDATE users SET verified = 1 WHERE email = :email", | ||
| %{"email" => "[email protected]"}, | ||
| [], | ||
| state | ||
| ) | ||
|
|
||
| {:ok, _, state} = EctoLibSql.handle_commit([], state) | ||
| ``` | ||
|
|
||
| **Benefits:** | ||
| - **Readability**: Clear parameter names make queries self-documenting | ||
| - **Maintainability**: Easier to refactor when parameter names are explicit | ||
| - **Type safety**: Parameter validation can check required parameters upfront | ||
| - **Flexibility**: Use any of three SQLite syntaxes interchangeably | ||
| - **Prevention**: Prevents SQL injection attacks through proper parameter binding | ||
|
|
||
| **Backward Compatibility:** | ||
| Positional parameters (`?`) still work unchanged: | ||
|
|
||
| ```elixir | ||
| # Positional parameters still work | ||
| {:ok, _, result, state} = EctoLibSql.handle_execute( | ||
| "SELECT * FROM users WHERE email = ? AND status = ?", | ||
| ["[email protected]", "active"], | ||
| [], | ||
| state | ||
| ) | ||
|
|
||
| # Named and positional can coexist in separate queries within the same codebase | ||
| ``` | ||
|
|
||
| **Avoiding Mixed Syntax:** | ||
| While SQLite technically permits mixing positional (`?`) and named (`:name`) parameters in a single statement, this is discouraged. Named parameters receive implicit numeric indices which can conflict with positional parameters, leading to unexpected binding order. This adapter's map-based approach naturally avoids this issue—pass a list for positional queries, or a map for named queries, but don't mix within a single statement. | ||
|
|
||
| #### How Statement Caching Works | ||
|
|
||
|
|
@@ -1345,6 +1453,100 @@ mix ecto.migrate # Run migrations | |
| mix ecto.rollback # Rollback last migration | ||
| ``` | ||
|
|
||
| #### STRICT Tables (Type Enforcement) | ||
|
|
||
| STRICT tables enforce strict type checking - columns must be one of the allowed SQLite types. This prevents accidental type mismatches and data corruption: | ||
|
|
||
| ```elixir | ||
| # Create a STRICT table for type safety | ||
| defmodule MyApp.Repo.Migrations.CreateUsers do | ||
| use Ecto.Migration | ||
|
|
||
| def change do | ||
| create table(:users, strict: true) do | ||
| add :id, :integer, primary_key: true | ||
| add :name, :string, null: false | ||
| add :email, :string, null: false | ||
| add :age, :integer | ||
| add :balance, :float, default: 0.0 | ||
| add :avatar, :binary | ||
| add :is_active, :boolean, default: true | ||
|
|
||
| timestamps() | ||
| end | ||
|
|
||
| create unique_index(:users, [:email]) | ||
| end | ||
| end | ||
| ``` | ||
|
|
||
| **Benefits:** | ||
| - **Type Safety**: Enforces that columns only accept their declared types (TEXT, INTEGER, REAL, BLOB, NULL) | ||
| - **Data Integrity**: Prevents accidental type coercion that could lead to bugs | ||
| - **Better Errors**: Clear error messages when incorrect types are inserted | ||
| - **Performance**: Can enable better query optimisation by knowing exact column types | ||
|
|
||
| **Allowed Types in STRICT Tables:** | ||
| - `INT`, `INTEGER` - Integer values only | ||
| - `TEXT` - Text values only | ||
| - `BLOB` - Binary data only | ||
| - `REAL` - Floating-point values only | ||
| - `NULL` - NULL values only (rarely used) | ||
|
|
||
| **Usage Examples:** | ||
|
|
||
| ```elixir | ||
| # STRICT table with various types | ||
| create table(:products, strict: true) do | ||
| add :sku, :string, null: false # Must be TEXT | ||
| add :name, :string, null: false # Must be TEXT | ||
| add :quantity, :integer, default: 0 # Must be INTEGER | ||
| add :price, :float, null: false # Must be REAL | ||
| add :description, :text # Must be TEXT | ||
| add :image_data, :binary # Must be BLOB | ||
| add :published_at, :utc_datetime # Stored as TEXT (ISO8601 format) | ||
| timestamps() | ||
| end | ||
|
|
||
| # Combining STRICT with RANDOM ROWID | ||
| create table(:api_keys, options: [strict: true, random_rowid: true]) do | ||
| add :user_id, references(:users, on_delete: :delete_all) # INTEGER | ||
| add :key, :string, null: false # TEXT | ||
| add :secret, :string, null: false # TEXT | ||
| add :last_used_at, :utc_datetime # TEXT | ||
| timestamps() | ||
| end | ||
| ``` | ||
|
|
||
| **Restrictions:** | ||
| - STRICT is a libSQL/SQLite 3.37+ extension (not available in older versions) | ||
| - Type affinity is enforced: generic types like `TEXT(50)` or `DATE` are not allowed | ||
| - Dynamic type changes (e.g., storing integers in TEXT columns) will fail with type errors | ||
| - Standard SQLite does not support STRICT tables | ||
|
|
||
| **SQL Output:** | ||
| ```sql | ||
| CREATE TABLE users ( | ||
| id INTEGER PRIMARY KEY, | ||
| name TEXT NOT NULL, | ||
| email TEXT NOT NULL, | ||
| age INTEGER, | ||
| balance REAL DEFAULT 0.0, | ||
| avatar BLOB, | ||
| is_active INTEGER DEFAULT 1, | ||
| inserted_at TEXT, | ||
| updated_at TEXT | ||
| ) STRICT | ||
| ``` | ||
|
|
||
| **Error Example:** | ||
| ```elixir | ||
| # This will fail on a STRICT table: | ||
| Repo.query!("INSERT INTO users (name, email, age) VALUES (?, ?, ?)", | ||
| [123, "[email protected]", "thirty"]) # ← age is string, not INTEGER | ||
| # Error: "Type mismatch" (SQLite enforces STRICT) | ||
| ``` | ||
|
|
||
| #### RANDOM ROWID Support (libSQL Extension) | ||
|
|
||
| For security and privacy, use RANDOM ROWID to generate pseudorandom row IDs instead of sequential integers: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,6 +9,56 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 | |
|
|
||
| ### Added | ||
|
|
||
| - **Named Parameters Execution Support** | ||
| - Full support for SQLite named parameter syntax in prepared statements and direct execution | ||
| - **Three SQLite syntaxes supported**: `:name`, `@name`, `$name` | ||
| - **Transparent conversion**: Map-based named parameters automatically converted to positional arguments for internal execution | ||
| - **Use cases**: Dynamic query building, parameter validation, better debuggability, API introspection | ||
| - **Execution paths**: Works with prepared statements, transactions, batch operations, and cursor streaming | ||
| - **Backward compatibility**: Existing positional parameter syntax (`?`) continues to work unchanged | ||
| - **Implementation**: Automatic parameter binding detection and conversion in both transactional and non-transactional paths | ||
| - **Usage examples**: | ||
| ```elixir | ||
| # Named parameters in prepared statements | ||
| {:ok, stmt_id} = EctoLibSql.Native.prepare( | ||
| state, | ||
| "SELECT * FROM users WHERE email = :email AND status = :status" | ||
| ) | ||
|
|
||
| # Execute with named parameters as map | ||
| {:ok, result} = EctoLibSql.Native.query_stmt( | ||
| state, | ||
| stmt_id, | ||
| %{"email" => "[email protected]", "status" => "active"} | ||
| ) | ||
|
|
||
| # Alternative syntaxes | ||
| "SELECT * FROM users WHERE email = @email" | ||
| "SELECT * FROM users WHERE email = $email" | ||
|
|
||
| # Works with direct execution | ||
| {:ok, _, result, state} = EctoLibSql.handle_execute( | ||
| "INSERT INTO users (name, email) VALUES (:name, :email)", | ||
| %{"name" => "Alice", "email" => "[email protected]"}, | ||
| [], | ||
| state | ||
| ) | ||
|
|
||
| # Works with transactions | ||
| {:ok, :begin, state} = EctoLibSql.handle_begin([], state) | ||
| {:ok, _, _, state} = EctoLibSql.handle_execute( | ||
| "UPDATE users SET status = :status WHERE id = :id", | ||
| %{"status" => "inactive", "id" => 123}, | ||
| [], | ||
| state | ||
| ) | ||
| {:ok, _, state} = EctoLibSql.handle_commit([], state) | ||
| ``` | ||
| - **Type handling**: All value types (strings, integers, floats, binaries, nil) properly converted | ||
| - **Parameter validation**: Uses `stmt_parameter_name/3` introspection for validation | ||
| - **Edge cases handled**: Empty parameter maps, missing parameters with proper error messages, mixed positional and named parameters | ||
| - **Added comprehensive test coverage** in `test/named_parameters_execution_test.exs` covering all SQLite syntaxes, CRUD operations, transactions, batch operations, and backward compatibility | ||
|
|
||
| - **Query-Based UPSERT Support (on_conflict with Ecto.Query)** | ||
| - Extended `on_conflict` support to handle query-based updates | ||
| - Allows using keyword list syntax for dynamic update operations: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.