Skip to content

Commit 4a431e4

Browse files
gruuyagruuya
authored
Merge pull request #662 from splitgraph/layered-checkout-write-support-cu-21phdxv
Layered checkout write support
2 parents 3f1ef20 + 01d093f commit 4a431e4

19 files changed

Lines changed: 871 additions & 131 deletions

File tree

‎.pre-commit-config.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
repos:
22
- repo: https://github.com/ambv/black
3-
rev: 22.1.0
3+
rev: 22.3.0
44
hooks:
55
- id: black
66
language_version: python3.8

‎poetry.lock‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎splitgraph/commandline/image_creation.py‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -35,15 +35,11 @@ def checkout_c(image_spec, force, uncheckout, layered):
3535
"""
3636
Check out a Splitgraph image into a Postgres schema.
3737
38-
This downloads the required physical objects and materializes all tables, unless ``-l`` or ``--layered`` is passed,
39-
in which case the objects are downloaded and a foreign data wrapper is set up on the engine to satisfy read-only
40-
queries by combining results from each table's fragments.
38+
This downloads the required physical objects and materializes all tables, unless ``-l`` or ``--layered`` is passed.
4139
42-
Tables checked out in this way are still presented as normal Postgres tables and can queried in the same way.
43-
Since the tables aren't materialized, layered querying is faster to set up, but since each query now results in a
44-
subquery to each object comprising the table, actual query execution is slower than to materialized Postgres tables.
45-
46-
Layered querying is only supported for read-only queries.
40+
In the layered querying mode we construct an overlay table structure inspired by OverlayFS to facilitate both reads
41+
and writes (INSERT/UPDATE/DELETE). Since the tables aren't materialized, layered querying is faster to set up and,
42+
depending on the kind of the query being run, can sometimes be faster than querying materialized PostgreSQL tables.
4743
4844
Image spec must be of the format ``[NAMESPACE/]REPOSITORY[:HASH_OR_TAG]``. Note that currently, the schema that the
4945
image is checked out into has to have the same name as the repository. If no image hash or tag is passed,

‎splitgraph/commandline/image_info.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -430,7 +430,7 @@ def _emit_repository_data(repositories, engine):
430430
if (
431431
engine.run_sql(
432432
"SELECT 1 FROM pg_foreign_server WHERE srvname = %s",
433-
(("%s_lq_checkout_server" % repo.to_schema())[:63],),
433+
(repo.lq_server_name(),),
434434
return_shape=ResultShape.ONE_ONE,
435435
)
436436
is not None

‎splitgraph/core/common.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,7 @@ def manage_audit_triggers(
8888
for r, head in get_current_repositories(engine)
8989
if head is not None
9090
for t in set(object_engine.get_all_tables(r.to_schema())) & set(head.get_tables())
91+
if not r.is_overlay_view(t)
9192
]
9293
tracked_tables = object_engine.get_tracked_tables()
9394

0 commit comments

Comments
 (0)