Skip to content

Multicorn agg pushdown v2 #613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gruuya merged 20 commits into from
Jan 21, 2022
Merged

Multicorn agg pushdown v2 #613

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
372b4e5
Incorporate new aggregation pushdown capabilites into the engine
gruuya Jan 14, 2022
e3aab81
Merge branch 'master' into multicorn-agg-pushdown-v2-cu-1z461e4
gruuya Jan 14, 2022
ec5fd86
Add tests for upperrel qual pushdown
gruuya Jan 14, 2022
4e5494b
Add support for pushdown of COUNT(*) and corresponding tests
gruuya Jan 17, 2022
a3be017
Actually add the tests
gruuya Jan 17, 2022
369a230
Bump ES to version with fixed wildcard match and correspodning test
gruuya Jan 17, 2022
20e5d8d
Fix pattern matching qual pushdown test assertion
gruuya Jan 17, 2022
447fd5c
Add test for WHERE-compatible HAVING clause pushdown
gruuya Jan 18, 2022
6146047
Fix MySQL GPG key in engine's Dockerfile
gruuya Jan 18, 2022
b91fc5d
Add test for pattern match pushdown
gruuya Jan 18, 2022
116bf3a
Extend ES FDW pattern matching test
gruuya Jan 19, 2022
e432df1
Fix pattern matching test
gruuya Jan 19, 2022
6ebe428
Add NOT LIKE pattern match test
gruuya Jan 19, 2022
2d3b56b
[WIP] Bump Multicorn and ES FDW submodules
gruuya Jan 19, 2022
f44264f
Extend test for elementary PG-to-ES pattern match conversions
gruuya Jan 19, 2022
6a5bfd5
Bump Multicorn to version with fixed qual operator pushability verifi…
gruuya Jan 20, 2022
fed09dd
Bump Multicorn submodule - py-to-pg list conversion polish
gruuya Jan 20, 2022
d4a805f
Add test for escaping special ES characters
gruuya Jan 21, 2022
c7a7653
Appease flake8 in pattern matching test
gruuya Jan 21, 2022
fa8dacc
Point Multicorn and ES submodules to master
gruuya Jan 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions engine/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ ENV LC_ALL C.UTF-8

# Fix taken from https://github.com/f-secure-foundry/usbarmory-debian-base_image/issues/9
RUN mkdir ~/.gnupg && echo "disable-ipv6" >> ~/.gnupg/dirmngr.conf && \
apt-key adv --homedir ~/.gnupg --keyserver hkp://keyserver.ubuntu.com --recv-keys 5072E1F5 && \
apt-key adv --homedir ~/.gnupg --keyserver hkp://keyserver.ubuntu.com --recv-keys 467B942D3A79BD29 && \
echo "deb http://repo.mysql.com/apt/debian/ buster mysql-8.0" > /etc/apt/sources.list.d/mysql.list && \
apt-get update -qq && apt-get install -y libmysqlclient-dev && rm -rf /var/lib/apt/lists/*

Expand Down Expand Up @@ -143,7 +143,7 @@ RUN --mount=type=cache,id=pip-cache,target=/root/.cache/pip /build/build_splitgr


RUN mkdir ~/.gnupg && echo "disable-ipv6" >> ~/.gnupg/dirmngr.conf && \
apt-key adv --homedir ~/.gnupg --keyserver hkp://keyserver.ubuntu.com --recv-keys 5072E1F5 && \
apt-key adv --homedir ~/.gnupg --keyserver hkp://keyserver.ubuntu.com --recv-keys 467B942D3A79BD29 && \
echo "deb http://repo.mysql.com/apt/debian/ buster mysql-8.0" > /etc/apt/sources.list.d/mysql.list && \
apt-get update -qq && apt-get install -y libmysqlclient-dev && rm -rf /var/lib/apt/lists/*

Expand Down
2 changes: 1 addition & 1 deletion engine/src/Multicorn
Open in desktop
Submodule Multicorn updated 6 files
+5 −0 python/multicorn/__init__.py
+23 −50 src/deparse.c
+70 −23 src/multicorn.c
+14 −8 src/multicorn.h
+42 −26 src/python.c
+4 −4 src/query.c
2 changes: 1 addition & 1 deletion engine/src/postgres-elasticsearch-fdw
Open in desktop
Submodule postgres-elasticsearch-fdw updated 2 files
+14 −1 pg_es_fdw/__init__.py
+48 −15 pg_es_fdw/_es_query.py
0 ...in_combinations/account_join_sub_aggs.yml → ...in_combinations/account_join_sub_aggs.yml
View file
Open in desktop
File renamed without changes.
0 ...ggregations_bare/account_count_by_age.yml → ...ggregations_bare/account_count_by_age.yml
View file
Open in desktop
File renamed without changes.
Loading