Skip to content

feat: Add Snowflake materialization engine#2948

Merged
achals merged 1 commit intofeast-dev:masterfrom
sfc-gh-madkins:snowflake_materialize
Aug 18, 2022
Merged

feat: Add Snowflake materialization engine#2948
achals merged 1 commit intofeast-dev:masterfrom
sfc-gh-madkins:snowflake_materialize

Conversation

@sfc-gh-madkins
Copy link
Collaborator

@sfc-gh-madkins sfc-gh-madkins commented Jul 16, 2022

What this PR does / why we need it:

Allows Snowflake to be a materialization option when Snowflake is used as an offline store

Which issue(s) this PR fixes:

Fixes #2633

@codecov-commenter
Copy link

codecov-commenter commented Jul 16, 2022

Codecov Report

Merging #2948 (5d0cf95) into master (83c5efb) will decrease coverage by 9.28%.
The diff coverage is 33.42%.

@@            Coverage Diff             @@
##           master    #2948      +/-   ##
==========================================
- Coverage   67.08%   57.80%   -9.29%     
==========================================
  Files         173      209      +36     
  Lines       15132    17380    +2248     
==========================================
- Hits        10152    10046     -106     
- Misses       4980     7334    +2354     
Flag Coverage Δ
integrationtests ?
unittests 57.80% <33.42%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...hon/feast/infra/offline_stores/snowflake_source.py 63.04% <14.70%> (-29.27%) ⬇️
...hon/feast/infra/utils/snowflake/snowflake_utils.py 18.96% <20.51%> (ø)
...on/feast/infra/materialization/snowflake_engine.py 34.37% <34.37%> (ø)
...ests/integration/materialization/test_snowflake.py 38.09% <38.09%> (ø)
sdk/python/feast/infra/offline_stores/snowflake.py 37.19% <46.15%> (-47.14%) ⬇️
sdk/python/feast/type_map.py 46.54% <46.66%> (-13.75%) ⬇️
...python/feast/infra/offline_stores/offline_store.py 67.64% <50.00%> (-15.36%) ⬇️
sdk/python/feast/repo_config.py 76.62% <50.00%> (-6.77%) ⬇️
sdk/python/feast/infra/online_stores/snowflake.py 47.66% <71.42%> (-50.51%) ⬇️
.../feature_repos/universal/data_sources/snowflake.py 51.21% <100.00%> (-48.79%) ⬇️
... and 165 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Member

@achals achals left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks reasonable to me

Copy link
Collaborator

@kevjumba kevjumba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just need some documentation and a few nits.

@sfc-gh-madkins sfc-gh-madkins force-pushed the snowflake_materialize branch 2 times, most recently from 905cdb4 to 88984d9 Compare July 30, 2022 13:13
@sfc-gh-madkins sfc-gh-madkins force-pushed the snowflake_materialize branch from 88984d9 to cf2e3a0 Compare July 30, 2022 14:20
@sfc-gh-madkins sfc-gh-madkins force-pushed the snowflake_materialize branch 2 times, most recently from db2a1b7 to 2c61948 Compare July 31, 2022 02:13
@sfc-gh-madkins sfc-gh-madkins force-pushed the snowflake_materialize branch 5 times, most recently from 747a85f to fe1834e Compare August 2, 2022 02:55
@sfc-gh-madkins sfc-gh-madkins force-pushed the snowflake_materialize branch 2 times, most recently from 456a520 to b42c452 Compare August 16, 2022 04:52
@sfc-gh-madkins sfc-gh-madkins force-pushed the snowflake_materialize branch 6 times, most recently from a0b28b7 to ff3c697 Compare August 17, 2022 17:43
@sfc-gh-madkins sfc-gh-madkins force-pushed the snowflake_materialize branch 4 times, most recently from 7d99d8b to 70eb0f1 Compare August 18, 2022 04:52
Copy link
Member

@achals achals left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@achals
Copy link
Member

achals commented Aug 18, 2022

Tests are red because of a couple of tests that aren't being skipped correctly (since pr-local-integration-tests is triggered on pull_request_target).

Going to force merge this in.

Copy link
Member

@achals achals left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: achals, kevjumba, sfc-gh-madkins

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@achals achals merged commit f3b522b into feast-dev:master Aug 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Snowflake type behavior is strange

6 participants