Release notes from feathr

v1.0.0

2023-02-28T10:04:15Z

We are excited to announce Feathr 1.0.0 is released, please refer to https://feathr-ai.github.io/feathr/release-announcements/v1.0.0.html for details

What's Changed

Clean up some links that's referring to LinkedIn by @xiaoyongzhu in #872
Insert test coverage check for python client into github pipeline by @enya-yx in #862
Create docs on how to update Feathr client and registry, and how to pass credentials by @xiaoyongzhu in #818
Add a custom pytest marker config to pyproject.toml by @loomlike in #786
fix broken link by @xiaoyongzhu in #874
Separate out snowflake source by @aabbasi-hbo in #836
Decouple build feature code by @xiaoyongzhu in #838
Refining example, add utilities, and fix xdist test error by @loomlike in #794
Add 'format' arg to get_result_df by @loomlike in #885
Fix test failure by @jaymo001 in #904
UI add feature the deletion for projects/features/dataSource by @Fendoe in #909
Fixing Bugs reported during oncall by @jainr in #908
Ignore 'registry_utils' in test coverage by @enya-yx in #907
Update azure_resource_provision.json by @xiaoyongzhu in #912
Exclude pegasus jars and release version by @rakeshkashyap123 in #913
Exclude pegasus data files explicitly by @rakeshkashyap123 in #916
Fix auto-tz casting bug by @bozhonghu in #905
Support printing features and returning keys when getting features from registry by @enya-yx in #886
Adding Continuous deployment == ON flag to webapp settings by @jainr in #927
Add use_env_var flag to client by @loomlike in #923
Format docs and add tech talks by @xiaoyongzhu in #931
Add is_synapse() by @loomlike in #929
publish fat jar in maven update action by @Yuqing-cat in #935
Add Spark Configuration Not Applied content by @Yuqing-cat in #928
Support get online features by composite keys by @enya-yx in #919
React best practice implementations in ui code by @blrchen in #938
fix numpy version conflict w/ pyspark by @loomlike in #940
Add presentation for Feathr community talk by @xiaoyongzhu in #941
Add FIRST aggregation for look up feature by @jaymo001 in #917
Disable auth if Feathr is deployed with RBAC set to off by @Fendoe in #925
Update sample notebooks (fraud detection and recommendation examples) to use latest feathr api by @loomlike in #921
Windoze/spark sql source by @windoze in #839
Updates on docs to remove out-of-date contents by @blrchen in #950
Lock python deps version for registry projects by @blrchen in #946
Add pytest cases and check test coverage for sql-registry and purview… by @enya-yx in #937
Fix delete entity function bug for #852 by @Yuqing-cat in #952
Remove hadoop dependency by @rakeshkashyap123 in #949
Create new feature form by @Fendoe in #936
set purview name environment variable in workflow by @enya-yx in #956
Bug fix - Fraud detection sample notebook chart error. by @loomlike in #948
Update registry start script to support REACT_APP_ENABLE_RBAC==false cases by @Yuqing-cat in #954
Improve error message for checking feature keys by @enya-yx in #959
Changed !pip to %pip by @ahlag in #963
Fix registry to support Source with types other than HDFS by @windoze in #965
Create Datasource UI and update create feature form by @Fendoe in #969
Bump version to 0.10.4-rc1 for rc bug bash by @Yuqing-cat in #971
Registry client support SparkSqlSource by @windoze in #970
[BUG] zsh: no matches found: feathr[notebook] during pip install feathr[notebook] by @ahlag in #961
Add a Sandbox for Feathr by @xiaoyongzhu in #966
Add Bacpac creation doc by @xiaoyongzhu in #957
Print out logs for local spark if the job is failed by @xiaoyongzhu in #973
Fix maven publish workflow always skip Gradle publish by @Yuqing-cat in #974
Lock python deps versions in sql registry by @blrchen in #977
Updating automated maven publish workflow to work with gradle creds by @jainr in #979
Fixing Maven Automated Publish by @jainr in #983
Fix config generation by @windoze in #986
Fix environment variable read issues and potential bugs by @xiaoyongzhu in #980
Add docs for Sandbox by @xiaoyongzhu in #981
Fix local submission issues. by @xiaoyongzhu in #988
Fix Dimension Type field is required by @Fendoe in #989
Publish Fat Jar to Blob Storage Steps Decouple and Fix by @Yuqing-cat in #990
Fix duplicated source check by @windoze in #995
Adding azure-cosmos-package to databricks spark config by @jainr in #972
Enable docker workflow to be triggered manually. by @windoze in #997
NYC sample notebook feature re-name by @loomlike in #991
Add docs about registry server test and label by @enya-yx in #993
Revert "Adding azure-cosmos-package to databricks spark config" by @blrchen in #1002
Update fraud detection sample notebook by @loomlike in #984
Add additional maven repo url to databricks submission param by @Yuqing-cat in #1001
Hard code json-schema dependency with comment as hot fix by @Yuqing-cat in #1004
Remove use_env_vars arg from FeathrCient by @loomlike in #999
Resolve pyspark / numpy conflicts by @loomlike in #992
Fix Hadoop S3 Key settings by @aabbasi-hbo in #1007
Add retry when data loading fails by @jaymo001 in #930
Fixing documentation error in Synapse sample notebook by @jainr in #1014
Adding CosmosDb as supported Online Store from deployment template by @jainr in #939
Adding documentation for azure.cosmos.spark package and managing spark libraries by @jainr in #1013
Adding dummy entries to feathr_config.yaml file for additional library entries by @jainr in #1012
Adding guidance for picking optimal SKU for Azure SQL DB by @jainr in #1015
Refactor config helper to be cleaner by @loomlike in #1011
Support external/alien MVEL UDF by @jaymo001 in #1018
Updating various sections of documentation based on customer feedback. by @jainr in #1016
Revert "Add retry when data loading fails (#930)" by @jaymo001 in #1021
Skip features optionally if feature data is missing by @rakeshkashyap123 in #1019
Fix bug when SWA feature datapath does not end in daily/hourly by @rakeshkashyap123 in #1027
Skip Anchored and derived features if the feature data is unavailable by @rakeshkashyap123 in #1026
Upgrade log4j from 1.x to 2.x by @Yuqing-cat in #1031
Add optional retry mechanism if data loading fails by @rakeshkashyap123 in #1034
Auto Delete Empty Anchors and Sources in SQL Registry by @Yuqing-cat in #1023
Exclude test_feathr_materialize_to_cosmosdb from ci Databricks test by @blrchen in #1037
Add registry endpoint setting description to the quickstart notebook by @loomlike in #1038
Fix sandbox issues and improve experience by @xiaoyongzhu in #1008
Fix data sample file link by @loomlike in #1042
Remove json schema validator by @windoze in #1043
Fix NPE of MVEL UDFs by @jaymo001 in #1039
Extend local spark retry timeout by @xiaoyongzhu in #1046
Cancel any in progress CI from the same PR, branch when a new CI is triggered by push by @blrchen in #1049
Fix upload config file issue by @enya-yx in #1044
Bump rc version number by @rakeshkashyap123 in #1047
Bump python client and ui version to 0.10.4-rc5 by @blrchen in #1051
Fix nyc_taxi data utils bug in databricks path by @loomlike in #1050
Fix #1055 by @windoze in #1056
Invoke mvelContext for SimpleMvelDerivedFeatures by @rakeshkashyap123 in #1054
Skip anchored and derived features by @rakeshkashyap123 in #1052
Suppress error caused by numpy breaking change by @windoze in #1057
Update jdbc-cosmos-notes.md by @xiaoyongzhu in #1058
Remove PRs that will fail due to lack of credentials by @xiaoyongzhu in #1060
Add test cases for sources and sinks by @enya-yx in #1045
Fix Selected project state should be persisted globally by @Fendoe in #1062
Add DSL generation api in python client by @windoze in #1029
Allow empty config by @jaymo001 in #1066
Add ability to extract features without any type conversion by @rakeshkashyap123 in #1059
Feature names conflicts check and auto-correction by @enya-yx in #1024
Bump version to 1.0.0-rc1 by @blrchen in #1065
Fix Feathr Build Failures by Disabling the irrelevant download by @xiaoyongzhu in #1068
Block merging PRs with test cases failures by @enya-yx in #1064
Debug and sanity mode by @jaymo001 in #1069
Fix data source issue by @rakeshkashyap123 in #1070
WebUI: Fix role management route by @Fendoe in #1071
Fix sandbox build issues by @xiaoyongzhu in #1077
Add code samples for time partition pattern and name conflicts by @enya-yx in #1073
Feathr 1.0.0 release note doc by @blrchen in #1079
Fix some markdown content render issue on doc site by @blrchen in #1080
Updates on release guide doc by @blrchen in #1072
Update local notebooks by @xiaoyongzhu in #1081
Fix notebook UI issues in Sandbox by @xiaoyongzhu in #1082
Minor fix for API cross domain issue by @xiaoyongzhu in #1083
Disable paths-ignore for pull-request-target by @enya-yx in #1078
Improve local sandbox setup and fix registry 404 regression by @blrchen in #1088
Adjust test status by @enya-yx in #1085
Update feathr-ui.png in README by @blrchen in #1089
Add utility to convert avro tensor to tensor format by @rakeshkashyap123 in #1086
Version bump by @rakeshkashyap123 in #1087
Fail test jobs with low coverage by @enya-yx in #1091
Bump version to 1.0.0-rc4 by @blrchen in #1094
Update the sandbox docker run command in docs to use release tag by @blrchen in #1095
Shorten GitHub Actions and Workflows names and fix broken links in docs by @blrchen in #1096
Clean up keys in redis cache created by tests to improve performance by @enya-yx in #1100
Bump version to 1.0.0 by @blrchen in #1104

Full Changelog: v0.9.0...v1.0.0

v1.0.0-rc4

2023-02-28T03:55:13Z

What's Changed

Clean up some links that's referring to LinkedIn by @xiaoyongzhu in #872
Insert test coverage check for python client into github pipeline by @enya-yx in #862
Create docs on how to update Feathr client and registry, and how to pass credentials by @xiaoyongzhu in #818
Add a custom pytest marker config to pyproject.toml by @loomlike in #786
fix broken link by @xiaoyongzhu in #874
Separate out snowflake source by @aabbasi-hbo in #836
Decouple build feature code by @xiaoyongzhu in #838
Refining example, add utilities, and fix xdist test error by @loomlike in #794
Add 'format' arg to get_result_df by @loomlike in #885
Fix test failure by @jaymo001 in #904
UI add feature the deletion for projects/features/dataSource by @Fendoe in #909
Fixing Bugs reported during oncall by @jainr in #908
Ignore 'registry_utils' in test coverage by @enya-yx in #907
Update azure_resource_provision.json by @xiaoyongzhu in #912
Exclude pegasus jars and release version by @rakeshkashyap123 in #913
Exclude pegasus data files explicitly by @rakeshkashyap123 in #916
Fix auto-tz casting bug by @bozhonghu in #905
Support printing features and returning keys when getting features from registry by @enya-yx in #886
Adding Continuous deployment == ON flag to webapp settings by @jainr in #927
Add use_env_var flag to client by @loomlike in #923
Format docs and add tech talks by @xiaoyongzhu in #931
Add is_synapse() by @loomlike in #929
publish fat jar in maven update action by @Yuqing-cat in #935
Add Spark Configuration Not Applied content by @Yuqing-cat in #928
Support get online features by composite keys by @enya-yx in #919
React best practice implementations in ui code by @blrchen in #938
fix numpy version conflict w/ pyspark by @loomlike in #940
Add presentation for Feathr community talk by @xiaoyongzhu in #941
Add FIRST aggregation for look up feature by @jaymo001 in #917
Disable auth if Feathr is deployed with RBAC set to off by @Fendoe in #925
Update sample notebooks (fraud detection and recommendation examples) to use latest feathr api by @loomlike in #921
Windoze/spark sql source by @windoze in #839
Updates on docs to remove out-of-date contents by @blrchen in #950
Lock python deps version for registry projects by @blrchen in #946
Add pytest cases and check test coverage for sql-registry and purview… by @enya-yx in #937
Fix delete entity function bug for #852 by @Yuqing-cat in #952
Remove hadoop dependency by @rakeshkashyap123 in #949
Create new feature form by @Fendoe in #936
set purview name environment variable in workflow by @enya-yx in #956
Bug fix - Fraud detection sample notebook chart error. by @loomlike in #948
Update registry start script to support REACT_APP_ENABLE_RBAC==false cases by @Yuqing-cat in #954
Improve error message for checking feature keys by @enya-yx in #959
Changed !pip to %pip by @ahlag in #963
Fix registry to support Source with types other than HDFS by @windoze in #965
Create Datasource UI and update create feature form by @Fendoe in #969
Bump version to 0.10.4-rc1 for rc bug bash by @Yuqing-cat in #971
Registry client support SparkSqlSource by @windoze in #970
[BUG] zsh: no matches found: feathr[notebook] during pip install feathr[notebook] by @ahlag in #961
Add a Sandbox for Feathr by @xiaoyongzhu in #966
Add Bacpac creation doc by @xiaoyongzhu in #957
Print out logs for local spark if the job is failed by @xiaoyongzhu in #973
Fix maven publish workflow always skip Gradle publish by @Yuqing-cat in #974
Lock python deps versions in sql registry by @blrchen in #977
Updating automated maven publish workflow to work with gradle creds by @jainr in #979
Fixing Maven Automated Publish by @jainr in #983
Fix config generation by @windoze in #986
Fix environment variable read issues and potential bugs by @xiaoyongzhu in #980
Add docs for Sandbox by @xiaoyongzhu in #981
Fix local submission issues. by @xiaoyongzhu in #988
Fix Dimension Type field is required by @Fendoe in #989
Publish Fat Jar to Blob Storage Steps Decouple and Fix by @Yuqing-cat in #990
Fix duplicated source check by @windoze in #995
Adding azure-cosmos-package to databricks spark config by @jainr in #972
Enable docker workflow to be triggered manually. by @windoze in #997
NYC sample notebook feature re-name by @loomlike in #991
Add docs about registry server test and label by @enya-yx in #993
Revert "Adding azure-cosmos-package to databricks spark config" by @blrchen in #1002
Update fraud detection sample notebook by @loomlike in #984
Add additional maven repo url to databricks submission param by @Yuqing-cat in #1001
Hard code json-schema dependency with comment as hot fix by @Yuqing-cat in #1004
Remove use_env_vars arg from FeathrCient by @loomlike in #999
Resolve pyspark / numpy conflicts by @loomlike in #992
Fix Hadoop S3 Key settings by @aabbasi-hbo in #1007
Add retry when data loading fails by @jaymo001 in #930
Fixing documentation error in Synapse sample notebook by @jainr in #1014
Adding CosmosDb as supported Online Store from deployment template by @jainr in #939
Adding documentation for azure.cosmos.spark package and managing spark libraries by @jainr in #1013
Adding dummy entries to feathr_config.yaml file for additional library entries by @jainr in #1012
Adding guidance for picking optimal SKU for Azure SQL DB by @jainr in #1015
Refactor config helper to be cleaner by @loomlike in #1011
Support external/alien MVEL UDF by @jaymo001 in #1018
Updating various sections of documentation based on customer feedback. by @jainr in #1016
Revert "Add retry when data loading fails (#930)" by @jaymo001 in #1021
Skip features optionally if feature data is missing by @rakeshkashyap123 in #1019
Fix bug when SWA feature datapath does not end in daily/hourly by @rakeshkashyap123 in #1027
Skip Anchored and derived features if the feature data is unavailable by @rakeshkashyap123 in #1026
Upgrade log4j from 1.x to 2.x by @Yuqing-cat in #1031
Add optional retry mechanism if data loading fails by @rakeshkashyap123 in #1034
Auto Delete Empty Anchors and Sources in SQL Registry by @Yuqing-cat in #1023
Exclude test_feathr_materialize_to_cosmosdb from ci Databricks test by @blrchen in #1037
Add registry endpoint setting description to the quickstart notebook by @loomlike in #1038

Full Changelog: v0.9.0...v0.10.4-rc5

What's Changed

Clean up some links that's referring to LinkedIn by @xiaoyongzhu in #872
Insert test coverage check for python client into github pipeline by @enya-yx in #862
Create docs on how to update Feathr client and registry, and how to pass credentials by @xiaoyongzhu in #818
Add a custom pytest marker config to pyproject.toml by @loomlike in #786
fix broken link by @xiaoyongzhu in #874
Separate out snowflake source by @aabbasi-hbo in #836
Decouple build feature code by @xiaoyongzhu in #838
Refining example, add utilities, and fix xdist test error by @loomlike in #794
Add 'format' arg to get_result_df by @loomlike in #885
Fix test failure by @jaymo001 in #904
UI add feature the deletion for projects/features/dataSource by @Fendoe in #909
Fixing Bugs reported during oncall by @jainr in #908
Ignore 'registry_utils' in test coverage by @enya-yx in #907
Update azure_resource_provision.json by @xiaoyongzhu in #912
Exclude pegasus jars and release version by @rakeshkashyap123 in #913
Exclude pegasus data files explicitly by @rakeshkashyap123 in #916
Fix auto-tz casting bug by @bozhonghu in #905
Support printing features and returning keys when getting features from registry by @enya-yx in #886
Adding Continuous deployment == ON flag to webapp settings by @jainr in #927
Add use_env_var flag to client by @loomlike in #923
Format docs and add tech talks by @xiaoyongzhu in #931
Add is_synapse() by @loomlike in #929
publish fat jar in maven update action by @Yuqing-cat in #935
Add Spark Configuration Not Applied content by @Yuqing-cat in #928
Support get online features by composite keys by @enya-yx in #919
React best practice implementations in ui code by @blrchen in #938
fix numpy version conflict w/ pyspark by @loomlike in #940
Add presentation for Feathr community talk by @xiaoyongzhu in #941
Add FIRST aggregation for look up feature by @jaymo001 in #917
Disable auth if Feathr is deployed with RBAC set to off by @Fendoe in #925
Update sample notebooks (fraud detection and recommendation examples) to use latest feathr api by @loomlike in #921
Windoze/spark sql source by @windoze in #839
Updates on docs to remove out-of-date contents by @blrchen in #950
Lock python deps version for registry projects by @blrchen in #946
Add pytest cases and check test coverage for sql-registry and purview… by @enya-yx in #937
Fix delete entity function bug for #852 by @Yuqing-cat in #952
Remove hadoop dependency by @rakeshkashyap123 in #949
Create new feature form by @Fendoe in #936
set purview name environment variable in workflow by @enya-yx in #956
Bug fix - Fraud detection sample notebook chart error. by @loomlike in #948
Update registry start script to support REACT_APP_ENABLE_RBAC==false cases by @Yuqing-cat in #954
Improve error message for checking feature keys by @enya-yx in #959
Changed !pip to %pip by @ahlag in #963
Fix registry to support Source with types other than HDFS by @windoze in #965
Create Datasource UI and update create feature form by @Fendoe in #969
Bump version to 0.10.4-rc1 for rc bug bash by @Yuqing-cat in #971
Registry client support SparkSqlSource by @windoze in #970
[BUG] zsh: no matches found: feathr[notebook] during pip install feathr[notebook] by @ahlag in #961
Add a Sandbox for Feathr by @xiaoyongzhu in #966
Add Bacpac creation doc by @xiaoyongzhu in #957
Print out logs for local spark if the job is failed by @xiaoyongzhu in #973
Fix maven publish workflow always skip Gradle publish by @Yuqing-cat in #974
Lock python deps versions in sql registry by @blrchen in #977
Updating automated maven publish workflow to work with gradle creds by @jainr in #979
Fixing Maven Automated Publish by @jainr in #983
Fix config generation by @windoze in #986
Fix environment variable read issues and potential bugs by @xiaoyongzhu in #980
Add docs for Sandbox by @xiaoyongzhu in #981
Fix local submission issues. by @xiaoyongzhu in #988
Fix Dimension Type field is required by @Fendoe in #989
Publish Fat Jar to Blob Storage Steps Decouple and Fix by @Yuqing-cat in #990
Fix duplicated source check by @windoze in #995
Adding azure-cosmos-package to databricks spark config by @jainr in #972
Enable docker workflow to be triggered manually. by @windoze in #997
NYC sample notebook feature re-name by @loomlike in #991
Add docs about registry server test and label by @enya-yx in #993
Revert "Adding azure-cosmos-package to databricks spark config" by @blrchen in #1002
Update fraud detection sample notebook by @loomlike in #984
Add additional maven repo url to databricks submission param by @Yuqing-cat in #1001
Hard code json-schema dependency with comment as hot fix by @Yuqing-cat in #1004
Remove use_env_vars arg from FeathrCient by @loomlike in #999
Resolve pyspark / numpy conflicts by @loomlike in #992
Fix Hadoop S3 Key settings by @aabbasi-hbo in #1007
Add retry when data loading fails by @jaymo001 in #930
Fixing documentation error in Synapse sample notebook by @jainr in #1014
Adding CosmosDb as supported Online Store from deployment template by @jainr in #939
Adding documentation for azure.cosmos.spark package and managing spark libraries by @jainr in #1013
Adding dummy entries to feathr_config.yaml file for additional library entries by @jainr in #1012
Adding guidance for picking optimal SKU for Azure SQL DB by @jainr in #1015
Refactor config helper to be cleaner by @loomlike in #1011
Support external/alien MVEL UDF by @jaymo001 in #1018
Updating various sections of documentation based on customer feedback. by @jainr in #1016
Revert "Add retry when data loading fails (#930)" by @jaymo001 in #1021
Skip features optionally if feature data is missing by @rakeshkashyap123 in #1019
Fix bug when SWA feature datapath does not end in daily/hourly by @rakeshkashyap123 in #1027
Skip Anchored and derived features if the feature data is unavailable by @rakeshkashyap123 in #1026
Upgrade log4j from 1.x to 2.x by @Yuqing-cat in #1031
Add optional retry mechanism if data loading fails by @rakeshkashyap123 in #1034
Auto Delete Empty Anchors and Sources in SQL Registry by @Yuqing-cat in #1023
Exclude test_feathr_materialize_to_cosmosdb from ci Databricks test by @blrchen in #1037
Add registry endpoint setting description to the quickstart notebook by @loomlike in #1038
Fix sandbox issues and improve experience by @xiaoyongzhu in #1008
Fix data sample file link by @loomlike in #1042
Remove json schema validator by @windoze in #1043
Fix NPE of MVEL UDFs by @jaymo001 in #1039
Extend local spark retry timeout by @xiaoyongzhu in #1046
Cancel any in progress CI from the same PR, branch when a new CI is triggered by push by @blrchen in #1049
Fix upload config file issue by @enya-yx in #1044
Bump rc version number by @rakeshkashyap123 in #1047
Bump python client and ui version to 0.10.4-rc5 by @blrchen in #1051
Fix nyc_taxi data utils bug in databricks path by @loomlike in #1050
Fix #1055 by @windoze in #1056
Invoke mvelContext for SimpleMvelDerivedFeatures by @rakeshkashyap123 in #1054
Skip anchored and derived features by @rakeshkashyap123 in #1052
Suppress error caused by numpy breaking change by @windoze in #1057
Update jdbc-cosmos-notes.md by @xiaoyongzhu in #1058
Remove PRs that will fail due to lack of credentials by @xiaoyongzhu in #1060
Add test cases for sources and sinks by @enya-yx in #1045
Fix Selected project state should be persisted globally by @Fendoe in #1062
Add DSL generation api in python client by @windoze in #1029
Allow empty config by @jaymo001 in #1066
Add ability to extract features without any type conversion by @rakeshkashyap123 in #1059
Feature names conflicts check and auto-correction by @enya-yx in #1024
Bump version to 1.0.0-rc1 by @blrchen in #1065

Full Changelog: v0.9.0...v1.0.0-rc1

v1.0.0-rc1

2023-02-14T06:04:18Z

Bump version to 1.0.0-rc1 (#1065)

v0.10.4-rc5

2023-02-08T09:19:09Z

Bump version for maven test

v0.10.4-rc3

2023-01-17T08:52:15Z

What's Changed

Clean up some links that's referring to LinkedIn by @xiaoyongzhu in #872
Insert test coverage check for python client into github pipeline by @enya-yx in #862
Create docs on how to update Feathr client and registry, and how to pass credentials by @xiaoyongzhu in #818
Add a custom pytest marker config to pyproject.toml by @loomlike in #786
fix broken link by @xiaoyongzhu in #874
Separate out snowflake source by @aabbasi-hbo in #836
Decouple build feature code by @xiaoyongzhu in #838
Refining example, add utilities, and fix xdist test error by @loomlike in #794
Add 'format' arg to get_result_df by @loomlike in #885
Fix test failure by @jaymo001 in #904
UI add feature the deletion for projects/features/dataSource by @Fendoe in #909
Fixing Bugs reported during oncall by @jainr in #908
Ignore 'registry_utils' in test coverage by @enya-yx in #907
Update azure_resource_provision.json by @xiaoyongzhu in #912
Exclude pegasus jars and release version by @rakeshkashyap123 in #913
Exclude pegasus data files explicitly by @rakeshkashyap123 in #916
Fix auto-tz casting bug by @bozhonghu in #905
Support printing features and returning keys when getting features from registry by @enya-yx in #886
Adding Continuous deployment == ON flag to webapp settings by @jainr in #927
Add use_env_var flag to client by @loomlike in #923
Format docs and add tech talks by @xiaoyongzhu in #931
Add is_synapse() by @loomlike in #929
publish fat jar in maven update action by @Yuqing-cat in #935
Add Spark Configuration Not Applied content by @Yuqing-cat in #928
Support get online features by composite keys by @enya-yx in #919
React best practice implementations in ui code by @blrchen in #938
fix numpy version conflict w/ pyspark by @loomlike in #940
Add presentation for Feathr community talk by @xiaoyongzhu in #941
Add FIRST aggregation for look up feature by @jaymo001 in #917
Disable auth if Feathr is deployed with RBAC set to off by @Fendoe in #925
Update sample notebooks (fraud detection and recommendation examples) to use latest feathr api by @loomlike in #921
Windoze/spark sql source by @windoze in #839
Updates on docs to remove out-of-date contents by @blrchen in #950
Lock python deps version for registry projects by @blrchen in #946
Add pytest cases and check test coverage for sql-registry and purview… by @enya-yx in #937
Fix delete entity function bug for #852 by @Yuqing-cat in #952
Remove hadoop dependency by @rakeshkashyap123 in #949
Create new feature form by @Fendoe in #936
set purview name environment variable in workflow by @enya-yx in #956
Bug fix - Fraud detection sample notebook chart error. by @loomlike in #948
Update registry start script to support REACT_APP_ENABLE_RBAC==false cases by @Yuqing-cat in #954
Improve error message for checking feature keys by @enya-yx in #959
Changed !pip to %pip by @ahlag in #963
Fix registry to support Source with types other than HDFS by @windoze in #965
Create Datasource UI and update create feature form by @Fendoe in #969
Bump version to 0.10.4-rc1 for rc bug bash by @Yuqing-cat in #971
Registry client support SparkSqlSource by @windoze in #970
[BUG] zsh: no matches found: feathr[notebook] during pip install feathr[notebook] by @ahlag in #961
Add a Sandbox for Feathr by @xiaoyongzhu in #966
Add Bacpac creation doc by @xiaoyongzhu in #957
Print out logs for local spark if the job is failed by @xiaoyongzhu in #973
Fix maven publish workflow always skip Gradle publish by @Yuqing-cat in #974

Full Changelog: v0.9.0...v0.10.4-rc3

0.10.4-rc2

2023-01-17T08:08:51Z

What's Changed

Clean up some links that's referring to LinkedIn by @xiaoyongzhu in #872
Insert test coverage check for python client into github pipeline by @enya-yx in #862
Create docs on how to update Feathr client and registry, and how to pass credentials by @xiaoyongzhu in #818
Add a custom pytest marker config to pyproject.toml by @loomlike in #786
fix broken link by @xiaoyongzhu in #874
Separate out snowflake source by @aabbasi-hbo in #836
Decouple build feature code by @xiaoyongzhu in #838
Refining example, add utilities, and fix xdist test error by @loomlike in #794
Add 'format' arg to get_result_df by @loomlike in #885
Fix test failure by @jaymo001 in #904
UI add feature the deletion for projects/features/dataSource by @Fendoe in #909
Fixing Bugs reported during oncall by @jainr in #908
Ignore 'registry_utils' in test coverage by @enya-yx in #907
Update azure_resource_provision.json by @xiaoyongzhu in #912
Exclude pegasus jars and release version by @rakeshkashyap123 in #913
Exclude pegasus data files explicitly by @rakeshkashyap123 in #916
Fix auto-tz casting bug by @bozhonghu in #905
Support printing features and returning keys when getting features from registry by @enya-yx in #886
Adding Continuous deployment == ON flag to webapp settings by @jainr in #927
Add use_env_var flag to client by @loomlike in #923
Format docs and add tech talks by @xiaoyongzhu in #931
Add is_synapse() by @loomlike in #929
publish fat jar in maven update action by @Yuqing-cat in #935
Add Spark Configuration Not Applied content by @Yuqing-cat in #928
Support get online features by composite keys by @enya-yx in #919
React best practice implementations in ui code by @blrchen in #938
fix numpy version conflict w/ pyspark by @loomlike in #940
Add presentation for Feathr community talk by @xiaoyongzhu in #941
Add FIRST aggregation for look up feature by @jaymo001 in #917
Disable auth if Feathr is deployed with RBAC set to off by @Fendoe in #925
Update sample notebooks (fraud detection and recommendation examples) to use latest feathr api by @loomlike in #921
Windoze/spark sql source by @windoze in #839
Updates on docs to remove out-of-date contents by @blrchen in #950
Lock python deps version for registry projects by @blrchen in #946
Add pytest cases and check test coverage for sql-registry and purview… by @enya-yx in #937
Fix delete entity function bug for #852 by @Yuqing-cat in #952
Remove hadoop dependency by @rakeshkashyap123 in #949
Create new feature form by @Fendoe in #936
set purview name environment variable in workflow by @enya-yx in #956
Bug fix - Fraud detection sample notebook chart error. by @loomlike in #948
Update registry start script to support REACT_APP_ENABLE_RBAC==false cases by @Yuqing-cat in #954
Improve error message for checking feature keys by @enya-yx in #959
Changed !pip to %pip by @ahlag in #963
Fix registry to support Source with types other than HDFS by @windoze in #965
Create Datasource UI and update create feature form by @Fendoe in #969
Bump version to 0.10.4-rc1 for rc bug bash by @Yuqing-cat in #971
Registry client support SparkSqlSource by @windoze in #970
[BUG] zsh: no matches found: feathr[notebook] during pip install feathr[notebook] by @ahlag in #961
Add a Sandbox for Feathr by @xiaoyongzhu in #966
Add Bacpac creation doc by @xiaoyongzhu in #957

Full Changelog: v0.9.0...v0.10.4-rc2

0.10.4-rc1

2023-01-17T07:21:21Z

What's Changed

Clean up some links that's referring to LinkedIn by @xiaoyongzhu in #872
Insert test coverage check for python client into github pipeline by @enya-yx in #862
Create docs on how to update Feathr client and registry, and how to pass credentials by @xiaoyongzhu in #818
Add a custom pytest marker config to pyproject.toml by @loomlike in #786
fix broken link by @xiaoyongzhu in #874
Separate out snowflake source by @aabbasi-hbo in #836
Decouple build feature code by @xiaoyongzhu in #838
Refining example, add utilities, and fix xdist test error by @loomlike in #794
Add 'format' arg to get_result_df by @loomlike in #885
Fix test failure by @jaymo001 in #904
UI add feature the deletion for projects/features/dataSource by @Fendoe in #909
Fixing Bugs reported during oncall by @jainr in #908
Ignore 'registry_utils' in test coverage by @enya-yx in #907
Update azure_resource_provision.json by @xiaoyongzhu in #912
Exclude pegasus jars and release version by @rakeshkashyap123 in #913
Exclude pegasus data files explicitly by @rakeshkashyap123 in #916
Fix auto-tz casting bug by @bozhonghu in #905
Support printing features and returning keys when getting features from registry by @enya-yx in #886
Adding Continuous deployment == ON flag to webapp settings by @jainr in #927
Add use_env_var flag to client by @loomlike in #923
Format docs and add tech talks by @xiaoyongzhu in #931
Add is_synapse() by @loomlike in #929
publish fat jar in maven update action by @Yuqing-cat in #935
Add Spark Configuration Not Applied content by @Yuqing-cat in #928
Support get online features by composite keys by @enya-yx in #919
React best practice implementations in ui code by @blrchen in #938
fix numpy version conflict w/ pyspark by @loomlike in #940
Add presentation for Feathr community talk by @xiaoyongzhu in #941
Add FIRST aggregation for look up feature by @jaymo001 in #917
Disable auth if Feathr is deployed with RBAC set to off by @Fendoe in #925
Update sample notebooks (fraud detection and recommendation examples) to use latest feathr api by @loomlike in #921
Windoze/spark sql source by @windoze in #839
Updates on docs to remove out-of-date contents by @blrchen in #950
Lock python deps version for registry projects by @blrchen in #946
Add pytest cases and check test coverage for sql-registry and purview… by @enya-yx in #937
Fix delete entity function bug for #852 by @Yuqing-cat in #952
Remove hadoop dependency by @rakeshkashyap123 in #949
Create new feature form by @Fendoe in #936
set purview name environment variable in workflow by @enya-yx in #956
Bug fix - Fraud detection sample notebook chart error. by @loomlike in #948
Update registry start script to support REACT_APP_ENABLE_RBAC==false cases by @Yuqing-cat in #954
Improve error message for checking feature keys by @enya-yx in #959
Changed !pip to %pip by @ahlag in #963
Fix registry to support Source with types other than HDFS by @windoze in #965
Create Datasource UI and update create feature form by @Fendoe in #969
Bump version to 0.10.4-rc1 for rc bug bash by @Yuqing-cat in #971
Registry client support SparkSqlSource by @windoze in #970
[BUG] zsh: no matches found: feathr[notebook] during pip install feathr[notebook] by @ahlag in #961
Add a Sandbox for Feathr by @xiaoyongzhu in #966

Full Changelog: v0.9.0...v0.10.4-rc1

Feathr Online Transform (alpha)

2022-12-15T16:04:08Z

Overview

As of v0.10.0, Feathr team is happy to introduce Online Transform (alpha). You might want to try Online Transform in the following scenarios:

Featurization source for transformation is only available at inference time.
Pre-compute features with offline transform might be a waste of storage and compute resources.
Decouple featurization work off the upstream online system.
Write Python programs to use the online transformation functions.

Note: Only use alpha build for evaluation. Alpha build is intented for early access and feedback, it might not as stable as GA build and might be changed at any time before GA without notice. Please use alpha build at your risk.

Running Feathr Online Transform locally

Feathr Online Transform can be run locally with docker. Please check out details on README.

Running Feathr Online Transform on Azure

To deploy Feathr Online Transform to Azure, you need to have a AKS cluseter setup on Azure first. Then follow README to deploy helm chart to AKS cluster.

Using Python library

Feathr Online Transform also has a Python library for further development, check out details from the project site

Roadmap

Define transformation once for both online and offline consumption.
Registry integration
Java Library

Additional Note

Source code for Online Transform (alpha) are hosted under personal accounts, Feathr team is working on moving these source codes to the official repo but it might take some time.

v0.9.0

2022-11-18T10:20:54Z

Breaking Changes

We have changed the execution engine for derived features to Spark SQL so this might introduce a little bit breaking changes for users who is not running the up-to-date sample notebooks. Specifically, they might face this failure:

Preprocessed DataFrames are: 
{'feature_user_age,feature_user_gift_card_balance,feature_user_has_valid_credit_card,feature_user_tax_rate': JavaObject id=o243}
Traceback (most recent call last):
  File "feathr_pyspark_driver.py", line 107, in <module>
    submit_spark_job(feature_names_funcs)
  File "feathr_pyspark_driver.py", line 85, in submit_spark_job
    py4j_feature_job.mainWithPreprocessedDataFrame(job_param_java_array, new_preprocessed_df_map)
  File "/home/trusted-service-user/cluster-env/env/lib/python3.8/site-packages/py4j/java_gateway.py", line 1304, in __call__
    return_value = get_return_value(
  File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 117, in deco
pyspark.sql.utils.AnalysisException: Undefined function: 'toBoolean'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.; line 1 pos 84
)

Users should change:

feature_user_purchasing_power = DerivedFeature(name="feature_user_purchasing_power",
                                      key=user_id,
                                      feature_type=FLOAT,
                                      input_features=[
                                          feature_user_gift_card_balance, feature_user_has_valid_credit_card],
                                      transform="feature_user_gift_card_balance + if_else(toBoolean(feature_user_has_valid_credit_card), 100, 0)")

feature_user_purchasing_power = DerivedFeature(name="feature_user_purchasing_power",
                                      key=user_id,
                                      feature_type=FLOAT,
                                      input_features=[
                                          feature_user_gift_card_balance, feature_user_has_valid_credit_card],
                                      transform="feature_user_gift_card_balance + if(boolean(feature_user_has_valid_credit_card), 100, 0)")

What's Changed

Fix a feature type bug by @jaymo001 in #701
Fix wheel building problem in Windows by @xiaoyongzhu in #702
Fix Purview+RBAC registry web app issue by @Yuqing-cat in #700
Remove hard coded resources in docs by @enya-yx in #696
Add e2e test for purview registry and rbac registry by @blrchen in #689
Update tests use runtime jar from maven for spark submission to cover Databricks by @blrchen in #706
Enhance databricks submission error message by @enya-yx in #710
Enhance purview registry error messages by @blrchen in #709
[WIP] hot fix databricks es dependency issue by @Yuqing-cat in #713
Fix materialize to sql e2e test failure by @blrchen in #717
Add Data Models in Feathr by @hyingyang-linkedin in #659
Revert "Enhance purview registry error messages (#709)" by @blrchen in #720
Improve Avro GenericRecord and SpecificRecord based row-level extractor performance by @jaymo001 in #723
Fix lookup feature missing issue when converting feature definition to HOCON files by @jaymo001 in #732
Fix function string parsing by @loomlike in #725
Apply a same credential within each sample [ Docs ] by @enya-yx in #718
Enable incremental for HDFS sink by @enya-yx in #695
#492 fix, fail only if different sources have same name by @windoze in #733
Remove unused credentials and deprecated purview settings by @enya-yx in #708
Revoke adb token submitted by mistaken by @blrchen in #730
Fix synapse errors not print out issue by @enya-yx in #734
Spark config passing bug fix for local spark submission by @loomlike in #729
Fix direct purview client missing transformation by @YihuiGuo in #736
Support SQL expression in derived feature transformation by @jaymo001 in #731
Support SWA with groupBy to 1d tensor conversion by @jaymo001 in #748
Rijai/armfix by @jainr in #742
bump version to 0.8.2 by @Yuqing-cat in #722
Added latest deltalake version by @ahlag in #735
Fix #474 Disable local mode by @windoze in #738
Allow recreating entities for PurView registry by @windoze in #691
Adding DevSkim linter to Github actions by @jainr in #657
Fix icons in UI cannot auto scale (#737) by @Fendoe in #744
Expose 'timePartitionPattern' in Python API [ WIP ] by @enya-yx in #714
Setting up component governance pipeline by @jainr in #655
Add docs to explain on feature materialization behavior by @xiaoyongzhu in #688
Fix protobuf version by @enya-yx in #711
Add some notes based on on-call issues by @enya-yx in #753
Refine spark runtime error message by @Yuqing-cat in #755
Serialization bug due to version incompatibility between azure-core and msrest by @jainr in #763
Unify Python SDK Build Version and decouple Feathr Maven Version by @Yuqing-cat in #746
Replace hard code string in notebook and align with others by @Yuqing-cat in #765
Add flag to enable generation non-agg features by @windoze in #719
roll back 0.8.2 version bump by @Yuqing-cat in #771
Refactor Product Recommendation sample notebook by @jainr in #743
Update role-management page in UI (#751) by @Fendoe in #764
Create Feature less module in UI code and import alias by @Fendoe in #768
Add extra dependencies to setup.py by @loomlike in #773
Fix Windows compatibility issues by @xiaoyongzhu in #776
UI: Replace logo icon by @Fendoe in #778
Refine example notebooks by @loomlike in #756
UI: Display version by @Fendoe in #779
Add nightly Notification to PR Test GitHub Action by @Yuqing-cat in #783
Fix broken links for #743 by @Yuqing-cat in #789
Update notebook image links for github rendering by @loomlike in #787
Revert 756 by @blrchen in #798
remove unnecessary spark job from registry test by @Yuqing-cat in #790
Revert "Expose 'timePartitionPattern' in Python API [ WIP ]" by @blrchen in #799
Update CONTRIBUTING.md with committers information by @hangfei in #793
Fix test_azure_spark_maven_e2e ci test error by @blrchen in #800
Add failure warning and run link to daily notification by @Yuqing-cat in #802
Minor documentation update to add info about maven automated workflow by @jainr in #795
Fix doc dead links by @blrchen in #805
Fix more dead links on docs by @blrchen in #807
Improve UI experience and clean up ui code warnings by @Fendoe in #801
Add release instructions for Release Candidate by @blrchen in #809
Bump version to 0.9.0-rc1 by @blrchen in #810
Fix bug in empty array dense tensor default value by @bozhonghu in #806
Fix sql-based derived feature by @jaymo001 in #812
Replacing webapp-deploy action with workflow-webhook action. by @jainr in #813
Fix passthrough feature reference in sql-based derived feature by @jaymo001 in #815
Revert databricks example notebook until fixing issues by @loomlike in #814
Add retry logic for purview project-ids logic by @Yuqing-cat in #821
Bump version to 0.9.0-rc2 by @blrchen in #822
Fix Not display management menu by @Fendoe in #826
Update text and link by @Fendoe in #828
fix sample issues due to derived feature engine change by @xiaoyongzhu in #829
Add exception if materialize features defined on 'INPUT_CONTEXT' by @enya-yx in #785
Fix only first Key will show even if multiple keys are added by @Fendoe in #837
Move the version information to the bottom of the sidemenu. by @Fendoe in #832
Fix key cannot read properties of undefined (reading 'map') by @Fendoe in #841
Model by @hyingyang-linkedin in #769
Bump loader-utils from 2.0.2 to 2.0.3 in /ui by @dependabot in #846
Maven Package Version Configuration Fix by @Yuqing-cat in #845
Copy/paste typo by @windoze in #849
Update outdated docs (WASB_ to BLOB_) by @loomlike in #850
Update registry nightly deploy CICD by @blrchen in #853
Windoze/purview registry error log by @windoze in #851
Fix duplicate action id in registry CICD by @blrchen in #854
Improve Feathr Client initialization logs by @blrchen in #856
Enhance error messages of synapse jobs by @enya-yx in #855
Fix avro files read failure under timePartitionPattern paths by @enya-yx in #808
Bump version to 0.9.0-rc3 by @blrchen in #860
Enhance sample notebook by @enya-yx in #848

New Contributors

@hyingyang-linkedin made their first contribution in #659
@loomlike made their first contribution in #725
@Fendoe made their first contribution in #744
@bozhonghu made their first contribution in #806

Full Changelog: v0.8.0...v0.9.0

v0.9.0-rc3: Enhance sample notebook (#848)

2022-11-16T03:37:16Z

Enhance sample notebook to solve some issues in synapse