All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Remove redundant log to reduce noise
- Upgraded internal dependency
redisto versionredis >=7.1.0, <8.0.0 - Bumped a supported SDK runtime version to minimum
3.10since new redis lib drops support for3.9
- Upgraded internal dependency
urllib3to versionurllib3 >= 2.6.3, <3.0.0since2.5.0has these vulnerabilities
- Fixed issue with
MAX_RETRY_COUNT
- Fix some deprecation warnings raised by Pydantic V2
- CI docs build & deploy condition
- Logging duplication on CloudWatch side
- Full migration from old
v1ofpydantictov2dependencies:pydanticitself:"pydantic >= 2.0, <3.0"pydantic-settings:"pydantic-settings >=2.0, <3.0"
- Revamp Redis, get rid of LUA scripts at all
- Add
py.typedfor type checking - Upgrading a bunch of dependencies
- Upgraded internal dependency
urllib3to version2.5.0
- Removed support of Python version
3.8
- added back Api attributes removed in a previous release, this should fix potential breaking changes introduced in 1.14.1;
- Session mechanism for significantly decrease number of an http load on data-api for apps with intensive calling
- Added possibility to adjust some params related to connection pool
POOL_CONNECTIONS_COUNT: Total pools countPOOL_MAX_SIZE: Max connections count per pool/hostPOOL_BLOCK: Wait until connection released or not (instantly raise an exception)MAX_RETRY_COUNT: If 0 then retires will be disabled, otherwise retrying logic will be used
- Move retrying logic from
tenacityto internalurllib3.util.Retry(...) - Removed redundant dependency
tenacityfrompython-sdk - Bump version for
py3.13topy3.13.3at CI version matrix in order to fix broken tests for logging - Bump version for
fakeredisto fix some tests
- merge_events parameter for scheduled data time apps should result in correct start/end times in a final app event.
- Documentation 404 at GitHub pages
- Upgrade a bunch of outdated core dependencies + test & lint dependencies
- Add app logic for meaningful error when mismatched app type is used either declared at
manifest.jsonor according to used event payload passed to app
- Unset the
CORVA_LOGGER.propagate = False, so the OTel handler will be able to collect and send those logs as well refs https://github.com/corva-ai/otel/pull/37
- Now "version" field represents not schema version but rather the version of rerun and can be any integer.
- Filter out records with
Nonedata from stream time records.
- Most of the fields in partial merge events should be optional.
- Fields
partitionandrerun_partitionare not expected in partial merge event payload. - Data structure under
datakey of partial merge event payload can accept additional fields.
- Optional merge_events parameter to @stream and @scheduled decorators. Default is False, if True - merge all incoming events into one.
- Added functionality to make retries for the failed HTTP requests.
- New handler to process partial rerun merge events
- issue with
insert_datamethod not working when trying to insert data into any dataset.
- Limit number of available Cache connections to 1
urllib3v2 not supported by AWS Lambda.
log_identifierfield to stream depth events.
- Reraise exceptions in
taskapps instead of suppressing them.
idfield to rerun data.
insert_datamethod toApi. SeeInsert datasection in docs.- Example usage for followable apps.
See
Followable appssection in docs.
produce_messagesmethod toApi. SeeProduce messagessection in docs.- Documentation on followable apps.
See
Followable appssection in docs.
cacheparameter toapp_runnerto reuse the cache object.
- Set status as completed for failed scheduled natural time apps.
- GitHub Pages
- GitHub Pages
- Rerun support.
StreamTimeEvent,StreamDepthEvent,ScheduledDataTimeEvent,ScheduledDepthEventandScheduledNaturalTimeEventgot newrerunfield which stores rerun metadata.
Cachemethods:set_manyget_manyget_alldelete_manydelete_all
- Wrong log formatting in Dev Center: multiline Python logs (like stack traces) being split into multiple CloudWatch log entries.
- Log internal SDK errors using
warninglevel instead ofexception.
get,setand new signature fordeletemethods toCache. See docs for more info.
Cachemethods:- old
deletesignature storeloadload_alldelete_allttlpttlexists
- old
Cacheunable to set unique expiry for each key.
- Wrong log formatting in Dev Center: multiline Python logs (like stack traces) being split into multiple CloudWatch log entries.
- Secrets support. See corresponding section in docs.
- Successful task app failing to update task status.
- Returning dict result from task app to get it stored in task payload. Send the request to update the payload explicitly in your app.
- Subtypes of scheduled event:
ScheduledDataTimeEvent,ScheduledDepthEventandScheduledNaturalTimeEvent
ScheduledEventusage
- Ability to pass custom logging handlers
to app decorators using
handlerkeyword argument. - Natural time event (single dict) parsing in
scheduledapps.
- Multiple logging of the same exception in stream and scheduled apps.
- Log message truncation for low
LOG_THRESHOLD_MESSAGE_SIZEvalues.
- Logging of internal errors.
- AWS Lambda context parsing.
corva.stream,corva.scheduledandcorva.taskapp decorators. See readme for usage examples.ScheduledEvent.company_idfield.
corva.Corvaclass.
- Events are allowed to have extra fields.
corva.Loggerobject, that should be used for app logging.LOG_THRESHOLD_MESSAGE_SIZEandLOG_THRESHOLD_MESSAGE_COUNTenv variables, that should be used to configure logging.
app_runnerfixture for testing apps.
StreamEventwas split intoStreamTimeEventandStreamDepthEvent, which have correspondingStreamTimeRecordandStreamDepthRecordrecords.- Deleted all unsued fields from
ScheduledEvent,TaskEvent,StreamTimeEventandStreamDepthEvent.
filter_modeparameter fromCorva.stream. Filtering is now automatic.
Corva.taskdecorator for task apps.
Testingsection toREADME.md.Api.get_datasetmethod.
ScheduledEvent.schedule_endfield is now optional.ScheduledEvent.schedule_endandScheduledEvent.schedule_startfield types fromdatetimetoint.
- Tools for testing apps.
TaskEventqueue event parsing.
###Changed
StreamEventmust have at least one record.StreamEventandScheduledEvent:- Added descriptions to fields.
- Simplified event structures.
ScheduledEventqueue event parsing.
Apiclass:- Deleted retries.
- Responses do not use
raise_for_statusanymore. - Lowered
default_timeoutto 30 seconds. - Fixed url build exceptions on Windows.
- Obsolete
StreamEventfields:app_version. - Obsolete
ScheduledEventfields:app_version.
api_keyextraction fromcontext.
StreamEventqueue event parsing.
- Required
contextparameter toCorva. - Documentation in
README.md.
Corvaclass, which containsstreamandscheduleddecorators for stream and scheduled apps.
StreamAppandScheduledAppclasses.
- Deployment to PyPI.
###Added
StreamAppto build stream apps.ScheduledAppto build scheduled apps.TaskAppto build task apps.Apiclass to access Platform and Data Corva APIs.Cacheclass to share data between app invokes.- Event classes:
StreamEvent,ScheduledEventandTaskEvent.
[Unreleased] https://github.com/corva-ai/python-sdk/compare/v2.1.1...master
[2.1.1] https://github.com/corva-ai/python-sdk/compare/v2.1.0...v2.1.1 [2.1.0] https://github.com/corva-ai/python-sdk/compare/v1.15.1...v1.16.1 [2.1.0] https://github.com/corva-ai/python-sdk/compare/v2.0.3...v2.1.0 [2.0.3] https://github.com/corva-ai/python-sdk/compare/v2.0.2...v2.0.3 [2.0.2] https://github.com/corva-ai/python-sdk/compare/v2.0.1...v2.0.2 [2.0.1] https://github.com/corva-ai/python-sdk/compare/v2.0.0...v2.0.1 [1.15.1] https://github.com/corva-ai/python-sdk/compare/v1.15.0...v1.15.1 [2.0.0] https://github.com/corva-ai/python-sdk/compare/v1.15.0...v2.0.0 [1.14.2] https://github.com/corva-ai/python-sdk/compare/v1.14.1...v1.14.2 [1.14.1] https://github.com/corva-ai/python-sdk/compare/v1.14.0...v1.14.1 [1.14.0] https://github.com/corva-ai/python-sdk/compare/v1.13.1...v1.14.0 [1.13.1] https://github.com/corva-ai/python-sdk/compare/v1.13.0...v1.13.1 [1.13.0] https://github.com/corva-ai/python-sdk/compare/v1.12.1...v1.13.0 [1.12.1] https://github.com/corva-ai/python-sdk/compare/v1.12.0...v1.12.1 [1.12.0] https://github.com/corva-ai/python-sdk/compare/v1.11.4...v1.12.0 [1.11.4] https://github.com/corva-ai/python-sdk/compare/v1.11.3...v1.11.2 [1.11.3] https://github.com/corva-ai/python-sdk/compare/v1.11.2...v1.11.3 [1.11.2] https://github.com/corva-ai/python-sdk/compare/v1.11.1...v1.11.2 [1.11.1] https://github.com/corva-ai/python-sdk/compare/v1.11.0...v1.11.1 [1.11.0] https://github.com/corva-ai/python-sdk/compare/v1.10.0...v1.11.0 [1.10.0] https://github.com/corva-ai/python-sdk/compare/v1.9.2...v1.10.0 [1.9.2] https://github.com/corva-ai/python-sdk/compare/v1.9.1...v1.9.2 [1.9.1] https://github.com/corva-ai/python-sdk/compare/v1.9.0...v1.9.1 [1.9.0] https://github.com/corva-ai/python-sdk/compare/v1.8.1...v1.9.0 [1.8.1] https://github.com/corva-ai/python-sdk/compare/v1.8.0...v1.8.1 [1.8.0] https://github.com/corva-ai/python-sdk/compare/v1.7.0...v1.8.0 [1.7.0] https://github.com/corva-ai/python-sdk/compare/v1.6.0...v1.7.0 [1.6.0] https://github.com/corva-ai/python-sdk/compare/v1.5.3...v1.6.0 [1.5.3] https://github.com/corva-ai/python-sdk/compare/v1.5.2...v1.5.3 [1.5.2] https://github.com/corva-ai/python-sdk/compare/v1.5.1...v1.5.2 [1.5.1] https://github.com/corva-ai/python-sdk/compare/v1.5.0...v1.5.1 [1.5.0] https://github.com/corva-ai/python-sdk/compare/v1.4.0...v1.5.0 [1.4.0] https://github.com/corva-ai/python-sdk/compare/v1.4.0-rc.1...v1.4.0 [1.4.0-rc.1] https://github.com/corva-ai/python-sdk/compare/v1.3.1...v1.4.0-rc.1 [1.3.1] https://github.com/corva-ai/python-sdk/compare/v1.3.0...v1.3.1 [1.3.0] https://github.com/corva-ai/python-sdk/compare/v1.2.2...v1.3.0 [1.2.2] https://github.com/corva-ai/python-sdk/compare/v1.2.1...v1.2.2 [1.2.1] https://github.com/corva-ai/python-sdk/compare/v1.2.0...v1.2.1 [1.2.0] https://github.com/corva-ai/python-sdk/compare/v1.1.0...v1.2.0 [1.1.0] https://github.com/corva-ai/python-sdk/compare/v1.0.3...v1.1.0 [1.0.3] https://github.com/corva-ai/python-sdk/compare/v1.0.2...v1.0.3 [1.0.2] https://github.com/corva-ai/python-sdk/compare/v1.0.1...v1.0.2 [1.0.1]: https://github.com/corva-ai/python-sdk/compare/v1.0.0...v1.0.1 [1.0.0]: https://github.com/corva-ai/python-sdk/compare/v0.0.18...v1.0.0 [0.0.18]: https://github.com/corva-ai/python-sdk/compare/v0.0.17...v0.0.18 [0.0.17]: https://github.com/corva-ai/python-sdk/compare/v0.0.16...v0.0.17 [0.0.16]: https://github.com/corva-ai/python-sdk/compare/v0.0.15...v0.0.16 [0.0.15]: https://github.com/corva-ai/python-sdk/compare/v0.0.14...v0.0.15 [0.0.14]: https://github.com/corva-ai/python-sdk/compare/v0.0.13...v0.0.14 [0.0.13]: https://github.com/corva-ai/python-sdk/compare/v0.0.12...v0.0.13 [0.0.12]: https://github.com/corva-ai/python-sdk/compare/v0.0.11...v0.0.12 [0.0.11]: https://github.com/corva-ai/python-sdk/compare/v0.0.10...v0.0.11 [0.0.10]: https://github.com/corva-ai/python-sdk/compare/v0.0.9...v0.0.10 [0.0.9]: https://github.com/corva-ai/python-sdk/compare/v0.0.8...v0.0.9 [0.0.8]: https://github.com/corva-ai/python-sdk/compare/v0.0.7...v0.0.8 [0.0.7]: https://github.com/corva-ai/python-sdk/compare/v0.0.5...v0.0.7 [0.0.5]: https://github.com/corva-ai/python-sdk/compare/v0.0.4...v0.0.5 [0.0.4]: https://github.com/corva-ai/python-sdk/compare/v0.0.3...v0.0.4 [0.0.3]: https://github.com/corva-ai/python-sdk/compare/v0.0.2...v0.0.3 [0.0.2]: https://github.com/corva-ai/python-sdk/releases/tag/v0.0.2