Is this related to an existing feature request or issue?
#4067
Which Powertools for AWS Lambda (Python) utility does this relate to?
Other
Summary
We are excited to announce that we are starting a new major version (v3) for Powertools for AWS Lambda (Python). We are expediting a new major version to align with the scheduled Pydantic v1 end-of-life on June 30th, signalling the switch to much anticipated Pydantic v2. Pydantic plays a vital role and we care deeply about supply chain security. This release highlights our dedication to providing a secure and stable library that our customers trust.
Today, we already support both Pydantic v1 and v2. In this new major version, we will make Pydantic v2 the default, removing Pydantic v1. We will take this opportunity to introduce minor breaking changes to address common pain points. As always, we will provide an extensive upgrade guide to make this transition as smooth as possible.
Community
We invite the entire Powertools community, including users and contributors, to actively participate in this RFC. We greatly value the community's contribution, ideas, and willingness to solve problems or validate proposed changes.
Use case
The primary use case is to offer Pydantic v2. That includes a new Lambda Layer for each Python version since Pydantic v2 contains non-Python code (Rust). It future-proof Layers to contain any additional compiled dependency (like cryptography) without a new major version.
Additional key items on v3 backlog
- Distribute Lambda Layers in the AWS GovCloud regions
- Return sensible defaults over
Optional in Event Source Data Classes to further reduce customers code
- Remove deprecated code, deprecate legacy code to be removed in v4, and reduce API surface
Proposal
The action plan for this release with scope and tasks is as follows:
- Update our documentation to comply with our Versioning Policy and inform customers that we have started work on a new major version
- Create a branch named v3
- Make adjustments to our GitHub actions to run checks on this new branch
- Update Powertools v3 milestone to reflect current situation
- Address and resolve the issues mapped out for the execution of this launch plan.
- If possible, release early alpha versions for testing
Quick summary (Work in progress - Items can be added or removed):
- ✅ = done
- 🚧 = Work in progress
- 🚫 = Not considered in v3
| Item |
Issue/PR |
Status |
Code change required |
Utility |
Notes |
Cross-Language |
refactor the SSMProvider class to implement the get_multiple method |
#3252 |
✅ |
Yes |
Parameters |
|
|
Return empty Dict or List in Event Source Data Classes instead of None |
#2605 |
✅ |
Yes |
Event Source Data Class |
|
|
event_parser envelopes should handle unions of BaseModels |
#2734 |
✅ |
No |
Parser |
|
|
Pydantic v2 managed layer |
#2939 |
✅ |
No |
Layer |
|
|
Deprecate tracing.base.BaseProvider |
#4105 |
🚧 |
No |
Tracer |
|
|
resolved_headers_field is not Optional |
#4147 |
✅ |
No |
Event Source Data Class |
|
|
Create a Powertools Lambda layer for each Python version |
#3859 |
✅ |
No |
Layer |
|
|
| Publish layer to Gov Cloud regions |
#1166 |
🚧 |
No |
Layer |
|
✅ |
Remove any specific code for Pydantic v1 |
#4191 |
✅ |
Yes |
Codebase |
|
|
Update Powertools documentation to add banner about v3 |
#4198 |
✅ |
No |
Documentation |
|
|
Add the aws-encryption-sdk dependency in the Powertools layer |
#4192 |
✅ |
No |
Layer |
Depends on #3859 |
|
Review the method extract_data_from_envelope in JMESPath Functions |
#4218 |
✅ |
No |
JMESPath Functions |
|
|
Split the code from the api_gateway.py file into several other files to ease maintenance |
#4194 |
🚫 |
Yes |
Event Handler |
|
|
Add support for v3 branch in the github actions |
#4220 |
✅ |
No |
CI |
|
|
| Release the new version of Tracer utility with support for external providers |
#2342 |
🚧 |
Yes |
Tracer |
|
|
Unmarshal DynamoDB fields when using DynamoDBStreamModel |
#4219 |
✅ |
Yes |
Parser |
Reference: #1619 |
✅ |
Remove legacy code in the Metrics utility? |
- |
🚫 |
Yes |
Metrics |
Needs discussion |
|
| Keep the compatibility with Pydantic v2 and Bedrock agent resolver |
#4196 |
🚧 |
No |
Event Handler |
|
|
Change the default TTL (cache) to 5 minutes |
#4193 |
✅ |
Yes |
Parameters |
|
✅ |
Update boto3 version to 1.34.32 |
#4195 |
✅ |
No |
Dependencies |
|
|
| Change behavior to fail batch on first error when using DynamoDB and Kinesis in batch processing |
NEED ISSUE |
🚧 |
Yes |
Batch Processing |
Reference: https://tinyurl.com/yc335m3s |
✅ |
Remove the flag log_uncaught_exceptions in the Logger utility |
#4199 |
🚫 |
Yes |
Logger |
See: #1798 (comment) |
|
| Support for retrieving batch of secrets |
#4200 |
🚧 |
No |
Parameters |
|
|
Deadlines
We plan to release the first version of v3 by the end of June/beginning of July.
Out of scope
We will not include in v3:
- Modularization to further reduce sizes
- Strict typing
- New utilities
Potential challenges
Adding Pydantic v2 in the Powertools layer requires layers per Python version, so the effort to implement this is not yet clear.
Dependencies and Integrations
No response
Alternative solutions
No response
Acknowledgment
Is this related to an existing feature request or issue?
#4067
Which Powertools for AWS Lambda (Python) utility does this relate to?
Other
Summary
We are excited to announce that we are starting a new major version (v3) for Powertools for AWS Lambda (Python). We are expediting a new major version to align with the scheduled Pydantic v1 end-of-life on June 30th, signalling the switch to much anticipated Pydantic v2. Pydantic plays a vital role and we care deeply about supply chain security. This release highlights our dedication to providing a secure and stable library that our customers trust.
Today, we already support both Pydantic v1 and v2. In this new major version, we will make Pydantic v2 the default, removing Pydantic v1. We will take this opportunity to introduce minor breaking changes to address common pain points. As always, we will provide an extensive upgrade guide to make this transition as smooth as possible.
Community
We invite the entire Powertools community, including users and contributors, to actively participate in this RFC. We greatly value the community's contribution, ideas, and willingness to solve problems or validate proposed changes.
Use case
The primary use case is to offer Pydantic v2. That includes a new Lambda Layer for each Python version since Pydantic v2 contains non-Python code (Rust). It future-proof Layers to contain any additional compiled dependency (like cryptography) without a new major version.
Additional key items on v3 backlog
Optionalin Event Source Data Classes to further reduce customers codeProposal
The action plan for this release with scope and tasks is as follows:
Quick summary (Work in progress - Items can be added or removed):
refactor theSSMProviderclass to implement theget_multiplemethodReturn emptyDictorListin Event Source Data Classes instead ofNoneevent_parserenvelopes should handle unions of BaseModelsPydantic v2 managed layertracing.base.BaseProviderresolved_headers_fieldis not OptionalCreate a Powertools Lambda layer for each Python versionRemove any specific code for Pydantic v1Update Powertools documentation to add banner about v3Add theaws-encryption-sdkdependency in the Powertools layerReview the methodextract_data_from_envelopein JMESPath FunctionsSplit the code from theapi_gateway.pyfile into several other files to ease maintenanceAdd support for v3 branch in the github actionsUnmarshal DynamoDB fields when usingDynamoDBStreamModelRemove legacy code in the Metrics utility?Change the default TTL (cache) to 5 minutesUpdate boto3 version to 1.34.32Remove the flaglog_uncaught_exceptionsin the Logger utilityDeadlines
We plan to release the first version of v3 by the end of June/beginning of July.
Out of scope
We will not include in v3:
Potential challenges
Adding Pydantic v2 in the Powertools layer requires layers per Python version, so the effort to implement this is not yet clear.
Dependencies and Integrations
No response
Alternative solutions
No response
Acknowledgment