Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
4a0e71c
chore: add plan for cloud events, written with Opus 4.6 Thinking
feO2x Feb 14, 2026
1b30f02
chore: add link to official Cloud Event spec to plan 15
feO2x Feb 14, 2026
61c1dda
chore: updated plan 15 to be more compliant with Cloud Events JSON spec
feO2x Feb 14, 2026
2ab13ca
chore: make plan 15 more explicit, especially regarding envelope attr…
feO2x Feb 14, 2026
52ba457
chore: introduce lroutcome into plan 15
feO2x Feb 14, 2026
7a44499
chore: further clarification on Cloud Event Reading
feO2x Feb 14, 2026
aaa606e
chore: add PreferSuccessPayload and SerializerOptions to plan 15
feO2x Feb 14, 2026
62ae97d
chore: add Composition Root integration for plan 15
feO2x Feb 14, 2026
5905974
feat: add initial Cloud Events integration
feO2x Feb 14, 2026
12ccea7
test: add additional tests for Cloud Events
feO2x Feb 14, 2026
dd06de4
chore: add additional XML comments to ReadOnlyMemoryCloudEventExtensions
feO2x Feb 14, 2026
332562a
refactor: properly use JsonSerializer.Deserialize in ReadOnlyMemoryCl…
feO2x Feb 14, 2026
715ff8b
chore: add lroutcome to list of known words
feO2x Feb 15, 2026
f31a9d5
perf: JsonSerializerOptions is now a singleton in LightResultsCloudEv…
feO2x Feb 15, 2026
48bea33
chore: add plan for optimizing reading of Cloud Events
feO2x Feb 15, 2026
a3cacc1
chore: add 'datacontenttype' to team-shared dictionary
feO2x Feb 15, 2026
d8b6256
perf: restructure Cloud Event reading to avoid byte[] allocations for…
feO2x Feb 15, 2026
eee65e3
chore: add additional plan for Cloud Events writing optimization
feO2x Feb 15, 2026
edc92c6
perf: optimized Cloud Event writing
feO2x Feb 15, 2026
0503aa4
chore: add plan for streamlined STJ use when writing Cloud Events
feO2x Feb 15, 2026
da10fec
chore: finish Cloud Events streamlining plan
feO2x Feb 16, 2026
a87f708
chore: add missing AI plans to .slnx
feO2x Feb 16, 2026
36f10b5
chore: added 'specversion' to team-shared dictionary
feO2x Feb 20, 2026
69390ee
fix: CloudEvents.Writing now properly calls JsonSerializer pipeline
feO2x Feb 20, 2026
104b074
feat: PooledByBufferWriter is now public, allow callers to safely use…
feO2x Feb 21, 2026
e405a0c
chore: adapt AGENTS.md for tests to use expected instance instead of …
feO2x Feb 21, 2026
10e5b35
test: increase code coverage for Light.Results
feO2x Feb 21, 2026
91b107f
feat: use UUIDv7 for CloudEvents
feO2x Feb 21, 2026
7cb9fa8
chore: add deviations from plan 15
feO2x Feb 21, 2026
1baacc1
chore: update IdResolver documentation to reflect UUIDv7 default gene…
feO2x Feb 21, 2026
d75c6f4
fix: restrain growth in PooledByteBufferWriter
feO2x Feb 21, 2026
1ee33eb
fix: PooledArray now delegates everything to PooledByteBufferWriter
feO2x Feb 21, 2026
3bc6ed0
fix: CloudEvents data is not written for non-generic Result if not re…
feO2x Feb 22, 2026
b397eda
fix: RentedArrayBufferWriter is now disposed when exception occurs du…
feO2x Feb 22, 2026
ac84fee
chore: update deviations from plan 15
feO2x Feb 22, 2026
2e68382
fix: MetadataValueAnnotationHelper.WithAnnotation now checks if the M…
feO2x Feb 22, 2026
9f7435f
fix: MetadataValueAnnotationHelper.WithAnnotation for MetadataValue n…
feO2x Feb 22, 2026
6ee34be
chore: extend XML comments for MetadataValueAnnotationHelper
feO2x Feb 22, 2026
51c7025
refactor: eliminate readOptions in ReadOnlyMemoryCloudEventExtensions
feO2x Feb 22, 2026
f50597a
chore: 'CloudEvents' instead of CloudEvent renaming
feO2x Feb 22, 2026
4ef6e44
chore: fix error messages in RentedArrayBufferWriter
feO2x Feb 22, 2026
44945d0
fix: CloudEventEnvelopeJsonReader now parsed CloudEvents time value w…
feO2x Feb 22, 2026
f4ce67e
fix: remove DataContentType from CloudEventEnvelopeForWriting because…
feO2x Feb 22, 2026
1c44f92
fix: metadata arrays are consistently forbidden for CloudEvents exten…
feO2x Feb 22, 2026
c2a5d6e
chore: fix XML comment for RentedArrayBufferWriter.FinishWriting
feO2x Feb 22, 2026
9332d74
chore: add code review for plan 15
feO2x Feb 22, 2026
fed4e8b
chore: documented deviations for MetadataValueAnnotation names
feO2x Feb 22, 2026
3d3f7a4
chore: use 'CloudEvents' consistently across the code base instead of…
feO2x Feb 22, 2026
efe5840
chore: removed mention of IBufferWriter overload from plan deviations
feO2x Feb 22, 2026
9437cf1
chore: add final code review for plan 15 to slnx
feO2x Feb 22, 2026
1d62289
chore: remove ReadRichErrorObject in ResultJsonReader
feO2x Feb 22, 2026
1d710ec
chore: remove duplicate IsPrimitive methods for MetadataKind
feO2x Feb 22, 2026
f605621
fix: CloudEvents reading implementation now rejects complex extension…
feO2x Feb 22, 2026
281b9a3
chore: add additional XML comments
feO2x Feb 22, 2026
45c2652
fix: add guard clauses in WriteMetadataPropertyAndValue
feO2x Feb 22, 2026
e8a6760
fix: CloudEventsEnvelopeJsonReader now checks whether bytesConsumed c…
feO2x Feb 23, 2026
08824c2
chore: fix XML comments in IRentedArray
feO2x Feb 23, 2026
db6d62f
chore: fix XML comment in MetadataExtensions
feO2x Feb 23, 2026
c089a51
fix: add Guard Clauses to ErrorsExtensions.WriteRichErrors
feO2x Feb 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
<PackageVersion Include="RequiredMemberAttribute" Version="1.0.0" />
<PackageVersion Include="System.Collections.Immutable" Version="10.0.2" />
<PackageVersion Include="System.Text.Json" Version="10.0.2" />
<PackageVersion Include="Ulid" Version="1.4.1" />
<PackageVersion Include="Verify.Http" Version="7.5.1" />
<PackageVersion Include="Verify.XunitV3" Version="31.10.0" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />
Expand Down
5 changes: 4 additions & 1 deletion Light.Results.sln.DotSettings
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<wpf:ResourceDictionary xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
<wpf:ResourceDictionary xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:s="clr-namespace:System;assembly=mscorlib"
xmlns:wpf="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xml:space="preserve">
<s:Boolean
Expand Down Expand Up @@ -231,6 +231,9 @@
x:Key="/Default/Environment/SettingsMigration/IsMigratorApplied/=JetBrains_002EReSharper_002EPsi_002ECSharp_002ECodeStyle_002ECSharpUseContinuousIndentInsideBracesMigration/@EntryIndexedValue">True</s:Boolean>
<s:Boolean
x:Key="/Default/Environment/SettingsMigration/IsMigratorApplied/=JetBrains_002EReSharper_002EPsi_002ECSharp_002ECodeStyle_002ESettingsUpgrade_002EMigrateBlankLinesAroundFieldToBlankLinesAroundProperty/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=datacontenttype/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=hmacsha/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=lroutcome/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=nupkg/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=specversion/@EntryIndexedValue">True</s:Boolean>
</wpf:ResourceDictionary>
6 changes: 6 additions & 0 deletions Light.Results.slnx
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@
<File Path="ai-plans\0009-updated-minimal-api-integration.md" />
<File Path="ai-plans\0011-http-response-deserialization.md" />
<File Path="ai-plans\0011-plan-deviations.md" />
<File Path="ai-plans\0013-mvc-integration.md" />
<File Path="ai-plans\0015-cloud-events-reading-performance-optimization.md" />
<File Path="ai-plans\0015-cloud-events-serialization.md" />
<File Path="ai-plans\0015-cloud-events-write-optimization.md" />
<File Path="ai-plans\0015-cloud-events-write-streamlining.md" />
<File Path="ai-plans\0015-final-code-review.md" />
<File Path="ai-plans\AGENTS.md" />
</Folder>
<Folder Name="/benchmarks/">
Expand Down
234 changes: 234 additions & 0 deletions ai-plans/0015-cloud-events-reading-performance-optimization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
# CloudEvents Reading Performance Optimization

## Rationale

After implementing the initial plan 0015-cloud-events-serialization, the current CloudEvent JSON reading implementation incurs unnecessary allocations when deserializing the `data` property. In `CloudEventEnvelopeJsonReader.ReadEnvelope`, when the `data` property is encountered, the implementation:

1. Parses the data subtree into a `JsonDocument`
2. Serializes it back to a `byte[]` via `JsonSerializer.SerializeToUtf8Bytes`
3. Stores this copy in `CloudEventEnvelopePayload.DataBytes`
4. Later deserializes from `DataBytes` to the actual payload type

This creates three allocations (JsonDocument, internal buffers, byte[] copy) that can be eliminated. Since the caller provides a `ReadOnlyMemory<byte>` containing the original JSON, we can track byte positions and use a slice of the original buffer instead of copying.

## Acceptance Criteria

- [x] `CloudEventEnvelopePayload` stores position-based tracking (`DataStart`, `DataLength`) instead of `byte[]` for the data segment
- [x] No `JsonDocument` is allocated when parsing the CloudEvent envelope
- [x] No intermediate `byte[]` copy is created for the data payload
- [x] The original buffer slice is used directly when deserializing the data payload
- [x] All existing CloudEvent reading functionality remains intact
- [x] Automated tests are written or updated to verify the new implementation
- [x] BenchmarkDotNet benchmarks are added to `./benchmarks/Benchmarks/` to measure the allocation reduction

## Technical Details

### Current Allocation Flow

```
ReadOnlyMemory<byte> cloudEvent (original buffer)
JsonSerializer.Deserialize<CloudEventEnvelopePayload>(cloudEvent.Span, options)
▼ (inside CloudEventEnvelopePayloadJsonConverter.Read)
CloudEventEnvelopeJsonReader.ReadEnvelope(ref Utf8JsonReader reader)
└─ When hitting "data" property:
├─ JsonDocument.ParseValue(ref reader) ← Allocation #1
└─ JsonSerializer.SerializeToUtf8Bytes() ← Allocation #2
byte[] dataBytes stored in payload
Later: JsonSerializer.Deserialize<T>(dataBytes) ← Re-parsing same content
```

### Zero-Copy Architecture (Position-Based)

The key insight is that `Utf8JsonReader.BytesConsumed` tracks the current byte position within the input buffer. By recording positions before and after skipping the `data` value, we can compute a slice of the original buffer **after** deserialization completes.

This approach maintains full STJ converter extensibility - users can still provide custom `JsonConverter` implementations.

#### 1. Modify `CloudEventEnvelopePayload`

Change the data storage from `byte[]?` to position tracking:

```csharp
public readonly struct CloudEventEnvelopePayload
{
// Remove: public byte[]? DataBytes { get; }
// Add:
public int DataStart { get; } // Byte offset where data value begins
public int DataLength { get; } // Length of data value in bytes

// HasData and IsDataNull remain for semantic clarity
}
```

#### 2. Modify `CloudEventEnvelopeJsonReader.ReadEnvelope`

Update the existing method to track byte positions using `BytesConsumed` instead of copying bytes:

```csharp
public static CloudEventEnvelopePayload ReadEnvelope(ref Utf8JsonReader reader)
{
// ... existing envelope attribute parsing ...

int dataStart = 0;
int dataLength = 0;
var hasData = false;
var isDataNull = false;

// When hitting "data" property:
else if (reader.ValueTextEquals("data"))
{
// Record position BEFORE reading the data value token
// BytesConsumed is the position after the property name
int positionBeforeDataValue = (int)reader.BytesConsumed;

if (!reader.Read())
{
throw new JsonException("Unexpected end of JSON while reading data.");
}

hasData = true;
if (reader.TokenType == JsonTokenType.Null)
{
isDataNull = true;
// dataStart and dataLength remain 0
}
else
{
// Skip the entire data subtree without parsing into JsonDocument
reader.Skip();

int positionAfterDataValue = (int)reader.BytesConsumed;

dataStart = positionBeforeDataValue;
dataLength = positionAfterDataValue - positionBeforeDataValue;
}
}

// ... rest of parsing ...

return new CloudEventEnvelopePayload(
type!,
source!,
id!,
subject,
time,
dataContentType,
dataSchema,
extensionAttributes,
hasData,
isDataNull,
dataStart,
dataLength
);
}
```

**Important:** `Utf8JsonReader.BytesConsumed` returns the number of bytes consumed *up to and including* the current token. We capture the position *after* reading the property name (before the value), then again after `Skip()` to get the complete range.

#### 3. Update Extension Methods

Modify `ReadOnlyMemoryCloudEventExtensions` to slice the original buffer after deserialization:

```csharp
public static CloudEventEnvelope ReadResultWithCloudEventEnvelope(
this ReadOnlyMemory<byte> cloudEvent,
LightResultsCloudEventReadOptions? options = null)
{
var readOptions = options ?? LightResultsCloudEventReadOptions.Default;

// Deserialize through STJ (maintains converter extensibility)
var parsedEnvelope = JsonSerializer.Deserialize<CloudEventEnvelopePayload>(
cloudEvent.Span,
readOptions.SerializerOptions
);

var isFailure = DetermineIsFailure(parsedEnvelope, readOptions);

// Slice the original buffer using tracked positions - ZERO COPY
var dataSegment = parsedEnvelope.HasData && !parsedEnvelope.IsDataNull
? cloudEvent.Slice(parsedEnvelope.DataStart, parsedEnvelope.DataLength)
: ReadOnlyMemory<byte>.Empty;

var result = ParseResultPayload(dataSegment, isFailure, readOptions);

// ... rest unchanged ...
}
```

#### 4. Update Payload Parsing Methods

Change signature to accept `ReadOnlyMemory<byte>` instead of extracting from payload:

```csharp
private static Result ParseResultPayload(
ReadOnlyMemory<byte> dataSegment,
bool isFailure,
LightResultsCloudEventReadOptions options)
{
if (dataSegment.IsEmpty)
{
if (isFailure)
{
throw new JsonException(
"CloudEvent failure payloads for non-generic Result must contain non-null data."
);
}
return Result.Ok();
}

if (isFailure)
{
var failurePayload = JsonSerializer.Deserialize<CloudEventFailurePayload>(
dataSegment.Span,
options.SerializerOptions
);
return Result.Fail(failurePayload.Errors, failurePayload.Metadata);
}

// ... etc ...
}
```

#### 5. Converter Extensibility Preserved

The `CloudEventEnvelopePayloadJsonConverter` continues to work through STJ's normal deserialization pipeline. Users can:
- Register custom converters for envelope parsing
- Override behavior via `JsonSerializerOptions`
- Extend without modifying core library code

The optimization is transparent to the converter - it simply stores positions instead of copying bytes.

### Benchmark Design

Create `CloudEventReadingBenchmarks.cs` with:

1. **Baseline:** Current implementation (JsonDocument + SerializeToUtf8Bytes copy)
2. **Optimized:** Position-based approach (BytesConsumed + Skip + slice)
3. **Test cases:**
- Small data payload (~100 bytes)
- Medium data payload (~1KB)
- Large data payload (~10KB)
- Success vs failure payloads

Measure both execution time and allocations using `[MemoryDiagnoser]`.

### Edge Cases

- **Empty data segment:** `DataStart = 0, DataLength = 0` when `IsDataNull` is true or `HasData` is false
- **Nested complex data:** `reader.Skip()` correctly handles any valid JSON subtree
- **Unicode and escapes:** The slice captures raw UTF-8 bytes which `JsonSerializer` handles correctly
- **Whitespace:** `BytesConsumed` includes any whitespace between tokens; this is fine since the slice is re-parsed by STJ

### Breaking Changes

Changing `byte[]? DataBytes` to `int DataStart` and `int DataLength` is a breaking change, but the library is not published yet, so there is no issue here.
Loading