Skip to content

Include metadata and partitionOffset into merge reply begin #599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jfallows merged 3 commits into from
Dec 1, 2023
Merged

Include metadata and partitionOffset into merge reply begin #599

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e2fcf2c
Include metadata and partitionOffset into merge reply begin
akrambek Dec 1, 2023
770f168
update scripts
akrambek Dec 1, 2023
0c322a0
Refactor code
akrambek Dec 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Include metadata and partitionOffset into merge reply begin
  • Loading branch information
@akrambek
akrambek committed Dec 1, 2023
commit e2fcf2cefb97550a184c45e3f301660c1802fe11
28 changes: 22 additions & 6 deletions ...main/java/io/aklivity/zilla/runtime/binding/kafka/internal/stream/KafkaMergedFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
import org.agrona.collections.Int2ObjectHashMap;
import org.agrona.collections.Long2LongHashMap;
import org.agrona.collections.MutableInteger;
import org.agrona.collections.MutableLong;
import org.agrona.collections.MutableReference;
import org.agrona.concurrent.UnsafeBuffer;

Expand Down Expand Up @@ -158,6 +159,8 @@ public final class KafkaMergedFactory implements BindingHandler
private final MutableInteger initialNoAckRW = new MutableInteger();
private final MutableInteger initialPadRW = new MutableInteger();
private final MutableInteger initialMaxRW = new MutableInteger();
private final MutableLong partitionOffsetRW = new MutableLong();
private final StringBuilder metadataRW = new StringBuilder();

private final int kafkaTypeId;
private final MutableDirectBuffer writeBuffer;
Expand Down Expand Up @@ -1614,12 +1617,25 @@ private Consumer<KafkaMergedBeginExFW.Builder> beginExToKafkaMerged()
return builder ->
{
builder.capabilities(c -> c.set(FETCH_ONLY)).topic(topic);
latestOffsetByPartitionId.longForEach((k, v) -> builder
.partitionsItem(i -> i
.partitionId((int) k)
.partitionOffset(0L)
.stableOffset(stableOffsetByPartitionId.get(k))
.latestOffset(v)));
latestOffsetByPartitionId.longForEach((k, v) ->
{
partitionOffsetRW.value = 0;
metadataRW.setLength(0);
if (!offsetsByPartitionId.isEmpty())
{
final KafkaPartitionOffset kafkaPartitionOffset = offsetsByPartitionId.get(k);
partitionOffsetRW.value = kafkaPartitionOffset.partitionOffset;
metadataRW.append(kafkaPartitionOffset.metadata);
}

builder
.partitionsItem(i -> i
.partitionId((int) k)
.partitionOffset(partitionOffsetRW.value)
.stableOffset(stableOffsetByPartitionId.get(k))
.latestOffset(v)
.metadata(metadataRW.toString()));
});
};
}

Expand Down