Describe the desired outcome from the user's perspective
Large messages are often fragmented into smaller partial messages, but should still be validated according to the same rules as if they are observed as a single unified value.
Acceptance criteria
- Fragmented HTTP request messages can be validated or rejected;
string, integer, json
- Fragmented MQTT publish messages can be validated or rejected;
string, integer, json
- Fragmented Kafka produce messages can be validated or rejected;
string, integer, json
- Fragmented messages should not require buffering into a complete message before validation
Additional context
Conversion from json to avro (for example) is much more challenging to implement incrementally without buffering the complete message, so we should separate the concepts of validator and converter, supporting converters only at the kafka cache where we already have a view of the complete message by design.
Tasks
Describe the desired outcome from the user's perspective
Large messages are often fragmented into smaller partial messages, but should still be validated according to the same rules as if they are observed as a single unified value.
Acceptance criteria
string,integer,jsonstring,integer,jsonstring,integer,jsonAdditional context
Conversion from
jsontoavro(for example) is much more challenging to implement incrementally without buffering the complete message, so we should separate the concepts ofvalidatorandconverter, supportingconvertersonly at thekafkacachewhere we already have a view of the complete message by design.Tasks
ValidatorandConverterin Java APIsavroconvertershould only be configurable inkafkacacheby schema (not other protocols)protobufconvertershould only be configurable inkafkacacheby schema (not other protocols)validatorsshould only be configurable onHTTPandMQTTby schemavalidatorsandconvertersshould be configurable onKafkaby schemastringfragmented validatorintegerfragmented validatorjsonfragmented validator