Skip to content

feat: Add dynamic source and sink kafka properties #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sumitaich1998 merged 28 commits into from
Sep 16, 2024
Merged

feat: Add dynamic source and sink kafka properties #31

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
328c44a
add kafka security module
ekawinataa Sep 2, 2024
1f6d09c
aDD SASL class callback config for producer and consumer
ekawinataa Sep 2, 2024
19574f9
Add config map
ekawinataa Sep 2, 2024
28e31de
remove build.gradle
ekawinataa Sep 2, 2024
6c33ff4
Add dynamic props
ekawinataa Sep 3, 2024
8ef873b
Update regex
ekawinataa Sep 3, 2024
c3d4556
rename var
ekawinataa Sep 3, 2024
f19b90b
Remove redundant imports
ekawinataa Sep 3, 2024
b731a38
Rename prefix
ekawinataa Sep 3, 2024
ef3d7e9
Remove unused import
ekawinataa Sep 3, 2024
e643dc6
Update test
ekawinataa Sep 3, 2024
d9cfc8a
Add implementation for sink dynamic props
ekawinataa Sep 5, 2024
fc26377
Add null checking for the additional props
ekawinataa Sep 5, 2024
d2c981d
Added validations on source config
ekawinataa Sep 10, 2024
11c8da5
Add docs and refactor pattern to enum
ekawinataa Sep 10, 2024
fb8695b
chECKSTYLE
ekawinataa Sep 10, 2024
cc2485f
Add readme
ekawinataa Sep 10, 2024
d110153
Make the pattern more specific and embedded to the enum
ekawinataa Sep 11, 2024
33b0cf9
Add more test
ekawinataa Sep 11, 2024
5f3a80c
bump version
ekawinataa Sep 11, 2024
62b4eca
Add unit tests
ekawinataa Sep 13, 2024
6c4e8ae
Use expected annotation
ekawinataa Sep 16, 2024
1063b25
Assert exception message. Add fail mechanism in case of not throwing …
ekawinataa Sep 16, 2024
07f5e53
Use rule for asserting exception
ekawinataa Sep 16, 2024
edf2e4d
Add more test case
ekawinataa Sep 16, 2024
0c2d621
add more unit test
ekawinataa Sep 16, 2024
a262868
feat: Enable multiple underscore parsing
ekawinataa Sep 16, 2024
d31f834
test: Add test on multiple underscore parsing
ekawinataa Sep 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: Enable multiple underscore parsing
  • Loading branch information
@ekawinataa
ekawinataa committed Sep 16, 2024
commit a262868f37bc1a1dfb5dbded37f13614bf72ff5a
2 changes: 1 addition & 1 deletion ...re/src/main/java/com/gotocompany/dagger/core/enumeration/KafkaConnectorTypesMetadata.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import java.util.regex.Pattern;

public enum KafkaConnectorTypesMetadata {
SOURCE("SOURCE_KAFKA_CONSUMER_CONFIG_"), SINK("SINK_KAFKA_PRODUCER_CONFIG_");
SOURCE("SOURCE_KAFKA_CONSUMER_CONFIG_+"), SINK("SINK_KAFKA_PRODUCER_CONFIG_+");

KafkaConnectorTypesMetadata(String prefixPattern) {
this.prefixPattern = prefixPattern;
Expand Down
2 changes: 1 addition & 1 deletion dagger-core/src/main/java/com/gotocompany/dagger/core/utils/KafkaConfigUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ public static Properties parseKafkaConfiguration(KafkaConnectorTypesMetadata kaf
if (matcher.find()) {
String kafkaConfigKey = matcher.group(1)
.toLowerCase()
.replaceAll("_", ".");
.replaceAll("_+", ".");
kafkaProperties.setProperty(kafkaConfigKey, properties.get(key).toString());
}
}
Expand Down