Merge branch 'develop' into feature/ztable

aklivity · jfallows · Dec 27, 2024 · Nov 12, 2024 · Nov 14, 2024 · Nov 14, 2024
commit 2a4815f6c4bc298dc45a9f459bfecd14bf936dab
diff --git a/.github/workflows/links-check.yml b/.github/workflows/links-check.yml
@@ -10,10 +10,10 @@ jobs:
   linkChecker:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Link Checker
-        uses: lycheeverse/lychee-action@v1.8.0
+        uses: lycheeverse/lychee-action@v2.1.0
         with:
           fail: true
           args: --exclude-mail --exclude .+localhost.+ README.md

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,81 @@
 # Changelog
 
+## [0.9.108](https://github.com/aklivity/zilla/tree/0.9.108) (2024-12-12)
+
+[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.107...0.9.108)
+
+**Merged pull requests:**
+
+- Bump alpine from 3.20.3 to 3.21.0 in /cloud/docker-image/src/main/docker [\#1346](https://github.com/aklivity/zilla/pull/1346) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump lycheeverse/lychee-action from 1.8.0 to 2.1.0 [\#1325](https://github.com/aklivity/zilla/pull/1325) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump actions/checkout from 3 to 4 [\#1219](https://github.com/aklivity/zilla/pull/1219) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump ubuntu from jammy-20240530 to jammy-20240808 in /cloud/docker-image/src/main/docker [\#1205](https://github.com/aklivity/zilla/pull/1205) ([dependabot[bot]](https://github.com/apps/dependabot))
+
+## [0.9.107](https://github.com/aklivity/zilla/tree/0.9.107) (2024-12-10)
+
+[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.106...0.9.107)
+
+**Merged pull requests:**
+
+- Bump org.apache.avro:avro from 1.11.3 to 1.11.4 [\#1284](https://github.com/aklivity/zilla/pull/1284) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Bump com.google.protobuf:protobuf-java from 3.24.4 to 3.25.5 in /runtime/model-protobuf [\#1257](https://github.com/aklivity/zilla/pull/1257) ([dependabot[bot]](https://github.com/apps/dependabot))
+
+## [0.9.106](https://github.com/aklivity/zilla/tree/0.9.106) (2024-12-04)
+
+[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.105...0.9.106)
+
+**Merged pull requests:**
+
+- Exclude package-info.class from delegate module to avoid empty packages [\#1343](https://github.com/aklivity/zilla/pull/1343) ([jfallows](https://github.com/jfallows))
+
+## [0.9.105](https://github.com/aklivity/zilla/tree/0.9.105) (2024-12-03)
+
+[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.104...0.9.105)
+
+**Merged pull requests:**
+
+- Json deserialization of nullable fields [\#1342](https://github.com/aklivity/zilla/pull/1342) ([akrambek](https://github.com/akrambek))
+
+## [0.9.104](https://github.com/aklivity/zilla/tree/0.9.104) (2024-12-02)
+
+[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.103...0.9.104)
+
+**Fixed bugs:**
+
+- `affinity mask must specify at least one bit` crash preventing Zilla from starting on Windows 11 + Docker Compose [\#1338](https://github.com/aklivity/zilla/issues/1338)
+
+**Merged pull requests:**
+
+- Json serialization of nullable fields [\#1341](https://github.com/aklivity/zilla/pull/1341) ([akrambek](https://github.com/akrambek))
+
+## [0.9.103](https://github.com/aklivity/zilla/tree/0.9.103) (2024-11-27)
+
+[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.102...0.9.103)
+
+**Fixed bugs:**
+
+- Mqtt selecting incorrect publish stream [\#1327](https://github.com/aklivity/zilla/issues/1327)
+
+**Merged pull requests:**
+
+- resolve nested `$ref` in schema [\#1337](https://github.com/aklivity/zilla/pull/1337) ([ankitk-me](https://github.com/ankitk-me))
+- Enhance test vault and test exporter [\#1333](https://github.com/aklivity/zilla/pull/1333) ([jfallows](https://github.com/jfallows))
+- validate `begin` frame type for `ws` client [\#1332](https://github.com/aklivity/zilla/pull/1332) ([ankitk-me](https://github.com/ankitk-me))
+- Support `directory` syntax & functionality [\#1323](https://github.com/aklivity/zilla/pull/1323) ([ankitk-me](https://github.com/ankitk-me))
+
+## [0.9.102](https://github.com/aklivity/zilla/tree/0.9.102) (2024-11-13)
+
+[Full Changelog](https://github.com/aklivity/zilla/compare/0.9.101...0.9.102)
+
+**Implemented enhancements:**
+
+- Support `risingwave` `pgsql` message transformations [\#1208](https://github.com/aklivity/zilla/issues/1208)
+
+**Merged pull requests:**
+
+- Fix MQTT binding selecting incorrect publish stream [\#1328](https://github.com/aklivity/zilla/pull/1328) ([bmaidics](https://github.com/bmaidics))
+- Support ALTER STREAM and ALTER TABLE transformation [\#1320](https://github.com/aklivity/zilla/pull/1320) ([akrambek](https://github.com/akrambek))
+
 ## [0.9.101](https://github.com/aklivity/zilla/tree/0.9.101) (2024-11-07)
 
 [Full Changelog](https://github.com/aklivity/zilla/compare/0.9.100...0.9.101)

diff --git a/cloud/docker-image/pom.xml b/cloud/docker-image/pom.xml
@@ -411,6 +411,9 @@
                     <platform>linux/amd64</platform>
                     <platform>linux/arm64</platform>
                   </platforms>
+                  <attestations>
+                    <provenance>false</provenance>
+                  </attestations>
                 </buildx>
                 <assemblies>
                   <assembly>
@@ -462,6 +465,9 @@
                       <platforms>
                         <platform>linux/amd64</platform>
                       </platforms>
+                      <attestations>
+                        <provenance>false</provenance>
+                      </attestations>
                     </buildx>
                     <assemblies>
                       <assembly>

diff --git a/cloud/docker-image/src/main/docker/Dockerfile b/cloud/docker-image/src/main/docker/Dockerfile
@@ -25,7 +25,7 @@ RUN cat zpm.json.template | sed "s/\${VERSION}/${project.version}/g" | tee zpm.j
 RUN ./zpmw install --debug --instrument --exclude-remote-repositories
 RUN ./zpmw clean --keep-image
 
-FROM ubuntu:jammy-20240530
+FROM ubuntu:jammy-20240808
 
 ENV ZILLA_VERSION ${project.version}
 

diff --git a/cloud/docker-image/src/main/docker/alpine.Dockerfile b/cloud/docker-image/src/main/docker/alpine.Dockerfile
@@ -27,7 +27,7 @@ RUN apk add --no-cache wget
 RUN ./zpmw install --debug --exclude-remote-repositories
 RUN ./zpmw clean --keep-image
 
-FROM alpine:3.20.3
+FROM alpine:3.21.0
 
 ENV ZILLA_VERSION ${project.version}
 

diff --git a/...y/zilla/specs/binding/pgsql/kafka/streams/pgsql/create.topic.with.generated.as/client.rpt b/...y/zilla/specs/binding/pgsql/kafka/streams/pgsql/create.topic.with.generated.as/client.rpt
@@ -20,7 +20,7 @@ connect "zilla://streams/app0"
 
 write zilla:begin.ext ${pgsql:beginEx()
                               .typeId(zilla:id("pgsql"))
-                              .parameter("user", "root")
+                              .parameter("user", "app")
                               .parameter("database", "dev")
                               .parameter("application_name", "psql")
                               .parameter("client_encoding", "UTF8")

diff --git a/...ty/zilla/runtime/binding/pgsql/kafka/internal/schema/PgsqlKafkaKeyAvroSchemaTemplate.java b/...ty/zilla/runtime/binding/pgsql/kafka/internal/schema/PgsqlKafkaKeyAvroSchemaTemplate.java
@@ -31,15 +31,15 @@ public PgsqlKafkaKeyAvroSchemaTemplate(
 
     public String generate(
         String database,
-        CreateTable createTable)
+        CreateTable command)
     {
         final String newNamespace = namespace.replace(DATABASE_PLACEHOLDER, database);
 
-        List<AvroField> fields = createTable.columns().stream()
+        List<AvroField> fields = command.columns().stream()
             .map(column -> new AvroField(column.name(), mapSqlTypeToAvroType(column.type()), null))
             .collect(Collectors.toList());
 
-        AvroSchema schema = new AvroSchema("record", createTable.name(), newNamespace, fields);
+        AvroSchema schema = new AvroSchema("record", command.name(), newNamespace, fields);
         AvroPayload payload = new AvroPayload("AVRO", jsonb.toJson(schema));
 
         return jsonbFormatted.toJson(payload);

diff --git a/.../zilla/runtime/binding/pgsql/kafka/internal/schema/PgsqlKafkaValueAvroSchemaTemplate.java b/.../zilla/runtime/binding/pgsql/kafka/internal/schema/PgsqlKafkaValueAvroSchemaTemplate.java
@@ -39,11 +39,11 @@ public PgsqlKafkaValueAvroSchemaTemplate(
     }
 
     public String generate(
-        CreateTable createTable)
+        CreateTable command)
     {
-        final String newNamespace = namespace.replace(DATABASE_PLACEHOLDER, createTable.schema());
+        final String newNamespace = namespace.replace(DATABASE_PLACEHOLDER, command.schema());
 
-        List<AvroField> fields = createTable.columns().stream()
+        List<AvroField> fields = command.columns().stream()
             .map(column ->
             {
                 String columnName = column.name();
@@ -65,7 +65,7 @@ public String generate(
             })
             .collect(Collectors.toList());
 
-        AvroSchema schema = new AvroSchema("record", createTable.name(), newNamespace, fields);
+        AvroSchema schema = new AvroSchema("record", command.name(), newNamespace, fields);
         AvroPayload payload = new AvroPayload("AVRO", jsonb.toJson(schema));
 
         return jsonbFormatted.toJson(payload);

diff --git a/...-pgsql/src/main/antlr4/io/aklivity/zilla/runtime/binding/pgsql/parser/PostgreSqlParser.g4 b/...-pgsql/src/main/antlr4/io/aklivity/zilla/runtime/binding/pgsql/parser/PostgreSqlParser.g4
@@ -1645,6 +1645,7 @@ show_object_type_name
     | VIEWS
     | MATERIALIZED VIEWS
     | TOPICS
+    | HEAD
     | ZTABLES
     | ZVIEWS
     ;
@@ -1672,6 +1673,7 @@ object_type_any_name
     | TOPIC
     | STREAM
     | ZVIEW
+    | HEAD
     | ZTABLE
     | INDEX
     | FOREIGN TABLE

diff --git a/...g-pgsql/src/test/java/io/aklivity/zilla/runtime/binding/pgsql/parser/PgsqlParserTest.java b/...g-pgsql/src/test/java/io/aklivity/zilla/runtime/binding/pgsql/parser/PgsqlParserTest.java
@@ -64,6 +64,7 @@ name VARCHAR(100),
                 created_at TIMESTAMP GENERATED ALWAYS AS NOW,
                 PRIMARY KEY (id, name)
             );""";
+
         CreateTable table = parser.parseCreateTable(sql);
 
         assertNotNull(table);
@@ -191,6 +192,7 @@ public void shouldHandleInvalidZCreateZtable()
     public void shouldDropSingleZtable()
     {
         String sql = "DROP ZTABLE test_table;";
+
         List<Drop> drops = parser.parseDrop(sql);
 
         assertEquals(1, drops.size());

diff --git a/...illa/specs/binding/risingwave/streams/effective/query.with.multiple.statements/client.rpt b/...illa/specs/binding/risingwave/streams/effective/query.with.multiple.statements/client.rpt
@@ -184,6 +184,20 @@ read advised zilla:flush ${pgsql:flushEx()
                                 .build()
                               .build()}
 
+connect "zilla://streams/app1"
+    option zilla:window 8192
+    option zilla:transmission "duplex"
+
+write zilla:begin.ext ${pgsql:beginEx()
+                              .typeId(zilla:id("pgsql"))
+                              .parameter("user", "app")
+                              .parameter("database", "dev")
+                              .parameter("application_name", "psql")
+                              .parameter("client_encoding", "UTF8")
+                              .build()}
+
+connected
+
 write zilla:data.ext ${pgsql:dataEx()
                               .typeId(zilla:id("pgsql"))
                               .query()

diff --git a/...illa/specs/binding/risingwave/streams/effective/query.with.multiple.statements/server.rpt b/...illa/specs/binding/risingwave/streams/effective/query.with.multiple.statements/server.rpt
@@ -182,6 +182,18 @@ write advise zilla:flush ${pgsql:flushEx()
                                 .build()
                               .build()}
 
+accepted
+
+read zilla:begin.ext ${pgsql:beginEx()
+                              .typeId(zilla:id("pgsql"))
+                              .parameter("user", "app")
+                              .parameter("database", "dev")
+                              .parameter("application_name", "psql")
+                              .parameter("client_encoding", "UTF8")
+                              .build()}
+
+connected
+
 read zilla:data.ext ${pgsql:dataEx()
                               .typeId(zilla:id("pgsql"))
                               .query()

diff --git a/...ripts/io/aklivity/zilla/specs/binding/risingwave/streams/effective/show.zviews/client.rpt b/...ripts/io/aklivity/zilla/specs/binding/risingwave/streams/effective/show.zviews/client.rpt
@@ -72,6 +72,7 @@ read  [0x00 0x02]                                                 # Field Count
       " SELECT id, city FROM weather GROUP BY city;"
       [0x00]
 
+
 read advised zilla:flush ${pgsql:flushEx()
                               .typeId(zilla:id("pgsql"))
                               .completion()

diff --git a/...ripts/io/aklivity/zilla/specs/binding/risingwave/streams/effective/show.zviews/server.rpt b/...ripts/io/aklivity/zilla/specs/binding/risingwave/streams/effective/show.zviews/server.rpt
@@ -73,6 +73,7 @@ write [0x00 0x02]                                                 # Field Count
       "CREATE MATERIALIZED VIEW IF NOT EXISTS distinct_cities AS" # Data
       " SELECT id, city FROM weather GROUP BY city;"
       [0x00]
+
 write flush
 
 

diff --git a/...y/zilla/runtime/binding/risingwave/internal/statement/RisingwaveCreateSourceTemplate.java b/...y/zilla/runtime/binding/risingwave/internal/statement/RisingwaveCreateSourceTemplate.java
@@ -61,7 +61,6 @@ public String generateStreamSource(
         includeBuilder.setLength(0);
         Map<String, String> includes = createStream.columns().entrySet().stream()
             .filter(e -> ZILLA_MAPPINGS_OLD.containsKey(e.getKey()))
-
             .collect(LinkedHashMap::new, (m, e) -> m.put(e.getKey(), e.getValue()), Map::putAll);
 
         if (!includes.isEmpty())
@@ -85,7 +84,6 @@ public String generateTableSource(
         List<TableColumn> includes = createTable.columns().stream()
             .filter(column -> column.constraints().stream()
                 .anyMatch(ZILLA_MAPPINGS::containsKey))
-
             .collect(Collectors.toCollection(ArrayList::new));
 
         if (!includes.isEmpty())

diff --git a/.../io/aklivity/zilla/runtime/binding/risingwave/internal/stream/RisingwaveProxyFactory.java b/.../io/aklivity/zilla/runtime/binding/risingwave/internal/stream/RisingwaveProxyFactory.java
@@ -855,6 +855,7 @@ private PgsqlClient(
             this.originId = originId;
             this.routedId = routedId;
             this.user = user;
+
         }
 
         private void onAppMessage(

diff --git a/manager/NOTICE b/manager/NOTICE
@@ -16,21 +16,45 @@ This project includes:
   Airline - Library under The Apache License, Version 2.0
   Apache Commons Collections under Apache License, Version 2.0
   Apache Commons Lang under Apache License, Version 2.0
-  Apache Ivy under The Apache Software License, Version 2.0
+  Apache HttpClient under Apache License, Version 2.0
+  Apache HttpCore under Apache License, Version 2.0
+  error-prone annotations under Apache 2.0
+  Gson under Apache-2.0
   Jakarta Dependency Injection under The Apache Software License, Version 2.0
   Jakarta JSON Processing API under Eclipse Public License 2.0 or GNU General Public License, version 2 with the GNU Classpath Exception
   javax.annotation API under CDDL + GPLv2 with classpath exception
+  javax.inject under The Apache Software License, Version 2.0
   JSON-B API under Eclipse Public License 2.0 or GNU General Public License, version 2 with the GNU Classpath Exception
   JSON-P Default Provider under Eclipse Public License 2.0 or GNU General Public License, version 2 with the GNU Classpath Exception
+  Maven Artifact under Apache-2.0
+  Maven Artifact Resolver API under Apache-2.0
+  Maven Artifact Resolver Connector Basic under Apache-2.0
+  Maven Artifact Resolver Implementation under Apache-2.0
+  Maven Artifact Resolver Instance Supplier Maven3 under Apache-2.0
+  Maven Artifact Resolver Named Locks under Apache-2.0
+  Maven Artifact Resolver Provider under Apache-2.0
+  Maven Artifact Resolver SPI under Apache-2.0
+  Maven Artifact Resolver Transport Apache under Apache-2.0
+  Maven Artifact Resolver Transport File under Apache-2.0
+  Maven Artifact Resolver Utilities under Apache-2.0
+  Maven Builder Support under Apache-2.0
+  Maven Model under Apache-2.0
+  Maven Model Builder under Apache-2.0
+  Maven Repository Metadata Model under Apache-2.0
+  Maven Settings under Apache-2.0
+  Maven Settings Builder under Apache-2.0
   org.eclipse.sisu.inject under Eclipse Public License, Version 1.0
   org.eclipse.sisu.plexus under Eclipse Public License, Version 1.0
   org.eclipse.yasson under Eclipse Public License v. 2.0 or Eclipse Distribution License v. 1.0
   Plexus :: Component Annotations under Apache License, Version 2.0
-  Plexus Cipher: encryption/decryption Component under Apache Public License 2.0
+  Plexus Cipher: encryption/decryption Component under Apache License, Version 2.0
   Plexus Classworlds under Apache License, Version 2.0
   Plexus Common Utilities under Apache License, Version 2.0
-  Plexus Security Dispatcher Component under Apache Public License 2.0
+  Plexus Interpolation API under Apache License, Version 2.0
+  Plexus Security Dispatcher Component under Apache License, Version 2.0
+  Plexus XML Utilities under Apache License, Version 2.0
   Sisu-Inject-Plexus : legacy wrapper under Eclipse Public License, Version 1.0
+  SLF4J API Module under MIT License
 
 
 This project also includes code under copyright of the following entities:
-Original file line number
+Diff line change
@@ Expand Up / @@ -73,6 +73,7 @@ write [0x00 0x02] # Field Count @@
           "CREATE MATERIALIZED VIEW IF NOT EXISTS distinct_cities AS" # Data
           " SELECT id, city FROM weather GROUP BY city;"
           [0x00]
     write flush
@@ Expand Down @@