diff --git a/README.md b/README.md index 8b1c2cea..4e6c5685 100644 --- a/README.md +++ b/README.md @@ -19,24 +19,24 @@ For information on Siddhi and i ## Latest API Docs -Latest API Docs is 2.3.3. +Latest API Docs is 2.3.4. ## Features -* http *(Sink)*

HTTP sink publishes messages via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML and JSON. It can also publish to endpoints protected by basic authentication or OAuth 2.0.

-* http-call *(Sink)*

The http-call sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-call-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

-* http-request *(Sink)*

_(Use http-call sink instead)._
The http-request sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

-* http-response *(Sink)*

_(Use http-service-response sink instead)._
The http-response sink send responses of the requests consumed by its corresponding http-request source, by mapping the response messages to formats such as text, XML and JSON.

-* http-service-response *(Sink)*

The http-service-response sink send responses of the requests consumed by its corresponding http-service source, by mapping the response messages to formats such as text, XML and JSON.

-* sse-server *(Sink)*

HTTP SSE sink sends events to all subscribers.

-* websubhub *(Sink)*

WebSubHubEventPublisher publishes messages via HTTP/HTTP according to the provided URL when subscribe to the WebSub hub. The table.name, hub.id and

-* http *(Source)*

HTTP source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON. It also supports basic authentication to ensure events are received from authorized users/systems.
The request headers and properties can be accessed via transport properties in the format trp:<header>.

-* http-call-response *(Source)*

The http-call-response source receives the responses for the calls made by its corresponding http-call sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-call-response sources are allowed to associate with a single http-call sink.
It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

-* http-request *(Source)*

_(Use http-service source instead)._
The http-request source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

-* http-response *(Source)*

_(Use http-call-response source instead)._
The http-response source receives the responses for the calls made by its corresponding http-request sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-response sources are allowed to associate with a single http-request sink. It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

-* http-service *(Source)*

The http-service source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-service-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-service-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

-* sse *(Source)*

HTTP SSE source send a request to a given url and listen to the response stream.

-* websubhub *(Source)*

WebSub Hub source receive subscription requests via Http and according to the request, the subscription details will be saved to the given table and against the callback and topic name. The subscription request MUST have a Content-Type header of application/x-www-form-urlencoded and following MUST provide as parameter body hub.callback      - REQUIRED. The subscriber's callback URL where content distribution notifications should be delivered. The callback URL SHOULD be an unguessable URL that is unique per subscription.
hub.mode      - REQUIRED. The literal string "subscribe" or "unsubscribe", depending on the goal of the request.
hub.topic      - REQUIRED. The topic URL that the subscriber wishes to subscribe to or unsubscribe from.
 hub.lease_seconds     - OPTIONAL. Number of seconds for which the subscriber would like to have the subscription active, given as a positive decimal integer.
hub.secret     - OPTIONAL. A subscriber-provided cryptographically random unique secret string that will be used to compute an HMAC digest for authorized content distribution. If not supplied, the HMAC digest will not be present for content distribution requests.

+* http *(Sink)*

HTTP sink publishes messages via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML and JSON. It can also publish to endpoints protected by basic authentication or OAuth 2.0.

+* http-call *(Sink)*

The http-call sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-call-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

+* http-request *(Sink)*

_(Use http-call sink instead)._
The http-request sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

+* http-response *(Sink)*

_(Use http-service-response sink instead)._
The http-response sink send responses of the requests consumed by its corresponding http-request source, by mapping the response messages to formats such as text, XML and JSON.

+* http-service-response *(Sink)*

The http-service-response sink send responses of the requests consumed by its corresponding http-service source, by mapping the response messages to formats such as text, XML and JSON.

+* sse-server *(Sink)*

HTTP SSE sink sends events to all subscribers.

+* websubhub *(Sink)*

WebSubHubEventPublisher publishes messages via HTTP/HTTP according to the provided URL when subscribe to the WebSub hub. The table.name, hub.id are mandatory when defining the websubhub source

+* http *(Source)*

HTTP source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON. It also supports basic authentication to ensure events are received from authorized users/systems.
The request headers and properties can be accessed via transport properties in the format trp:<header>.

+* http-call-response *(Source)*

The http-call-response source receives the responses for the calls made by its corresponding http-call sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-call-response sources are allowed to associate with a single http-call sink.
It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

+* http-request *(Source)*

_(Use http-service source instead)._
The http-request source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

+* http-response *(Source)*

_(Use http-call-response source instead)._
The http-response source receives the responses for the calls made by its corresponding http-request sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-response sources are allowed to associate with a single http-request sink. It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

+* http-service *(Source)*

The http-service source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-service-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-service-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

+* sse *(Source)*

HTTP SSE source send a request to a given url and listen to the response stream.

+* websubhub *(Source)*

WebSub Hub source receive subscription requests via Http and according to the request, the subscription details will be saved to the given table and against the callback and topic name. The subscription request **MUST** have a Content-Type header of **application/x-www-form-urlencoded** and following **MUST** provide as parameter body.
      hub.callback      - REQUIRED.     The subscriber's callback URL where content distribution notifications should be delivered. The callback URL SHOULD be an unguessable URL that is unique per subscription.
     hub.mode      - REQUIRED.     The literal string "subscribe" or "unsubscribe", depending on the goal of the request.
     hub.topic      - REQUIRED.     The topic URL that the subscriber wishes to subscribe to or unsubscribe from.
      hub.lease_seconds     - OPTIONAL.     Number of seconds for which the subscriber would like to have the subscription active, given as a positive decimal integer.
     hub.secret     - OPTIONAL.     A subscriber-provided cryptographically random unique secret string that will be used to compute an HMAC digest for authorized content distribution. If not supplied, the HMAC digest will not be present for content distribution requests.

## Dependencies diff --git a/component/pom.xml b/component/pom.xml index 7b49c7be..8c4c4ce7 100644 --- a/component/pom.xml +++ b/component/pom.xml @@ -18,7 +18,7 @@ siddhi-io-http-parent io.siddhi.extension.io.http - 2.3.3 + 2.3.4 ../pom.xml 4.0.0 @@ -49,10 +49,6 @@ io.siddhi siddhi-query-compiler - - org.wso2.carbon.messaging - org.wso2.carbon.messaging - apache-collections commons-collections @@ -98,6 +94,18 @@ org.wso2.carbon.analytics org.wso2.carbon.si.metrics.core + + com.squareup.okhttp3 + okhttp + + + org.apache.logging.log4j + log4j-core + + + org.apache.logging.log4j + log4j-api + @@ -141,7 +149,7 @@ !org.wso2.transport.http.netty.*, - !io.netty.*, + !io.netty.*;version="${netty.version.range}", io.siddhi.core.*;version="${siddhi.version.range}", io.siddhi.annotation.*;version="${siddhi.version.range}", io.siddhi.query.api.*;version="${siddhi.version.range}", diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/HttpCallSink.java b/component/src/main/java/io/siddhi/extension/io/http/sink/HttpCallSink.java index f37ceafc..49348730 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/HttpCallSink.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/HttpCallSink.java @@ -40,7 +40,8 @@ import io.siddhi.extension.io.http.util.HttpConstants; import io.siddhi.query.api.definition.Attribute; import io.siddhi.query.api.definition.StreamDefinition; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.carbon.messaging.Header; import org.wso2.transport.http.netty.contract.Constants; import org.wso2.transport.http.netty.contract.HttpResponseFuture; @@ -435,7 +436,7 @@ ) public class HttpCallSink extends HttpSink { - private static final Logger log = Logger.getLogger(HttpCallSink.class); + private static final Logger log = LogManager.getLogger(HttpCallSink.class); private String sinkId; private boolean isDownloadEnabled; private Option downloadPath; diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/HttpServiceResponseSink.java b/component/src/main/java/io/siddhi/extension/io/http/sink/HttpServiceResponseSink.java index 96e3bb2c..94b6927b 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/HttpServiceResponseSink.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/HttpServiceResponseSink.java @@ -35,7 +35,6 @@ import io.siddhi.extension.io.http.util.HTTPSourceRegistry; import io.siddhi.extension.io.http.util.HttpConstants; import io.siddhi.query.api.definition.StreamDefinition; -import org.apache.log4j.Logger; import org.wso2.carbon.messaging.Header; import java.util.List; @@ -118,7 +117,6 @@ ) public class HttpServiceResponseSink extends Sink { - private static final Logger log = Logger.getLogger(HttpServiceResponseSink.class); private Option messageIdOption; private String sourceId; private Option httpHeaderOption; diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/HttpSink.java b/component/src/main/java/io/siddhi/extension/io/http/sink/HttpSink.java index 8ae04fe6..f45fcc12 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/HttpSink.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/HttpSink.java @@ -53,7 +53,8 @@ import io.siddhi.extension.io.http.util.HttpConstants; import io.siddhi.extension.io.http.util.HttpIoUtil; import io.siddhi.query.api.definition.StreamDefinition; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.carbon.messaging.Header; import org.wso2.carbon.si.metrics.core.internal.MetricsDataHolder; import org.wso2.transport.http.netty.contract.Constants; @@ -165,6 +166,19 @@ type = {DataType.STRING}, optional = true, defaultValue = "-"), + @Parameter( + name = "body.consumer.key", + description = "Consumer key used for calling endpoints protected by OAuth 2.0 if it's " + + "required to be sent in token request body", + type = {DataType.STRING}, + optional = true, + defaultValue = "-"), + @Parameter( + name = "body.consumer.secret", + description = "Consumer secret used for calling endpoints protected by OAuth 2.0", + type = {DataType.STRING}, + optional = true, + defaultValue = "-"), @Parameter( name = "token.url", description = "Token URL to generate a new access tokens " + @@ -179,6 +193,13 @@ type = {DataType.STRING}, optional = true, defaultValue = "-"), + @Parameter( + name = HttpConstants.OAUTH2_SCOPE_PARAMETER_NAME, + description = "Standard OAuth 2.0 scope parameter", + type = {DataType.STRING}, + optional = true, + defaultValue = "default" + ), @Parameter( name = "headers", description = "HTTP request headers in format `\"':',':'\"`.\n" + @@ -299,6 +320,12 @@ type = {DataType.INT}, optional = true, defaultValue = "100"), + @Parameter( + name = "executor.service.threads", + description = "Thread count for the executor service.", + type = {DataType.INT}, + optional = true, + defaultValue = "20"), @Parameter( name = "min.evictable.idle.time", description = "Minimum time (in millis) a connection may sit idle in the " + @@ -431,18 +458,23 @@ } ) public class HttpSink extends Sink { - private static final Logger log = Logger.getLogger(HttpSink.class); - String mapType; - Option httpHeaderOption; - Option httpMethodOption; + private static final Logger log = LogManager.getLogger(HttpSink.class); protected String streamID; protected String consumerKey; protected String consumerSecret; - private String authorizationHeader; protected String userName; protected String userPassword; protected ClientConnector staticClientConnector; protected Option publisherURLOption; + protected SiddhiAppContext siddhiAppContext; + protected String oauthUsername; + protected String oauthUserPassword; + protected String authType; + protected String tokenURL; + String mapType; + Option httpHeaderOption; + Option httpMethodOption; + private String authorizationHeader; private String clientStoreFile; private String clientStorePass; private int socketIdleTimeout; @@ -452,13 +484,8 @@ public class HttpSink extends Sink { private String parametersList; private String clientBootstrapConfiguration; private ConfigReader configReader; - protected SiddhiAppContext siddhiAppContext; - protected String oauthUsername; - protected String oauthUserPassword; private Option refreshToken; - protected String authType; private AccessTokenCache accessTokenCache = AccessTokenCache.getInstance(); - protected String tokenURL; private String hostnameVerificationEnabled; private String sslVerificationDisabled; private Executor executor = null; @@ -466,6 +493,9 @@ public class HttpSink extends Sink { protected SinkMetrics metrics; protected long startTime; protected long endTime; + private String bodyConsumerKey; + private String bodyConsumerSecret; + private String oauth2Scope; private DefaultHttpWsConnectorFactory httpConnectorFactory; private ProxyServerConfiguration proxyServerConfiguration; @@ -519,6 +549,9 @@ protected StateFactory init(StreamDefinition outputStreamDefinition, OptionHolde this.httpMethodOption = optionHolder.getOrCreateOption(HttpConstants.METHOD, HttpConstants.DEFAULT_METHOD); this.consumerKey = optionHolder.validateAndGetStaticValue(HttpConstants.CONSUMER_KEY, EMPTY_STRING); this.consumerSecret = optionHolder.validateAndGetStaticValue(HttpConstants.CONSUMER_SECRET, EMPTY_STRING); + this.bodyConsumerKey = optionHolder.validateAndGetStaticValue(HttpConstants.BODY_CONSUMER_KEY, EMPTY_STRING); + this.bodyConsumerSecret = optionHolder.validateAndGetStaticValue(HttpConstants.BODY_CONSUMER_SECRET, + EMPTY_STRING); this.oauthUsername = optionHolder.validateAndGetStaticValue(HttpConstants.RECEIVER_OAUTH_USERNAME, EMPTY_STRING); this.oauthUserPassword = optionHolder.validateAndGetStaticValue(HttpConstants.RECEIVER_OAUTH_PASSWORD, @@ -527,6 +560,8 @@ protected StateFactory init(StreamDefinition outputStreamDefinition, OptionHolde this.tokenURL = optionHolder.validateAndGetStaticValue(HttpConstants.TOKEN_URL, EMPTY_STRING); this.clientStoreFile = optionHolder.validateAndGetStaticValue(HttpConstants.CLIENT_TRUSTSTORE_PATH_PARAM, HttpSinkUtil.trustStorePath(configReader)); + this.oauth2Scope = optionHolder.validateAndGetStaticValue(HttpConstants.OAUTH2_SCOPE_PARAMETER_NAME, + "default"); clientStorePass = optionHolder.validateAndGetStaticValue(HttpConstants.CLIENT_TRUSTSTORE_PASSWORD_PARAM, HttpSinkUtil.trustStorePassword(configReader)); socketIdleTimeout = Integer.parseInt(optionHolder.validateAndGetStaticValue @@ -538,6 +573,8 @@ protected StateFactory init(StreamDefinition outputStreamDefinition, OptionHolde //pool configurations connectionPoolConfiguration = createPoolConfigurations(optionHolder); + executor = Executors.newFixedThreadPool(connectionPoolConfiguration.getExecutorServiceThreads()); + parametersList = optionHolder.validateAndGetStaticValue(HttpConstants.SINK_PARAMETERS, EMPTY_STRING); clientBootstrapConfiguration = optionHolder @@ -554,6 +591,8 @@ protected StateFactory init(StreamDefinition outputStreamDefinition, OptionHolde authType = HttpConstants.BASIC_AUTH; } else if ((!HttpConstants.EMPTY_STRING.equals(consumerKey) && !HttpConstants.EMPTY_STRING.equals(consumerSecret)) || + (!HttpConstants.EMPTY_STRING.equals(bodyConsumerKey) + && !HttpConstants.EMPTY_STRING.equals(bodyConsumerSecret)) || (!HttpConstants.EMPTY_STRING.equals(oauthUsername) && !HttpConstants.EMPTY_STRING.equals(oauthUserPassword))) { authType = HttpConstants.OAUTH; @@ -676,6 +715,7 @@ protected int sendRequest(Object payload, DynamicOptions dynamicOptions, List bootStrapProperties = HttpSinkUtil .populateTransportConfiguration(clientBootstrapConfiguration); diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/SSEConnectorRegistry.java b/component/src/main/java/io/siddhi/extension/io/http/sink/SSEConnectorRegistry.java index 8c58ffcf..5f15b3cb 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/SSEConnectorRegistry.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/SSEConnectorRegistry.java @@ -25,7 +25,8 @@ import io.siddhi.extension.io.http.source.exception.HttpSourceAdaptorRuntimeException; import io.siddhi.extension.io.http.source.util.HttpSourceUtil; import io.siddhi.extension.io.http.util.HttpConstants; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.transport.http.netty.contract.ServerConnector; import org.wso2.transport.http.netty.contract.ServerConnectorFuture; import org.wso2.transport.http.netty.contract.config.InboundMsgSizeValidationConfig; @@ -52,7 +53,7 @@ */ public class SSEConnectorRegistry { private static SSEConnectorRegistry instance = new SSEConnectorRegistry(); - private final Logger log = Logger.getLogger(SSEConnectorRegistry.class); + private static final Logger log = LogManager.getLogger(SSEConnectorRegistry.class); protected TransportsConfiguration trpConfig; protected DefaultHttpWsConnectorFactory httpConnectorFactory; private Map serverConnectorPool = new ConcurrentHashMap<>(); diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/SSEServerSink.java b/component/src/main/java/io/siddhi/extension/io/http/sink/SSEServerSink.java index d2a53296..21e422f5 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/SSEServerSink.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/SSEServerSink.java @@ -47,7 +47,8 @@ import io.siddhi.extension.io.http.util.HttpConstants; import io.siddhi.extension.io.http.util.HttpIoUtil; import io.siddhi.query.api.definition.StreamDefinition; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.carbon.messaging.Header; import org.wso2.carbon.si.metrics.core.internal.MetricsDataHolder; import org.wso2.transport.http.netty.contract.HttpConnectorListener; @@ -171,8 +172,7 @@ } ) public class SSEServerSink extends Sink { - private static final Logger logger = Logger.getLogger(SSEServerSink.class); - + private static final Logger logger = LogManager.getLogger(SSEServerSink.class); private String siddhiAppName; private String streamId; private String listenerUrl; diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/SSESyncConnectorRegistry.java b/component/src/main/java/io/siddhi/extension/io/http/sink/SSESyncConnectorRegistry.java index c70561fc..26c204aa 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/SSESyncConnectorRegistry.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/SSESyncConnectorRegistry.java @@ -23,7 +23,6 @@ import io.siddhi.extension.io.http.source.HttpConnectorPortBindingListener; import io.siddhi.extension.io.http.source.util.HttpSourceUtil; import io.siddhi.extension.io.http.util.HttpConstants; -import org.apache.log4j.Logger; import org.wso2.transport.http.netty.contract.ServerConnectorFuture; import org.wso2.transport.http.netty.contractimpl.DefaultHttpWsConnectorFactory; @@ -36,7 +35,6 @@ public class SSESyncConnectorRegistry extends SSEConnectorRegistry { private static SSESyncConnectorRegistry instance = new SSESyncConnectorRegistry(); - private final Logger log = Logger.getLogger(SSESyncConnectorRegistry.class); private Map sourceListenersMap = new ConcurrentHashMap<>(); private SSESyncConnectorRegistry() { diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/SSEWorkerThread.java b/component/src/main/java/io/siddhi/extension/io/http/sink/SSEWorkerThread.java index 70ef0c99..ceb813ad 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/SSEWorkerThread.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/SSEWorkerThread.java @@ -58,9 +58,10 @@ public void run() { try { buf.close(); - carbonMessage.waitAndReleaseAllEntities(); } catch (IOException e) { logger.error("Error occurred when closing the byte buffer in source " + streamID, e); + } finally { + carbonMessage.waitAndReleaseAllEntities(); } } } diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/WebSubHubSink.java b/component/src/main/java/io/siddhi/extension/io/http/sink/WebSubHubSink.java index 903e773c..697c9d5a 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/WebSubHubSink.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/WebSubHubSink.java @@ -38,7 +38,6 @@ import io.siddhi.core.event.stream.StreamEvent; import io.siddhi.core.exception.ConnectionUnavailableException; import io.siddhi.core.exception.SiddhiAppCreationException; -import io.siddhi.core.exception.SiddhiAppRuntimeException; import io.siddhi.core.query.OnDemandQueryRuntime; import io.siddhi.core.stream.ServiceDeploymentInfo; import io.siddhi.core.stream.output.sink.Sink; @@ -71,7 +70,8 @@ import io.siddhi.query.api.expression.condition.Compare; import io.siddhi.query.api.expression.constant.StringConstant; import org.apache.commons.codec.digest.DigestUtils; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.carbon.messaging.Header; import org.wso2.carbon.si.metrics.core.internal.MetricsDataHolder; import org.wso2.transport.http.netty.contract.Constants; @@ -89,8 +89,6 @@ import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.InputStream; -import java.io.UnsupportedEncodingException; -import java.net.URLEncoder; import java.nio.charset.Charset; import java.nio.charset.StandardCharsets; import java.util.ArrayList; @@ -131,7 +129,7 @@ @Extension(name = "websubhub", namespace = "sink", description = "" + "WebSubHubEventPublisher publishes messages via HTTP/HTTP according to the provided URL when " + - "subscribe to the WebSub hub. The table.name, hub.id and ", + "subscribe to the WebSub hub. The table.name, hub.id are mandatory when defining the websubhub source ", parameters = { @Parameter( name = "hub.id", @@ -299,6 +297,12 @@ type = {DataType.INT}, optional = true, defaultValue = "100"), + @Parameter( + name = "executor.service.threads", + description = "Thread count for the executor service.", + type = {DataType.INT}, + optional = true, + defaultValue = "20"), @Parameter( name = "min.evictable.idle.time", description = "Minimum time (in millis) a connection may sit idle in the " + @@ -399,7 +403,7 @@ ) public class WebSubHubSink extends Sink { - private static final Logger log = Logger.getLogger(WebSubHubSink.class); + private static final Logger log = LogManager.getLogger(WebSubHubSink.class); private final String[] outputColumns = new String[]{HUB_CALLBACK, HUB_TOPIC, HUB_SECRET, HUB_LEASE_SECONDS, REQUEST_TIMESTAMP}; protected String streamID; @@ -490,6 +494,7 @@ protected StateFactory init(StreamDefinition outputStreamDefinition, OptionHolde //pool configurations connectionPoolConfiguration = createPoolConfigurations(optionHolder); + executor = Executors.newFixedThreadPool(connectionPoolConfiguration.getExecutorServiceThreads()); parametersList = optionHolder.validateAndGetStaticValue(HttpConstants.SINK_PARAMETERS, EMPTY_STRING); clientBootstrapConfiguration = optionHolder .validateAndGetStaticValue(HttpConstants.CLIENT_BOOTSTRAP_CONFIGURATION, EMPTY_STRING); @@ -534,7 +539,7 @@ public void publish(Object payload, DynamicOptions dynamicOptions, State state) if (mapType == null) { mapType = getMapper().getType(); } - sendRequest(payload, dynamicOptions, headersList, clientConnector, publisherURL); + sendRequest(payloadMap, dynamicOptions, headersList, clientConnector, publisherURL); } else { log.debug("Added to expired subscription list " + webSubDTO.getCallback() + " : " + webSubDTO.getTopic()); @@ -701,9 +706,7 @@ HttpCarbonMessage generateCarbonMessage(List
headers, String contentType String getMessageBody(Object payload) { if (HttpConstants.MAP_KEYVALUE.equals(mapType)) { Map params = (HashMap) payload; - return params.entrySet().stream() - .map(p -> encodeMessage(p.getKey()) + "=" + encodeMessage(p.getValue())) - .reduce("", (p1, p2) -> p1 + "&" + p2); + return params.get("payload").toString(); } else { return (String) payload; } @@ -760,9 +763,6 @@ public ClientConnector createClientConnector(String publisherURL) { senderConfig.disableSsl(); } - executor = Executors.newFixedThreadPool( - senderConfig.getPoolConfiguration().getExecutorServiceThreads()); - //overwrite default transport configuration Map bootStrapProperties = HttpSinkUtil .populateTransportConfiguration(clientBootstrapConfiguration); @@ -770,15 +770,6 @@ public ClientConnector createClientConnector(String publisherURL) { httpConnectorFactory.createHttpClientConnector(bootStrapProperties, senderConfig)); } - private String encodeMessage(Object s) { - try { - return URLEncoder.encode((String) s, HttpConstants.DEFAULT_ENCODING); - } catch (UnsupportedEncodingException e) { - throw new SiddhiAppRuntimeException("Execution of Siddhi app " + siddhiAppContext.getName() + - " failed due to " + e.getMessage(), e); - } - } - private Table getSubscriptionTable(String tableName) { TableDefinition tableDefinition = TableDefinition.id(tableName); List annotationsInSourceDefinition = outputStreamDefinition.getAnnotations(); @@ -823,7 +814,6 @@ private void setOnDemandQueryRuntimeForFindSubscription() { siddhiAppContext, tableMap, windowMap, aggregationRuntimeMap); } - public void setWebSubSubscriptionMap(Map> webSubSubscriptionMap) { this.webSubSubscriptionMap = webSubSubscriptionMap; } diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/updatetoken/HttpRequest.java b/component/src/main/java/io/siddhi/extension/io/http/sink/updatetoken/HttpRequest.java index b5200f2a..56500e69 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/updatetoken/HttpRequest.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/updatetoken/HttpRequest.java @@ -18,87 +18,52 @@ */ package io.siddhi.extension.io.http.sink.updatetoken; -import io.netty.buffer.Unpooled; -import io.netty.handler.codec.http.DefaultHttpRequest; -import io.netty.handler.codec.http.DefaultLastHttpContent; -import io.netty.handler.codec.http.HttpMethod; -import io.netty.handler.codec.http.HttpVersion; +import io.netty.handler.codec.http.HttpHeaderValues; import io.siddhi.extension.io.http.util.HttpConstants; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.wso2.transport.http.netty.contract.Constants; -import org.wso2.transport.http.netty.contract.HttpClientConnector; -import org.wso2.transport.http.netty.contract.HttpResponseFuture; -import org.wso2.transport.http.netty.message.HttpCarbonMessage; -import org.wso2.transport.http.netty.message.HttpMessageDataStreamer; +import okhttp3.Headers; +import okhttp3.MediaType; +import okhttp3.OkHttpClient; +import okhttp3.Request; +import okhttp3.RequestBody; +import okhttp3.Response; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; -import java.io.BufferedReader; import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.nio.ByteBuffer; -import java.nio.charset.Charset; +import java.nio.charset.StandardCharsets; import java.util.ArrayList; +import java.util.List; import java.util.Map; -import java.util.concurrent.CountDownLatch; -import java.util.concurrent.TimeUnit; -import java.util.stream.Collectors; + +import static io.siddhi.extension.io.http.util.HttpConstants.HTTP_CONTENT_TYPE; +import static io.siddhi.extension.io.http.util.HttpConstants.HTTP_METHOD_POST; /** * {@code HttpRequest} Handle the HTTP request for invalid access token. */ public class HttpRequest { - private static final Logger LOG = LoggerFactory.getLogger(HttpRequest.class); + private static final Logger log = LogManager.getLogger(HttpRequest.class); + public static List getResponse(String tokenUrl, String encodedAuth, String payload, OkHttpClient client, + Map headers) { + List responses = new ArrayList<>(); + MediaType mediaType = MediaType.parse(String.valueOf(HttpHeaderValues.APPLICATION_X_WWW_FORM_URLENCODED)); + RequestBody requestBody = RequestBody.create(payload.getBytes(StandardCharsets.UTF_8), mediaType); + headers.put(HTTP_CONTENT_TYPE, String.valueOf(HttpHeaderValues.APPLICATION_X_WWW_FORM_URLENCODED)); + headers.put(HttpConstants.AUTHORIZATION_HEADER, encodedAuth); - public static ArrayList sendPostRequest(HttpClientConnector httpClientConnector, String serverScheme, - String serverHost, int serverPort, String serverPath, - String payload, Map headers) { - ArrayList responses = new ArrayList<>(); + Headers headerbuilder = Headers.of(headers); - HttpCarbonMessage msg = createHttpPostReq(serverScheme, serverHost, serverPort, serverPath, payload); - for (Map.Entry entry : headers.entrySet()) { - msg.setHeader(entry.getKey(), entry.getValue()); - } - CountDownLatch latch = new CountDownLatch(1); - DefaultListener listener = new DefaultListener(latch, HttpConstants.OAUTH); - HttpResponseFuture responseFuture = httpClientConnector.send(msg); - responseFuture.setHttpConnectorListener(listener); + Request request = new Request.Builder().url(tokenUrl).method(HTTP_METHOD_POST, requestBody) + .headers(headerbuilder).build(); try { - boolean latchCount = latch.await(30, TimeUnit.SECONDS); - if (!latchCount) { - LOG.debug("Time out due to getting new access token. "); - } - } catch (InterruptedException e) { - LOG.debug("Time out due to getting new access token. " + e); - } - HttpCarbonMessage response = listener.getHttpResponseMessage(); - String statusCode = Integer.toString(response.getNettyHttpResponse().status().code()); - responses.add(statusCode); - InputStream httpMessageDataStreamer = new HttpMessageDataStreamer(response).getInputStream(); - InputStreamReader inputStreamReader = new InputStreamReader(httpMessageDataStreamer, Charset.defaultCharset()); - try (BufferedReader buffer = new BufferedReader(inputStreamReader)) { - String responsePayload = buffer.lines().collect(Collectors.joining("\n")); - responses.add(responsePayload); + Response response = client.newCall(request).execute(); + responses.add(String.valueOf(response.code())); + responses.add(response.body().string()); + response.close(); } catch (IOException e) { - LOG.debug("There was an error in reading the file while generating new access token. " + e); + log.error("Error occurred while generating a new access token ", e); } return responses; } - - private static HttpCarbonMessage createHttpPostReq(String serverScheme, String serverHost, int serverPort, - String serverPath, String payload) { - HttpCarbonMessage httpPostRequest = new HttpCarbonMessage( - new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.POST, serverPath)); - httpPostRequest.setProperty(Constants.PROTOCOL, serverScheme); - httpPostRequest.setProperty(Constants.HTTP_HOST, serverHost); - httpPostRequest.setProperty(Constants.HTTP_PORT, serverPort); - httpPostRequest.setProperty(Constants.TO, serverPath); - httpPostRequest.setHttpMethod(Constants.HTTP_POST_METHOD); - // this header is mandatory otherwise the request would result in 400 status - httpPostRequest.setHeader(Constants.HTTP_HOST, serverHost); - ByteBuffer byteBuffer = ByteBuffer.wrap(payload.getBytes(Charset.forName("UTF-8"))); - httpPostRequest.addHttpContent(new DefaultLastHttpContent(Unpooled.wrappedBuffer(byteBuffer))); - return httpPostRequest; - } } diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/updatetoken/HttpsClient.java b/component/src/main/java/io/siddhi/extension/io/http/sink/updatetoken/HttpsClient.java index 443b8334..5cb29ca4 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/updatetoken/HttpsClient.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/updatetoken/HttpsClient.java @@ -19,23 +19,31 @@ package io.siddhi.extension.io.http.sink.updatetoken; import io.netty.handler.codec.http.HttpHeaderValues; -import io.siddhi.extension.io.http.sink.util.HttpSinkUtil; import io.siddhi.extension.io.http.util.HttpConstants; +import okhttp3.OkHttpClient; import org.json.JSONObject; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import org.wso2.transport.http.netty.contract.Constants; -import org.wso2.transport.http.netty.contract.HttpClientConnector; -import org.wso2.transport.http.netty.contract.config.SenderConfiguration; -import org.wso2.transport.http.netty.contractimpl.DefaultHttpWsConnectorFactory; +import org.wso2.transport.http.netty.contractimpl.common.Util; +import java.io.File; +import java.io.FileInputStream; +import java.io.IOException; import java.io.UnsupportedEncodingException; import java.net.URLEncoder; -import java.util.ArrayList; +import java.security.KeyManagementException; +import java.security.KeyStore; +import java.security.KeyStoreException; +import java.security.NoSuchAlgorithmException; +import java.security.SecureRandom; +import java.security.UnrecoverableKeyException; +import java.security.cert.CertificateException; import java.util.HashMap; +import java.util.List; import java.util.Map; - -import static org.wso2.transport.http.netty.contract.Constants.HTTPS_SCHEME; +import javax.net.ssl.KeyManagerFactory; +import javax.net.ssl.SSLContext; +import javax.net.ssl.TrustManagerFactory; /** @@ -44,16 +52,6 @@ public class HttpsClient { private static final Logger LOG = LoggerFactory.getLogger(HttpsClient.class); private AccessTokenCache accessTokenCache = AccessTokenCache.getInstance(); - private Map tokenURLProperties; - - private static SenderConfiguration getSenderConfigurationForHttp(String trustStorePath, String trustStorePassword) { - SenderConfiguration senderConfiguration = new SenderConfiguration(); - senderConfiguration.setTrustStoreFile(trustStorePath); - senderConfiguration.setTrustStorePass(trustStorePassword); - senderConfiguration.setScheme(HTTPS_SCHEME); - senderConfiguration.setHostNameVerificationEnabled(false); - return senderConfiguration; - } private static String encodeMessage(Object s) { try { @@ -79,56 +77,100 @@ private static HashMap setHeaders(String encodedAuth) { return headers; } + private static OkHttpClient getOkHttpClient(String trustStorePath, String trustStorePassword) { + KeyStore keyStore; + try { + keyStore = readKeyStore(trustStorePath, trustStorePassword); + SSLContext sslContext = SSLContext.getInstance("SSL"); + TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance( + TrustManagerFactory.getDefaultAlgorithm()); + trustManagerFactory.init(keyStore); + KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance( + KeyManagerFactory.getDefaultAlgorithm()); + keyManagerFactory.init(keyStore, "keystore_pass".toCharArray()); + sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), + new SecureRandom()); + + return new OkHttpClient().newBuilder() + .sslSocketFactory(sslContext.getSocketFactory()) + .hostnameVerifier((host, sslSession) -> true).build(); + } catch (IOException | NoSuchAlgorithmException | UnrecoverableKeyException | KeyStoreException + | KeyManagementException e) { + LOG.error("Error occurred while initializing the http client, Returning normal client", e); + return new OkHttpClient(); + } + } + + private static KeyStore readKeyStore(String trustStorePath, String trustStorePassword) + throws IOException, KeyStoreException { + KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType()); + + // Get user password and file input stream + char[] password = trustStorePassword.toCharArray(); + FileInputStream fis = null; + try { + File file = new File(Util.substituteVariables(trustStorePath)); + fis = new FileInputStream(file); + ks.load(fis, password); + } catch (CertificateException | NoSuchAlgorithmException | IOException e) { + LOG.error(e.getMessage(), e); + } finally { + if (fis != null) { + fis.close(); + } + } + return ks; + } + public void getPasswordGrantAccessToken(String tokenUrl, String trustStorePath, String trustStorePassword, - String username, String password, String encodedAuth) { - tokenURLProperties = HttpSinkUtil.getURLProperties(tokenUrl); - DefaultHttpWsConnectorFactory factory = new DefaultHttpWsConnectorFactory(); - HttpClientConnector httpClientConnector = factory - .createHttpClientConnector(new HashMap<>(), getSenderConfigurationForHttp(trustStorePath, - trustStorePassword)); + String username, String password, String encodedAuth, String consumerKey, + String consumerSecret, String oAuth2Scope) { final Map refreshTokenBody = new HashMap<>(); refreshTokenBody.put(HttpConstants.GRANT_TYPE, HttpConstants.GRANT_PASSWORD); refreshTokenBody.put(HttpConstants.USERNAME, username); refreshTokenBody.put(HttpConstants.PASSWORD, password); - String payload = getPayload(refreshTokenBody); + if (!HttpConstants.EMPTY_STRING.equals(oAuth2Scope)) { + refreshTokenBody.put(HttpConstants.OAUTH2_SCOPE_PARAMETER_KEY, oAuth2Scope); + } + if (!HttpConstants.EMPTY_STRING.equals(consumerKey)) { + refreshTokenBody.put(HttpConstants.OAUTH_CLIENT_ID, consumerKey); + } + if (!HttpConstants.EMPTY_STRING.equals(consumerSecret)) { + refreshTokenBody.put(HttpConstants.OAUTH_CLIENT_SECRET, consumerSecret); + } + + OkHttpClient client = getOkHttpClient(trustStorePath, trustStorePassword); Map headers = setHeaders(encodedAuth); - ArrayList response = HttpRequest.sendPostRequest(httpClientConnector, - tokenURLProperties.get(HttpConstants.PROTOCOL), tokenURLProperties.get(Constants.HTTP_HOST), - Integer.parseInt(tokenURLProperties.get(Constants.HTTP_PORT)), - tokenURLProperties.get(HttpConstants.PATH), payload, + List response = HttpRequest.getResponse(tokenUrl, encodedAuth, getPayload(refreshTokenBody), client, headers); + client.dispatcher().executorService().shutdown(); JSONObject jsonObject = new JSONObject(response.get(1)); int statusCode = Integer.parseInt(response.get(0)); if (statusCode == HttpConstants.SUCCESS_CODE) { String accessToken = jsonObject.getString(HttpConstants.ACCESS_TOKEN); accessTokenCache.setAccessToken(encodedAuth, HttpConstants.BEARER + accessToken); - if (jsonObject.has(HttpConstants.REFRESH_TOKEN)) { - String newRefreshToken = jsonObject.getString(HttpConstants.REFRESH_TOKEN); + String newRefreshToken = jsonObject.getString(HttpConstants.REFRESH_TOKEN); + if (newRefreshToken != null) { accessTokenCache.setRefreshtoken(encodedAuth, newRefreshToken); } accessTokenCache.setResponseCode(encodedAuth, statusCode); } else { accessTokenCache.setResponseCode(encodedAuth, statusCode); } - factory.shutdownNow(); } public void getRefreshGrantAccessToken(String url, String trustStorePath, String trustStorePassword, - String encodedAuth, String refreshToken) { - tokenURLProperties = HttpSinkUtil.getURLProperties(url); - DefaultHttpWsConnectorFactory factory = new DefaultHttpWsConnectorFactory(); - HttpClientConnector httpClientConnector = factory - .createHttpClientConnector(new HashMap<>(), getSenderConfigurationForHttp(trustStorePath, - trustStorePassword)); + String encodedAuth, String refreshToken, String oauthUsername, + String oauthUserPassword, String bodyConsumerKey, String bodyConsumerSecret, + String oauth2Scope) { final Map refreshTokenBody = new HashMap<>(); Map headers = setHeaders(encodedAuth); refreshTokenBody.put(HttpConstants.GRANT_TYPE, HttpConstants.GRANT_REFRESHTOKEN); refreshTokenBody.put(HttpConstants.GRANT_REFRESHTOKEN, refreshToken); - String payload = getPayload(refreshTokenBody); - ArrayList response = HttpRequest.sendPostRequest(httpClientConnector, - tokenURLProperties.get(HttpConstants.PROTOCOL), tokenURLProperties.get(Constants.HTTP_HOST), - Integer.parseInt(tokenURLProperties.get(Constants.HTTP_PORT)), - tokenURLProperties.get(HttpConstants.PATH), payload, headers); + OkHttpClient client = getOkHttpClient(trustStorePath, trustStorePassword); + List response = HttpRequest.getResponse(url, encodedAuth, getPayload(refreshTokenBody), client, + headers); + client.dispatcher().executorService().shutdown(); int statusCode = Integer.parseInt(response.get(0)); JSONObject jsonObject = new JSONObject(response.get(1)); if (statusCode == HttpConstants.SUCCESS_CODE) { @@ -137,30 +179,29 @@ public void getRefreshGrantAccessToken(String url, String trustStorePath, String accessTokenCache.setAccessToken(encodedAuth, HttpConstants.BEARER + accessToken); accessTokenCache.setRefreshtoken(encodedAuth, newRefreshToken); accessTokenCache.setResponseCode(encodedAuth, statusCode); + } else if ((statusCode == HttpConstants.AUTHENTICATION_FAIL_CODE + || statusCode == HttpConstants.PERSISTENT_ACCESS_FAIL_CODE) && + (!HttpConstants.EMPTY_STRING.equals(oauthUsername) && + !HttpConstants.EMPTY_STRING.equals(oauthUserPassword))) { + getPasswordGrantAccessToken(url, trustStorePath, trustStorePassword, oauthUsername, + oauthUserPassword, encodedAuth, bodyConsumerKey, bodyConsumerSecret, oauth2Scope); } else if (statusCode == HttpConstants.AUTHENTICATION_FAIL_CODE || statusCode == HttpConstants.PERSISTENT_ACCESS_FAIL_CODE) { getClientGrantAccessToken(url, trustStorePath, trustStorePassword, encodedAuth); } else { accessTokenCache.setResponseCode(encodedAuth, statusCode); } - factory.shutdownNow(); } public void getClientGrantAccessToken(String url, String trustStorePath, String trustStorePassword, String encodedAuth) { - tokenURLProperties = HttpSinkUtil.getURLProperties(url); - DefaultHttpWsConnectorFactory factory = new DefaultHttpWsConnectorFactory(); - HttpClientConnector httpClientConnector = factory - .createHttpClientConnector(new HashMap<>(), getSenderConfigurationForHttp(trustStorePath, - trustStorePassword)); final Map refreshTokenBody = new HashMap<>(); refreshTokenBody.put(HttpConstants.GRANT_TYPE, HttpConstants.GRANT_CLIENTTOKEN); - String payload = getPayload(refreshTokenBody); + OkHttpClient client = getOkHttpClient(trustStorePath, trustStorePassword); Map headers = setHeaders(encodedAuth); - ArrayList response = HttpRequest.sendPostRequest(httpClientConnector, - tokenURLProperties.get(HttpConstants.PROTOCOL), tokenURLProperties.get(Constants.HTTP_HOST), - Integer.parseInt(tokenURLProperties.get(Constants.HTTP_PORT)), - tokenURLProperties.get(HttpConstants.PATH), payload, headers); + List response = HttpRequest.getResponse(url, encodedAuth, getPayload(refreshTokenBody), client, + headers); + client.dispatcher().executorService().shutdown(); JSONObject jsonObject = new JSONObject(response.get(1)); int statusCode = Integer.parseInt(response.get(0)); if (statusCode == HttpConstants.SUCCESS_CODE) { @@ -170,6 +211,5 @@ public void getClientGrantAccessToken(String url, String trustStorePath, String } else { accessTokenCache.setResponseCode(encodedAuth, statusCode); } - factory.shutdownNow(); } } diff --git a/component/src/main/java/io/siddhi/extension/io/http/sink/util/HttpSinkUtil.java b/component/src/main/java/io/siddhi/extension/io/http/sink/util/HttpSinkUtil.java index 74deb725..9be92b34 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/sink/util/HttpSinkUtil.java +++ b/component/src/main/java/io/siddhi/extension/io/http/sink/util/HttpSinkUtil.java @@ -24,7 +24,8 @@ import io.siddhi.extension.io.http.sink.exception.HttpSinkAdaptorRuntimeException; import io.siddhi.extension.io.http.util.HttpConstants; import io.siddhi.extension.io.http.util.TrpPropertyTypes; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.carbon.messaging.Header; import org.wso2.transport.http.netty.contract.Constants; import org.wso2.transport.http.netty.contract.config.ProxyServerConfiguration; @@ -59,7 +60,7 @@ * {@code HttpSinkUtil } responsible of the all configuration reading and input formatting of http transport. */ public class HttpSinkUtil { - private static final Logger log = Logger.getLogger(HttpSinkUtil.class); + private static final Logger log = LogManager.getLogger(HttpSinkUtil.class); private HttpSinkUtil() { } @@ -223,7 +224,7 @@ public static String getContentType(String mapType, List
headersList) { case HttpConstants.MAP_JSON: return HttpConstants.APPLICATION_JSON; - + case HttpConstants.MAP_KEYVALUE: return HttpConstants.APPLICATION_URL_ENCODED; @@ -295,6 +296,8 @@ private static Map trpPropertyTypeMap() { } public static PoolConfiguration createPoolConfigurations(OptionHolder optionHolder) { + int executorServiceThreads = Integer.parseInt(optionHolder.validateAndGetStaticValue( + HttpConstants.EXECUTOR_SERVICE_THREAD_COUNT, HttpConstants.DEFAULT_EXECUTOR_SERVICE_THREAD_COUNT)); int maxIdlePerPool = Integer.parseInt(optionHolder.validateAndGetStaticValue( HttpConstants.MAX_IDLE_CONNECTIONS_PER_POOL, HttpConstants.DEFAULT_MAX_IDLE_CONNECTIONS_PER_POOL)); int minIdlePerPool = Integer.parseInt(optionHolder.validateAndGetStaticValue( @@ -305,7 +308,7 @@ public static PoolConfiguration createPoolConfigurations(OptionHolder optionHold HttpConstants.DEFAULT_TEST_ON_BORROW)); boolean testWhileIdle = Boolean.parseBoolean( optionHolder.validateAndGetStaticValue(HttpConstants.TEST_WHILE_IDLE, - HttpConstants.DEFAULT_TEST_WHILE_IDLE)); + HttpConstants.DEFAULT_TEST_WHILE_IDLE)); long timeBetweenEvictionRuns = Long.parseLong(optionHolder.validateAndGetStaticValue( HttpConstants.TIME_BETWEEN_EVICTION_RUNS, HttpConstants.DEFAULT_TIME_BETWEEN_EVICTION_RUNS)); long minEvictableIdleTime = Long.parseLong(optionHolder.validateAndGetStaticValue( @@ -315,6 +318,7 @@ public static PoolConfiguration createPoolConfigurations(OptionHolder optionHold int maxWaitTime = Integer.parseInt(optionHolder.validateAndGetStaticValue( HttpConstants.MAX_WAIT_TIME, HttpConstants.DEFAULT_MAX_WAIT_TIME)); PoolConfiguration connectionPoolConfiguration = new PoolConfiguration(); + connectionPoolConfiguration.setExecutorServiceThreads(executorServiceThreads); connectionPoolConfiguration.setMaxActivePerPool(maxActivePerPool); connectionPoolConfiguration.setMinIdlePerPool(minIdlePerPool); connectionPoolConfiguration.setMaxIdlePerPool(maxIdlePerPool); diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/HttpCallResponseSource.java b/component/src/main/java/io/siddhi/extension/io/http/source/HttpCallResponseSource.java index ea4d81b2..6a80c921 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/HttpCallResponseSource.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/HttpCallResponseSource.java @@ -32,7 +32,6 @@ import io.siddhi.core.util.transport.OptionHolder; import io.siddhi.extension.io.http.util.HTTPSourceRegistry; import io.siddhi.extension.io.http.util.HttpConstants; -import org.apache.log4j.Logger; import static io.siddhi.extension.io.http.util.HttpConstants.DEFAULT_WORKER_COUNT; @@ -128,7 +127,6 @@ ) public class HttpCallResponseSource extends Source { - private static final Logger log = Logger.getLogger(HttpCallResponseSource.class); private String sinkId; private SourceEventListener sourceEventListener; private String[] requestedTransportPropertyNames; diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/HttpConnectorRegistry.java b/component/src/main/java/io/siddhi/extension/io/http/source/HttpConnectorRegistry.java index 14f6b25b..89e68b89 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/HttpConnectorRegistry.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/HttpConnectorRegistry.java @@ -27,7 +27,8 @@ import io.siddhi.extension.io.http.source.exception.HttpSourceAdaptorRuntimeException; import io.siddhi.extension.io.http.source.util.HttpSourceUtil; import io.siddhi.extension.io.http.util.HttpConstants; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.transport.http.netty.contract.ServerConnector; import org.wso2.transport.http.netty.contract.ServerConnectorFuture; import org.wso2.transport.http.netty.contract.config.InboundMsgSizeValidationConfig; @@ -55,7 +56,7 @@ */ public class HttpConnectorRegistry { private static HttpConnectorRegistry instance = new HttpConnectorRegistry(); - private final Logger log = Logger.getLogger(HttpConnectorRegistry.class); + private static final Logger log = LogManager.getLogger(HttpConnectorRegistry.class); protected TransportsConfiguration trpConfig; protected DefaultHttpWsConnectorFactory httpConnectorFactory; private Map serverConnectorPool = new ConcurrentHashMap<>(); diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/HttpServiceSource.java b/component/src/main/java/io/siddhi/extension/io/http/source/HttpServiceSource.java index 7dadc911..028648e7 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/HttpServiceSource.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/HttpServiceSource.java @@ -42,7 +42,8 @@ import io.siddhi.extension.io.http.source.util.HttpSourceUtil; import io.siddhi.extension.io.http.util.HTTPSourceRegistry; import io.siddhi.extension.io.http.util.HttpConstants; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.carbon.messaging.Header; import org.wso2.transport.http.netty.contract.exceptions.ServerConnectorException; import org.wso2.transport.http.netty.message.HttpCarbonMessage; @@ -299,11 +300,11 @@ ) public class HttpServiceSource extends HttpSource { - private static final Logger log = Logger.getLogger(HttpServiceSource.class); + private static final Logger log = LogManager.getLogger(HttpServiceSource.class); private HttpSyncConnectorRegistry httpConnectorRegistry; private String sourceId; private long connectionTimeout; - + private Map requestContainerMap = new ConcurrentHashMap<>(); private HashedWheelTimer timer; diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/HttpSource.java b/component/src/main/java/io/siddhi/extension/io/http/source/HttpSource.java index e6ce2c20..4c6b7f29 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/HttpSource.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/HttpSource.java @@ -36,7 +36,8 @@ import io.siddhi.extension.io.http.source.util.HttpSourceUtil; import io.siddhi.extension.io.http.util.HttpConstants; import io.siddhi.extension.io.http.util.HttpIoUtil; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.carbon.si.metrics.core.internal.MetricsDataHolder; import org.wso2.transport.http.netty.contract.config.ListenerConfiguration; @@ -267,7 +268,7 @@ } ) public class HttpSource extends Source { - private static final Logger log = Logger.getLogger(HttpSource.class); + private static final Logger log = LogManager.getLogger(HttpSource.class); protected String listenerUrl; protected Boolean isAuth; protected int workerThread; diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/HttpSyncConnectorRegistry.java b/component/src/main/java/io/siddhi/extension/io/http/source/HttpSyncConnectorRegistry.java index 1c8beb18..5bcd57e4 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/HttpSyncConnectorRegistry.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/HttpSyncConnectorRegistry.java @@ -24,7 +24,6 @@ import io.siddhi.extension.io.http.metrics.SourceMetrics; import io.siddhi.extension.io.http.source.util.HttpSourceUtil; import io.siddhi.extension.io.http.util.HttpConstants; -import org.apache.log4j.Logger; import org.wso2.transport.http.netty.contract.ServerConnectorFuture; import org.wso2.transport.http.netty.contractimpl.DefaultHttpWsConnectorFactory; @@ -37,7 +36,6 @@ public class HttpSyncConnectorRegistry extends HttpConnectorRegistry { private static HttpSyncConnectorRegistry instance = new HttpSyncConnectorRegistry(); - private final Logger log = Logger.getLogger(HttpSyncConnectorRegistry.class); private Map sourceListenersMap = new ConcurrentHashMap<>(); private HttpSyncConnectorRegistry() { diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/HttpSyncWorkerThread.java b/component/src/main/java/io/siddhi/extension/io/http/source/HttpSyncWorkerThread.java index 92c71e7e..5ee904ce 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/HttpSyncWorkerThread.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/HttpSyncWorkerThread.java @@ -94,13 +94,13 @@ public void run() { } finally { try { buf.close(); - carbonMessage.waitAndReleaseAllEntities(); } catch (IOException e) { if (metrics != null) { metrics.getTotalHttpErrorsMetric().inc(); } - logger.error("Error occurred when closing the byte buffer in source " + sourceID, e); + } finally { + carbonMessage.waitAndReleaseAllEntities(); } } } diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/HttpWebSubResponseProcessor.java b/component/src/main/java/io/siddhi/extension/io/http/source/HttpWebSubResponseProcessor.java index def03714..18bef003 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/HttpWebSubResponseProcessor.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/HttpWebSubResponseProcessor.java @@ -49,6 +49,7 @@ import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; +import java.io.UnsupportedEncodingException; import java.nio.charset.Charset; import java.nio.charset.StandardCharsets; import java.util.ArrayList; @@ -135,7 +136,10 @@ public void run() { List attributeList = table.getTableDefinition().getAttributeList(); if (payloadMap.get(HUB_MODE).toString().equalsIgnoreCase(HUB_MODE_SUBSCRIBE)) { for (Attribute attribute : attributeList) { - if (attribute.getName().equals(HttpConstants.HUB_ID_COLUMN_NAME)) { + if (attribute.getName().equalsIgnoreCase(HUB_CALLBACK)) { + event.add(java.net.URLDecoder.decode(payloadMap.get(attribute.getName()).toString(), + StandardCharsets.UTF_8.name())); + } else if (attribute.getName().equals(HttpConstants.HUB_ID_COLUMN_NAME)) { event.add(hubId); } else if (attribute.getName().equals(REQUEST_TIMESTAMP)) { event.add(System.currentTimeMillis()); @@ -186,18 +190,19 @@ public void run() { logger.debug("Empty payload event, hence dropping the event chunk at source " + sourceID); } } - } catch (RuntimeException e) { + } catch (RuntimeException | UnsupportedEncodingException e) { HttpIoUtil.handleFailure(carbonMessage, null, INTERNAL_SERVER_FAIL_CODE, e.getMessage()); logger.error("Error occurred while processing the payload ", e); } finally { try { buf.close(); - carbonMessage.waitAndReleaseAllEntities(); } catch (IOException e) { if (metrics != null) { metrics.getTotalHttpErrorsMetric().inc(); } logger.error("Error occurred when closing the byte buffer in source " + sourceID, e); + } finally { + carbonMessage.waitAndReleaseAllEntities(); } } } diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/HttpWebSubSource.java b/component/src/main/java/io/siddhi/extension/io/http/source/HttpWebSubSource.java index d45aaea7..f15c5035 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/HttpWebSubSource.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/HttpWebSubSource.java @@ -42,7 +42,8 @@ import io.siddhi.query.api.annotation.Annotation; import io.siddhi.query.api.definition.Attribute; import io.siddhi.query.api.definition.TableDefinition; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.carbon.si.metrics.core.internal.MetricsDataHolder; import org.wso2.transport.http.netty.contract.config.ListenerConfiguration; @@ -68,20 +69,20 @@ @Extension(name = "websubhub", namespace = "source", description = " WebSub Hub source receive subscription requests via Http and according to the request," + " the subscription details will be saved to the given table and against the callback and topic name. " + - "The subscription request MUST have a Content-Type header of application/x-www-form-urlencoded and " + - "following MUST provide as parameter body " + - "hub.callback \t - REQUIRED. The subscriber's callback URL where content distribution notifications " + - "should be delivered. The callback URL SHOULD be an unguessable URL that is unique per " + + "The subscription request **MUST** have a Content-Type header of " + + "**application/x-www-form-urlencoded** and following **MUST** provide as parameter body. \n " + + "\t hub.callback \t - REQUIRED.\t The subscriber's callback URL where content distribution " + + "notifications should be delivered. The callback URL SHOULD be an unguessable URL that is unique per " + "subscription.\n" + - "hub.mode \t - REQUIRED. The literal string \"subscribe\" or \"unsubscribe\", depending on the " + + "\t hub.mode \t - REQUIRED.\t The literal string \"subscribe\" or \"unsubscribe\", depending on the " + "goal of the request.\n" + - "hub.topic \t - REQUIRED. The topic URL that the subscriber wishes to subscribe to or unsubscribe " + - "from.\n " + - "hub.lease_seconds\t - OPTIONAL. Number of seconds for which the subscriber would like to have " + + "\t hub.topic \t - REQUIRED.\t The topic URL that the subscriber wishes to subscribe to or " + + "unsubscribe from.\n " + + "\t hub.lease_seconds\t - OPTIONAL.\t Number of seconds for which the subscriber would like to have " + "the subscription active, given as a positive decimal integer. \n" + - "hub.secret\t - OPTIONAL. A subscriber-provided cryptographically random unique secret string that " + - "will be used to compute an HMAC digest for authorized content distribution. If not supplied, the " + - "HMAC digest will not be present for content distribution requests. ", + "\t hub.secret\t - OPTIONAL.\t A subscriber-provided cryptographically random unique secret string " + + "that will be used to compute an HMAC digest for authorized content distribution. If not supplied, " + + "the HMAC digest will not be present for content distribution requests. ", parameters = { @Parameter(name = "hub.id", @@ -291,7 +292,7 @@ } ) public class HttpWebSubSource extends Source { - private static final Logger log = Logger.getLogger(HttpWebSubSource.class); + private static final Logger log = LogManager.getLogger(HttpWebSubSource.class); public String siddhiAppName; protected SourceMetrics metrics; private HttpConnectorRegistry httpConnectorRegistry; @@ -420,6 +421,7 @@ public void connect(ConnectionCallback connectionCallback, State state) throws C this.httpConnectorRegistry.registerSourceListener(sourceEventListener, listenerUrl, workerThread, isAuth, requestedTransportPropertyNames, siddhiAppName, metrics, webSubMetaTable, hubId, siddhiAppContext, topics); + webSubMetaTable.connectWithRetry(); } protected void initConnectorRegistry(OptionHolder optionHolder, ConfigReader configReader) { diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/HttpWorkerThread.java b/component/src/main/java/io/siddhi/extension/io/http/source/HttpWorkerThread.java index ef54d44e..9c611307 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/HttpWorkerThread.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/HttpWorkerThread.java @@ -88,13 +88,13 @@ public void run() { } finally { try { buf.close(); - carbonMessage.waitAndReleaseAllEntities(); } catch (IOException e) { if (metrics != null) { metrics.getTotalHttpErrorsMetric().inc(); } - logger.error("Error occurred when closing the byte buffer in source " + sourceID, e); + } finally { + carbonMessage.waitAndReleaseAllEntities(); } } } diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/SSESource.java b/component/src/main/java/io/siddhi/extension/io/http/source/SSESource.java index 9a8cb780..5bbd7bb2 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/SSESource.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/SSESource.java @@ -44,7 +44,8 @@ import io.siddhi.extension.io.http.sink.util.HttpSinkUtil; import io.siddhi.extension.io.http.util.HTTPSourceRegistry; import io.siddhi.extension.io.http.util.HttpConstants; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.carbon.messaging.Header; import org.wso2.carbon.si.metrics.core.internal.MetricsDataHolder; import org.wso2.transport.http.netty.contract.Constants; @@ -183,7 +184,7 @@ } ) public class SSESource extends Source { - private static final Logger log = Logger.getLogger(SSESource.class); + private static final Logger log = LogManager.getLogger(SSESource.class); private int workerThread; private String siddhiAppName; private String streamID; diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/internal/ServiceComponent.java b/component/src/main/java/io/siddhi/extension/io/http/source/internal/ServiceComponent.java index c6824c93..e9752dbc 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/internal/ServiceComponent.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/internal/ServiceComponent.java @@ -19,7 +19,8 @@ package io.siddhi.extension.io.http.source.internal; import io.siddhi.extension.io.http.source.exception.HttpSourceAdaptorException; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.osgi.framework.BundleContext; import org.osgi.service.component.annotations.Activate; import org.osgi.service.component.annotations.Component; @@ -38,7 +39,7 @@ immediate = true ) public class ServiceComponent { - private static final Logger log = Logger.getLogger(ServiceComponent.class); + private static final Logger log = LogManager.getLogger(ServiceComponent.class); /** * This is the activation method of Http io declarative service. This will be called when its references are diff --git a/component/src/main/java/io/siddhi/extension/io/http/source/util/HttpSourceUtil.java b/component/src/main/java/io/siddhi/extension/io/http/source/util/HttpSourceUtil.java index ec82e788..e3fb88d2 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/source/util/HttpSourceUtil.java +++ b/component/src/main/java/io/siddhi/extension/io/http/source/util/HttpSourceUtil.java @@ -25,7 +25,8 @@ import io.siddhi.extension.io.http.util.HttpConstants; import io.siddhi.extension.io.http.util.HttpIoUtil; import io.siddhi.extension.io.http.util.TrpPropertyTypes; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.transport.http.netty.contract.config.ListenerConfiguration; import org.wso2.transport.http.netty.contract.config.TransportProperty; import org.wso2.transport.http.netty.message.HttpCarbonMessage; @@ -67,7 +68,7 @@ * Handles the http source util functions. */ public class HttpSourceUtil { - private static final Logger log = Logger.getLogger(HttpSourceUtil.class); + private static final Logger log = LogManager.getLogger(HttpSourceUtil.class); private HttpSourceUtil() { } diff --git a/component/src/main/java/io/siddhi/extension/io/http/util/HttpConstants.java b/component/src/main/java/io/siddhi/extension/io/http/util/HttpConstants.java index 81948b05..5714270a 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/util/HttpConstants.java +++ b/component/src/main/java/io/siddhi/extension/io/http/util/HttpConstants.java @@ -59,6 +59,8 @@ public class HttpConstants { public static final String DEFAULT_ENCODING = "UTF-8"; public static final String DOWNLOAD_ENABLED = "downloading.enabled"; public static final String DEFAULT_DOWNLOAD_ENABLED_VALUE = "false"; + public static final String OAUTH2_SCOPE_PARAMETER_NAME = "oauth.scope"; + public static final String OAUTH2_SCOPE_PARAMETER_KEY = "scope"; //Common util values public static final String HTTP_METHOD_GET = "GET"; public static final String HTTP_METHOD_POST = "POST"; //method name @@ -197,6 +199,8 @@ public class HttpConstants { public static final String TOKEN_URL = "token.url"; public static final String CONSUMER_SECRET = "consumer.secret"; public static final String CONSUMER_KEY = "consumer.key"; + public static final String BODY_CONSUMER_SECRET = "body.consumer.secret"; + public static final String BODY_CONSUMER_KEY = "body.consumer.key"; public static final String GRANT_TYPE = "grant_type"; public static final String GRANT_PASSWORD = "password"; public static final String GRANT_REFRESHTOKEN = "refresh_token"; @@ -210,6 +214,8 @@ public class HttpConstants { public static final int AUTHENTICATION_FAIL_CODE = 401; public static final int PERSISTENT_ACCESS_FAIL_CODE = 400; public static final int INTERNAL_SERVER_FAIL_CODE = 500; + public static final String OAUTH_CLIENT_ID = "client_id"; + public static final String OAUTH_CLIENT_SECRET = "client_secret"; public static final String RECEIVER_OAUTH_USERNAME = "oauth.username"; public static final String RECEIVER_OAUTH_PASSWORD = "oauth.password"; public static final String RECEIVER_REFRESH_TOKEN = "refresh.token"; @@ -225,6 +231,8 @@ public class HttpConstants { public static final String BLOCKING_IO = "blocking.io"; //pool configurations + public static final String EXECUTOR_SERVICE_THREAD_COUNT = "executor.service.threads"; + public static final String DEFAULT_EXECUTOR_SERVICE_THREAD_COUNT = "20"; public static final String MAX_ACTIVE_CONNECTIONS_PER_POOL = "max.pool.active.connections"; public static final String DEFAULT_MAX_ACTIVE_CONNECTIONS_PER_POOL = "-1"; // unlimited public static final String MIN_IDLE_CONNECTIONS_PER_POOL = "min.pool.idle.connections"; diff --git a/component/src/main/java/io/siddhi/extension/io/http/util/HttpIoUtil.java b/component/src/main/java/io/siddhi/extension/io/http/util/HttpIoUtil.java index e1911566..27ba80ad 100644 --- a/component/src/main/java/io/siddhi/extension/io/http/util/HttpIoUtil.java +++ b/component/src/main/java/io/siddhi/extension/io/http/util/HttpIoUtil.java @@ -38,7 +38,8 @@ import io.siddhi.query.api.expression.Variable; import io.siddhi.query.api.expression.condition.And; import io.siddhi.query.api.expression.condition.Compare; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.wso2.transport.http.netty.contract.config.Parameter; import org.wso2.transport.http.netty.contract.exceptions.ServerConnectorException; import org.wso2.transport.http.netty.contractimpl.DefaultHttpWsConnectorFactory; @@ -73,7 +74,7 @@ * Util class which is use for handle HTTP util function. */ public class HttpIoUtil { - private static final Logger log = Logger.getLogger(HttpIoUtil.class); + private static final Logger log = LogManager.getLogger(HttpIoUtil.class); private static String clientStoreFile; private static String clientStorePass; private static ConfigReader configReader; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpAuthTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpAuthTestCase.java index 625b8f18..02711a30 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpAuthTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpAuthTestCase.java @@ -26,7 +26,8 @@ import io.siddhi.extension.io.http.sink.util.HttpServerListenerHandler; import io.siddhi.extension.map.xml.sinkmapper.XMLSinkMapper; import io.siddhi.query.api.exception.SiddhiAppValidationException; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.Test; @@ -34,7 +35,7 @@ * test cases for basic authentication. */ public class HttpAuthTestCase { - private static final Logger log = Logger.getLogger(HttpAuthTestCase.class); + private static final Logger log = LogManager.getLogger(HttpAuthTestCase.class); /** * Creating test for publishing events wth basic authentication false. diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpCallResponseTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpCallResponseTestCase.java index f82a99fb..8092277a 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpCallResponseTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpCallResponseTestCase.java @@ -26,7 +26,8 @@ import io.siddhi.core.util.SiddhiTestHelper; import io.siddhi.extension.io.http.sink.util.HttpFileServerListenerHandler; import io.siddhi.extension.io.http.sink.util.HttpServerListenerHandler; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeClass; import org.testng.annotations.BeforeMethod; @@ -36,7 +37,7 @@ import java.util.concurrent.atomic.AtomicInteger; public class HttpCallResponseTestCase { - private static final Logger log = Logger.getLogger(HttpCallResponseTestCase.class); + private static final Logger log = LogManager.getLogger(HttpCallResponseTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); private String downloadPath; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpMultipleEventSinkTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpMultipleEventSinkTestCase.java index afd5026e..3e67c670 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpMultipleEventSinkTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpMultipleEventSinkTestCase.java @@ -24,9 +24,18 @@ import io.siddhi.extension.io.http.sink.util.HttpServerListener; import io.siddhi.extension.io.http.sink.util.HttpServerListenerHandler; import io.siddhi.extension.map.xml.sinkmapper.XMLSinkMapper; -import org.apache.log4j.AppenderSkeleton; -import org.apache.log4j.Logger; -import org.apache.log4j.spi.LoggingEvent; +import org.apache.logging.log4j.Level; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.core.Appender; +import org.apache.logging.log4j.core.Core; +import org.apache.logging.log4j.core.Filter; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.core.Logger; +import org.apache.logging.log4j.core.appender.AbstractAppender; +import org.apache.logging.log4j.core.config.plugins.Plugin; +import org.apache.logging.log4j.core.config.plugins.PluginAttribute; +import org.apache.logging.log4j.core.config.plugins.PluginElement; +import org.apache.logging.log4j.core.config.plugins.PluginFactory; import org.testng.Assert; import org.testng.annotations.Test; @@ -37,7 +46,7 @@ * Test cases for multiple event sink synchronously. */ public class HttpMultipleEventSinkTestCase { - private static final Logger logger = Logger.getLogger(HttpMultipleEventSinkTestCase.class); + private static final Logger logger = (Logger) LogManager.getLogger(HttpMultipleEventSinkTestCase.class); /** * Test cases for multiple event sink synchronously. @@ -47,12 +56,14 @@ public class HttpMultipleEventSinkTestCase { @Test public void testHTTPMultipleEvents() throws Exception { logger.info("Creating test for multiple event sink synchronously."); - final TestAppender appender = new TestAppender(); - final Logger logger = Logger.getLogger(HttpServerListener.class); + final TestAppender appender = new TestAppender("TestAppender", null); + Logger logger = (Logger) LogManager.getLogger(HttpServerListener.class); + logger.setLevel(Level.ALL); logger.addAppender(appender); SiddhiManager siddhiManager = new SiddhiManager(); siddhiManager.setExtension("xml-output-mapper", XMLSinkMapper.class); + String inStreamDefinition1 = "Define stream FooStreamA (message String,method String,headers String);" + "@sink(type='http',publisher.url='http://localhost:8005/abc',method='{{method}}'," + "headers='{{headers}}'," @@ -100,38 +111,54 @@ public void testHTTPMultipleEvents() throws Exception { fooStream.send(new Object[]{event1, "POST", "'Name:John','Age:23'"}); fooStream2.send(new Object[]{event2, "POST", "'Name:John','Age:23'"}); Thread.sleep(1000); - final List log = appender.getLog(); + final List loggedEvents = ((TestAppender) logger.getAppenders(). + get("TestAppender")).getLog(); List logMessages = new ArrayList<>(); - for (LoggingEvent logEvent : log) { - logMessages.add(logEvent.getMessage().toString()); + for (String logEvent : loggedEvents) { + String message = String.valueOf(logEvent); + if (message.contains(":")) { + message = message.split(":")[1].trim(); + } + logMessages.add(message); } - Assert.assertEquals(logMessages.contains("Event Arrived: " + event1 + "\n"), true); - Assert.assertEquals(logMessages.contains("Event Arrived: " + event2 + "\n"), true); + Assert.assertEquals(logMessages.contains(event1), true); + Assert.assertEquals(logMessages.contains(event2), true); Thread.sleep(500); siddhiAppRuntime.shutdown(); lst.shutdown(); + logger.removeAppender(appender); } - private static class TestAppender extends AppenderSkeleton { + @Plugin(name = "TestAppender", + category = Core.CATEGORY_NAME, elementType = Appender.ELEMENT_TYPE) + private static class TestAppender extends AbstractAppender { - private final List log = new ArrayList<>(); + private final List log = new ArrayList<>(); - @Override - public boolean requiresLayout() { - return false; + public TestAppender(String name, Filter filter) { + + super(name, filter, null); } - @Override - protected void append(final LoggingEvent loggingEvent) { - log.add(loggingEvent); + @PluginFactory + public static TestAppender createAppender( + @PluginAttribute("name") String name, + @PluginElement("Filter") Filter filter) { + + return new TestAppender(name, filter); } @Override - public void close() { + public void append(LogEvent event) { + + log.add(event.getMessage().getFormattedMessage()); + } - List getLog() { - return new ArrayList<>(log); + public List getLog() { + List clone = new ArrayList<>(log); + log.clear(); + return clone; } } } diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpOAuthTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpOAuthTestCase.java index da3d8da4..ba367a3f 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpOAuthTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpOAuthTestCase.java @@ -26,7 +26,8 @@ import io.siddhi.core.util.SiddhiTestHelper; import io.siddhi.extension.io.http.sink.util.HttpOAuthListenerHandler; import io.siddhi.extension.map.xml.sinkmapper.XMLSinkMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -37,7 +38,7 @@ public class HttpOAuthTestCase { public static final String CARBON_HOME = "carbon.home"; - private static final Logger log = Logger.getLogger(HttpAuthTestCase.class); + private static final Logger log = LogManager.getLogger(HttpAuthTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); @BeforeMethod diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpRequestResponseTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpRequestResponseTestCase.java index 98470ff7..e07dc780 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpRequestResponseTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpRequestResponseTestCase.java @@ -27,7 +27,8 @@ import io.siddhi.core.util.SiddhiTestHelper; import io.siddhi.extension.io.http.sink.util.HttpFileServerListenerHandler; import io.siddhi.extension.io.http.sink.util.HttpServerListenerHandler; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeClass; import org.testng.annotations.BeforeMethod; @@ -37,7 +38,7 @@ import java.util.concurrent.atomic.AtomicInteger; public class HttpRequestResponseTestCase { - private static final Logger log = Logger.getLogger(HttpRequestResponseTestCase.class); + private static final Logger log = LogManager.getLogger(HttpRequestResponseTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); private String downloadPath; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkCustomConfigurationTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkCustomConfigurationTestCase.java index 55bb516e..21614cd9 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkCustomConfigurationTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkCustomConfigurationTestCase.java @@ -23,7 +23,8 @@ import io.siddhi.core.stream.input.InputHandler; import io.siddhi.extension.io.http.sink.util.HttpServerListenerHandler; import io.siddhi.extension.map.xml.sinkmapper.XMLSinkMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.Test; @@ -32,7 +33,7 @@ * Test case for mapping type. */ public class HttpSinkCustomConfigurationTestCase { - private static final Logger log = Logger.getLogger(HttpSinkCustomConfigurationTestCase.class); + private static final Logger log = LogManager.getLogger(HttpSinkCustomConfigurationTestCase.class); /** * Creating test for publishing events with XML mapping. diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkMappingTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkMappingTestCase.java index c5522ab0..d47c33a8 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkMappingTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkMappingTestCase.java @@ -25,7 +25,8 @@ import io.siddhi.extension.map.json.sinkmapper.JsonSinkMapper; import io.siddhi.extension.map.text.sinkmapper.TextSinkMapper; import io.siddhi.extension.map.xml.sinkmapper.XMLSinkMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.Test; @@ -34,7 +35,7 @@ * Test case for mapping type. */ public class HttpSinkMappingTestCase { - private static final Logger log = Logger.getLogger(HttpSinkMappingTestCase.class); + private static final Logger log = LogManager.getLogger(HttpSinkMappingTestCase.class); /** * Creating test for publishing events with XML mapping. diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkMethodTypeTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkMethodTypeTestCase.java index 281975b8..f70a302f 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkMethodTypeTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkMethodTypeTestCase.java @@ -23,7 +23,8 @@ import io.siddhi.core.stream.input.InputHandler; import io.siddhi.extension.io.http.sink.util.HttpServerListenerHandler; import io.siddhi.extension.map.xml.sinkmapper.XMLSinkMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; @@ -33,7 +34,7 @@ * Test cases for different method types. */ public class HttpSinkMethodTypeTestCase { - private static final Logger log = Logger.getLogger(HttpSinkMethodTypeTestCase.class); + private static final Logger log = LogManager.getLogger(HttpSinkMethodTypeTestCase.class); private String payload; private String expected; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkTestCase.java index 839b398a..ba583b24 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpSinkTestCase.java @@ -26,7 +26,9 @@ import io.siddhi.extension.io.http.sink.util.HttpServerListenerHandler; import io.siddhi.extension.io.http.sink.util.UnitTestAppender; import io.siddhi.extension.map.xml.sinkmapper.XMLSinkMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.Level; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.core.Logger; import org.testng.Assert; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; @@ -39,7 +41,7 @@ * Test cases for content type header. */ public class HttpSinkTestCase { - private static final Logger log = Logger.getLogger(HttpSinkTestCase.class); + private static final Logger log = (Logger) LogManager.getLogger(HttpSinkTestCase.class); private String payload; private String expected; private AtomicInteger eventCount = new AtomicInteger(0); @@ -166,6 +168,12 @@ public void testHTTPContentTypeAtHeaders() throws Exception { public void testHTTPConnectionFailure() throws InterruptedException { log.info("Creating test for publishing events to invalid HTTP endpoint."); SiddhiManager siddhiManager = new SiddhiManager(); + UnitTestAppender appender = new UnitTestAppender("UnitTestAppender", null); + Logger logger = (Logger) LogManager.getLogger(Sink.class); + logger.setLevel(Level.ALL); + logger.addAppender(appender); + appender.start(); + siddhiManager.setExtension("xml-output-mapper", XMLSinkMapper.class); String inStreamDefinition = "Define stream FooStream (message String,method String,headers String);" + "@sink(type='http', publisher.url='http://localhost:8010/abcd',method='{{method}}'," @@ -182,9 +190,6 @@ public void testHTTPConnectionFailure() throws InterruptedException { SiddhiAppRuntime siddhiAppRuntime = siddhiManager.createSiddhiAppRuntime(inStreamDefinition + query); InputHandler fooStream = siddhiAppRuntime.getInputHandler("FooStream"); siddhiAppRuntime.start(); - Logger logger = Logger.getLogger(Sink.class); - UnitTestAppender appender = new UnitTestAppender(); - logger.addAppender(appender); try { fooStream.send(1566562744069L, new Object[]{payload, "POST", "'Name:John','Age:23'"}); @@ -192,7 +197,8 @@ public void testHTTPConnectionFailure() throws InterruptedException { String expectedMessage = "Dropping event at Sink 'http' at 'BarStream' as its still trying to " + "reconnect!, events dropped 'WSO255.645" + "100'"; - Assert.assertTrue(appender.getMessages().contains(expectedMessage)); + Assert.assertTrue(((UnitTestAppender) logger.getAppenders(). + get("UnitTestAppender")).getMessages().contains(expectedMessage)); } finally { logger.removeAppender(appender); siddhiAppRuntime.shutdown(); diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpsSinkTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpsSinkTestCase.java index ee53aa95..65ef52df 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/HttpsSinkTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/HttpsSinkTestCase.java @@ -23,7 +23,8 @@ import io.siddhi.core.stream.input.InputHandler; import io.siddhi.extension.io.http.sink.util.HttpsServerListenerHandler; import io.siddhi.extension.map.xml.sinkmapper.XMLSinkMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; @@ -36,7 +37,7 @@ */ public class HttpsSinkTestCase { public static final String CARBON_HOME = "carbon.home"; - private static final Logger logger = Logger.getLogger(HttpsSinkTestCase.class); + private static final Logger logger = LogManager.getLogger(HttpsSinkTestCase.class); private String payload; private String expected; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/SSEServerSinkTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/sink/SSEServerSinkTestCase.java index 407b6c1a..b24ba52f 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/SSEServerSinkTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/SSEServerSinkTestCase.java @@ -28,7 +28,6 @@ import io.siddhi.core.util.persistence.PersistenceStore; import io.siddhi.extension.map.json.sinkmapper.JsonSinkMapper; import io.siddhi.extension.map.json.sourcemapper.JsonSourceMapper; -import org.apache.log4j.Logger; import org.testng.Assert; import org.testng.annotations.Test; @@ -41,7 +40,6 @@ * Tests http sse sink. */ public class SSEServerSinkTestCase { - private static final Logger log = Logger.getLogger(SSEServerSinkTestCase.class); private static final int SLEEP_TIME = 50; private static final int EVENT_COUNT = 10; private static final int TIMEOUT = 30000; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpFileServerListener.java b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpFileServerListener.java index cdcb8ccb..277ea7c5 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpFileServerListener.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpFileServerListener.java @@ -22,7 +22,8 @@ import com.sun.net.httpserver.HttpExchange; import com.sun.net.httpserver.HttpHandler; import org.apache.commons.io.IOUtils; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import java.io.File; import java.io.FileInputStream; @@ -35,7 +36,7 @@ * Test handler for file downloading feature. */ public class HttpFileServerListener implements HttpHandler { - private static final Logger logger = Logger.getLogger(HttpFileServerListener.class); + private static final Logger logger = LogManager.getLogger(HttpFileServerListener.class); private AtomicBoolean isEventArrived = new AtomicBoolean(false); private Headers headers; private String filePath; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpFileServerListenerHandler.java b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpFileServerListenerHandler.java index 5a621f67..1ca5bfd4 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpFileServerListenerHandler.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpFileServerListenerHandler.java @@ -19,7 +19,8 @@ package io.siddhi.extension.io.http.sink.util; import com.sun.net.httpserver.HttpServer; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import java.io.IOException; import java.net.InetSocketAddress; @@ -28,7 +29,7 @@ * Http test sever listener for file downloading feature. */ public class HttpFileServerListenerHandler implements Runnable { - private static final Logger logger = Logger.getLogger(HttpFileServerListenerHandler.class); + private static final Logger logger = LogManager.getLogger(HttpFileServerListenerHandler.class); private HttpFileServerListener fileServerListener; private HttpServer server; private int port; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpOAuthListenerHandler.java b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpOAuthListenerHandler.java index c5a749ab..8fe27df3 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpOAuthListenerHandler.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpOAuthListenerHandler.java @@ -21,7 +21,8 @@ import com.sun.net.httpserver.HttpsConfigurator; import com.sun.net.httpserver.HttpsParameters; import com.sun.net.httpserver.HttpsServer; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import java.io.FileInputStream; import java.io.IOException; @@ -41,7 +42,7 @@ * Http test sever listener handler for OAuth requests. */ public class HttpOAuthListenerHandler implements Runnable { - private static final Logger logger = Logger.getLogger(HttpServerListenerHandler.class); + private static final Logger logger = LogManager.getLogger(HttpServerListenerHandler.class); private HttpOAuthTokenEndpointListener httpOAuthTokenEndpointListener; private HttpOAuthEndpointListener httpOAuthEndpointListener; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpServerListener.java b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpServerListener.java index 7a467c7d..81347def 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpServerListener.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpServerListener.java @@ -21,7 +21,8 @@ import com.sun.net.httpserver.Headers; import com.sun.net.httpserver.HttpExchange; import com.sun.net.httpserver.HttpHandler; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import java.io.BufferedReader; import java.io.IOException; @@ -34,7 +35,7 @@ * Test Server Listener Manger. */ public class HttpServerListener implements HttpHandler { - private static final Logger logger = Logger.getLogger(HttpServerListener.class); + private static final Logger logger = LogManager.getLogger(HttpServerListener.class); private AtomicBoolean isEventArrived = new AtomicBoolean(false); private StringBuilder stringBuilder; private Headers headers; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpServerListenerHandler.java b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpServerListenerHandler.java index 53a055ae..c4cc9740 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpServerListenerHandler.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpServerListenerHandler.java @@ -19,7 +19,8 @@ package io.siddhi.extension.io.http.sink.util; import com.sun.net.httpserver.HttpServer; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import java.io.IOException; import java.net.InetSocketAddress; @@ -28,7 +29,7 @@ * Http test sever listener. */ public class HttpServerListenerHandler implements Runnable { - private static final Logger logger = Logger.getLogger(HttpServerListenerHandler.class); + private static final Logger logger = LogManager.getLogger(HttpServerListenerHandler.class); private HttpServerListener serverListener; private HttpServer server; private int port; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpsServerListenerHandler.java b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpsServerListenerHandler.java index 45ce1894..79cdf03f 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpsServerListenerHandler.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/util/HttpsServerListenerHandler.java @@ -21,7 +21,8 @@ import com.sun.net.httpserver.HttpsConfigurator; import com.sun.net.httpserver.HttpsParameters; import com.sun.net.httpserver.HttpsServer; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import java.io.FileInputStream; import java.io.IOException; @@ -41,7 +42,7 @@ * Https test sever listener. */ public class HttpsServerListenerHandler implements Runnable { - private static final Logger logger = Logger.getLogger(HttpsServerListenerHandler.class); + private static final Logger logger = LogManager.getLogger(HttpsServerListenerHandler.class); private HttpServerListener serverListener; private int port; private KeyStore keyStore; diff --git a/component/src/test/java/io/siddhi/extension/io/http/sink/util/UnitTestAppender.java b/component/src/test/java/io/siddhi/extension/io/http/sink/util/UnitTestAppender.java index 949057ea..13d7e150 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/sink/util/UnitTestAppender.java +++ b/component/src/test/java/io/siddhi/extension/io/http/sink/util/UnitTestAppender.java @@ -18,28 +18,46 @@ package io.siddhi.extension.io.http.sink.util; -import org.apache.log4j.AppenderSkeleton; -import org.apache.log4j.spi.LoggingEvent; +import org.apache.logging.log4j.core.Appender; +import org.apache.logging.log4j.core.Core; +import org.apache.logging.log4j.core.Filter; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.core.appender.AbstractAppender; +import org.apache.logging.log4j.core.config.plugins.Plugin; +import org.apache.logging.log4j.core.config.plugins.PluginAttribute; +import org.apache.logging.log4j.core.config.plugins.PluginElement; +import org.apache.logging.log4j.core.config.plugins.PluginFactory; +import org.mvel2.util.StringAppender; + +@Plugin(name = "UnitTestAppender", + category = Core.CATEGORY_NAME, elementType = Appender.ELEMENT_TYPE) +public class UnitTestAppender extends AbstractAppender { + + private StringAppender messages = new StringAppender(); + + public UnitTestAppender(String name, Filter filter) { + + super(name, filter, null); + } -public class UnitTestAppender extends AppenderSkeleton { - private String messages; + @PluginFactory + public static UnitTestAppender createAppender( + @PluginAttribute("name") String name, + @PluginElement("Filter") Filter filter) { - public String getMessages() { - return messages; + return new UnitTestAppender(name, filter); } - @Override - protected void append(LoggingEvent loggingEvent) { - messages = loggingEvent.getRenderedMessage(); + public String getMessages() { + + String results = messages.toString(); + return results; } @Override - public void close() { + public void append(LogEvent event) { + messages.append(event.getMessage().getFormattedMessage()); } - @Override - public boolean requiresLayout() { - return false; - } } diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/HttpBasicTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/HttpBasicTestCase.java index 6e978d51..f7346a7c 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/HttpBasicTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/HttpBasicTestCase.java @@ -27,9 +27,18 @@ import io.siddhi.core.util.persistence.PersistenceStore; import io.siddhi.extension.io.http.source.util.HttpTestUtil; import io.siddhi.extension.map.xml.sourcemapper.XmlSourceMapper; -import org.apache.log4j.AppenderSkeleton; -import org.apache.log4j.Logger; -import org.apache.log4j.spi.LoggingEvent; +import org.apache.logging.log4j.Level; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.core.Appender; +import org.apache.logging.log4j.core.Core; +import org.apache.logging.log4j.core.Filter; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.core.Logger; +import org.apache.logging.log4j.core.appender.AbstractAppender; +import org.apache.logging.log4j.core.config.plugins.Plugin; +import org.apache.logging.log4j.core.config.plugins.PluginAttribute; +import org.apache.logging.log4j.core.config.plugins.PluginElement; +import org.apache.logging.log4j.core.config.plugins.PluginFactory; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -44,7 +53,7 @@ * Basic test cases for http source functions. */ public class HttpBasicTestCase { - private static final Logger logger = Logger.getLogger(HttpBasicTestCase.class); + private static final Logger logObj = (Logger) LogManager.getLogger(HttpBasicTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); private int waitTime = 50; private int timeout = 30000; @@ -61,7 +70,7 @@ public void init() { */ @Test public void testHTTPInputTransportWithoutURL() throws Exception { - logger.info(" Creating test for publishing events without URL."); + logObj.info(" Creating test for publishing events without URL."); URI baseURI = URI.create(String.format("http://%s:%d", "0.0.0.0", 8280)); List receivedEventNameList = new ArrayList<>(2); PersistenceStore persistenceStore = new InMemoryPersistenceStore(); @@ -122,10 +131,13 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { */ @Test(dependsOnMethods = "testHTTPInputTransportWithoutURL") public void testHTTPInputTransportPutMethod() throws Exception { - logger.info("Test case for put method"); - final TestAppender appender = new TestAppender(); - final Logger logger = Logger.getRootLogger(); + logObj.info("Test case for put method"); + final HttpBasicTestCaseTestAppender appender = new + HttpBasicTestCaseTestAppender("HttpBasicTestCaseTestAppender", null); + final Logger logger = (Logger) LogManager.getRootLogger(); + logger.setLevel(Level.ALL); logger.addAppender(appender); + appender.start(); logger.info("Creating test for publishing events from PUT method."); URI baseURI = URI.create(String.format("http://%s:%d", "localhost", 8005)); List receivedEventNameList = new ArrayList<>(2); @@ -169,16 +181,14 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { HttpTestUtil.httpPublishEvent(event1, baseURI, "/endpoints/RecPro", "PUT"); HttpTestUtil.httpPublishEvent(event2, baseURI, "/endpoints/RecPro", "PUT"); - final List log = appender.getLog(); - List logMessages = new ArrayList<>(); - for (LoggingEvent logEvent : log) { - logMessages.add(String.valueOf(logEvent.getMessage())); - } + final List loggedEvents = ((HttpBasicTestCaseTestAppender) logger.getAppenders(). + get("HttpBasicTestCaseTestAppender")).getLog(); SiddhiTestHelper.waitForEvents(waitTime, 0, eventCount, timeout); - Assert.assertEquals(logMessages.contains("Event response code 400"), true); - Assert.assertEquals(Collections.frequency(logMessages, "Event response code 400"), 2); + Assert.assertEquals(loggedEvents.contains("Event response code 400"), true); + Assert.assertEquals(Collections.frequency(loggedEvents, "Event response code 400"), 2); Assert.assertEquals(receivedEventNameList.toString(), expected.toString()); siddhiAppRuntime.shutdown(); + logger.removeAppender(appender); } /** @@ -188,10 +198,14 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { */ @Test(dependsOnMethods = "testHTTPInputTransportPutMethod") public void testMultipleListenersSameURL() throws Exception { - logger.info("Creating test for same url in different execution plain."); - final TestAppender appender = new TestAppender(); - final Logger logger = Logger.getRootLogger(); + logObj.info("Creating test for same url in different execution plain."); + final HttpBasicTestCaseTestAppender appender = new + HttpBasicTestCaseTestAppender("HttpBasicTestCaseTestAppender", null); + final Logger logger = (Logger) LogManager.getRootLogger(); + logger.setLevel(Level.ALL); logger.addAppender(appender); + appender.start(); + logger.info("ADDED the HttpBasicTestCaseTestAppender appender in testMultipleListenersSameURL"); URI baseURI = URI.create(String.format("http://%s:%d", "localhost", 8008)); List receivedEventNameList = new ArrayList<>(2); PersistenceStore persistenceStore = new InMemoryPersistenceStore(); @@ -272,17 +286,14 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { "POST"); HttpTestUtil.httpPublishEvent(event2, baseURI, "/endpoints/abc", "POST"); - final List log = appender.getLog(); - List logMessages = new ArrayList<>(); - for (LoggingEvent logEvent : log) { - logMessages.add(String.valueOf(logEvent.getMessage())); - } + final List loggedEvents = ((HttpBasicTestCaseTestAppender) logger.getAppenders(). + get("HttpBasicTestCaseTestAppender")).getLog(); SiddhiTestHelper.waitForEvents(waitTime, 2, eventCount, timeout); - Assert.assertEquals(logMessages.contains("Error on '" + siddhiAppRuntime2.getName() + + Assert.assertEquals(loggedEvents.contains("Error on '" + siddhiAppRuntime2.getName() + "'. Listener URL http://localhost:8008/endpoints/abc already connected " + "Error while connecting at Source 'http' at 'inputStream2'.") , true); - Assert.assertEquals(Collections.frequency(logMessages, "Error on '" + siddhiAppRuntime2.getName() + + Assert.assertEquals(Collections.frequency(loggedEvents, "Error on '" + siddhiAppRuntime2.getName() + "'. Listener URL http://localhost:8008/endpoints/abc already connected " + "Error while connecting at Source 'http' at 'inputStream2'."), 1); } catch (InterruptedException e) { @@ -294,6 +305,7 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { logger.error(t.getMessage(), t); } finally { siddhiAppRuntime.shutdown(); + logger.removeAppender(appender); } Assert.assertEquals(receivedEventNameList.toString(), expected.toString()); @@ -304,10 +316,13 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { */ @Test(dependsOnMethods = "testMultipleListenersSameURL") public void testMultipleListenersSameURLInSameExecutionPlan() throws InterruptedException { - logger.info("Creating test for publishing events same url in same execution plain."); - final TestAppender appender = new TestAppender(); - final Logger logger = Logger.getRootLogger(); + logObj.info("Creating test for publishing events same url in same execution plain."); + final HttpBasicTestCaseTestAppender appender = new + HttpBasicTestCaseTestAppender("HttpBasicTestCaseTestAppender", null); + final Logger logger = (Logger) LogManager.getRootLogger(); + logger.setLevel(Level.ALL); logger.addAppender(appender); + appender.start(); SiddhiManager siddhiManager = new SiddhiManager(); List receivedEventNameListA = new ArrayList<>(2); @@ -357,22 +372,20 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { try { siddhiAppRuntime.start(); //To check weather only one is deployed - final List log = appender.getLog(); - List logMessages = new ArrayList<>(); - for (LoggingEvent logEvent : log) { - logMessages.add(String.valueOf(logEvent.getMessage())); - } + final List loggedEvents = ((HttpBasicTestCaseTestAppender) logger.getAppenders(). + get("HttpBasicTestCaseTestAppender")).getLog(); SiddhiTestHelper.waitForEvents(waitTime, 0, eventCount, timeout); - Assert.assertEquals(logMessages.contains("Error on '" + siddhiAppRuntime.getName() + "'. Listener URL " + + Assert.assertEquals(loggedEvents.contains("Error on '" + siddhiAppRuntime.getName() + "'. Listener URL " + "http://localhost:8006/endpoints/RecPro already connected Error while " + "connecting at Source 'http' at 'inputStreamA'."), true); - Assert.assertEquals(Collections.frequency(logMessages, "Error on '" + siddhiAppRuntime.getName() + + Assert.assertEquals(Collections.frequency(loggedEvents, "Error on '" + siddhiAppRuntime.getName() + "'. Listener URL http://localhost:8006/endpoints/RecPro already connected Error while " + "connecting at Source 'http' at 'inputStreamA'."), 1); } catch (InterruptedException t) { logger.error(t.getMessage(), t); } finally { siddhiAppRuntime.shutdown(); + logger.removeAppender(appender); } } @@ -383,7 +396,7 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { */ @Test(dependsOnMethods = "testMultipleListenersSameURLInSameExecutionPlan") public void testHTTPInputTransportDifferentURL() throws Exception { - logger.info("Creating test for publishing events with different url with same context."); + logObj.info("Creating test for publishing events with different url with same context."); URI baseURIA = URI.create(String.format("http://%s:%d", "localhost", 8005)); URI baseURIB = URI.create(String.format("http://%s:%d", "localhost", 8009)); List receivedEventNameListA = new ArrayList<>(2); @@ -487,7 +500,7 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { */ @Test(dependsOnMethods = "testHTTPInputTransportDifferentURL") public void testHTTPInputTransportEmployPayload() throws Exception { - logger.info("Creating test for publishing events with empty payload."); + logObj.info("Creating test for publishing events with empty payload."); URI baseURI = URI.create(String.format("http://%s:%d", "localhost", 8005)); List receivedEventNameList = new ArrayList<>(2); SiddhiManager siddhiManager = new SiddhiManager(); @@ -523,26 +536,35 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { siddhiAppRuntime.shutdown(); } - private static class TestAppender extends AppenderSkeleton { + @Plugin(name = "HttpBasicTestCaseTestAppender", + category = Core.CATEGORY_NAME, elementType = Appender.ELEMENT_TYPE) + private static class HttpBasicTestCaseTestAppender extends AbstractAppender { - private final List log = new ArrayList<>(); + private final List log = new ArrayList<>(); - @Override - public boolean requiresLayout() { - return false; + public HttpBasicTestCaseTestAppender(String name, Filter filter) { + + super(name, filter, null); } - @Override - protected void append(final LoggingEvent loggingEvent) { - log.add(loggingEvent); + @PluginFactory + public static HttpBasicTestCaseTestAppender createAppender( + @PluginAttribute("name") String name, + @PluginElement("Filter") Filter filter) { + + return new HttpBasicTestCaseTestAppender(name, filter); } @Override - public void close() { + public void append(LogEvent event) { + log.add(event.getMessage().getFormattedMessage()); + } - List getLog() { - return new ArrayList<>(log); + public List getLog() { + List clone = new ArrayList<>(log); + log.clear(); + return clone; } } } diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/HttpCustomConfigTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/HttpCustomConfigTestCase.java index 900a05c6..935588e1 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/HttpCustomConfigTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/HttpCustomConfigTestCase.java @@ -30,7 +30,8 @@ import io.siddhi.core.util.persistence.PersistenceStore; import io.siddhi.extension.io.http.source.util.HttpTestUtil; import io.siddhi.extension.map.xml.sourcemapper.XmlSourceMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -44,7 +45,7 @@ * Http test case with custom configurations. */ public class HttpCustomConfigTestCase { - private static final Logger logger = Logger.getLogger(HttpCustomConfigTestCase.class); + private static final Logger logger = LogManager.getLogger(HttpCustomConfigTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); private int waitTime = 500; private int timeout = 30000; diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/HttpCustomThreadPoolConfigTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/HttpCustomThreadPoolConfigTestCase.java index 7bd3de0d..8f9dfc7a 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/HttpCustomThreadPoolConfigTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/HttpCustomThreadPoolConfigTestCase.java @@ -28,7 +28,8 @@ import io.siddhi.core.util.persistence.PersistenceStore; import io.siddhi.extension.io.http.source.util.HttpTestUtil; import io.siddhi.extension.map.xml.sourcemapper.XmlSourceMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -42,7 +43,7 @@ * Test case for HTTPS protocol. */ public class HttpCustomThreadPoolConfigTestCase { - private static final Logger logger = Logger.getLogger(HttpCustomThreadPoolConfigTestCase.class); + private static final Logger logger = LogManager.getLogger(HttpCustomThreadPoolConfigTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); @BeforeMethod diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/HttpMultipleFormatSourcesTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/HttpMultipleFormatSourcesTestCase.java index c5f20366..9e65e4e6 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/HttpMultipleFormatSourcesTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/HttpMultipleFormatSourcesTestCase.java @@ -27,7 +27,8 @@ import io.siddhi.core.util.persistence.PersistenceStore; import io.siddhi.extension.io.http.source.util.HttpTestUtil; import io.siddhi.extension.map.xml.sourcemapper.XmlSourceMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -42,7 +43,7 @@ */ public class HttpMultipleFormatSourcesTestCase { - private static final Logger logger = Logger.getLogger(HttpMultipleFormatSourcesTestCase.class); + private static final Logger logger = LogManager.getLogger(HttpMultipleFormatSourcesTestCase.class); private AtomicInteger eventCountA = new AtomicInteger(0); private AtomicInteger eventCountB = new AtomicInteger(0); private AtomicInteger eventCountC = new AtomicInteger(0); diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/HttpOrderlyProcessTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/HttpOrderlyProcessTestCase.java index a4f5058f..cd89b3a3 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/HttpOrderlyProcessTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/HttpOrderlyProcessTestCase.java @@ -28,7 +28,8 @@ import io.siddhi.core.util.persistence.PersistenceStore; import io.siddhi.extension.io.http.source.util.HttpTestUtil; import io.siddhi.extension.map.xml.sourcemapper.XmlSourceMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -42,7 +43,7 @@ * Test case for HTTPS protocol. */ public class HttpOrderlyProcessTestCase { - private static final Logger logger = Logger.getLogger(HttpOrderlyProcessTestCase.class); + private static final Logger logger = LogManager.getLogger(HttpOrderlyProcessTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); @BeforeMethod diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceBasicAuthTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceBasicAuthTestCase.java index 576bb883..8a4c39a0 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceBasicAuthTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceBasicAuthTestCase.java @@ -28,7 +28,8 @@ import io.siddhi.core.util.persistence.PersistenceStore; import io.siddhi.extension.io.http.source.util.HttpTestUtil; import io.siddhi.extension.map.xml.sourcemapper.XmlSourceMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -42,7 +43,7 @@ * Test cases for basic authentication. */ public class HttpSourceBasicAuthTestCase { - private static final Logger logger = Logger.getLogger(HttpSourceBasicAuthTestCase.class); + private static final Logger logger = LogManager.getLogger(HttpSourceBasicAuthTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); private int waitTime = 50; private int timeout = 30000; diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceConflictsTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceConflictsTestCase.java index 48abb13c..cb4c45ad 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceConflictsTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceConflictsTestCase.java @@ -26,7 +26,8 @@ import io.siddhi.core.util.config.InMemoryConfigManager; import io.siddhi.extension.io.http.source.util.HttpTestUtil; import io.siddhi.extension.map.xml.sourcemapper.XmlSourceMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -42,7 +43,7 @@ */ public class HttpSourceConflictsTestCase { - private static final Logger logger = Logger.getLogger(HttpSourceConflictsTestCase.class); + private static final Logger logger = LogManager.getLogger(HttpSourceConflictsTestCase.class); private AtomicInteger eventCountA = new AtomicInteger(0); private AtomicInteger eventCountB = new AtomicInteger(0); private int waitTime = 50; diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceMappingTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceMappingTestCase.java index 7793e4db..b86bdeab 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceMappingTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/HttpSourceMappingTestCase.java @@ -30,6 +30,8 @@ import io.siddhi.extension.map.json.sourcemapper.JsonSourceMapper; import io.siddhi.extension.map.text.sourcemapper.TextSourceMapper; import io.siddhi.extension.map.xml.sourcemapper.XmlSourceMapper; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -43,8 +45,7 @@ * Test cases for mapping types. */ public class HttpSourceMappingTestCase { - private static final org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger - (HttpSourceMappingTestCase.class); + private static final Logger logger = LogManager.getLogger(HttpSourceMappingTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); private int waitTime = 50; private int timeout = 30000; diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/HttpsSSLSourceTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/HttpsSSLSourceTestCase.java index 588fce83..8419a2d4 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/HttpsSSLSourceTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/HttpsSSLSourceTestCase.java @@ -27,10 +27,18 @@ import io.siddhi.core.util.config.InMemoryConfigManager; import io.siddhi.extension.io.http.source.util.HttpTestUtil; import io.siddhi.extension.map.xml.sourcemapper.XmlSourceMapper; -import org.apache.log4j.AppenderSkeleton; -import org.apache.log4j.Level; -import org.apache.log4j.Logger; -import org.apache.log4j.spi.LoggingEvent; +import org.apache.logging.log4j.Level; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.core.Appender; +import org.apache.logging.log4j.core.Core; +import org.apache.logging.log4j.core.Filter; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.core.Logger; +import org.apache.logging.log4j.core.appender.AbstractAppender; +import org.apache.logging.log4j.core.config.plugins.Plugin; +import org.apache.logging.log4j.core.config.plugins.PluginAttribute; +import org.apache.logging.log4j.core.config.plugins.PluginElement; +import org.apache.logging.log4j.core.config.plugins.PluginFactory; import org.testng.Assert; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; @@ -46,7 +54,7 @@ * Test case for HTTPS protocol. */ public class HttpsSSLSourceTestCase { - private static final Logger logger = Logger.getLogger(HttpsSSLSourceTestCase.class); + private static final Logger logger = (Logger) LogManager.getLogger(HttpsSSLSourceTestCase.class); private AtomicInteger eventCount = new AtomicInteger(0); private int waitTime = 50; private int timeout = 30000; @@ -129,8 +137,10 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { */ @Test(dependsOnMethods = "testHTTPSInputTransport") public void testHTTPSInputTransportInvalidKeyStore() throws Exception { - final TestAppender appender = new TestAppender(); - final Logger logger = Logger.getRootLogger(); + final HttpsSSLSourceTestCaseTestAppender appender = new + HttpsSSLSourceTestCaseTestAppender("HttpsSSLSourceTestCaseTestAppender", null); + final Logger logger = (Logger) LogManager.getRootLogger(); + logger.setLevel(Level.ALL); logger.addAppender(appender); logger.info("Creating test for publishing events with https protocol with invalid keystore."); HttpTestUtil.setCarbonHome(); @@ -186,16 +196,19 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { + ""; HttpTestUtil.httpsPublishEvent(event1); HttpTestUtil.httpsPublishEvent(event2); - final List log = appender.getLog(); - List logMessages = new ArrayList<>(); - for (LoggingEvent logEvent : log) { - logMessages.add(logEvent.getLevel()); - } - Assert.assertEquals(logMessages.contains(Level.ERROR), true); - Assert.assertEquals(Collections.frequency(logMessages, Level.ERROR), 4); + final List loggedEvents = ((HttpsSSLSourceTestCaseTestAppender) logger.getAppenders(). + get("HttpsSSLSourceTestCaseTestAppender")).getLog(); + Assert.assertEquals(loggedEvents.contains("Error on '" + siddhiAppRuntime.getName() + + "'. Failed to initialize the SSLContext: Keystore was tampered with, or password was " + + "incorrect Error while connecting at Source 'http' at 'inputStream'.") + , true); + Assert.assertEquals(Collections.frequency(loggedEvents, "Error on '" + siddhiAppRuntime.getName() + + "'. Failed to initialize the SSLContext: Keystore was tampered with, or password was " + + "incorrect Error while connecting at Source 'http' at 'inputStream'."), 1); SiddhiTestHelper.waitForEvents(waitTime, 0, eventCount, timeout); Assert.assertEquals(receivedEventNameList.toString(), expected.toString()); siddhiAppRuntime.shutdown(); + logger.removeAppender(appender); } /** @@ -205,8 +218,10 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { */ @Test(dependsOnMethods = "testHTTPSInputTransportInvalidKeyStore") public void testHTTPSInputTransportInvalidKeyStorePass() throws Exception { - final TestAppender appender = new TestAppender(); - final Logger logger = Logger.getRootLogger(); + final HttpsSSLSourceTestCaseTestAppender appender = new + HttpsSSLSourceTestCaseTestAppender("HttpsSSLSourceTestCaseTestAppender", null); + final Logger logger = (Logger) LogManager.getRootLogger(); + logger.setLevel(Level.ALL); logger.addAppender(appender); logger.info("Creating test for publishing events with https protocol with invalid keystore pass."); HttpTestUtil.setCarbonHome(); @@ -263,18 +278,21 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { + ""; HttpTestUtil.httpsPublishEvent(event1); HttpTestUtil.httpsPublishEvent(event2); - final List log = appender.getLog(); - List logMessages = new ArrayList<>(); - for (LoggingEvent logEvent : log) { - logMessages.add(logEvent.getLevel()); - } - Assert.assertEquals(logMessages.contains(Level.ERROR), true); - Assert.assertEquals(Collections.frequency(logMessages, Level.ERROR), 4); + final List loggedEvents = ((HttpsSSLSourceTestCaseTestAppender) logger.getAppenders(). + get("HttpsSSLSourceTestCaseTestAppender")).getLog(); + Assert.assertEquals(loggedEvents.contains("Error on '" + siddhiAppRuntime.getName() + + "'. Failed to initialize the SSLContext: Keystore was tampered with, or password was " + + "incorrect Error while connecting at Source 'http' at 'inputStream'.") + , true); + Assert.assertEquals(Collections.frequency(loggedEvents, "Error on '" + siddhiAppRuntime.getName() + + "'. Failed to initialize the SSLContext: Keystore was tampered with, or password was " + + "incorrect Error while connecting at Source 'http' at 'inputStream'."), 1); SiddhiTestHelper.waitForEvents(waitTime, 0, eventCount, timeout); } catch (InterruptedException t) { logger.error(t.getMessage(), t); } finally { siddhiAppRuntime.shutdown(); + logger.removeAppender(appender); } Assert.assertEquals(receivedEventNameList.toString(), expected.toString()); } @@ -286,8 +304,10 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { */ @Test(enabled = false, dependsOnMethods = "testHTTPSInputTransportInvalidKeyStorePass") public void testHTTPSInputTransportInvalidCertPassword() throws Exception { - final TestAppender appender = new TestAppender(); - final Logger logger = Logger.getRootLogger(); + final HttpsSSLSourceTestCaseTestAppender appender = new + HttpsSSLSourceTestCaseTestAppender("HttpsSSLSourceTestCaseTestAppender", null); + final Logger logger = (Logger) LogManager.getRootLogger(); + logger.setLevel(Level.ALL); logger.addAppender(appender); logger.info(" Creating test for publishing events with https protocol with invalid cert pass."); HttpTestUtil.setCarbonHome(); @@ -341,37 +361,50 @@ public void receive(long timeStamp, Event[] inEvents, Event[] removeEvents) { + ""; HttpTestUtil.httpsPublishEvent(event1); HttpTestUtil.httpsPublishEvent(event2); - final List log = appender.getLog(); - List logMessages = new ArrayList<>(); - for (LoggingEvent logEvent : log) { - logMessages.add(logEvent.getLevel()); - } - Assert.assertEquals(logMessages.contains(Level.ERROR), true); - Assert.assertEquals(Collections.frequency(logMessages, Level.ERROR), 2); + final List loggedEvents = ((HttpsSSLSourceTestCaseTestAppender) logger.getAppenders(). + get("HttpsSSLSourceTestCaseTestAppender")).getLog(); + Assert.assertEquals(loggedEvents.contains("Error on '" + siddhiAppRuntime.getName() + + "'. Failed to initialize the SSLContext: Keystore was tampered with, or password was " + + "incorrect Error while connecting at Source 'http' at 'inputStream'.") + , true); + Assert.assertEquals(Collections.frequency(loggedEvents, "Error on '" + siddhiAppRuntime.getName() + + "'. Failed to initialize the SSLContext: Keystore was tampered with, or password was " + + "incorrect Error while connecting at Source 'http' at 'inputStream'."), 1); SiddhiTestHelper.waitForEvents(waitTime, 0, eventCount, timeout); Assert.assertEquals(receivedEventNameList.toString(), expected.toString()); siddhiAppRuntime.shutdown(); + logger.removeAppender(appender); } - private class TestAppender extends AppenderSkeleton { - private final List log = new ArrayList<>(); + @Plugin(name = "HttpsSSLSourceTestCaseTestAppender", + category = Core.CATEGORY_NAME, elementType = Appender.ELEMENT_TYPE) + private class HttpsSSLSourceTestCaseTestAppender extends AbstractAppender { - @Override - public boolean requiresLayout() { - return false; + private final List log = new ArrayList<>(); + + public HttpsSSLSourceTestCaseTestAppender(String name, Filter filter) { + + super(name, filter, null); } - @Override - protected void append(final LoggingEvent loggingEvent) { - log.add(loggingEvent); + @PluginFactory + public HttpsSSLSourceTestCaseTestAppender createAppender( + @PluginAttribute("name") String name, + @PluginElement("Filter") Filter filter) { + + return new HttpsSSLSourceTestCaseTestAppender(name, filter); } @Override - public void close() { + public void append(LogEvent event) { + log.add(event.getMessage().getFormattedMessage()); + } - List getLog() { - return new ArrayList(log); + public List getLog() { + List clone = new ArrayList<>(log); + log.clear(); + return clone; } } } diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/SSESourceTestCase.java b/component/src/test/java/io/siddhi/extension/io/http/source/SSESourceTestCase.java index 1cdc7b4c..1e000964 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/SSESourceTestCase.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/SSESourceTestCase.java @@ -31,7 +31,8 @@ import io.siddhi.extension.io.http.util.HttpConstants; import io.siddhi.extension.map.json.sinkmapper.JsonSinkMapper; import io.siddhi.extension.map.json.sourcemapper.JsonSourceMapper; -import org.apache.log4j.Logger; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import org.testng.Assert; import org.testng.annotations.AfterMethod; import org.testng.annotations.BeforeMethod; @@ -50,7 +51,7 @@ * Tests http sse source. */ public class SSESourceTestCase { - private static final Logger log = Logger.getLogger(SSESourceTestCase.class); + private static final Logger log = LogManager.getLogger(SSESourceTestCase.class); private static final int SSE_SERVER_PORT = 8010; private static final int SLEEP_TIME = 50; private static final int EVENT_COUNT = 10; diff --git a/component/src/test/java/io/siddhi/extension/io/http/source/util/HttpTestUtil.java b/component/src/test/java/io/siddhi/extension/io/http/source/util/HttpTestUtil.java index ccb84d53..fed03d23 100644 --- a/component/src/test/java/io/siddhi/extension/io/http/source/util/HttpTestUtil.java +++ b/component/src/test/java/io/siddhi/extension/io/http/source/util/HttpTestUtil.java @@ -19,6 +19,8 @@ package io.siddhi.extension.io.http.source.util; import io.netty.handler.codec.http.HttpMethod; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import java.io.BufferedInputStream; import java.io.BufferedReader; @@ -47,7 +49,7 @@ * Util class for test cases. */ public class HttpTestUtil { - private static final org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger(HttpTestUtil.class); + private static final Logger logger = LogManager.getLogger(HttpTestUtil.class); private static final String CARBON_HOME = "carbon.home"; public static void setCarbonHome() { diff --git a/component/src/test/resources/log4j.properties b/component/src/test/resources/log4j.properties deleted file mode 100644 index fdc34345..00000000 --- a/component/src/test/resources/log4j.properties +++ /dev/null @@ -1,36 +0,0 @@ -# -# Copyright (c) 2018, WSO2 Inc. (http://www.wso2.org) All Rights Reserved. -# -# WSO2 Inc. licenses this file to you under the Apache License, -# Version 2.0 (the "License"); you may not use this file except -# in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. -# -# For the general syntax of property based configuration files see the -# documenation of org.apache.log4j.PropertyConfigurator. -# The root category uses the appender called A1. Since no priority is -# specified, the root category assumes the default priority for root -# which is DEBUG in log4j. The root category is the only category that -# has a default priority. All other categories need not be assigned a -# priority in which case they inherit their priority from the -# hierarchy. -#log4j.rootLogger=DEBUG, stdout -log4j.rootLogger=INFO, stdout -log4j.appender.stdout=org.apache.log4j.ConsoleAppender -log4j.appender.stdout.layout=org.apache.log4j.PatternLayout -log4j.appender.stdout.layout.ConversionPattern=[%t] %-5p %c %x - %m%n -log4j.logger.org.apache.zookeeper=ERROR, stdout -log4j.logger.kafka.consumer=ERROR, stdout -log4j.logger.kafka.utils=ERROR, stdout -log4j.logger.org.I0Itec.zkclient=ERROR, stdout -log4j.logger.org.apache.zookeeper.ZooKeeper=ERROR, stdout -log4j.logger.kafka.producer=ERROR, stdout diff --git a/component/src/test/resources/log4j2.xml b/component/src/test/resources/log4j2.xml new file mode 100644 index 00000000..fa29e9bf --- /dev/null +++ b/component/src/test/resources/log4j2.xml @@ -0,0 +1,35 @@ + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/coverage-report/pom.xml b/coverage-report/pom.xml index d1c4507e..8ff38e22 100644 --- a/coverage-report/pom.xml +++ b/coverage-report/pom.xml @@ -21,7 +21,7 @@ siddhi-io-http-parent io.siddhi.extension.io.http - 2.3.3 + 2.3.4 ../pom.xml 4.0.0 diff --git a/docs/api/2.2.2.md b/docs/api/2.2.2.md index c496f68b..570ae4c5 100644 --- a/docs/api/2.2.2.md +++ b/docs/api/2.2.2.md @@ -12,7 +12,7 @@ Syntax ``` -@sink(type="http", publisher.url="", basic.auth.username="", basic.auth.password="", https.truststore.file="", https.truststore.password="", oauth.username="", oauth.password="", consumer.key="", consumer.secret="", token.url="", refresh.token="", headers="", method="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", tls.store.type="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) +@sink(type="http", publisher.url="", basic.auth.username="", basic.auth.password="", https.truststore.file="", https.truststore.password="", oauth.username="", oauth.password="", consumer.key="", consumer.secret="", body.consumer.key="", body.consumer.secret="", token.url="", refresh.token="", oauth.scope="", headers="", method="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", tls.store.type="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) ``` QUERY PARAMETERS @@ -97,6 +97,22 @@ Yes No + + body.consumer.key +

Consumer key used for calling endpoints protected by OAuth 2.0 if it's required to be sent in token request body

+ - + STRING + Yes + No + + + body.consumer.secret +

Consumer secret used for calling endpoints protected by OAuth 2.0

+ - + STRING + Yes + No + token.url

Token URL to generate a new access tokens when calling endpoints protected by OAuth 2.0

@@ -113,6 +129,14 @@ Yes No + + oauth.scope +

Standard OAuth 2.0 scope parameter

+ default + STRING + Yes + No + headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When Content-Type header is not provided the system derives the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

diff --git a/docs/api/2.3.4.md b/docs/api/2.3.4.md new file mode 100644 index 00000000..a1c88ee8 --- /dev/null +++ b/docs/api/2.3.4.md @@ -0,0 +1,2974 @@ +# API Docs - v2.3.4 + +!!! Info "Tested Siddhi Core version: *5.1.21*" + It could also support other Siddhi Core minor versions. + +## Sink + +### http *(Sink)* +

+

HTTP sink publishes messages via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML and JSON. It can also publish to endpoints protected by basic authentication or OAuth 2.0.

+

+Syntax + +``` +@sink(type="http", publisher.url="", basic.auth.username="", basic.auth.password="", https.truststore.file="", https.truststore.password="", oauth.username="", oauth.password="", consumer.key="", consumer.secret="", body.consumer.key="", body.consumer.secret="", token.url="", refresh.token="", oauth.scope="", headers="", method="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", tls.store.type="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", executor.service.threads="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
publisher.url

The URL to which the outgoing events should be published.
Examples:
http://localhost:8080/endpoint,
https://localhost:8080/endpoint

STRINGNoNo
basic.auth.username

The username to be included in the authentication header when calling endpoints protected by basic authentication. basic.auth.password property should be also set when using this property.

-STRINGYesNo
basic.auth.password

The password to be included in the authentication header when calling endpoints protected by basic authentication. basic.auth.username property should be also set when using this property.

-STRINGYesNo
https.truststore.file

The file path of the client truststore when sending messages through https protocol.

`${carbon.home}/resources/security/client-truststore.jks`STRINGYesNo
https.truststore.password

The password for the client-truststore.

wso2carbonSTRINGYesNo
oauth.username

The username to be included in the authentication header when calling endpoints protected by OAuth 2.0. oauth.password property should be also set when using this property.

-STRINGYesNo
oauth.password

The password to be included in the authentication header when calling endpoints protected by OAuth 2.0. oauth.username property should be also set when using this property.

-STRINGYesNo
consumer.key

Consumer key used for calling endpoints protected by OAuth 2.0

-STRINGYesNo
consumer.secret

Consumer secret used for calling endpoints protected by OAuth 2.0

-STRINGYesNo
body.consumer.key

Consumer key used for calling endpoints protected by OAuth 2.0 if it's required to be sent in token request body

-STRINGYesNo
body.consumer.secret

Consumer secret used for calling endpoints protected by OAuth 2.0

-STRINGYesNo
token.url

Token URL to generate a new access tokens when calling endpoints protected by OAuth 2.0

-STRINGYesNo
refresh.token

Refresh token used for generating new access tokens when calling endpoints protected by OAuth 2.0

-STRINGYesNo
oauth.scope

Standard OAuth 2.0 scope parameter

defaultSTRINGYesNo
headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When Content-Type header is not provided the system derives the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

Content-Type and Content-Length headersSTRINGYesNo
method

The HTTP method used for calling the endpoint.

POSTSTRINGYesNo
socket.idle.timeout

Socket timeout in millis.

6000INTYesNo
chunk.disabled

Disable chunked transfer encoding.

falseBOOLYesNo
ssl.protocol

SSL/TLS protocol.

TLSSTRINGYesNo
ssl.verification.disabled

Disable SSL verification.

falseBOOLYesNo
tls.store.type

TLS store type.

JKSSTRINGYesNo
ssl.configurations

SSL/TSL configurations in format "'<key>:<value>','<key>:<value>'".
Some supported parameters:
 - SSL/TLS protocols: 'sslEnabledProtocols:TLSv1.1,TLSv1.2'
 - List of ciphers: 'ciphers:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256'
 - Enable session creation: 'client.enable.session.creation:true'
 - Supported server names: 'server.suported.server.names:server'
 - Add HTTP SNIMatcher: 'server.supported.snimatchers:SNIMatcher'

-STRINGYesNo
proxy.host

Proxy server host

-STRINGYesNo
proxy.port

Proxy server port

-STRINGYesNo
proxy.username

Proxy server username

-STRINGYesNo
proxy.password

Proxy server password

-STRINGYesNo
client.bootstrap.configurations

Client bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Client connect timeout in millis: 'client.bootstrap.connect.timeout:15000'
 - Client socket timeout in seconds: 'client.bootstrap.socket.timeout:15'
 - Client socket reuse: 'client.bootstrap.socket.reuse:true'
 - Enable TCP no delay: 'client.bootstrap.nodelay:true'
 - Enable client keep alive: 'client.bootstrap.keepalive:true'
 - Send buffer size: 'client.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'client.bootstrap.recievebuffersize:1048576'

-STRINGYesNo
max.pool.active.connections

Maximum possible number of active connection per client pool.

-1INTYesNo
min.pool.idle.connections

Minimum number of idle connections that can exist per client pool.

0INTYesNo
max.pool.idle.connections

Maximum number of idle connections that can exist per client pool.

100INTYesNo
executor.service.threads

Thread count for the executor service.

20INTYesNo
min.evictable.idle.time

Minimum time (in millis) a connection may sit idle in the client pool before it become eligible for eviction.

300000STRINGYesNo
time.between.eviction.runs

Time between two eviction operations (in millis) on the client pool.

30000STRINGYesNo
max.wait.time

The maximum time (in millis) the pool will wait (when there are no available connections) for a connection to be returned to the pool.

60000STRINGYesNo
test.on.borrow

Enable connections to be validated before being borrowed from the client pool.

trueBOOLYesNo
test.while.idle

Enable connections to be validated during the eviction operation (if any).

trueBOOLYesNo
exhausted.action

Action that should be taken when the maximum number of active connections are being used. This action should be indicated as an int and possible action values are following.
0 - Fail the request.
1 - Block the request, until a connection returns to the pool.
2 - Grow the connection pool size.

1 (Block when exhausted)INTYesNo
hostname.verification.enabled

Enable hostname verification.

trueBOOLYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
clientBootstrapClientGroupSize

Number of client threads to perform non-blocking read and write to one or more channels.

(Number of available processors) * 2Any positive integer
clientBootstrapBossGroupSize

Number of boss threads to accept incoming connections.

Number of available processorsAny positive integer
clientBootstrapWorkerGroupSize

Number of worker threads to accept the connections from boss threads and perform non-blocking read and write from one or more channels.

(Number of available processors) * 2Any positive integer
trustStoreLocation

The default truststore file path.

`${carbon.home}/resources/security/client-truststore.jks`Path to client truststore `.jks` file
trustStorePassword

The default truststore password.

wso2carbonTruststore password as string
+ +Examples +EXAMPLE 1 +``` +@sink(type = 'http', publisher.url = 'http://stocks.com/stocks', + @map(type = 'json')) +define stream StockStream (symbol string, price float, volume long); +``` +

+

Events arriving on the StockStream will be published to the HTTP endpoint http://stocks.com/stocks using POST method with Content-Type application/json by converting those events to the default JSON format as following:

{
+  "event": {
+    "symbol": "FB",
+    "price": 24.5,
+    "volume": 5000
+  }
+}

+

+EXAMPLE 2 +``` +@sink(type='http', publisher.url = 'http://localhost:8009/foo', + client.bootstrap.configurations = "'client.bootstrap.socket.timeout:20'", + max.pool.active.connections = '1', headers = "{{headers}}", + @map(type='xml', @payload(""" +{{payloadBody}} +"""))) +define stream FooStream (payloadBody String, headers string); +``` +

+

Events arriving on FooStream will be published to the HTTP endpoint http://localhost:8009/foo using POST method with Content-Type application/xml and setting payloadBody and header attribute values.
If the payloadBody contains

<symbol>WSO2</symbol>
+<price>55.6</price>
+<volume>100</volume>

and header contains 'topic:foobar' values, then the system will generate an output with the body:

<stock>
+<symbol>WSO2</symbol>
+<price>55.6</price>
+<volume>100</volume>
+</stock>

and HTTP headers:
Content-Length:xxx,
Content-Location:'xxx',
Content-Type:'application/xml',
HTTP_METHOD:'POST'

+

+### http-call *(Sink)* +

+

The http-call sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-call-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

+

+Syntax + +``` +@sink(type="http-call", publisher.url="", sink.id="", basic.auth.username="", basic.auth.password="", https.truststore.file="", https.truststore.password="", oauth.username="", oauth.password="", consumer.key="", consumer.secret="", token.url="", refresh.token="", headers="", method="", downloading.enabled="", download.path="", blocking.io="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
publisher.url

The URL which should be called.
Examples:
http://localhost:8080/endpoint,
https://localhost:8080/endpoint

STRINGNoNo
sink.id

Identifier to correlate the http-call sink to its corresponding http-call-response sources to retrieved the responses.

STRINGNoNo
basic.auth.username

The username to be included in the authentication header when calling endpoints protected by basic authentication. basic.auth.password property should be also set when using this property.

-STRINGYesNo
basic.auth.password

The password to be included in the authentication header when calling endpoints protected by basic authentication. basic.auth.username property should be also set when using this property.

-STRINGYesNo
https.truststore.file

The file path of the client truststore when sending messages through https protocol.

`${carbon.home}/resources/security/client-truststore.jks`STRINGYesNo
https.truststore.password

The password for the client-truststore.

wso2carbonSTRINGYesNo
oauth.username

The username to be included in the authentication header when calling endpoints protected by OAuth 2.0. oauth.password property should be also set when using this property.

-STRINGYesNo
oauth.password

The password to be included in the authentication header when calling endpoints protected by OAuth 2.0. oauth.username property should be also set when using this property.

-STRINGYesNo
consumer.key

Consumer key used for calling endpoints protected by OAuth 2.0

-STRINGYesNo
consumer.secret

Consumer secret used for calling endpoints protected by OAuth 2.0

-STRINGYesNo
token.url

Token URL to generate a new access tokens when calling endpoints protected by OAuth 2.0

-STRINGYesNo
refresh.token

Refresh token used for generating new access tokens when calling endpoints protected by OAuth 2.0

-STRINGYesNo
headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When the Content-Type header is not provided the system decides the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

Content-Type and Content-Length headersSTRINGYesNo
method

The HTTP method used for calling the endpoint.

POSTSTRINGYesNo
downloading.enabled

Enable response received by the http-call-response source to be written to a file. When this is enabled the download.path property should be also set.

falseBOOLYesNo
download.path

The absolute file path along with the file name where the downloads should be saved.

-STRINGYesYes
blocking.io

Blocks the request thread until a response it received from HTTP call-response source before sending any other request.

falseBOOLYesNo
socket.idle.timeout

Socket timeout in millis.

6000INTYesNo
chunk.disabled

Disable chunked transfer encoding.

falseBOOLYesNo
ssl.protocol

SSL/TLS protocol.

TLSSTRINGYesNo
ssl.verification.disabled

Disable SSL verification.

falseBOOLYesNo
ssl.configurations

SSL/TSL configurations.
Expected format "'<key>:<value>','<key>:<value>'".
Some supported parameters:
 - SSL/TLS protocols: 'sslEnabledProtocols:TLSv1.1,TLSv1.2'
 - List of ciphers: 'ciphers:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256'
 - Enable session creation: 'client.enable.session.creation:true'
 - Supported server names: 'server.suported.server.names:server'
 - Add HTTP SNIMatcher: 'server.supported.snimatchers:SNIMatcher'

-STRINGYesNo
proxy.host

Proxy server host

-STRINGYesNo
proxy.port

Proxy server port

-STRINGYesNo
proxy.username

Proxy server username

-STRINGYesNo
proxy.password

Proxy server password

-STRINGYesNo
client.bootstrap.configurations

Client bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Client connect timeout in millis: 'client.bootstrap.connect.timeout:15000'
 - Client socket timeout in seconds: 'client.bootstrap.socket.timeout:15'
 - Client socket reuse: 'client.bootstrap.socket.reuse:true'
 - Enable TCP no delay: 'client.bootstrap.nodelay:true'
 - Enable client keep alive: 'client.bootstrap.keepalive:true'
 - Send buffer size: 'client.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'client.bootstrap.recievebuffersize:1048576'

-STRINGYesNo
max.pool.active.connections

Maximum possible number of active connection per client pool.

-1INTYesNo
min.pool.idle.connections

Minimum number of idle connections that can exist per client pool.

0INTYesNo
max.pool.idle.connections

Maximum number of idle connections that can exist per client pool.

100INTYesNo
min.evictable.idle.time

Minimum time (in millis) a connection may sit idle in the client pool before it become eligible for eviction.

300000STRINGYesNo
time.between.eviction.runs

Time between two eviction operations (in millis) on the client pool.

30000STRINGYesNo
max.wait.time

The maximum time (in millis) the pool will wait (when there are no available connections) for a connection to be returned to the pool.

60000STRINGYesNo
test.on.borrow

Enable connections to be validated before being borrowed from the client pool.

trueBOOLYesNo
test.while.idle

Enable connections to be validated during the eviction operation (if any).

trueBOOLYesNo
exhausted.action

Action that should be taken when the maximum number of active connections are being used. This action should be indicated as an int and possible action values are following.
0 - Fail the request.
1 - Block the request, until a connection returns to the pool.
2 - Grow the connection pool size.

1 (Block when exhausted)INTYesNo
hostname.verification.enabled

Enable hostname verification

trueBOOLYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
clientBootstrapClientGroupSize

Number of client threads to perform non-blocking read and write to one or more channels.

(Number of available processors) * 2Any positive integer
clientBootstrapBossGroupSize

Number of boss threads to accept incoming connections.

Number of available processorsAny positive integer
clientBootstrapWorkerGroupSize

Number of worker threads to accept the connections from boss threads and perform non-blocking read and write from one or more channels.

(Number of available processors) * 2Any positive integer
trustStoreLocation

The default truststore file path.

`${carbon.home}/resources/security/client-truststore.jks`Path to client truststore `.jks` file
trustStorePassword

The default truststore password.

wso2carbonTruststore password as string
+ +Examples +EXAMPLE 1 +``` +@sink(type='http-call', sink.id='foo', + publisher.url='http://localhost:8009/foo', + @map(type='xml', @payload('{{payloadBody}}'))) +define stream FooStream (payloadBody string); + +@source(type='http-call-response', sink.id='foo', + @map(type='text', regex.A='((.|\n)*)', + @attributes(headers='trp:headers', message='A[1]'))) +define stream ResponseStream(message string, headers string); +``` +

+

When events arrive in FooStream, http-call sink makes calls to endpoint on url http://localhost:8009/foo with POST method and Content-Type application/xml.
If the event payloadBody attribute contains following XML:

<item>
+    <name>apple</name>
+    <price>55</price>
+    <quantity>5</quantity>
+</item>

the http-call sink maps that and sends it to the endpoint.
When endpoint sends a response it will be consumed by the corresponding http-call-response source correlated via the same sink.id foo and that will map the response message and send it via ResponseStream steam by assigning the message body as message attribute and response headers as headers attribute of the event.

+

+EXAMPLE 2 +``` +@sink(type='http-call', publisher.url='http://localhost:8005/files/{{name}}' + downloading.enabled='true', download.path='{{downloadPath}}{{name}}', + method='GET', sink.id='download', @map(type='json')) +define stream DownloadRequestStream(name String, id int, downloadPath string); + +@source(type='http-call-response', sink.id='download', + http.status.code='2\\d+', + @map(type='text', regex.A='((.|\n)*)', + @attributes(name='trp:name', id='trp:id', file='A[1]'))) +define stream ResponseStream2xx(name string, id string, file string); + +@source(type='http-call-response', sink.id='download', + http.status.code='4\\d+', + @map(type='text', regex.A='((.|\n)*)', @attributes(errorMsg='A[1]'))) +define stream ResponseStream4xx(errorMsg string); +``` +

+

When events arrive in DownloadRequestStream with name:foo.txt, id:75 and downloadPath:/user/download/ the http-call sink sends a GET request to the url http://localhost:8005/files/foo.txt to download the file to the given path /user/download/foo.txt and capture the response via its corresponding http-call-response source based on the response status code.
If the response status code is in the range of 200 the message will be received by the http-call-response source associated with the ResponseStream2xx stream which expects http.status.code with regex 2\\d+ while downloading the file to the local file system on the path /user/download/foo.txt and mapping the response message having the absolute file path to event's file attribute.
If the response status code is in the range of 400 then the message will be received by the http-call-response source associated with the ResponseStream4xx stream which expects http.status.code with regex 4\\d+ while mapping the error response to the errorMsg attribute of the event.

+

+### http-request *(Sink)* +

Deprecated

+

+

_(Use http-call sink instead)._
The http-request sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

+

+Syntax + +``` +@sink(type="http-request", publisher.url="", sink.id="", basic.auth.username="", basic.auth.password="", https.truststore.file="", https.truststore.password="", oauth.username="", oauth.password="", consumer.key="", consumer.secret="", token.url="", refresh.token="", headers="", method="", downloading.enabled="", download.path="", blocking.io="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
publisher.url

The URL which should be called.
Examples:
http://localhost:8080/endpoint,
https://localhost:8080/endpoint

STRINGNoNo
sink.id

Identifier to correlate the http-request sink to its corresponding http-response sources to retrieved the responses.

STRINGNoNo
basic.auth.username

The username to be included in the authentication header when calling endpoints protected by basic authentication. basic.auth.password property should be also set when using this property.

-STRINGYesNo
basic.auth.password

The password to be included in the authentication header when calling endpoints protected by basic authentication. basic.auth.username property should be also set when using this property.

-STRINGYesNo
https.truststore.file

The file path of the client truststore when sending messages through https protocol.

`${carbon.home}/resources/security/client-truststore.jks`STRINGYesNo
https.truststore.password

The password for the client-truststore.

wso2carbonSTRINGYesNo
oauth.username

The username to be included in the authentication header when calling endpoints protected by OAuth 2.0. oauth.password property should be also set when using this property.

-STRINGYesNo
oauth.password

The password to be included in the authentication header when calling endpoints protected by OAuth 2.0. oauth.username property should be also set when using this property.

-STRINGYesNo
consumer.key

Consumer key used for calling endpoints protected by OAuth 2.0

-STRINGYesNo
consumer.secret

Consumer secret used for calling endpoints protected by OAuth 2.0

-STRINGYesNo
token.url

Token URL to generate a new access tokens when calling endpoints protected by OAuth 2.0

-STRINGYesNo
refresh.token

Refresh token used for generating new access tokens when calling endpoints protected by OAuth 2.0

-STRINGYesNo
headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When the Content-Type header is not provided the system decides the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

Content-Type and Content-Length headersSTRINGYesNo
method

The HTTP method used for calling the endpoint.

POSTSTRINGYesNo
downloading.enabled

Enable response received by the http-response source to be written to a file. When this is enabled the download.path property should be also set.

falseBOOLYesNo
download.path

The absolute file path along with the file name where the downloads should be saved.

-STRINGYesYes
blocking.io

Blocks the request thread until a response it received from HTTP call-response source before sending any other request.

falseBOOLYesNo
socket.idle.timeout

Socket timeout in millis.

6000INTYesNo
chunk.disabled

Disable chunked transfer encoding.

falseBOOLYesNo
ssl.protocol

SSL/TLS protocol.

TLSSTRINGYesNo
ssl.verification.disabled

Disable SSL verification.

falseBOOLYesNo
ssl.configurations

SSL/TSL configurations in format "'<key>:<value>','<key>:<value>'".
Some supported parameters:
 - SSL/TLS protocols: 'sslEnabledProtocols:TLSv1.1,TLSv1.2'
 - List of ciphers: 'ciphers:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256'
 - Enable session creation: 'client.enable.session.creation:true'
 - Supported server names: 'server.suported.server.names:server'
 - Add HTTP SNIMatcher: 'server.supported.snimatchers:SNIMatcher'

-STRINGYesNo
proxy.host

Proxy server host

-STRINGYesNo
proxy.port

Proxy server port

-STRINGYesNo
proxy.username

Proxy server username

-STRINGYesNo
proxy.password

Proxy server password

-STRINGYesNo
client.bootstrap.configurations

Client bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Client connect timeout in millis: 'client.bootstrap.connect.timeout:15000'
 - Client socket timeout in seconds: 'client.bootstrap.socket.timeout:15'
 - Client socket reuse: 'client.bootstrap.socket.reuse:true'
 - Enable TCP no delay: 'client.bootstrap.nodelay:true'
 - Enable client keep alive: 'client.bootstrap.keepalive:true'
 - Send buffer size: 'client.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'client.bootstrap.recievebuffersize:1048576'

-STRINGYesNo
max.pool.active.connections

Maximum possible number of active connection per client pool.

-1INTYesNo
min.pool.idle.connections

Minimum number of idle connections that can exist per client pool.

0INTYesNo
max.pool.idle.connections

Maximum number of idle connections that can exist per client pool.

100INTYesNo
min.evictable.idle.time

Minimum time (in millis) a connection may sit idle in the client pool before it become eligible for eviction.

300000STRINGYesNo
time.between.eviction.runs

Time between two eviction operations (in millis) on the client pool.

30000STRINGYesNo
max.wait.time

The maximum time (in millis) the pool will wait (when there are no available connections) for a connection to be returned to the pool.

60000STRINGYesNo
test.on.borrow

Enable connections to be validated before being borrowed from the client pool.

trueBOOLYesNo
test.while.idle

Enable connections to be validated during the eviction operation (if any).

trueBOOLYesNo
exhausted.action

Action that should be taken when the maximum number of active connections are being used. This action should be indicated as an int and possible action values are following.
0 - Fail the request.
1 - Block the request, until a connection returns to the pool.
2 - Grow the connection pool size.

1 (Block when exhausted)INTYesNo
hostname.verification.enabled

Enable hostname verification

trueBOOLYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
clientBootstrapClientGroupSize

Number of client threads to perform non-blocking read and write to one or more channels.

(Number of available processors) * 2Any positive integer
clientBootstrapBossGroupSize

Number of boss threads to accept incoming connections.

Number of available processorsAny positive integer
clientBootstrapWorkerGroupSize

Number of worker threads to accept the connections from boss threads and perform non-blocking read and write from one or more channels.

(Number of available processors) * 2Any positive integer
trustStoreLocation

The default truststore file path.

`${carbon.home}/resources/security/client-truststore.jks`Path to client truststore `.jks` file
trustStorePassword

The default truststore password.

wso2carbonTruststore password as string
+ +Examples +EXAMPLE 1 +``` +@sink(type='http-request', sink.id='foo', + publisher.url='http://localhost:8009/foo', + @map(type='xml', @payload('{{payloadBody}}'))) +define stream FooStream (payloadBody string); + +@source(type='http-response', sink.id='foo', + @map(type='text', regex.A='((.|\n)*)', + @attributes(headers='trp:headers', message='A[1]'))) +define stream ResponseStream(message string, headers string); +``` +

+

When events arrive in FooStream, http-request sink makes calls to endpoint on url http://localhost:8009/foo with POST method and Content-Type application/xml.
If the event payloadBody attribute contains following XML:

<item>
+    <name>apple</name>
+    <price>55</price>
+    <quantity>5</quantity>
+</item>

the http-request sink maps that and sends it to the endpoint.
When endpoint sends a response it will be consumed by the corresponding http-response source correlated via the same sink.id foo and that will map the response message and send it via ResponseStream steam by assigning the message body as message attribute and response headers as headers attribute of the event.

+

+EXAMPLE 2 +``` +@sink(type='http-request', publisher.url='http://localhost:8005/files/{{name}}' + downloading.enabled='true', download.path='{{downloadPath}}{{name}}', + method='GET', sink.id='download', @map(type='json')) +define stream DownloadRequestStream(name String, id int, downloadPath string); + +@source(type='http-response', sink.id='download', + http.status.code='2\\d+', + @map(type='text', regex.A='((.|\n)*)', + @attributes(name='trp:name', id='trp:id', file='A[1]'))) +define stream ResponseStream2xx(name string, id string, file string); + +@source(type='http-response', sink.id='download', + http.status.code='4\\d+', + @map(type='text', regex.A='((.|\n)*)', @attributes(errorMsg='A[1]'))) +define stream ResponseStream4xx(errorMsg string); +``` +

+

When events arrive in DownloadRequestStream with name:foo.txt, id:75 and downloadPath:/user/download/ the http-request sink sends a GET request to the url http://localhost:8005/files/foo.txt to download the file to the given path /user/download/foo.txt and capture the response via its corresponding http-response source based on the response status code.
If the response status code is in the range of 200 the message will be received by the http-response source associated with the ResponseStream2xx stream which expects http.status.code with regex 2\\d+ while downloading the file to the local file system on the path /user/download/foo.txt and mapping the response message having the absolute file path to event's file attribute.
If the response status code is in the range of 400 then the message will be received by the http-response source associated with the ResponseStream4xx stream which expects http.status.code with regex 4\\d+ while mapping the error response to the errorMsg attribute of the event.

+

+### http-response *(Sink)* +

Deprecated

+

+

_(Use http-service-response sink instead)._
The http-response sink send responses of the requests consumed by its corresponding http-request source, by mapping the response messages to formats such as text, XML and JSON.

+

+Syntax + +``` +@sink(type="http-response", source.id="", message.id="", headers="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
source.id

Identifier to correlate the http-response sink to its corresponding http-request source which consumed the request.

STRINGNoNo
message.id

Identifier to correlate the response with the request received by http-request source.

STRINGNoYes
headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When the Content-Type header is not provided the system decides the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

Content-Type and Content-Length headersSTRINGYesNo
+ +Examples +EXAMPLE 1 +``` +@source(type='http-request', receiver.url='http://localhost:5005/add', + source.id='adder', + @map(type='json, @attributes(messageId='trp:messageId', + value1='$.event.value1', + value2='$.event.value2'))) +define stream AddStream (messageId string, value1 long, value2 long); + +@sink(type='http-response', source.id='adder', + message.id='{{messageId}}', @map(type = 'json')) +define stream ResultStream (messageId string, results long); + +@info(name = 'query1') +from AddStream +select messageId, value1 + value2 as results +insert into ResultStream; +``` +

+

The http-request source on stream AddStream listens on url http://localhost:5005/stocks for JSON messages with format:

{
+  "event": {
+    "value1": 3,
+    "value2": 4
+  }
+}


and when events arrive it maps to AddStream events and pass them to query query1 for processing. The query results produced on ResultStream are sent as a response via http-response sink with format:

{
+  "event": {
+    "results": 7
+  }
+}

Here the request and response are correlated by passing the messageId produced by the http-request to the respective http-response sink.

+

+### http-service-response *(Sink)* +

+

The http-service-response sink send responses of the requests consumed by its corresponding http-service source, by mapping the response messages to formats such as text, XML and JSON.

+

+Syntax + +``` +@sink(type="http-service-response", source.id="", message.id="", headers="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
source.id

Identifier to correlate the http-service-response sink to its corresponding http-service source which consumed the request.

STRINGNoNo
message.id

Identifier to correlate the response with the request received by http-service source.

STRINGNoYes
headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When the Content-Type header is not provided the system decides the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

Content-Type and Content-Length headersSTRINGYesNo
+ +Examples +EXAMPLE 1 +``` +@source(type='http-service', receiver.url='http://localhost:5005/add', + source.id='adder', + @map(type='json, @attributes(messageId='trp:messageId', + value1='$.event.value1', + value2='$.event.value2'))) +define stream AddStream (messageId string, value1 long, value2 long); + +@sink(type='http-service-response', source.id='adder', + message.id='{{messageId}}', @map(type = 'json')) +define stream ResultStream (messageId string, results long); + +@info(name = 'query1') +from AddStream +select messageId, value1 + value2 as results +insert into ResultStream; +``` +

+

The http-service source on stream AddStream listens on url http://localhost:5005/stocks for JSON messages with format:

{
+  "event": {
+    "value1": 3,
+    "value2": 4
+  }
+}


and when events arrive it maps to AddStream events and pass them to query query1 for processing. The query results produced on ResultStream are sent as a response via http-service-response sink with format:

{
+  "event": {
+    "results": 7
+  }
+}

Here the request and response are correlated by passing the messageId produced by the http-service to the respective http-service-response sink.

+

+### sse-server *(Sink)* +

+

HTTP SSE sink sends events to all subscribers.

+

+Syntax + +``` +@sink(type="sse-server", server.url="", worker.count="", headers="", https.truststore.file="", https.truststore.password="", client.bootstrap.configurations="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
server.url

The listening URL of the SSE server which clients need to connect to receive events. If not provided url will be constructed using siddhi app name and stream name as the context by default with port 8280. eg :- http://0.0.0.0:8280/{app_name}/{stream_name}

STRINGNoNo
worker.count

The number of active worker threads to serve the incoming events. By default the value is set to 1 to ensure events are processed in the same order they arrived. By increasing this value, higher performance can be achieved in the expense of loosing event ordering.

1INTYesNo
headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When the Content-Type header is not provided the system decides the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

Content-Type and Content-Length headersSTRINGYesNo
https.truststore.file

The file path of the client truststore when sending messages through https protocol.

`${carbon.home}/resources/security/client-truststore.jks`STRINGYesNo
https.truststore.password

The password for the client-truststore.

wso2carbonSTRINGYesNo
client.bootstrap.configurations

Client bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Client connect timeout in millis: 'client.bootstrap.connect.timeout:15000'
 - Client socket timeout in seconds: 'client.bootstrap.socket.timeout:15'
 - Client socket reuse: 'client.bootstrap.socket.reuse:true'
 - Enable TCP no delay: 'client.bootstrap.nodelay:true'
 - Enable client keep alive: 'client.bootstrap.keepalive:true'
 - Send buffer size: 'client.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'client.bootstrap.recievebuffersize:1048576'

-STRINGYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
defaultScheme

The default protocol.

httphttp
https
defaultHttpPort

The default HTTP port when default scheme is http.

8280Any valid port
defaultHost

The default host of the transport.

0.0.0.0Any valid host
+ +Examples +EXAMPLE 1 +``` +@Source(type='sse-server', server.url='http://localhost:8080/sse', @map(type='json')) define stream PublishingStream (param1 string); +``` +

+

External clients can listen to the server.url

+

+### websubhub *(Sink)* +

+

WebSubHubEventPublisher publishes messages via HTTP/HTTP according to the provided URL when subscribe to the WebSub hub. The table.name, hub.id are mandatory when defining the websubhub source

+

+Syntax + +``` +@sink(type="websubhub", hub.id="", table.name="", https.truststore.file="", https.truststore.password="", consumer.key="", consumer.secret="", token.url="", refresh.token="", headers="", method="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", tls.store.type="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", executor.service.threads="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
hub.id

Id of the hub that the messages needed to process

STRINGNoNo
table.name

Name of the table which subscription data holds related to the hub

STRINGNoNo
https.truststore.file

The file path of the client truststore when sending messages through https protocol.

`${carbon.home}/resources/security/client-truststore.jks`STRINGYesNo
https.truststore.password

The password for the client-truststore.

wso2carbonSTRINGYesNo
consumer.key

Consumer key used for calling endpoints protected by OAuth 2.0

-STRINGYesNo
consumer.secret

Consumer secret used for calling endpoints protected by OAuth 2.0

-STRINGYesNo
token.url

Token URL to generate a new access tokens when calling endpoints protected by OAuth 2.0

-STRINGYesNo
refresh.token

Refresh token used for generating new access tokens when calling endpoints protected by OAuth 2.0

-STRINGYesNo
headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When Content-Type header is not provided the system derives the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

Content-Type and Content-Length headersSTRINGYesNo
method

The HTTP method used for calling the endpoint.

POSTSTRINGYesNo
socket.idle.timeout

Socket timeout in millis.

6000INTYesNo
chunk.disabled

Disable chunked transfer encoding.

falseBOOLYesNo
ssl.protocol

SSL/TLS protocol.

TLSSTRINGYesNo
ssl.verification.disabled

Disable SSL verification.

falseBOOLYesNo
tls.store.type

TLS store type.

JKSSTRINGYesNo
ssl.configurations

SSL/TSL configurations in format "'<key>:<value>','<key>:<value>'".
Some supported parameters:
 - SSL/TLS protocols: 'sslEnabledProtocols:TLSv1.1,TLSv1.2'
 - List of ciphers: 'ciphers:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256'
 - Enable session creation: 'client.enable.session.creation:true'
 - Supported server names: 'server.suported.server.names:server'
 - Add HTTP SNIMatcher: 'server.supported.snimatchers:SNIMatcher'

-STRINGYesNo
proxy.host

Proxy server host

-STRINGYesNo
proxy.port

Proxy server port

-STRINGYesNo
proxy.username

Proxy server username

-STRINGYesNo
proxy.password

Proxy server password

-STRINGYesNo
client.bootstrap.configurations

Client bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Client connect timeout in millis: 'client.bootstrap.connect.timeout:15000'
 - Client socket timeout in seconds: 'client.bootstrap.socket.timeout:15'
 - Client socket reuse: 'client.bootstrap.socket.reuse:true'
 - Enable TCP no delay: 'client.bootstrap.nodelay:true'
 - Enable client keep alive: 'client.bootstrap.keepalive:true'
 - Send buffer size: 'client.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'client.bootstrap.recievebuffersize:1048576'

-STRINGYesNo
max.pool.active.connections

Maximum possible number of active connection per client pool.

-1INTYesNo
min.pool.idle.connections

Minimum number of idle connections that can exist per client pool.

0INTYesNo
max.pool.idle.connections

Maximum number of idle connections that can exist per client pool.

100INTYesNo
executor.service.threads

Thread count for the executor service.

20INTYesNo
min.evictable.idle.time

Minimum time (in millis) a connection may sit idle in the client pool before it become eligible for eviction.

300000STRINGYesNo
time.between.eviction.runs

Time between two eviction operations (in millis) on the client pool.

30000STRINGYesNo
max.wait.time

The maximum time (in millis) the pool will wait (when there are no available connections) for a connection to be returned to the pool.

60000STRINGYesNo
test.on.borrow

Enable connections to be validated before being borrowed from the client pool.

trueBOOLYesNo
test.while.idle

Enable connections to be validated during the eviction operation (if any).

trueBOOLYesNo
exhausted.action

Action that should be taken when the maximum number of active connections are being used. This action should be indicated as an int and possible action values are following.
0 - Fail the request.
1 - Block the request, until a connection returns to the pool.
2 - Grow the connection pool size.

1 (Block when exhausted)INTYesNo
hostname.verification.enabled

Enable hostname verification.

trueBOOLYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
clientBootstrapClientGroupSize

Number of client threads to perform non-blocking read and write to one or more channels.

(Number of available processors) * 2Any positive integer
clientBootstrapBossGroupSize

Number of boss threads to accept incoming connections.

Number of available processorsAny positive integer
clientBootstrapWorkerGroupSize

Number of worker threads to accept the connections from boss threads and perform non-blocking read and write from one or more channels.

(Number of available processors) * 2Any positive integer
trustStoreLocation

The default truststore file path.

`${carbon.home}/resources/security/client-truststore.jks`Path to client truststore `.jks` file
trustStorePassword

The default truststore password.

wso2carbonTruststore password as string
+ +Examples +EXAMPLE 1 +``` +@store(type='rdbms' , jdbc.url='jdbc:mysql://localhost:3306/production?useSSL=false', username='root', password='root', jdbc.driver.name='com.mysql.jdbc.Driver') +@sink(type='websubhubeventpublisher', hub.id="anu_123" , table.name='SessionTable',publisher.url="mysql://localhost:3306/production?useSSL=false", +@map(type='keyvalue',implicit.cast.enable='true')) +define stream LowProductionAlertStream (topic string, payload string); +``` +

+

Subscribed users will received the messages generated through the hub and will publish to the callback url when subscribe.

+

+## Source + +### http *(Source)* +

+

HTTP source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON. It also supports basic authentication to ensure events are received from authorized users/systems.
The request headers and properties can be accessed via transport properties in the format trp:<header>.

+

+Syntax + +``` +@source(type="http", receiver.url="", basic.auth.enabled="", worker.count="", socket.idle.timeout="", ssl.verify.client="", ssl.protocol="", tls.store.type="", ssl.configurations="", request.size.validation.configurations="", header.validation.configurations="", server.bootstrap.configurations="", trace.log.enabled="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
receiver.url

The URL on which events should be received. To enable SSL use https protocol in the url.

`http://0.0.0.0:9763//`STRINGYesNo
basic.auth.enabled

This only works in VM, Docker and Kubernetes.
Where when enabled it authenticates each request using the Authorization:'Basic encodeBase64(username:Password)' header.

falseSTRINGYesNo
worker.count

The number of active worker threads to serve the incoming events. By default the value is set to 1 to ensure events are processed in the same order they arrived. By increasing this value, higher performance can be achieved in the expense of loosing event ordering.

1INTYesNo
socket.idle.timeout

Idle timeout for HTTP connection in millis.

120000INTYesNo
ssl.verify.client

The type of client certificate verification. Supported values are require, optional.

-STRINGYesNo
ssl.protocol

SSL/TLS protocol.

TLSSTRINGYesNo
tls.store.type

TLS store type.

JKSSTRINGYesNo
ssl.configurations

SSL/TSL configurations in format "'<key>:<value>','<key>:<value>'".
Some supported parameters:
 - SSL/TLS protocols: 'sslEnabledProtocols:TLSv1.1,TLSv1.2'
 - List of ciphers: 'ciphers:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256'
 - Enable session creation: 'client.enable.session.creation:true'
 - Supported server names: 'server.suported.server.names:server'
 - Add HTTP SNIMatcher: 'server.supported.snimatchers:SNIMatcher'

-STRINGYesNo
request.size.validation.configurations

Configurations to validate the HTTP request size.
Expected format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Enable request size validation: 'request.size.validation:true'
 If request size is validated
 - Maximum request size: 'request.size.validation.maximum.value:2048'
 - Response status code when request size validation fails: 'request.size.validation.reject.status.code:401'
 - Response message when request size validation fails: 'request.size.validation.reject.message:Message is bigger than the valid size'
 - Response Content-Type when request size validation fails: 'request.size.validation.reject.message.content.type:plain/text'

-STRINGYesNo
header.validation.configurations

Configurations to validate HTTP headers.
Expected format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Enable header size validation: 'header.size.validation:true'
 If header size is validated
 - Maximum length of initial line: 'header.validation.maximum.request.line:4096'
 - Maximum length of all headers: 'header.validation.maximum.size:8192'
 - Maximum length of the content or each chunk: 'header.validation.maximum.chunk.size:8192'
 - Response status code when header validation fails: 'header.validation.reject.status.code:401'
 - Response message when header validation fails: 'header.validation.reject.message:Message header is bigger than the valid size'
 - Response Content-Type when header validation fails: 'header.validation.reject.message.content.type:plain/text'

-STRINGYesNo
server.bootstrap.configurations

Server bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Server connect timeout in millis: 'server.bootstrap.connect.timeout:15000'
 - Server socket timeout in seconds: 'server.bootstrap.socket.timeout:15'
 - Enable TCP no delay: 'server.bootstrap.nodelay:true'
 - Enable server keep alive: 'server.bootstrap.keepalive:true'
 - Send buffer size: 'server.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'server.bootstrap.recievebuffersize:1048576'
 - Number of connections queued: 'server.bootstrap.socket.backlog:100'

-STRINGYesNo
trace.log.enabled

Enable trace log for traffic monitoring.

falseBOOLYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
serverBootstrapBossGroupSize

Number of boss threads to accept incoming connections.

Number of available processorsAny positive integer
serverBootstrapWorkerGroupSize

Number of worker threads to accept the connections from boss threads and perform non-blocking read and write from one or more channels.

(Number of available processors) * 2Any positive integer
serverBootstrapClientGroupSize

Number of client threads to perform non-blocking read and write to one or more channels.

(Number of available processors) * 2Any positive integer
defaultHost

The default host of the transport.

0.0.0.0Any valid host
defaultScheme

The default protocol.

httphttp
https
defaultHttpPort

The default HTTP port when default scheme is http.

8280Any valid port
defaultHttpsPort

The default HTTPS port when default scheme is https.

8243Any valid port
keyStoreLocation

The default keystore file path.

`${carbon.home}/resources/security/wso2carbon.jks`Path to `.jks` file
keyStorePassword

The default keystore password.

wso2carbonKeystore password as string
+ +Examples +EXAMPLE 1 +``` +@app.name('StockProcessor') + +@source(type='http', @map(type = 'json')) +define stream StockStream (symbol string, price float, volume long); + +``` +

+

Above HTTP source listeners on url http://0.0.0.0:9763/StockProcessor/StockStream for JSON messages on the format:

{
+  "event": {
+    "symbol": "FB",
+    "price": 24.5,
+    "volume": 5000
+  }
+}

It maps the incoming messages and sends them to StockStream for processing.

+

+EXAMPLE 2 +``` +@source(type='http', receiver.url='http://localhost:5005/stocks', + @map(type = 'xml')) +define stream StockStream (symbol string, price float, volume long); + +``` +

+

Above HTTP source listeners on url http://localhost:5005/stocks for JSON messages on the format:

<events>
+    <event>
+        <symbol>Fb</symbol>
+        <price>55.6</price>
+        <volume>100</volume>
+    </event>
+</events>


It maps the incoming messages and sends them to StockStream for processing.

+

+### http-call-response *(Source)* +

+

The http-call-response source receives the responses for the calls made by its corresponding http-call sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-call-response sources are allowed to associate with a single http-call sink.
It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

+

+Syntax + +``` +@source(type="http-call-response", sink.id="", http.status.code="", allow.streaming.responses="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
sink.id

Identifier to correlate the http-call-response source with its corresponding http-call sink that published the messages.

STRINGNoNo
http.status.code

The matching http responses status code regex, that is used to filter the the messages which will be processed by the source.Eg: http.status.code = '200',
http.status.code = '4\\d+'

200STRINGYesNo
allow.streaming.responses

Enable consuming responses on a streaming manner.

falseBOOLYesNo
+ +Examples +EXAMPLE 1 +``` +@sink(type='http-call', method='POST', + publisher.url='http://localhost:8005/registry/employee', + sink.id='employee-info', @map(type='json')) +define stream EmployeeRequestStream (name string, id int); + +@source(type='http-call-response', sink.id='employee-info', + http.status.code='2\\d+', + @map(type='json', + @attributes(name='trp:name', id='trp:id', + location='$.town', age='$.age'))) +define stream EmployeeResponseStream(name string, id int, + location string, age int); + +@source(type='http-call-response', sink.id='employee-info', + http.status.code='4\\d+', + @map(type='text', regex.A='((.|\n)*)', + @attributes(error='A[1]'))) +define stream EmployeeErrorStream(error string); +``` +

+

When events arrive in EmployeeRequestStream, http-call sink makes calls to endpoint on url http://localhost:8005/registry/employee with POST method and Content-Type application/json.
If the arriving event has attributes name:John and id:1423 it will send a message with default JSON mapping as follows:

{
+  "event": {
+    "name": "John",
+    "id": 1423
+  }
+}

When the endpoint responds with status code in the range of 200 the message will be received by the http-call-response source associated with the EmployeeResponseStream stream, because it is correlated with the sink by the same sink.id employee-info and as that expects messages with http.status.code in regex format 2\\d+. If the response message is in the format

{
+  "town": "NY",
+  "age": 24
+}

the source maps the location and age attributes by executing JSON path on the message and maps the name and id attributes by extracting them from the request event via as transport properties.
If the response status code is in the range of 400 then the message will be received by the http-call-response source associated with the EmployeeErrorStream stream, because it is correlated with the sink by the same sink.id employee-info and it expects messages with http.status.code in regex format 4\\d+, and maps the error response to the error attribute of the event.

+

+### http-request *(Source)* +

Deprecated

+

+

_(Use http-service source instead)._
The http-request source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

+

+Syntax + +``` +@source(type="http-request", receiver.url="", source.id="", connection.timeout="", basic.auth.enabled="", worker.count="", socket.idle.timeout="", ssl.verify.client="", ssl.protocol="", tls.store.type="", ssl.configurations="", request.size.validation.configurations="", header.validation.configurations="", server.bootstrap.configurations="", trace.log.enabled="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
receiver.url

The URL on which events should be received. To enable SSL use https protocol in the url.

`http://0.0.0.0:9763//`STRINGYesNo
source.id

Identifier to correlate the http-request source to its corresponding http-response sinks to send responses.

STRINGNoNo
connection.timeout

Connection timeout in millis. The system will send a timeout, if a corresponding response is not sent by an associated http-response sink within the given time.

120000INTYesNo
basic.auth.enabled

This only works in VM, Docker and Kubernetes.
Where when enabled it authenticates each request using the Authorization:'Basic encodeBase64(username:Password)' header.

falseSTRINGYesNo
worker.count

The number of active worker threads to serve the incoming events. By default the value is set to 1 to ensure events are processed in the same order they arrived. By increasing this value, higher performance can be achieved in the expense of loosing event ordering.

1INTYesNo
socket.idle.timeout

Idle timeout for HTTP connection in millis.

120000INTYesNo
ssl.verify.client

The type of client certificate verification. Supported values are require, optional.

-STRINGYesNo
ssl.protocol

SSL/TLS protocol.

TLSSTRINGYesNo
tls.store.type

TLS store type.

JKSSTRINGYesNo
ssl.configurations

SSL/TSL configurations in format "'<key>:<value>','<key>:<value>'".
Some supported parameters:
 - SSL/TLS protocols: 'sslEnabledProtocols:TLSv1.1,TLSv1.2'
 - List of ciphers: 'ciphers:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256'
 - Enable session creation: 'client.enable.session.creation:true'
 - Supported server names: 'server.suported.server.names:server'
 - Add HTTP SNIMatcher: 'server.supported.snimatchers:SNIMatcher'

-STRINGYesNo
request.size.validation.configurations

Configurations to validate the HTTP request size.
Expected format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Enable request size validation: 'request.size.validation:true'
 If request size is validated
 - Maximum request size: 'request.size.validation.maximum.value:2048'
 - Response status code when request size validation fails: 'request.size.validation.reject.status.code:401'
 - Response message when request size validation fails: 'request.size.validation.reject.message:Message is bigger than the valid size'
 - Response Content-Type when request size validation fails: 'request.size.validation.reject.message.content.type:plain/text'

-STRINGYesNo
header.validation.configurations

Configurations to validate HTTP headers.
Expected format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Enable header size validation: 'header.size.validation:true'
 If header size is validated
 - Maximum length of initial line: 'header.validation.maximum.request.line:4096'
 - Maximum length of all headers: 'header.validation.maximum.size:8192'
 - Maximum length of the content or each chunk: 'header.validation.maximum.chunk.size:8192'
 - Response status code when header validation fails: 'header.validation.reject.status.code:401'
 - Response message when header validation fails: 'header.validation.reject.message:Message header is bigger than the valid size'
 - Response Content-Type when header validation fails: 'header.validation.reject.message.content.type:plain/text'

-STRINGYesNo
server.bootstrap.configurations

Server bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Server connect timeout in millis: 'server.bootstrap.connect.timeout:15000'
 - Server socket timeout in seconds: 'server.bootstrap.socket.timeout:15'
 - Enable TCP no delay: 'server.bootstrap.nodelay:true'
 - Enable server keep alive: 'server.bootstrap.keepalive:true'
 - Send buffer size: 'server.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'server.bootstrap.recievebuffersize:1048576'
 - Number of connections queued: 'server.bootstrap.socket.backlog:100'

-STRINGYesNo
trace.log.enabled

Enable trace log for traffic monitoring.

falseBOOLYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
serverBootstrapBossGroupSize

Number of boss threads to accept incoming connections.

Number of available processorsAny positive integer
serverBootstrapWorkerGroupSize

Number of worker threads to accept the connections from boss threads and perform non-blocking read and write from one or more channels.

(Number of available processors) * 2Any positive integer
serverBootstrapClientGroupSize

Number of client threads to perform non-blocking read and write to one or more channels.

(Number of available processors) * 2Any positive integer
defaultHost

The default host of the transport.

0.0.0.0Any valid host
defaultScheme

The default protocol.

httphttp
https
defaultHttpPort

The default HTTP port when default scheme is http.

8280Any valid port
defaultHttpsPort

The default HTTPS port when default scheme is https.

8243Any valid port
keyStoreLocation

The default keystore file path.

`${carbon.home}/resources/security/wso2carbon.jks`Path to `.jks` file
keyStorePassword

The default keystore password.

wso2carbonKeystore password as string
+ +Examples +EXAMPLE 1 +``` +@source(type='http-request', receiver.url='http://localhost:5005/add', + source.id='adder', + @map(type='json, @attributes(messageId='trp:messageId', + value1='$.event.value1', + value2='$.event.value2'))) +define stream AddStream (messageId string, value1 long, value2 long); + +@sink(type='http-response', source.id='adder', + message.id='{{messageId}}', @map(type = 'json')) +define stream ResultStream (messageId string, results long); + +@info(name = 'query1') +from AddStream +select messageId, value1 + value2 as results +insert into ResultStream; +``` +

+

Above sample listens events on http://localhost:5005/stocks url for JSON messages on the format:

{
+  "event": {
+    "value1": 3,
+    "value2": 4
+  }
+}


Map the vents into AddStream, process the events through query query1, and sends the results produced on ResultStream via http-response sink on the message format:

{
+  "event": {
+    "results": 7
+  }
+}

+

+### http-response *(Source)* +

Deprecated

+

+

_(Use http-call-response source instead)._
The http-response source receives the responses for the calls made by its corresponding http-request sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-response sources are allowed to associate with a single http-request sink. It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

+

+Syntax + +``` +@source(type="http-response", sink.id="", http.status.code="", allow.streaming.responses="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
sink.id

Identifier to correlate the http-response source with its corresponding http-request sink that published the messages.

STRINGNoNo
http.status.code

The matching http responses status code regex, that is used to filter the the messages which will be processed by the source.Eg: http.status.code = '200',
http.status.code = '4\\d+'

200STRINGYesNo
allow.streaming.responses

Enable consuming responses on a streaming manner.

falseBOOLYesNo
+ +Examples +EXAMPLE 1 +``` +@sink(type='http-request', method='POST', + publisher.url='http://localhost:8005/registry/employee', + sink.id='employee-info', @map(type='json')) +define stream EmployeeRequestStream (name string, id int); + +@source(type='http-response', sink.id='employee-info', + http.status.code='2\\d+', + @map(type='json', + @attributes(name='trp:name', id='trp:id', + location='$.town', age='$.age'))) +define stream EmployeeResponseStream(name string, id int, + location string, age int); + +@source(type='http-response', sink.id='employee-info', + http.status.code='4\\d+', + @map(type='text', regex.A='((.|\n)*)', + @attributes(error='A[1]'))) +define stream EmployeeErrorStream(error string); +``` +

+

When events arrive in EmployeeRequestStream, http-request sink makes calls to endpoint on url http://localhost:8005/registry/employee with POST method and Content-Type application/json.
If the arriving event has attributes name:John and id:1423 it will send a message with default JSON mapping as follows:

{
+  "event": {
+    "name": "John",
+    "id": 1423
+  }
+}

When the endpoint responds with status code in the range of 200 the message will be received by the http-response source associated with the EmployeeResponseStream stream, because it is correlated with the sink by the same sink.id employee-info and as that expects messages with http.status.code in regex format 2\\d+. If the response message is in the format

{
+  "town": "NY",
+  "age": 24
+}

the source maps the location and age attributes by executing JSON path on the message and maps the name and id attributes by extracting them from the request event via as transport properties.
If the response status code is in the range of 400 then the message will be received by the http-response source associated with the EmployeeErrorStream stream, because it is correlated with the sink by the same sink.id employee-info and it expects messages with http.status.code in regex format 4\\d+, and maps the error response to the error attribute of the event.

+

+### http-service *(Source)* +

+

The http-service source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-service-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-service-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

+

+Syntax + +``` +@source(type="http-service", receiver.url="", source.id="", connection.timeout="", basic.auth.enabled="", worker.count="", socket.idle.timeout="", ssl.verify.client="", ssl.protocol="", tls.store.type="", ssl.configurations="", request.size.validation.configurations="", header.validation.configurations="", server.bootstrap.configurations="", trace.log.enabled="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
receiver.url

The URL on which events should be received. To enable SSL use https protocol in the url.

`http://0.0.0.0:9763//`STRINGYesNo
source.id

Identifier to correlate the http-service source to its corresponding http-service-response sinks to send responses.

STRINGNoNo
connection.timeout

Connection timeout in millis. The system will send a timeout, if a corresponding response is not sent by an associated http-service-response sink within the given time.

120000INTYesNo
basic.auth.enabled

This only works in VM, Docker and Kubernetes.
Where when enabled it authenticates each request using the Authorization:'Basic encodeBase64(username:Password)' header.

falseSTRINGYesNo
worker.count

The number of active worker threads to serve the incoming events. By default the value is set to 1 to ensure events are processed in the same order they arrived. By increasing this value, higher performance can be achieved in the expense of loosing event ordering.

1INTYesNo
socket.idle.timeout

Idle timeout for HTTP connection in millis.

120000INTYesNo
ssl.verify.client

The type of client certificate verification. Supported values are require, optional.

-STRINGYesNo
ssl.protocol

SSL/TLS protocol.

TLSSTRINGYesNo
tls.store.type

TLS store type.

JKSSTRINGYesNo
ssl.configurations

SSL/TSL configurations in format "'<key>:<value>','<key>:<value>'".
Some supported parameters:
 - SSL/TLS protocols: 'sslEnabledProtocols:TLSv1.1,TLSv1.2'
 - List of ciphers: 'ciphers:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256'
 - Enable session creation: 'client.enable.session.creation:true'
 - Supported server names: 'server.suported.server.names:server'
 - Add HTTP SNIMatcher: 'server.supported.snimatchers:SNIMatcher'

-STRINGYesNo
request.size.validation.configurations

Configurations to validate the HTTP request size.
Expected format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Enable request size validation: 'request.size.validation:true'
 If request size is validated
 - Maximum request size: 'request.size.validation.maximum.value:2048'
 - Response status code when request size validation fails: 'request.size.validation.reject.status.code:401'
 - Response message when request size validation fails: 'request.size.validation.reject.message:Message is bigger than the valid size'
 - Response Content-Type when request size validation fails: 'request.size.validation.reject.message.content.type:plain/text'

-STRINGYesNo
header.validation.configurations

Configurations to validate HTTP headers.
Expected format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Enable header size validation: 'header.size.validation:true'
 If header size is validated
 - Maximum length of initial line: 'header.validation.maximum.request.line:4096'
 - Maximum length of all headers: 'header.validation.maximum.size:8192'
 - Maximum length of the content or each chunk: 'header.validation.maximum.chunk.size:8192'
 - Response status code when header validation fails: 'header.validation.reject.status.code:401'
 - Response message when header validation fails: 'header.validation.reject.message:Message header is bigger than the valid size'
 - Response Content-Type when header validation fails: 'header.validation.reject.message.content.type:plain/text'

-STRINGYesNo
server.bootstrap.configurations

Server bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Server connect timeout in millis: 'server.bootstrap.connect.timeout:15000'
 - Server socket timeout in seconds: 'server.bootstrap.socket.timeout:15'
 - Enable TCP no delay: 'server.bootstrap.nodelay:true'
 - Enable server keep alive: 'server.bootstrap.keepalive:true'
 - Send buffer size: 'server.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'server.bootstrap.recievebuffersize:1048576'
 - Number of connections queued: 'server.bootstrap.socket.backlog:100'

-STRINGYesNo
trace.log.enabled

Enable trace log for traffic monitoring.

falseBOOLYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
serverBootstrapBossGroupSize

Number of boss threads to accept incoming connections.

Number of available processorsAny positive integer
serverBootstrapWorkerGroupSize

Number of worker threads to accept the connections from boss threads and perform non-blocking read and write from one or more channels.

(Number of available processors) * 2Any positive integer
serverBootstrapClientGroupSize

Number of client threads to perform non-blocking read and write to one or more channels.

(Number of available processors) * 2Any positive integer
defaultHost

The default host of the transport.

0.0.0.0Any valid host
defaultScheme

The default protocol.

httphttp
https
defaultHttpPort

The default HTTP port when default scheme is http.

8280Any valid port
defaultHttpsPort

The default HTTPS port when default scheme is https.

8243Any valid port
keyStoreLocation

The default keystore file path.

`${carbon.home}/resources/security/wso2carbon.jks`Path to `.jks` file
keyStorePassword

The default keystore password.

wso2carbonKeystore password as string
+ +Examples +EXAMPLE 1 +``` +@source(type='http-service', receiver.url='http://localhost:5005/add', + source.id='adder', + @map(type='json, @attributes(messageId='trp:messageId', + value1='$.event.value1', + value2='$.event.value2'))) +define stream AddStream (messageId string, value1 long, value2 long); + +@sink(type='http-service-response', source.id='adder', + message.id='{{messageId}}', @map(type = 'json')) +define stream ResultStream (messageId string, results long); + +@info(name = 'query1') +from AddStream +select messageId, value1 + value2 as results +insert into ResultStream; +``` +

+

Above sample listens events on http://localhost:5005/stocks url for JSON messages on the format:

{
+  "event": {
+    "value1": 3,
+    "value2": 4
+  }
+}


Map the vents into AddStream, process the events through query query1, and sends the results produced on ResultStream via http-service-response sink on the message format:

{
+  "event": {
+    "results": 7
+  }
+}

+

+### sse *(Source)* +

+

HTTP SSE source send a request to a given url and listen to the response stream.

+

+Syntax + +``` +@source(type="sse", receiver.url="", basic.auth.username="", basic.auth.password="", worker.count="", headers="", https.truststore.file="", https.truststore.password="", client.bootstrap.configurations="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
receiver.url

The sse endpoint url which should be listened.

STRINGNoNo
basic.auth.username

The username to be included in the authentication header when calling endpoints protected by basic authentication. basic.auth.password property should be also set when using this property.

-STRINGYesNo
basic.auth.password

The password to be included in the authentication header when calling endpoints protected by basic authentication. basic.auth.username property should be also set when using this property.

-STRINGYesNo
worker.count

The number of active worker threads to serve the incoming events. By default the value is set to 1 to ensure events are processed in the same order they arrived. By increasing this value, higher performance can be achieved in the expense of loosing event ordering.

1INTYesNo
headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When the Content-Type header is not provided the system decides the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

Content-Type and Content-Length headersSTRINGYesNo
https.truststore.file

The file path of the client truststore when sending messages through https protocol.

`${carbon.home}/resources/security/client-truststore.jks`STRINGYesNo
https.truststore.password

The password for the client-truststore.

wso2carbonSTRINGYesNo
client.bootstrap.configurations

Client bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Client connect timeout in millis: 'client.bootstrap.connect.timeout:15000'
 - Client socket timeout in seconds: 'client.bootstrap.socket.timeout:15'
 - Client socket reuse: 'client.bootstrap.socket.reuse:true'
 - Enable TCP no delay: 'client.bootstrap.nodelay:true'
 - Enable client keep alive: 'client.bootstrap.keepalive:true'
 - Send buffer size: 'client.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'client.bootstrap.recievebuffersize:1048576'

-STRINGYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
defaultScheme

The default protocol.

httphttp
https
defaultHttpPort

The default HTTP port when default scheme is http.

8280Any valid port
defaultHost

The default host of the transport.

0.0.0.0Any valid host
+ +Examples +EXAMPLE 1 +``` +@Source(type='sse', receiver.url='http://localhost:8080/sse', @map(type='json')) define stream IncomingStream (param1 string); +``` +

+

This subscribes to the events which gets published by the SSE server at receiver.url

+

+### websubhub *(Source)* +

+

WebSub Hub source receive subscription requests via Http and according to the request, the subscription details will be saved to the given table and against the callback and topic name. The subscription request **MUST** have a Content-Type header of **application/x-www-form-urlencoded** and following **MUST** provide as parameter body.
      hub.callback      - REQUIRED.     The subscriber's callback URL where content distribution notifications should be delivered. The callback URL SHOULD be an unguessable URL that is unique per subscription.
     hub.mode      - REQUIRED.     The literal string "subscribe" or "unsubscribe", depending on the goal of the request.
     hub.topic      - REQUIRED.     The topic URL that the subscriber wishes to subscribe to or unsubscribe from.
      hub.lease_seconds     - OPTIONAL.     Number of seconds for which the subscriber would like to have the subscription active, given as a positive decimal integer.
     hub.secret     - OPTIONAL.     A subscriber-provided cryptographically random unique secret string that will be used to compute an HMAC digest for authorized content distribution. If not supplied, the HMAC digest will not be present for content distribution requests.

+

+Syntax + +``` +@source(type="websubhub", hub.id="", table.name="", receiver.url="", topic.list="", basic.auth.enabled="", worker.count="", socket.idle.timeout="", ssl.verify.client="", ssl.protocol="", tls.store.type="", ssl.configurations="", request.size.validation.configurations="", header.validation.configurations="", server.bootstrap.configurations="", trace.log.enabled="", @map(...))) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
hub.id

Unique id for the WebSub Hub

STRINGNoNo
table.name

Table name to store the subscription details related to the hub

STRINGNoNo
receiver.url

The URL on which events should be received. To enable SSL use https protocol in the url.

`http://0.0.0.0:9763//`STRINGYesNo
topic.list

topics allowed in the websub hub

emptySTRINGNoNo
basic.auth.enabled

This only works in VM, Docker and Kubernetes.
Where when enabled it authenticates each request using the Authorization:'Basic encodeBase64(username:Password)' header.

falseSTRINGYesNo
worker.count

The number of active worker threads to serve the incoming events. By default the value is set to 1 to ensure events are processed in the same order they arrived. By increasing this value, higher performance can be achieved in the expense of loosing event ordering.

1INTYesNo
socket.idle.timeout

Idle timeout for HTTP connection in millis.

120000INTYesNo
ssl.verify.client

The type of client certificate verification. Supported values are require, optional.

-STRINGYesNo
ssl.protocol

SSL/TLS protocol.

TLSSTRINGYesNo
tls.store.type

TLS store type.

JKSSTRINGYesNo
ssl.configurations

SSL/TSL configurations in format "'<key>:<value>','<key>:<value>'".
Some supported parameters:
 - SSL/TLS protocols: 'sslEnabledProtocols:TLSv1.1,TLSv1.2'
 - List of ciphers: 'ciphers:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256'
 - Enable session creation: 'client.enable.session.creation:true'
 - Supported server names: 'server.suported.server.names:server'
 - Add HTTP SNIMatcher: 'server.supported.snimatchers:SNIMatcher'

-STRINGYesNo
request.size.validation.configurations

Configurations to validate the HTTP request size.
Expected format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Enable request size validation: 'request.size.validation:true'
 If request size is validated
 - Maximum request size: 'request.size.validation.maximum.value:2048'
 - Response status code when request size validation fails: 'request.size.validation.reject.status.code:401'
 - Response message when request size validation fails: 'request.size.validation.reject.message:Message is bigger than the valid size'
 - Response Content-Type when request size validation fails: 'request.size.validation.reject.message.content.type:plain/text'

-STRINGYesNo
header.validation.configurations

Configurations to validate HTTP headers.
Expected format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Enable header size validation: 'header.size.validation:true'
 If header size is validated
 - Maximum length of initial line: 'header.validation.maximum.request.line:4096'
 - Maximum length of all headers: 'header.validation.maximum.size:8192'
 - Maximum length of the content or each chunk: 'header.validation.maximum.chunk.size:8192'
 - Response status code when header validation fails: 'header.validation.reject.status.code:401'
 - Response message when header validation fails: 'header.validation.reject.message:Message header is bigger than the valid size'
 - Response Content-Type when header validation fails: 'header.validation.reject.message.content.type:plain/text'

-STRINGYesNo
server.bootstrap.configurations

Server bootstrap configurations in format "'<key>:<value>','<key>:<value>'".
Some supported configurations :
 - Server connect timeout in millis: 'server.bootstrap.connect.timeout:15000'
 - Server socket timeout in seconds: 'server.bootstrap.socket.timeout:15'
 - Enable TCP no delay: 'server.bootstrap.nodelay:true'
 - Enable server keep alive: 'server.bootstrap.keepalive:true'
 - Send buffer size: 'server.bootstrap.sendbuffersize:1048576'
 - Receive buffer size: 'server.bootstrap.recievebuffersize:1048576'
 - Number of connections queued: 'server.bootstrap.socket.backlog:100'

-STRINGYesNo
trace.log.enabled

Enable trace log for traffic monitoring.

falseBOOLYesNo
+ +System Parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Parameters
serverBootstrapBossGroupSize

Number of boss threads to accept incoming connections.

Number of available processorsAny positive integer
serverBootstrapWorkerGroupSize

Number of worker threads to accept the connections from boss threads and perform non-blocking read and write from one or more channels.

(Number of available processors) * 2Any positive integer
serverBootstrapClientGroupSize

Number of client threads to perform non-blocking read and write to one or more channels.

(Number of available processors) * 2Any positive integer
defaultHost

The default host of the transport.

0.0.0.0Any valid host
defaultScheme

The default protocol.

httphttp
https
defaultHttpPort

The default HTTP port when default scheme is http.

8280Any valid port
defaultHttpsPort

The default HTTPS port when default scheme is https.

8243Any valid port
keyStoreLocation

The default keystore file path.

`${carbon.home}/resources/security/wso2carbon.jks`Path to `.jks` file
keyStorePassword

The default keystore password.

wso2carbonKeystore password as string
+ +Examples +EXAMPLE 1 +``` +@app.name('StockProcessor') + +@store(type='rdbms' , jdbc.url='jdbc:mysql://localhost:3306/production?useSSL=false', username='root', password='root', jdbc.driver.name='com.mysql.jdbc.Driver') +@source(type='websubhub' , hub.id='anu_123',table.name='SessionTable', receiver.url='http://localhost:8006/productionStream',basic.auth.enabled='false', @map(type='keyvalue',implicit.cast.enable='true')) +define stream webSubStream(callback string, lease_seconds long, secret string, topic string, mode string); + +``` +

+

Above WebSubHub listening on http://localhost:8006/productionStream for thesubscription requests.

+

diff --git a/docs/api/latest.md b/docs/api/latest.md index cd49e3d2..a1c88ee8 100644 --- a/docs/api/latest.md +++ b/docs/api/latest.md @@ -1,6 +1,6 @@ -# API Docs - v2.3.3 +# API Docs - v2.3.4 -!!! Info "Tested Siddhi Core version: *5.1.19*" +!!! Info "Tested Siddhi Core version: *5.1.21*" It could also support other Siddhi Core minor versions. ## Sink @@ -12,7 +12,7 @@ Syntax ``` -@sink(type="http", publisher.url="", basic.auth.username="", basic.auth.password="", https.truststore.file="", https.truststore.password="", oauth.username="", oauth.password="", consumer.key="", consumer.secret="", token.url="", refresh.token="", headers="", method="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", tls.store.type="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) +@sink(type="http", publisher.url="", basic.auth.username="", basic.auth.password="", https.truststore.file="", https.truststore.password="", oauth.username="", oauth.password="", consumer.key="", consumer.secret="", body.consumer.key="", body.consumer.secret="", token.url="", refresh.token="", oauth.scope="", headers="", method="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", tls.store.type="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", executor.service.threads="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) ``` QUERY PARAMETERS @@ -97,6 +97,22 @@ Yes No + + body.consumer.key +

Consumer key used for calling endpoints protected by OAuth 2.0 if it's required to be sent in token request body

+ - + STRING + Yes + No + + + body.consumer.secret +

Consumer secret used for calling endpoints protected by OAuth 2.0

+ - + STRING + Yes + No + token.url

Token URL to generate a new access tokens when calling endpoints protected by OAuth 2.0

@@ -113,6 +129,14 @@ Yes No + + oauth.scope +

Standard OAuth 2.0 scope parameter

+ default + STRING + Yes + No + headers

HTTP request headers in format "'<key>:<value>','<key>:<value>'".
When Content-Type header is not provided the system derives the Content-Type based on the provided sink mapper as following:
 - @map(type='xml'): application/xml
 - @map(type='json'): application/json
 - @map(type='text'): plain/text
 - @map(type='keyvalue'): application/x-www-form-urlencoded
 - For all other cases system defaults to plain/text
Also the Content-Length header need not to be provided, as the system automatically defines it by calculating the size of the payload.

@@ -241,6 +265,14 @@ Yes No + + executor.service.threads +

Thread count for the executor service.

+ 20 + INT + Yes + No + min.evictable.idle.time

Minimum time (in millis) a connection may sit idle in the client pool before it become eligible for eviction.

@@ -1436,12 +1468,12 @@ insert into ResultStream;

### websubhub *(Sink)*

-

WebSubHubEventPublisher publishes messages via HTTP/HTTP according to the provided URL when subscribe to the WebSub hub. The table.name, hub.id and

+

WebSubHubEventPublisher publishes messages via HTTP/HTTP according to the provided URL when subscribe to the WebSub hub. The table.name, hub.id are mandatory when defining the websubhub source

Syntax ``` -@sink(type="websubhub", hub.id="", table.name="", https.truststore.file="", https.truststore.password="", consumer.key="", consumer.secret="", token.url="", refresh.token="", headers="", method="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", tls.store.type="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) +@sink(type="websubhub", hub.id="", table.name="", https.truststore.file="", https.truststore.password="", consumer.key="", consumer.secret="", token.url="", refresh.token="", headers="", method="", socket.idle.timeout="", chunk.disabled="", ssl.protocol="", ssl.verification.disabled="", tls.store.type="", ssl.configurations="", proxy.host="", proxy.port="", proxy.username="", proxy.password="", client.bootstrap.configurations="", max.pool.active.connections="", min.pool.idle.connections="", max.pool.idle.connections="", executor.service.threads="", min.evictable.idle.time="", time.between.eviction.runs="", max.wait.time="", test.on.borrow="", test.while.idle="", exhausted.action="", hostname.verification.enabled="", @map(...))) ``` QUERY PARAMETERS @@ -1646,6 +1678,14 @@ insert into ResultStream; Yes No + + executor.service.threads +

Thread count for the executor service.

+ 20 + INT + Yes + No + min.evictable.idle.time

Minimum time (in millis) a connection may sit idle in the client pool before it become eligible for eviction.

@@ -2715,7 +2755,7 @@ insert into ResultStream;

### websubhub *(Source)*

-

WebSub Hub source receive subscription requests via Http and according to the request, the subscription details will be saved to the given table and against the callback and topic name. The subscription request MUST have a Content-Type header of application/x-www-form-urlencoded and following MUST provide as parameter body hub.callback      - REQUIRED. The subscriber's callback URL where content distribution notifications should be delivered. The callback URL SHOULD be an unguessable URL that is unique per subscription.
hub.mode      - REQUIRED. The literal string "subscribe" or "unsubscribe", depending on the goal of the request.
hub.topic      - REQUIRED. The topic URL that the subscriber wishes to subscribe to or unsubscribe from.
 hub.lease_seconds     - OPTIONAL. Number of seconds for which the subscriber would like to have the subscription active, given as a positive decimal integer.
hub.secret     - OPTIONAL. A subscriber-provided cryptographically random unique secret string that will be used to compute an HMAC digest for authorized content distribution. If not supplied, the HMAC digest will not be present for content distribution requests.

+

WebSub Hub source receive subscription requests via Http and according to the request, the subscription details will be saved to the given table and against the callback and topic name. The subscription request **MUST** have a Content-Type header of **application/x-www-form-urlencoded** and following **MUST** provide as parameter body.
      hub.callback      - REQUIRED.     The subscriber's callback URL where content distribution notifications should be delivered. The callback URL SHOULD be an unguessable URL that is unique per subscription.
     hub.mode      - REQUIRED.     The literal string "subscribe" or "unsubscribe", depending on the goal of the request.
     hub.topic      - REQUIRED.     The topic URL that the subscriber wishes to subscribe to or unsubscribe from.
      hub.lease_seconds     - OPTIONAL.     Number of seconds for which the subscriber would like to have the subscription active, given as a positive decimal integer.
     hub.secret     - OPTIONAL.     A subscriber-provided cryptographically random unique secret string that will be used to compute an HMAC digest for authorized content distribution. If not supplied, the HMAC digest will not be present for content distribution requests.

Syntax diff --git a/docs/index.md b/docs/index.md index 8b1c2cea..4e6c5685 100644 --- a/docs/index.md +++ b/docs/index.md @@ -19,24 +19,24 @@ For information on Siddhi and i ## Latest API Docs -Latest API Docs is 2.3.3. +Latest API Docs is 2.3.4. ## Features -* http *(Sink)*

HTTP sink publishes messages via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML and JSON. It can also publish to endpoints protected by basic authentication or OAuth 2.0.

-* http-call *(Sink)*

The http-call sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-call-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

-* http-request *(Sink)*

_(Use http-call sink instead)._
The http-request sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

-* http-response *(Sink)*

_(Use http-service-response sink instead)._
The http-response sink send responses of the requests consumed by its corresponding http-request source, by mapping the response messages to formats such as text, XML and JSON.

-* http-service-response *(Sink)*

The http-service-response sink send responses of the requests consumed by its corresponding http-service source, by mapping the response messages to formats such as text, XML and JSON.

-* sse-server *(Sink)*

HTTP SSE sink sends events to all subscribers.

-* websubhub *(Sink)*

WebSubHubEventPublisher publishes messages via HTTP/HTTP according to the provided URL when subscribe to the WebSub hub. The table.name, hub.id and

-* http *(Source)*

HTTP source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON. It also supports basic authentication to ensure events are received from authorized users/systems.
The request headers and properties can be accessed via transport properties in the format trp:<header>.

-* http-call-response *(Source)*

The http-call-response source receives the responses for the calls made by its corresponding http-call sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-call-response sources are allowed to associate with a single http-call sink.
It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

-* http-request *(Source)*

_(Use http-service source instead)._
The http-request source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

-* http-response *(Source)*

_(Use http-call-response source instead)._
The http-response source receives the responses for the calls made by its corresponding http-request sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-response sources are allowed to associate with a single http-request sink. It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

-* http-service *(Source)*

The http-service source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-service-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-service-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

-* sse *(Source)*

HTTP SSE source send a request to a given url and listen to the response stream.

-* websubhub *(Source)*

WebSub Hub source receive subscription requests via Http and according to the request, the subscription details will be saved to the given table and against the callback and topic name. The subscription request MUST have a Content-Type header of application/x-www-form-urlencoded and following MUST provide as parameter body hub.callback      - REQUIRED. The subscriber's callback URL where content distribution notifications should be delivered. The callback URL SHOULD be an unguessable URL that is unique per subscription.
hub.mode      - REQUIRED. The literal string "subscribe" or "unsubscribe", depending on the goal of the request.
hub.topic      - REQUIRED. The topic URL that the subscriber wishes to subscribe to or unsubscribe from.
 hub.lease_seconds     - OPTIONAL. Number of seconds for which the subscriber would like to have the subscription active, given as a positive decimal integer.
hub.secret     - OPTIONAL. A subscriber-provided cryptographically random unique secret string that will be used to compute an HMAC digest for authorized content distribution. If not supplied, the HMAC digest will not be present for content distribution requests.

+* http *(Sink)*

HTTP sink publishes messages via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML and JSON. It can also publish to endpoints protected by basic authentication or OAuth 2.0.

+* http-call *(Sink)*

The http-call sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-call-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

+* http-request *(Sink)*

_(Use http-call sink instead)._
The http-request sink publishes messages to endpoints via HTTP or HTTPS protocols using methods such as POST, GET, PUT, and DELETE on formats text, XML or JSON and consume responses through its corresponding http-response source. It also supports calling endpoints protected with basic authentication or OAuth 2.0.

+* http-response *(Sink)*

_(Use http-service-response sink instead)._
The http-response sink send responses of the requests consumed by its corresponding http-request source, by mapping the response messages to formats such as text, XML and JSON.

+* http-service-response *(Sink)*

The http-service-response sink send responses of the requests consumed by its corresponding http-service source, by mapping the response messages to formats such as text, XML and JSON.

+* sse-server *(Sink)*

HTTP SSE sink sends events to all subscribers.

+* websubhub *(Sink)*

WebSubHubEventPublisher publishes messages via HTTP/HTTP according to the provided URL when subscribe to the WebSub hub. The table.name, hub.id are mandatory when defining the websubhub source

+* http *(Source)*

HTTP source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON. It also supports basic authentication to ensure events are received from authorized users/systems.
The request headers and properties can be accessed via transport properties in the format trp:<header>.

+* http-call-response *(Source)*

The http-call-response source receives the responses for the calls made by its corresponding http-call sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-call-response sources are allowed to associate with a single http-call sink.
It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

+* http-request *(Source)*

_(Use http-service source instead)._
The http-request source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

+* http-response *(Source)*

_(Use http-call-response source instead)._
The http-response source receives the responses for the calls made by its corresponding http-request sink, and maps them from formats such as text, XML and JSON.
To handle messages with different http status codes having different formats, multiple http-response sources are allowed to associate with a single http-request sink. It allows accessing the attributes of the event that initiated the call, and the response headers and properties via transport properties in the format trp:<attribute name> and trp:<header/property> respectively.

+* http-service *(Source)*

The http-service source receives POST requests via HTTP and HTTPS protocols in format such as text, XML and JSON and sends responses via its corresponding http-service-response sink correlated through a unique source.id.
For request and response correlation, it generates a messageId upon each incoming request and expose it via transport properties in the format trp:messageId to correlate them with the responses at the http-service-response sink.
The request headers and properties can be accessed via transport properties in the format trp:<header>.
It also supports basic authentication to ensure events are received from authorized users/systems.

+* sse *(Source)*

HTTP SSE source send a request to a given url and listen to the response stream.

+* websubhub *(Source)*

WebSub Hub source receive subscription requests via Http and according to the request, the subscription details will be saved to the given table and against the callback and topic name. The subscription request **MUST** have a Content-Type header of **application/x-www-form-urlencoded** and following **MUST** provide as parameter body.
      hub.callback      - REQUIRED.     The subscriber's callback URL where content distribution notifications should be delivered. The callback URL SHOULD be an unguessable URL that is unique per subscription.
     hub.mode      - REQUIRED.     The literal string "subscribe" or "unsubscribe", depending on the goal of the request.
     hub.topic      - REQUIRED.     The topic URL that the subscriber wishes to subscribe to or unsubscribe from.
      hub.lease_seconds     - OPTIONAL.     Number of seconds for which the subscriber would like to have the subscription active, given as a positive decimal integer.
     hub.secret     - OPTIONAL.     A subscriber-provided cryptographically random unique secret string that will be used to compute an HMAC digest for authorized content distribution. If not supplied, the HMAC digest will not be present for content distribution requests.

## Dependencies diff --git a/findbugs-exclude.xml b/findbugs-exclude.xml new file mode 100644 index 00000000..f67627a8 --- /dev/null +++ b/findbugs-exclude.xml @@ -0,0 +1,25 @@ + + + + + + + + + diff --git a/mkdocs.yml b/mkdocs.yml index b400a234..6c40e0b9 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -34,6 +34,7 @@ pages: - Information: index.md - API Docs: - latest: api/latest.md + - 2.3.4: api/2.3.4.md - 2.3.3: api/2.3.3.md - 2.3.2: api/2.3.2.md - 2.3.1: api/2.3.1.md diff --git a/pom.xml b/pom.xml index ded80343..e98d6d24 100644 --- a/pom.xml +++ b/pom.xml @@ -27,7 +27,7 @@ https://github.com/siddhi-io/siddhi-io-http.git scm:git:https://github.com/siddhi-io/siddhi-io-http.git - v2.3.3 + v2.3.4 component @@ -35,7 +35,7 @@ io.siddhi.extension.io.http siddhi-io-http-parent - 2.3.3 + 2.3.4 Siddhi IO Extension - HTTP IO Aggregator http://wso2.org @@ -95,27 +95,46 @@ org.apache.commons commons-lang3 ${commons.lang3.version} - - - org.wso2.carbon.messaging - org.wso2.carbon.messaging - ${org.wso2.carbon.messaging.version} + + + log4j + log4j + + org.wso2.transport.http org.wso2.transport.http.netty ${org.wso2.transport.http.version} + + + log4j + log4j + + io.netty netty-all ${io.netty.version} + + + log4j + log4j + + commons-io.wso2 commons-io ${commons-io.version} + + + log4j + log4j + + org.testng @@ -126,56 +145,146 @@ org.wso2.carbon.analytics-common org.wso2.carbon.analytics.idp.client ${carbon.analytics.common.version} + + + log4j + log4j + + apache-collections commons-collections ${commons.collections.version} + + + log4j + log4j + + org.ops4j.pax.tipi org.ops4j.pax.tipi.hamcrest.core ${org.ops4j.pax.tipi.version} + + + log4j + log4j + + org.wso2.msf4j msf4j-core ${msf4j.version} + + + log4j + log4j + + org.json.wso2 json ${org.json.wso2.version} + + + log4j + log4j + + org.wso2.carbon.analytics org.wso2.carbon.si.metrics.core ${carbon.analytics.version} + + + log4j + log4j + + commons-codec commons-codec ${commons.codec.version} + + + com.squareup.okhttp3 + okhttp + ${okhttp.client.version} + + + org.apache.logging.log4j + log4j-core + ${log4j.version} + + + javax.mail + mail + + + javax.jms + jms + + + com.sun.jdmk + jmxtools + + + com.sun.jmx + jmxri + + + + + org.apache.logging.log4j + log4j-api + ${log4j.version} + + + javax.mail + mail + + + javax.jms + jms + + + com.sun.jdmk + jmxtools + + + com.sun.jmx + jmxri + + + 2.7.6 + 2.17.1 3.0.0.wso2v1 - 5.0.6 - 2.0.4 - 5.0.4 - 5.1.19 + 5.2.2 + 2.1.1 + 5.2.2 + 5.1.21 [5.0.0,6.0.0) - 6.1.30 + [4.1.70.Final,5.0.0) + 6.1.57 3.1 1.3.0.1 - 4.1.42.Final + 4.1.73.Final 6.11 2.4.1 0.7.5.201505241946 1.2.4 - 6.3.22 + 6.3.30 [6.0.0,7.0.0) 2.4.0.wso2v1 3.7 @@ -193,6 +302,8 @@ 2.22.1 3.0.49 1.15 + 4.8.1 + findbugs-exclude.xml @@ -291,6 +402,9 @@ org.codehaus.mojo findbugs-maven-plugin + + ${mavan.findbugsplugin.exclude.file} + none