[Closed] Name for the part of the selector that comes after a ?

[p-avital]

Currently, Selector is defined by a key_selector and a value_selector.

We recently decided to expose the value_selector as a simple string for 0.6.0, with the intent of providing good parsers in zenoh-ext later.

However, we might as well take this chance to rename these fields:

• key_selector becomes key_expr easily, making the use of key_expr consistent everywhere when you want to access something's key expression.
• value_selector is very poorly named: it used to mirror key_selector, but that section of the selector can have arbitrary content and is often used to carry arguments rather than act as a selector that looks at the value.

[kydos]

I don't agree that value_selector is poorly named. It clearly states that it is about how you pick the values you are interested in and that includes both the predicate as well as the projection. Properties can also be seen as additional context to specify how to choose the value. Additionally if makes a nice tandem with key_selector. I am not convinced we need to change it and I find that the e proposal above make less intuitive for a user to get a hint of what it does.

[Mallets]

As part of the basic Zenoh API, what comes after ? is treated as a string with no constraints on the carried meaning.
One option is to use that part as value_selector but it doesn't have to. It can be used to pass any arbitrary information to the queryable like the time budget available to respond to a query or whether the queryable should silently fail in case of errors.
As a result, in zenoh-ext we can then define a semantic that acts as value_selector with specific syntax for express projects and filter.
A different semantic could be based on GraphQL, on SQL, or any other domain specific language if necessary.

So, the fact that it might look "less intuitive for a user to get a hint of what it does" to me is actually proving the point that value_selector actually does nothing since it is not defined. In the basic Zenoh API it carries no semantic meaning nor syntactic constraints. It is really implementation dependent and, to be clear, I'm completely OK to have an out-of-the-box system working in zenoh-ext.

Finally, I think that our key_expr is already acting as key_selector. I.e. its form can be already used to address a set of keys for subscriptions, publications, and queries. As of today, I believe a key_selector is just an alias for key_expr since they don't present any difference in semantic nor syntax.

[JEnoch]

Summary of a discussion between @Mallets, @OlivierHecart and @JEnoch today:

The parsing and partial interpretation of the value_selector part on API side is currently required only because the default ConsolidationStrategy (only the latest sample per key) doesn't suit to a query with starttime/stoptime that implies no consolidation (i.e. several samples per key).
@Mallets point's is that this parsing/interpretation of value_selector has a cost on querier side (including in zenoh-pico which is the most problematic), whatever the queryable - InfluxDB or not.

@Mallets suggested as a solution to move this parsing and interpretation of value_selector out of the zenoh base API, into zenoh-ext. If zenoh-ext is not used, the selector string is converted into a Selector object, without parsing of the value_selector.
If zenoh-ext is used, the selector string is converted into an intermediate type that will parse/interpret the value_selector and eventually be converted into a Selector object which also embed a modification of the configuration to be applied on the get() (i.e. deactivation of the ConsolidationStrategy).

@OlivierHecart and I found this solution to complex (forcing usage of zenoh-ext for this use case, and not clear how this will be implemented in other languages than Rust).
We suggested 2 simpler solutions:

1. Either change the default ConsolidationStrategy from reception() to none() (i.e. querier possibly receives several samples per key). The impact of this change would be for queries on multiple non-complete queryables that would now possibly receive several samples per-key by default.
2. Either let the queryable to influence the ConsolidationStrategy rather than the querier. In a sense, that's up to the queryable to interpret the value_selector and to decide if it has to reply several sample per key or only 1. This would require a change in the protocol, adding a flag to the Reply message to indicate if there are several sample per-key and thus to deactivate the consolidation in the router and in the querier API.

[p-avital]

Solution 2 appears simpler, but it pushes back complexity onto every user who would then have to beware of rogue queryables that may not respect their consolidation strategy and thus their applicative assertions.

Solution 1 makes little sense to me, as this forces every beginner to handle consolidation themselves until they figure out that ConsolidationStrategy exists, or worse, lets them work under the assumption that they'll only receive one value per key (a senseful assumption), until they scale out and discover that that assumption didn't hold later in the dev cycle.

Time-series queryables are cool, but I very much doubt that they are the default expected behavior, and already require the application to be built around them to be useful to begin with, so I'd side with making Reception the default strategy for ALL queries. Since we're planning on letting queryables send errors, time-series could peek the consolidation strategy and send an error whenever they'd have sent more than a single value back.

[p-avital]

zenoh-ext could then provide facilities to easily query time series, facilities that of course would set the consolidation strategy appropriately.

[kydos]

@Mallets I think it is a thin balance between giving sufficient context and structure that makes life easier for 99% of the users and still allowing the flexibility to that 1% of users who are going to define themselves the syntax and semantics of the value_selector. Please recall, that storages were rather integrated with zenoh when we had 2-levels of API. Now there is a bit "looser" integration, but once again, it is a matter of balance. I am not happy to push on zenoh-ext something that for 99% of the uses is bread and butter. I suggest that before this thread becomes too long we may get some discussion at the next core-team meeting.

Zenoh strikes for efficiency, ergonomicisy and easy of use. Let's keep that in mind.

[p-avital]

@kydos I think we need to rebalance our view of zenoh-ext. zenoh-ext is not zenoh-expert (such a crate would make no sense, and wouldn't have its place in the zenoh monorepo), but rather a vent for what is not core zenoh primitives, but rather common and useful patterns that we expect users to find uses for.

There are thousands of ways to implement storages for zenoh, zenoh-ext should be the place where we provide an efficient, safe and senseful implementation of the tools that let us make our own implementations and interact with them.

Although we currently are the main provider of queryables through our plugins, biasing our core API for our own implementations of queryables seems overzealous to me.

[JEnoch]

The value_selector purpose and syntax was discussed during a zenoh team meeting on 24/06/2022.
Here are the outcomes of this discussion:

A Selector is currently used only as query argument (get() operation). However, in the future, we want to allow it for subscriptions to support filtered subscriptions (possibly implemented at different levels: subscriber, publisher or infrastructure).
Another idea of Selector usage with subscriptions is to pass a stoptime for time-bounded subscriptions.
For those use case, the value_selector needs to be interpreted (fully or partially) by the zenoh API itself (not zenoh-ext).

So we need the value_selector to have a well defined syntax for the zenoh API to interpret some parts of it, but also extensible for the user's queryables or for the storages to possibly define their own DSL.
At the same time, as the REST API maps the URLs to a Selector (for GET methods), it would be nice to have its value_selector part easily mapped from the URL's query syntax, that is defined in RFC3986 as such:

query         = *( pchar / "/" / "?" )
pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"
unreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~"
sub-delims    = "!" / "$" / "&" / "'" / "(" / ")"/ "*" / "+" / "," / ";" / "="
pct-encoded   = "%" HEXDIG HEXDIG

Even if any character can be represented via percent-encoding, it would me more performant and user-friendly to stick with those characters.
Moreover, we think supporting the application/x-www-form-urlencoded format (i.e. a key/value list using = and & as separators) is a good idea since:

• it's a well known and defined format with lot of implementations in various languages
• it's extensible as it's a set of key/values where user can define it's own keys and DSL for the values
• it's quick to parse for the zenoh API and infrastructure to search for the keys it must support
• In Rust, the form_urlencoded crate is widely used and provide quick parsing with no allocation if no percent-encoding character is used (and no + character which is transformed as a ' '(whitespace)). Still, it supports any UTF-8 character, even if not in the set of RFC3986.

Therefore, the new format of the value_selector will be aligned with application/x-www-form-urlencoded format:

• & as separator between each key/value string
• = as separator between a key and its value
• if a key/value string doesn't contain a = or ends with a =, it means the key has an empty string as value
• if a key/value string starts with a =, it means the key is an empty string.
• any UTF-8 character is accepted in all APIs, with the exception of the REST API where the string must be percent-encoded
• the keys starting with a _ are reserved for zenoh usage (i.e. might be interpreted by the zenoh APIs, infrastructure or storages)

At this stage, we plan to make use of those reserved keys:

• _time to specify a time-range for values' timestamps. This is in replacement of the starttime and stoptime properties used in v0.5.0. E.g.:

  • /home/room/*?_time=[2022-06-23T09:00:00Z..2022-06-24T09:00:00Z[ (interval between 2 dates, start is inclusive, end is exclusive)
  • /home/room/*?_time=[2022-06-23T09:05:06Z;1h] (1 hour interval starting at 2022-06-23T09:05:06Z)
  • /home/room/*?_time=[now(-1h)..] (un-bounded interval starting 1 hour ago)
  • /home/room/*?_time=]now(-1h)..now()] (1 hour interval starting 1 hour ago (exclusive) and ending now)
  • /home/room/*?_time=[now(-1h);1m] (1 minute interval starting 1 hour ago)
  • /home/room/*?_time=[..] (un-bounded time interval, for instance to query a complete history)

• _filter to specify a filter on values (e.g.: /home/room/*?_filter=temp>20ANDhum>50). The exact DSL the filter has to be defined, knowing that & and + can't be used.

• _proj to specify a projection on values (e.g.: /home/room/*?_proj=temp;hum). The exact DSL the has to be defined.

[JEnoch]

Wrt. the original question: "Name for the part of the selector that comes after a ?" :
most of the usage we foresee for this part is to do a selection on the values the user would like to receive in its query result or subscriber. Thus value_selector makes sense.
However, to cover the cases where this part is used to pass arbitrary arguments to a user queryable, I suggest to rename this part with the more generic parameters name. Moreover, that's the name commonly used for URL's query string

[kydos]

As discussed the consensus is to change the name to parameter.