GETKEYSANDFLAGS sub-command, new module apis, and key-spec flags changes (redis#1784)

doc changes for redis/redis#10237

Co-authored-by: Itamar Haber <[email protected]>

Author: oranagra

commands.json

@@ -582,7 +582,8 @@
                 },
                 "RW": true,
                 "access": true,
-                "update": true
+                "update": true,
+                "variable_flags": true
             }
         ],
         "arguments": [
@@ -2854,6 +2855,21 @@
             "stale"
         ]
     },
+    "COMMAND GETKEYSANDFLAGS": {
+        "summary": "Extract keys given a full Redis command",
+        "since": "7.0.0",
+        "group": "server",
+        "complexity": "O(N) where N is the number of arguments to the command",
+        "acl_categories": [
+            "@slow",
+            "@connection"
+        ],
+        "arity": -4,
+        "command_flags": [
+            "loading",
+            "stale"
+        ]
+    },
     "COMMAND HELP": {
         "summary": "Show helpful text about the different subcommands",
         "since": "5.0.0",
@@ -4244,7 +4260,7 @@
         "arity": 3,
         "arguments": [
             {
-                "name": "function-name",
+                "name": "library-name",
                 "type": "string"
             }
         ],
@@ -11111,7 +11127,8 @@
                 },
                 "RW": true,
                 "access": true,
-                "update": true
+                "update": true,
+                "variable_flags": true
             }
         ],
         "arguments": [

commands/command-getkeys.md

@@ -3,10 +3,9 @@ Returns @array-reply of keys from a full Redis command.
 `COMMAND GETKEYS` is a helper command to let you find the keys
 from a full Redis command.
 
-`COMMAND` shows some commands as having movablekeys meaning
-the entire command must be parsed to discover storage or retrieval
-keys.  You can use `COMMAND GETKEYS` to discover key positions
-directly from how Redis parses the commands.
+`COMMAND` provides information on how to find the key names of each command (see `firstkey`, [key specifications](/topics/key-specs#logical-operation-flags), and `movablekeys`),
+but in some cases it's not possible to find keys of certain commands and then the entire command must be parsed to discover some / all key names.
+You can use `COMMAND GETKEYS` or `COMMAND GETKEYSANDFLAGS` to discover key names directly from how Redis parses the commands.
 
 
 @return

commands/command-getkeysandflags.md

@@ -0,0 +1,22 @@
+Returns @array-reply of keys from a full Redis command and their usage flags.
+
+`COMMAND GETKEYSANDFLAGS` is a helper command to let you find the keys from a full Redis command together with flags indicating what each key is used for.
+
+`COMMAND` provides information on how to find the key names of each command (see `firstkey`, [key specifications](/topics/key-specs#logical-operation-flags), and `movablekeys`),
+but in some cases it's not possible to find keys of certain commands and then the entire command must be parsed to discover some / all key names.
+You can use `COMMAND GETKEYS` or `COMMAND GETKEYSANDFLAGS` to discover key names directly from how Redis parses the commands.
+
+Refer to [key specifications](/topics/key-specs#logical-operation-flags) for information about the meaning of the key flags.
+
+@return
+
+@array-reply: list of keys from your command.
+Each element of the array is an array containing key name in the first entry, and flags in the second.
+
+@examples
+
+```cli
+COMMAND GETKEYS MSET a b c d e f
+COMMAND GETKEYS EVAL "not consulted" 3 key1 key2 key3 arg1 arg2 arg3 argN
+COMMAND GETKEYSANDFLAGS LMOST mylist1 mylist2 left left
+```

topics/key-specs.md

@@ -156,7 +156,8 @@ Key specifications may have the following flags:
 * **channel:** this flag indicates that the specification isn't about keys at all.
   Instead, the specification relates to the name of a sharded Pub/Sub channel.
   Please refer to the `SPUBLISH` command for further details about sharded Pub/Sub.
-* **incomplete:** this flag is explained in the following section.
+* **incomplete:** this flag is explained below.
+* **variable_flags:** this flag is explained below.
 
 ### incomplete
 
@@ -180,6 +181,13 @@ The difficulty arises, for example, because the string _"STORE"_ is both a keywo
 the only commands with _incomplete_ key specifications are `SORT` and `MIGRATE`.
 We don't expect the addition of such commands in the future.
 
+### variable_flags
+
+In some commands, the flags for the same key name argument can depend on other arguments.
+For example, consider the `SET` command and its optional  _GET_ argument.
+Without the _GET_ argument, `SET` is write-only, but it becomes a read and write command with it.
+When this flag is present, it means that the key specification flags cover all possible options, but the effective flags depend on other arguments.
+
 ## Examples
 
 ### `SET`'s key specifications

topics/modules-api-ref.md

@@ -152,28 +152,42 @@ Return non-zero if a module command, that was declared with the
 flag "getkeys-api", is called in a special way to get the keys positions
 and not to get executed. Otherwise zero is returned.
 
-<span id="RedisModule_KeyAtPos"></span>
+<span id="RedisModule_KeyAtPosWithFlags"></span>
 
-### `RedisModule_KeyAtPos`
+### `RedisModule_KeyAtPosWithFlags`
 
-    void RedisModule_KeyAtPos(RedisModuleCtx *ctx, int pos);
-
-**Available since:** 4.0.0
+    void RedisModule_KeyAtPosWithFlags(RedisModuleCtx *ctx, int pos, int flags);
 
 When a module command is called in order to obtain the position of
 keys, since it was flagged as "getkeys-api" during the registration,
 the command implementation checks for this special call using the
 [`RedisModule_IsKeysPositionRequest()`](#RedisModule_IsKeysPositionRequest) API and uses this function in
-order to report keys, like in the following example:
+order to report keys.
+
+The supported flags are the ones used by [`RedisModule_SetCommandInfo`](#RedisModule_SetCommandInfo), see `REDISMODULE_CMD_KEY_`*.
+
+
+The following is an example of how it could be used:
 
     if (RedisModule_IsKeysPositionRequest(ctx)) {
-        RedisModule_KeyAtPos(ctx,1);
-        RedisModule_KeyAtPos(ctx,2);
+        RedisModule_KeyAtPosWithFlags(ctx, 2, REDISMODULE_CMD_KEY_RO | REDISMODULE_CMD_KEY_ACCESS);
+        RedisModule_KeyAtPosWithFlags(ctx, 1, REDISMODULE_CMD_KEY_RW | REDISMODULE_CMD_KEY_UPDATE | REDISMODULE_CMD_KEY_ACCESS);
     }
 
- Note: in the example below the get keys API would not be needed since
- keys are at fixed positions. This interface is only used for commands
- with a more complex structure.
+ Note: in the example above the get keys API could have been handled by key-specs (preferred).
+ Implementing the getkeys-api is required only when is it not possible to declare key-specs that cover all keys.
+
+<span id="RedisModule_KeyAtPos"></span>
+
+### `RedisModule_KeyAtPos`
+
+    void RedisModule_KeyAtPos(RedisModuleCtx *ctx, int pos);
+
+**Available since:** 4.0.0
+
+This API existed before [`RedisModule_KeyAtPosWithFlags`](#RedisModule_KeyAtPosWithFlags) was added, now deprecated and
+can be used for compatibility with older versions, before key-specs and flags
+were introduced.
 
 <span id="RedisModule_CreateCommand"></span>
 
@@ -408,8 +422,9 @@ All fields except `version` are optional. Explanation of the fields:
     versions where RM_SetCommandInfo is not available.
 
     Note that key-specs don't fully replace the "getkeys-api" (see
-    RM_CreateCommand, RM_IsKeysPositionRequest and RM_KeyAtPos) so it may be
-    a good idea to supply both key-specs and a implement the getkeys-api.
+    RM_CreateCommand, RM_IsKeysPositionRequest and RM_KeyAtPosWithFlags) so
+    it may be a good idea to supply both key-specs and implement the
+    getkeys-api.
 
     A key-spec has the following structure:
 
@@ -6472,21 +6487,24 @@ such as:
 
 If `old_value` is non-NULL, the old value is returned by reference.
 
-<span id="RedisModule_GetCommandKeys"></span>
-
-### `RedisModule_GetCommandKeys`
+<span id="RedisModule_GetCommandKeysWithFlags"></span>
 
-    int *RedisModule_GetCommandKeys(RedisModuleCtx *ctx,
-                                    RedisModuleString **argv,
-                                    int argc,
-                                    int *num_keys);
+### `RedisModule_GetCommandKeysWithFlags`
 
-**Available since:** 6.0.9
+    int *RedisModule_GetCommandKeysWithFlags(RedisModuleCtx *ctx,
+                                             RedisModuleString **argv,
+                                             int argc,
+                                             int *num_keys,
+                                             int **out_flags);
 
 For a specified command, parse its arguments and return an array that
 contains the indexes of all key name arguments. This function is
 essentially a more efficient way to do `COMMAND GETKEYS`.
 
+The `out_flags` argument is optional, and can be set to NULL.
+When provided it is filled with `REDISMODULE_CMD_KEY_` flags in matching
+indexes with the key indexes of the returned array.
+
 A NULL return value indicates the specified command has no keys, or
 an error condition. Error conditions are indicated by setting errno
 as follows:
@@ -6496,7 +6514,21 @@ as follows:
 
 NOTE: The returned array is not a Redis Module object so it does not
 get automatically freed even when auto-memory is used. The caller
-must explicitly call [`RedisModule_Free()`](#RedisModule_Free) to free it.
+must explicitly call [`RedisModule_Free()`](#RedisModule_Free) to free it, same as the `out_flags` pointer if
+used.
+
+<span id="RedisModule_GetCommandKeys"></span>
+
+### `RedisModule_GetCommandKeys`
+
+    int *RedisModule_GetCommandKeys(RedisModuleCtx *ctx,
+                                    RedisModuleString **argv,
+                                    int argc,
+                                    int *num_keys);
+
+**Available since:** 6.0.9
+
+Identinal to [`RedisModule_GetCommandKeysWithFlags`](#RedisModule_GetCommandKeysWithFlags) when flags are not needed.
 
 <span id="RedisModule_GetCurrentCommandName"></span>
 
@@ -6762,6 +6794,7 @@ There is no guarantee that this info is always available, so this may return -1.
 * [`RedisModule_GetClusterSize`](#RedisModule_GetClusterSize)
 * [`RedisModule_GetCommand`](#RedisModule_GetCommand)
 * [`RedisModule_GetCommandKeys`](#RedisModule_GetCommandKeys)
+* [`RedisModule_GetCommandKeysWithFlags`](#RedisModule_GetCommandKeysWithFlags)
 * [`RedisModule_GetContextFlags`](#RedisModule_GetContextFlags)
 * [`RedisModule_GetContextFlagsAll`](#RedisModule_GetContextFlagsAll)
 * [`RedisModule_GetCurrentCommandName`](#RedisModule_GetCurrentCommandName)
@@ -6814,6 +6847,7 @@ There is no guarantee that this info is always available, so this may return -1.
 * [`RedisModule_IsModuleNameBusy`](#RedisModule_IsModuleNameBusy)
 * [`RedisModule_IsSubEventSupported`](#RedisModule_IsSubEventSupported)
 * [`RedisModule_KeyAtPos`](#RedisModule_KeyAtPos)
+* [`RedisModule_KeyAtPosWithFlags`](#RedisModule_KeyAtPosWithFlags)
 * [`RedisModule_KeyExists`](#RedisModule_KeyExists)
 * [`RedisModule_KeyType`](#RedisModule_KeyType)
 * [`RedisModule_KillForkChild`](#RedisModule_KillForkChild)