Unverified Commit 86781600 authored by guybe7's avatar guybe7 Committed by GitHub
Browse files

Auto-generate the command table from JSON files (#9656)

Delete the hardcoded command table and replace it with an auto-generated table, based
on a JSON file that describes the commands (each command must have a JSON file).

These JSON files are the SSOT of everything there is to know about Redis commands,
and it is reflected fully in COMMAND INFO.

These JSON files are used to generate commands.c (using a python script), which is then
committed to the repo and compiled.

The purpose is:
* Clients and proxies will be able to get much more info from redis, instead of relying on hard coded logic.
* drop the dependency between Redis-user and the commands.json in redis-doc.
* delete help.h and have redis-cli learn everything it needs to know just by issuing COMMAND (will be
  done in a separate PR)
* redis.io should stop using commands.json and learn everything from Redis (ultimately one of the release
  artifacts should be a large JSON, containing all the information about all of the commands, which will be
  generated from COMMAND's reply)
* the byproduct of this is:
  * module commands will be able to provide that info and possibly be more of a first-class citizens
  * in theory, one may be able to generate a redis client library for a strictly typed language, by using this info.

### Interface changes

#### COMMAND INFO's reply change (and arg-less COMMAND)

Before this commit the reply at index 7 contained the key-specs list
and reply at index 8 contained the sub-commands list (Both unreleased).
Now, reply at index 7 is a map of:
- summary - short command description
- since - debut version
- group - command group
- complexity - complexity string
- doc-flags - flags used for documentation (e.g. "deprecated")
- deprecated-since - if deprecated, from which version?
- replaced-by - if deprecated, which command replaced it?
- history - a list of (version, what-changed) tuples
- hints - a list of strings, meant to provide hints for clients/proxies. see https://github.com/redis/redis/issues/9876
- arguments - an array of arguments. each element is a map, with the possibility of nesting (sub-arguments)
- key-specs - an array of keys specs (already in unstable, just changed location)
- subcommands - a list of sub-commands (already in unstable, just changed location)
- reply-schema - will be added in the future (see https://github.com/redis/redis/issues/9845)

more details on these can be found in https://github.com/redis/redis-doc/pull/1697

only the first three fields are mandatory 

#### API changes (unreleased API obviously)

now they take RedisModuleCommand opaque pointer instead of looking up the command by name

- RM_CreateSubcommand
- RM_AddCommandKeySpec
- RM_SetCommandKeySpecBeginSearchIndex
- RM_SetCommandKeySpecBeginSearchKeyword
- RM_SetCommandKeySpecFindKeysRange
- RM_SetCommandKeySpecFindKeysKeynum

Currently, we did not add module API to provide additional information about their commands because
we couldn't agree on how the API should look like, see https://github.com/redis/redis/issues/9944

.

### Somehow related changes
1. Literals should be in uppercase while placeholder in lowercase. Now all the GEO* command
   will be documented with M|KM|FT|MI and can take both lowercase and uppercase

### Unrelated changes
1. Bugfix: no_madaory_keys was absent in COMMAND's reply
2. expose CMD_MODULE as "module" via COMMAND
3. have a dedicated uint64 for ACL categories (instead of having them in the same uint64 as command flags)
Co-authored-by: default avatarItamar Haber <itamar@garantiadata.com>
parent fbfdf513
{
"XADD": {
"summary": "Appends a new entry to a stream",
"complexity": "O(1) when adding a new entry, O(N) when trimming where N being the number of entries evicted.",
"group": "stream",
"since": "5.0.0",
"arity": -5,
"function": "xaddCommand",
"history": [
[
"6.2",
"Added the `NOMKSTREAM` option, `MINID` trimming strategy and the `LIMIT` option."
],
[
"7.0",
"Added support for the `<ms>-*` explicit ID form."
]
],
"command_flags": [
"WRITE",
"DENYOOM",
"RANDOM",
"FAST"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"WRITE"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"token": "NOMKSTREAM",
"name": "nomkstream",
"type": "pure-token",
"optional": true,
"since": "6.2"
},
{
"name": "trim",
"type": "block",
"optional": true,
"arguments": [
{
"name": "strategy",
"type": "oneof",
"arguments": [
{
"name": "maxlen",
"type": "pure-token",
"token": "MAXLEN"
},
{
"name": "minid",
"type": "pure-token",
"token": "MINID",
"since": "6.2"
}
]
},
{
"name": "operator",
"type": "oneof",
"optional": true,
"arguments": [
{
"name": "equal",
"type": "pure-token",
"token": "="
},
{
"name": "approximately",
"type": "pure-token",
"token": "~"
}
]
},
{
"name": "threshold",
"type": "string"
},
{
"token": "LIMIT",
"name": "count",
"type": "integer",
"optional": true,
"since": "6.2"
}
]
},
{
"name": "id_or_auto",
"type": "oneof",
"arguments": [
{
"name": "auto_id",
"type": "pure-token",
"token": "*"
},
{
"name": "ID",
"type": "string"
}
]
},
{
"name": "field_value",
"type": "block",
"multiple": true,
"arguments": [
{
"name": "field",
"type": "string"
},
{
"name": "value",
"type": "string"
}
]
}
]
}
}
{
"XAUTOCLAIM": {
"summary": "Changes (or acquires) ownership of messages in a consumer group, as if the messages were delivered to the specified consumer.",
"complexity": "O(1) if COUNT is small.",
"group": "stream",
"since": "6.2.0",
"arity": -6,
"function": "xautoclaimCommand",
"command_flags": [
"WRITE",
"RANDOM",
"FAST"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"WRITE"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "group",
"type": "string"
},
{
"name": "consumer",
"type": "string"
},
{
"name": "min-idle-time",
"type": "string"
},
{
"name": "start",
"type": "string"
},
{
"token": "COUNT",
"name": "count",
"type": "integer",
"optional": true
},
{
"name": "justid",
"token": "JUSTID",
"type": "pure-token",
"optional": true
}
]
}
}
{
"XCLAIM": {
"summary": "Changes (or acquires) ownership of a message in a consumer group, as if the message was delivered to the specified consumer.",
"complexity": "O(log N) with N being the number of messages in the PEL of the consumer group.",
"group": "stream",
"since": "5.0.0",
"arity": -6,
"function": "xclaimCommand",
"command_flags": [
"WRITE",
"RANDOM",
"FAST"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"WRITE"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "group",
"type": "string"
},
{
"name": "consumer",
"type": "string"
},
{
"name": "min-idle-time",
"type": "string"
},
{
"name": "ID",
"type": "string",
"multiple": true
},
{
"token": "IDLE",
"name": "ms",
"type": "integer",
"optional": true
},
{
"token": "TIME",
"name": "ms-unix-time",
"type": "integer",
"optional": true
},
{
"token": "RETRYCOUNT",
"name": "count",
"type": "integer",
"optional": true
},
{
"name": "force",
"token": "FORCE",
"type": "pure-token",
"optional": true
},
{
"name": "justid",
"token": "JUSTID",
"type": "pure-token",
"optional": true
}
]
}
}
{
"XDEL": {
"summary": "Removes the specified entries from the stream. Returns the number of items actually deleted, that may be different from the number of IDs passed in case certain IDs do not exist.",
"complexity": "O(1) for each single item to delete in the stream, regardless of the stream size.",
"group": "stream",
"since": "5.0.0",
"arity": -3,
"function": "xdelCommand",
"command_flags": [
"WRITE",
"FAST"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"WRITE"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "ID",
"type": "string",
"multiple": true
}
]
}
}
{
"CREATE": {
"summary": "Create a consumer group.",
"complexity": "O(1)",
"group": "stream",
"since": "5.0.0",
"arity": -5,
"container": "XGROUP",
"function": "xgroupCommand",
"command_flags": [
"WRITE",
"DENYOOM"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"WRITE"
],
"begin_search": {
"index": {
"pos": 2
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "groupname",
"type": "string"
},
{
"name": "id",
"type": "oneof",
"arguments": [
{
"name": "ID",
"type": "string"
},
{
"name": "new_id",
"type": "pure-token",
"token": "$"
}
]
},
{
"token": "MKSTREAM",
"name": "mkstream",
"type": "pure-token",
"optional": true
}
]
}
}
{
"CREATECONSUMER": {
"summary": "Create a consumer in a consumer group.",
"complexity": "O(1)",
"group": "stream",
"since": "6.2.0",
"arity": 5,
"container": "XGROUP",
"function": "xgroupCommand",
"command_flags": [
"WRITE",
"DENYOOM"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"WRITE"
],
"begin_search": {
"index": {
"pos": 2
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "groupname",
"type": "string"
},
{
"name": "consumername",
"type": "string"
}
]
}
}
{
"DELCONSUMER": {
"summary": "Delete a consumer from a consumer group.",
"complexity": "O(1)",
"group": "stream",
"since": "5.0.0",
"arity": 5,
"container": "XGROUP",
"function": "xgroupCommand",
"command_flags": [
"WRITE"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"WRITE"
],
"begin_search": {
"index": {
"pos": 2
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "groupname",
"type": "string"
},
{
"name": "consumername",
"type": "string"
}
]
}
}
{
"DESTROY": {
"summary": "Destroy a consumer group.",
"complexity": "O(N) where N is the number of entries in the group's pending entries list (PEL).",
"group": "stream",
"since": "5.0.0",
"arity": 4,
"container": "XGROUP",
"function": "xgroupCommand",
"command_flags": [
"WRITE"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"WRITE"
],
"begin_search": {
"index": {
"pos": 2
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "groupname",
"type": "string"
}
]
}
}
{
"HELP": {
"summary": "Show helpful text about the different subcommands",
"complexity": "O(1)",
"group": "stream",
"since": "5.0.0",
"arity": 2,
"container": "XGROUP",
"function": "xgroupCommand",
"command_flags": [
"LOADING",
"STALE"
],
"acl_categories": [
"STREAM"
]
}
}
{
"SETID": {
"summary": "Set a consumer group to an arbitrary last delivered ID value.",
"complexity": "O(1)",
"group": "stream",
"since": "5.0.0",
"arity": 5,
"container": "XGROUP",
"function": "xgroupCommand",
"command_flags": [
"WRITE"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"WRITE"
],
"begin_search": {
"index": {
"pos": 2
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "groupname",
"type": "string"
},
{
"name": "id",
"type": "oneof",
"arguments": [
{
"name": "ID",
"type": "string"
},
{
"name": "new_id",
"type": "pure-token",
"token": "$"
}
]
}
]
}
}
{
"XGROUP": {
"summary": "A container for consumer groups commands",
"complexity": "Depends on subcommand.",
"group": "stream",
"since": "5.0.0",
"arity": -2
}
}
{
"CONSUMERS": {
"summary": "List the consumers in a consumer group",
"complexity": "O(1)",
"group": "stream",
"since": "5.0.0",
"arity": 4,
"container": "XINFO",
"function": "xinfoCommand",
"command_flags": [
"READONLY",
"RANDOM"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"READ"
],
"begin_search": {
"index": {
"pos": 2
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "groupname",
"type": "string"
}
]
}
}
{
"GROUPS": {
"summary": "List the consumer groups of a stream",
"complexity": "O(1)",
"group": "stream",
"since": "5.0.0",
"arity": 3,
"container": "XINFO",
"function": "xinfoCommand",
"command_flags": [
"READONLY"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"READ"
],
"begin_search": {
"index": {
"pos": 2
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
}
]
}
}
{
"HELP": {
"summary": "Show helpful text about the different subcommands",
"complexity": "O(1)",
"group": "stream",
"since": "5.0.0",
"arity": 2,
"container": "XINFO",
"function": "xinfoCommand",
"command_flags": [
"LOADING",
"STALE"
],
"acl_categories": [
"STREAM"
]
}
}
{
"STREAM": {
"summary": "Get information about a stream",
"complexity": "O(1)",
"group": "stream",
"since": "5.0.0",
"arity": -3,
"container": "XINFO",
"function": "xinfoCommand",
"command_flags": [
"READONLY"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"READ"
],
"begin_search": {
"index": {
"pos": 2
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "full",
"token": "FULL",
"type": "block",
"optional": true,
"arguments": [
{
"token": "COUNT",
"name": "count",
"type": "integer",
"optional": true
}
]
}
]
}
}
{
"XINFO": {
"summary": "A container for stream introspection commands",
"complexity": "Depends on subcommand.",
"group": "stream",
"since": "5.0.0",
"arity": -2
}
}
{
"XLEN": {
"summary": "Return the number of entries in a stream",
"complexity": "O(1)",
"group": "stream",
"since": "5.0.0",
"arity": 2,
"function": "xlenCommand",
"command_flags": [
"READONLY",
"FAST"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"READ"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
}
]
}
}
{
"XPENDING": {
"summary": "Return information and entries from a stream consumer group pending entries list, that are messages fetched but never acknowledged.",
"complexity": "O(N) with N being the number of elements returned, so asking for a small fixed number of entries per call is O(1). O(M), where M is the total number of entries scanned when used with the IDLE filter. When the command returns just the summary and the list of consumers is small, it runs in O(1) time; otherwise, an additional O(N) time for iterating every consumer.",
"group": "stream",
"since": "5.0.0",
"arity": -3,
"function": "xpendingCommand",
"history": [
[
"6.2.0",
"Added the `IDLE` option and exclusive range intervals."
]
],
"command_flags": [
"READONLY",
"RANDOM"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"READ"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "group",
"type": "string"
},
{
"name": "filters",
"type": "block",
"optional": true,
"arguments": [
{
"token": "IDLE",
"name": "min-idle-time",
"type": "integer",
"optional": true
},
{
"name": "start",
"type": "string"
},
{
"name": "end",
"type": "string"
},
{
"name": "count",
"type": "integer"
},
{
"name": "consumer",
"type": "string",
"optional": true
}
]
}
]
}
}
{
"XRANGE": {
"summary": "Return a range of elements in a stream, with IDs matching the specified IDs interval",
"complexity": "O(N) with N being the number of elements being returned. If N is constant (e.g. always asking for the first 10 elements with COUNT), you can consider it O(1).",
"group": "stream",
"since": "5.0.0",
"arity": -4,
"function": "xrangeCommand",
"command_flags": [
"READONLY"
],
"acl_categories": [
"STREAM"
],
"key_specs": [
{
"flags": [
"READ"
],
"begin_search": {
"index": {
"pos": 1
}
},
"find_keys": {
"range": {
"lastkey": 0,
"step": 1,
"limit": 0
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "start",
"type": "string"
},
{
"name": "end",
"type": "string"
},
{
"token": "COUNT",
"name": "count",
"type": "integer",
"optional": true
}
]
}
}
{
"XREAD": {
"summary": "Return never seen elements in multiple streams, with IDs greater than the ones reported by the caller for each stream. Can block.",
"complexity": "For each stream mentioned: O(N) with N being the number of elements being returned, it means that XREAD-ing with a fixed COUNT is O(1). Note that when the BLOCK option is used, XADD will pay O(M) time in order to serve the M clients blocked on the stream getting new data.",
"group": "stream",
"since": "5.0.0",
"arity": -4,
"function": "xreadCommand",
"get_keys_function": "xreadGetKeys",
"command_flags": [
"READONLY"
],
"acl_categories": [
"STREAM",
"BLOCKING"
],
"key_specs": [
{
"flags": [
"READ"
],
"begin_search": {
"keyword": {
"keyword": "STREAMS",
"startfrom": 1
}
},
"find_keys": {
"range": {
"lastkey": -1,
"step": 1,
"limit": 2
}
}
}
],
"arguments": [
{
"token": "COUNT",
"name": "count",
"type": "integer",
"optional": true
},
{
"token": "BLOCK",
"name": "milliseconds",
"type": "integer",
"optional": true
},
{
"name": "streams",
"token": "STREAMS",
"type": "block",
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0,
"multiple": true
},
{
"name": "ID",
"type": "string",
"multiple": true
}
]
}
]
}
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment