Unverified Commit 64f61596 authored by Oran Agra's avatar Oran Agra Committed by GitHub
Browse files

Merge Redis Functions PR (#9780)

# Redis Function
This PR added the Redis Functions capabilities that were suggested on #8693.
The PR also introduce a big refactoring to the current Lua implementation
(i.e `scripting.c`). The main purpose of the refactoring is to have better
code sharing between the Lua implementation that exists today on Redis
(`scripting.c`) and the new Lua engine that is introduced on this PR.
The refactoring includes code movements and file name changes as well as some
logic changes that need to be carefully reviewed. To make the review easier,
the PR was split into multiple commits. Each commit is deeply described later on
but the main concept is that some commits are just moving code around without
making any logical changes, those commits are less likely to cause any issues
or regressions and can be reviewed fast. Other commits, which perform code and
logic changes, need to be reviewed carefully, but those commits were created
after the code movements so it's pretty easy to see what was changed. To sum up,
it is highly recommended to review this PR commit by commit as it will be easier
to see the changes, it is also recommended to read each commit description
(written below) to understand what was changed on the commit and whether or not
it's just a huge code movement or a logic changes.

## Terminology
Currently, the terminology in Redis is not clearly defined. Scripts refer to Lua
scripts and eval also refers only to Lua. Introducing Redis Function requires
redefining those terms to be able to clearly understand what is been discussed
on each context.
* eval - legacy Lua script implementation.
* Function - new scripting implementation (currently implemented in Lua but in
  the future, it might be other languages like javascript).
* Engine - the component that is responsible for executing functions.
* Script - Function or legacy Lua (executed with `eval` or `evalsha`)

## Refactoring New Structure
Today, the entire scripting logic is located on `scripting.c`. This logic can
be split into 3 main groups:
1. Script management - responsible for storing the scripts that were sent to
   Redis and retrieving them when they need to be run (base on the script sha
   on the current implementation).
2. Script invocation - invoke the script given on `eval` or `evalsha` command
   (this part includes finding the relevant script, preparing the arguments, ..)
3. Interact back with Redis (command invocation)

Those 3 groups are tightly coupled on `scripting.c`. Redis Functions also need
to use those groups logics, for example,  to interact back with Redis or to
execute Lua code. The refactoring attempts to split those 3 groups and define
APIs so that we can reuse the code both on legacy Lua scripts and Redis Functions.

In order to do so we define the following units:
1. script.c: responsible for interaction with Redis from within a script.
2. script_lua.c: responsible to execute Lua code, uses `script.c` to interact
   with Redis from within the Lua code.
3. function_lua.c: contains the Lua engine implementation, uses `script_lua.c`
   to execute the Lua code.
4. functions.c: Contains Redis Functions implementation (`FUNCTION` command,),
   uses `functions_lua.c` if the function it wants to invoke needs the Lua
   engine.
4. eval.c: the original `scripting.c` contains the Lua legacy implementation and
   was refactored to use `script_lua.c` to invoke the Lua code.

## Commits breakdown
Notice: Some small commits are omitted from this list as they are small and
insignificant (for example build fixes)

### First commit - code movements
This commit rename `scripting.c` -> `eval.c` and introduce the new `script_lua.c`
unit. The commit moves relevant code from `eval.c` (`scripting.c`) to
`script_lua.c`, the purpose of moving the code is so that later we will be able
to re-use the code on the Lua engine (`function_lua.c`). The commit only moves
the code without modifying even a single line, so there is a very low risk of
breaking anything and it also makes it much easier to see the changes on the
following commits.
Because the commit does not change the code (only moves it), it does not compile.
But we do not care about it as the only purpose here is to make the review
processes simpler.

### Second commit - move legacy Lua variables into `eval.c`
Today, all Lua-related variables are located on the server struct. The commit
attempt to identify those variable and take them out from the server struct,
leaving only script related variables (variables that later need to be used
also by engines)
The following variable where renamed and left on the server struct:
   * lua_caller 			-> script_caller
   * lua_time_limit 		-> script_time_limit
   * lua_timedout 		-> script_timedout
   * lua_oom 			-> script_oom
   * lua_disable_deny_script 	-> script_disable_deny_script
   * in_eval			-> in_script

The following variables where moved to lctx under eval.c
   * lua
   * lua_client
   * lua_cur_script
   * lua_scripts
   * lua_scripts_mem
   * lua_replicate_commands
   * lua_write_dirty
   * lua_random_dirty
   * lua_multi_emitted
   * lua_repl
   * lua_kill
   * lua_time_start
   * lua_time_snapshot

This commit is in a low risk of introducing any issues and it is just moving
variables around and not changing any logic.

### Third commit - introducing script unit
This commit introduces the `script.c` unit. Its purpose (as described above) is
to provide an API for scripts to interact with Redis. Interaction includes
mostly executing commands, but also other functionalities. The interaction is
done using a `ScriptRunCtx` object that needs to be created by the user and
initialized using `scriptPrepareForRun`. A detailed list of functionalities
expose by the unit:
1. Calling commands (including all the validation checks such as
   acl, cluster, read only run, ...)
2. Set Resp
3. Set Replication method (AOF/REPLICATION/NONE)
4. Call Redis back on long-running scripts to allow Redis to reply to clients
   and perform script kill

The commit introduces the new unit and uses it on eval commands to interact with
Redis.

### Fourth commit - Moved functionality of invoke Lua code to `script_lua.c`
This commit moves the logic of invoking the Lua code into `script_lua.c` so
later it can be used also by Lua engine (`function_lua.c`). The code is located
on `callFunction` function and assumes the Lua function already located on the
top of the Lua stack. This commit also change `eval.c` to use the new
functionality to invoke Lua code.

### Fith commit - Added Redis Functions unit (`functions.c`) and Lua engine
(`function_lua.c`)
Added Redis Functions unit under `functions.c`, included:
1. FUNCTION command:
     * FUNCTION CREATE
     * FUNCTION CALL
     * FUNCTION DELETE
     * FUNCTION KILL
     * FUNCTION INFO
     * FUNCTION STATS
2. Register engines

In addition, this commit introduces the first engine that uses the Redis
Functions capabilities, the Lua engine (`function_lua.c`)

## API Changes
### `lua-time-limit`
configuration was renamed to `script-time-limit` (keep `lua-time-limit` as alias
for backward compatibility).

### Error log changes
When integrating with Redis from within a Lua script, the `Lua` term was removed
from all the error messages and instead we write only `script`. For example:
`Wrong number of args calling Redis command From Lua script` -> `Wrong number
of args calling Redis command From script`

### `info memory` changes:
Before stating all the changes made to memory stats we will try to explain the
reason behind them and what we want to see on those metrics:
* memory metrics should show both totals (for all scripting frameworks), as well
  as a breakdown per framework / vm.
* The totals metrics should have "human" metrics while the breakdown shouldn't.
* We did try to maintain backward compatibility in some way, that said we did
  make some repurpose to existing metrics where it looks reasonable.
* We separate between memory used by the script framework (part of redis's
  used_memory), and memory used by the VM (not part of redis's used_memory)

A full breakdown of `info memory` changes:
* `used_memory_lua` and `used_memory_lua_human` was deprecated,
  `used_memory_vm_eval` has the same meaning as `used_memory_lua`
* `used_memory_scripts` was renamed to `used_memory_scripts_eval`
* `used_memory_scripts` and `used_memory_scripts_human` were repurposed and now
  return the total memory used by functions and eval (not including vm memory,
  only code cache, and structs).
* `used_memory_vm_function` was added and represents the total memory used by
  functions vm's
* `used_memory_functions` was added and represents the total memory by functions
  (not including vm memory, only code cache, and structs)
* `used_memory_vm_total` and `used_memory_vm_total_human` was added and
  represents the total memory used by vm's (functions and eval combined)

### `functions.caches`
`functions.caches` field was added to `memory stats`, representing the memory
used by engines that are not functions (this memory includes data structures
like dictionaries, arrays, ...)

## New API
### FUNCTION CREATE

Usage: FUNCTION CREATE `ENGINE` `NAME` `[REPLACE]` `[DESC <DESCRIPTION>]` `<CODE>`

* `ENGINE` - The name of the engine to use to create the script.
* `NAME` - the name of the function that can be used later to call the function
  using `FUNCTION CALL` command.
* `REPLACE` - if given, replace the given function with the existing function
  (if exists).
* `DESCRIPTION` - optional argument describing the function and what it does
* `CODE` - function code.

The command will return `OK` if created successfully or error in the following
cases:
* The given engine name does not exist
* The function name is already taken and `REPLACE` was not used.
* The given function failed on the compilation.

### FCALL and FCALL_RO

Usage: FCALL/FCALL_RO `NAME` `NUM_KEYS key1 key2` … ` arg1 arg2`

Call and execute the function specified by `NAME`. The function will receive
all arguments given after `NUM_KEYS`. The return value from the function will
be returned to the user as a result.

* `NAME` - Name of the function to run.
* The rest is as today with EVALSHA command.

The command will return an error in the following cases:
* `NAME` does not exist
* The function itself returned an error.

The `FCALL_RO` is equivalent to `EVAL_RO` and allows only read-only commands to
be invoked from the script.

### FUNCTION DELETE

Usage: FUNCTION DELETE `NAME`

Delete a function identified by `NAME`. Return `OK` on success or error on one
of the following:
* The given function does not exist

### FUNCTION INFO

Usage: FUNCTION INFO `NAME` [WITHCODE]

Return information about a function by function name:
* Function name
* Engine name
* Description
* Raw code (only if WITHCODE argument is given)

### FUNCTION LIST

Usage: FUNCTION LIST

Return general information about all the functions:
* Function name
* Engine name
* Description

### FUNCTION STATS

Usage: FUNCTION STATS

Return information about the current running function:
* Function name
* Command that was used to invoke the function
* Duration in MS that the function is already running

If no function is currently running, this section is just a RESP nil.

Additionally, return a list of all the available engines.

### FUNCTION KILL

Usage: `FUNCTION KILL`

Kill the currently executing function. The command will fail if the function
already initiated a write command.

## Notes
Note: Function creation/deletion is replicated to AOF but AOFRW is not
implemented sense its going to be removed: #9794
parents e57a4db5 cbd46317
/*
* Copyright (c) 2009-2021, Redis Ltd.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __SCRIPT_H_
#define __SCRIPT_H_
/*
* Script.c unit provides an API for functions and eval
* to interact with Redis. Interaction includes mostly
* executing commands, but also functionalities like calling
* Redis back on long scripts or check if the script was killed.
*
* The interaction is done using a scriptRunCtx object that
* need to be created by the user and initialized using scriptPrepareForRun.
*
* Detailed list of functionalities expose by the unit:
* 1. Calling commands (including all the validation checks such as
* acl, cluster, read only run, ...)
* 2. Set Resp
* 3. Set Replication method (AOF/REPLICATION/NONE)
* 4. Call Redis back to on long running scripts to allow Redis reply
* to clients and perform script kill
*/
/*
* scriptInterrupt function will return one of those value,
*
* - SCRIPT_KILL - kill the current running script.
* - SCRIPT_CONTINUE - keep running the current script.
*/
#define SCRIPT_KILL 1
#define SCRIPT_CONTINUE 2
/* runCtx flags */
#define SCRIPT_WRITE_DIRTY (1ULL<<0) /* indicate that the current script already performed a write command */
#define SCRIPT_RANDOM_DIRTY (1ULL<<1) /* indicate that the current script already performed a random reply command.
Thanks to this flag we'll raise an error every time a write command
is called after a random command and prevent none deterministic
replication or AOF. */
#define SCRIPT_MULTI_EMMITED (1ULL<<2) /* indicate that we already wrote a multi command to replication/aof */
#define SCRIPT_TIMEDOUT (1ULL<<3) /* indicate that the current script timedout */
#define SCRIPT_KILLED (1ULL<<4) /* indicate that the current script was marked to be killed */
#define SCRIPT_READ_ONLY (1ULL<<5) /* indicate that the current script should only perform read commands */
#define SCRIPT_EVAL_REPLICATION (1ULL<<6) /* mode for eval, indicate that we replicate the
script invocation and not the effects */
#define SCRIPT_EVAL_MODE (1ULL<<7) /* Indicate that the current script called from legacy Lua */
typedef struct scriptRunCtx scriptRunCtx;
struct scriptRunCtx {
const char *funcname;
client *c;
client *original_client;
int flags;
int repl_flags;
monotime start_time;
mstime_t snapshot_time;
};
void scriptPrepareForRun(scriptRunCtx *r_ctx, client *engine_client, client *caller, const char *funcname);
void scriptResetRun(scriptRunCtx *r_ctx);
int scriptSetResp(scriptRunCtx *r_ctx, int resp);
int scriptSetRepl(scriptRunCtx *r_ctx, int repl);
void scriptCall(scriptRunCtx *r_ctx, robj **argv, int argc, sds *err);
int scriptInterrupt(scriptRunCtx *r_ctx);
void scriptKill(client *c, int is_eval);
int scriptIsRunning();
const char* scriptCurrFunction();
int scriptIsEval();
int scriptIsTimedout();
client* scriptGetClient();
client* scriptGetCaller();
mstime_t scriptTimeSnapshot();
long long scriptRunDuration();
#endif /* __SCRIPT_H_ */
/*
* Copyright (c) 2009-2021, Redis Ltd.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "script_lua.h"
#include "server.h"
#include "sha1.h"
#include "rand.h"
#include "cluster.h"
#include "monotonic.h"
#include "resp_parser.h"
#include <lauxlib.h>
#include <lualib.h>
#include <ctype.h>
#include <math.h>
static int redis_math_random (lua_State *L);
static int redis_math_randomseed (lua_State *L);
static void redisProtocolToLuaType_Int(void *ctx, long long val, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_BulkString(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_NullBulkString(void *ctx, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_NullArray(void *ctx, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_Status(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_Error(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_Array(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
static void redisProtocolToLuaType_Map(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
static void redisProtocolToLuaType_Set(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
static void redisProtocolToLuaType_Null(void *ctx, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_Bool(void *ctx, int val, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_Double(void *ctx, double d, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_BigNumber(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_VerbatimString(void *ctx, const char *format, const char *str, size_t len, const char *proto, size_t proto_len);
static void redisProtocolToLuaType_Attribute(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
static void luaReplyToRedisReply(client *c, client* script_client, lua_State *lua);
/*
* Save the give pointer on Lua registry, used to save the Lua context and
* function context so we can retrieve them from lua_State.
*/
void luaSaveOnRegistry(lua_State* lua, const char* name, void* ptr) {
lua_pushstring(lua, name);
if (ptr) {
lua_pushlightuserdata(lua, ptr);
} else {
lua_pushnil(lua);
}
lua_settable(lua, LUA_REGISTRYINDEX);
}
/*
* Get a saved pointer from registry
*/
void* luaGetFromRegistry(lua_State* lua, const char* name) {
lua_pushstring(lua, name);
lua_gettable(lua, LUA_REGISTRYINDEX);
/* must be light user data */
serverAssert(lua_islightuserdata(lua, -1));
void* ptr = (void*) lua_topointer(lua, -1);
serverAssert(ptr);
/* pops the value */
lua_pop(lua, 1);
return ptr;
}
/* ---------------------------------------------------------------------------
* Redis reply to Lua type conversion functions.
* ------------------------------------------------------------------------- */
/* Take a Redis reply in the Redis protocol format and convert it into a
* Lua type. Thanks to this function, and the introduction of not connected
* clients, it is trivial to implement the redis() lua function.
*
* Basically we take the arguments, execute the Redis command in the context
* of a non connected client, then take the generated reply and convert it
* into a suitable Lua type. With this trick the scripting feature does not
* need the introduction of a full Redis internals API. The script
* is like a normal client that bypasses all the slow I/O paths.
*
* Note: in this function we do not do any sanity check as the reply is
* generated by Redis directly. This allows us to go faster.
*
* Errors are returned as a table with a single 'err' field set to the
* error string.
*/
static const ReplyParserCallbacks DefaultLuaTypeParserCallbacks = {
.null_array_callback = redisProtocolToLuaType_NullArray,
.bulk_string_callback = redisProtocolToLuaType_BulkString,
.null_bulk_string_callback = redisProtocolToLuaType_NullBulkString,
.error_callback = redisProtocolToLuaType_Error,
.simple_str_callback = redisProtocolToLuaType_Status,
.long_callback = redisProtocolToLuaType_Int,
.array_callback = redisProtocolToLuaType_Array,
.set_callback = redisProtocolToLuaType_Set,
.map_callback = redisProtocolToLuaType_Map,
.bool_callback = redisProtocolToLuaType_Bool,
.double_callback = redisProtocolToLuaType_Double,
.null_callback = redisProtocolToLuaType_Null,
.big_number_callback = redisProtocolToLuaType_BigNumber,
.verbatim_string_callback = redisProtocolToLuaType_VerbatimString,
.attribute_callback = redisProtocolToLuaType_Attribute,
.error = NULL,
};
static void redisProtocolToLuaType(lua_State *lua, char* reply) {
ReplyParser parser = {.curr_location = reply, .callbacks = DefaultLuaTypeParserCallbacks};
parseReply(&parser, lua);
}
static void redisProtocolToLuaType_Int(void *ctx, long long val, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 1)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_pushnumber(lua,(lua_Number)val);
}
static void redisProtocolToLuaType_NullBulkString(void *ctx, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 1)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_pushboolean(lua,0);
}
static void redisProtocolToLuaType_NullArray(void *ctx, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 1)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_pushboolean(lua,0);
}
static void redisProtocolToLuaType_BulkString(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 1)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_pushlstring(lua,str,len);
}
static void redisProtocolToLuaType_Status(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 3)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_newtable(lua);
lua_pushstring(lua,"ok");
lua_pushlstring(lua,str,len);
lua_settable(lua,-3);
}
static void redisProtocolToLuaType_Error(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 3)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_newtable(lua);
lua_pushstring(lua,"err");
lua_pushlstring(lua,str,len);
lua_settable(lua,-3);
}
static void redisProtocolToLuaType_Map(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
UNUSED(proto);
lua_State *lua = ctx;
if (lua) {
if (!lua_checkstack(lua, 3)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_newtable(lua);
lua_pushstring(lua, "map");
lua_newtable(lua);
}
for (size_t j = 0; j < len; j++) {
parseReply(parser,lua);
parseReply(parser,lua);
if (lua) lua_settable(lua,-3);
}
if (lua) lua_settable(lua,-3);
}
static void redisProtocolToLuaType_Set(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
UNUSED(proto);
lua_State *lua = ctx;
if (lua) {
if (!lua_checkstack(lua, 3)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_newtable(lua);
lua_pushstring(lua, "set");
lua_newtable(lua);
}
for (size_t j = 0; j < len; j++) {
parseReply(parser,lua);
if (lua) {
if (!lua_checkstack(lua, 1)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic.
* Notice that here we need to check the stack again because the recursive
* call to redisProtocolToLuaType might have use the room allocated in the stack*/
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_pushboolean(lua,1);
lua_settable(lua,-3);
}
}
if (lua) lua_settable(lua,-3);
}
static void redisProtocolToLuaType_Array(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
UNUSED(proto);
lua_State *lua = ctx;
if (lua){
if (!lua_checkstack(lua, 2)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_newtable(lua);
}
for (size_t j = 0; j < len; j++) {
if (lua) lua_pushnumber(lua,j+1);
parseReply(parser,lua);
if (lua) lua_settable(lua,-3);
}
}
static void redisProtocolToLuaType_Attribute(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
UNUSED(proto);
/* Parse the attribute reply.
* Currently, we do not expose the attribute to the Lua script so
* we just need to continue parsing and ignore it (the NULL ensures that the
* reply will be ignored). */
for (size_t j = 0; j < len; j++) {
parseReply(parser,NULL);
parseReply(parser,NULL);
}
/* Parse the reply itself. */
parseReply(parser,ctx);
}
static void redisProtocolToLuaType_VerbatimString(void *ctx, const char *format, const char *str, size_t len, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 5)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_newtable(lua);
lua_pushstring(lua,"verbatim_string");
lua_newtable(lua);
lua_pushstring(lua,"string");
lua_pushlstring(lua,str,len);
lua_settable(lua,-3);
lua_pushstring(lua,"format");
lua_pushlstring(lua,format,3);
lua_settable(lua,-3);
lua_settable(lua,-3);
}
static void redisProtocolToLuaType_BigNumber(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 3)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_newtable(lua);
lua_pushstring(lua,"big_number");
lua_pushlstring(lua,str,len);
lua_settable(lua,-3);
}
static void redisProtocolToLuaType_Null(void *ctx, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 1)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_pushnil(lua);
}
static void redisProtocolToLuaType_Bool(void *ctx, int val, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 1)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_pushboolean(lua,val);
}
static void redisProtocolToLuaType_Double(void *ctx, double d, const char *proto, size_t proto_len) {
UNUSED(proto);
UNUSED(proto_len);
if (!ctx) {
return;
}
lua_State *lua = ctx;
if (!lua_checkstack(lua, 3)) {
/* Increase the Lua stack if needed, to make sure there is enough room
* to push elements to the stack. On failure, exit with panic. */
serverPanic("lua stack limit reach when parsing redis.call reply");
}
lua_newtable(lua);
lua_pushstring(lua,"double");
lua_pushnumber(lua,d);
lua_settable(lua,-3);
}
/* This function is used in order to push an error on the Lua stack in the
* format used by redis.pcall to return errors, which is a lua table
* with a single "err" field set to the error string. Note that this
* table is never a valid reply by proper commands, since the returned
* tables are otherwise always indexed by integers, never by strings. */
static void luaPushError(lua_State *lua, char *error) {
lua_Debug dbg;
/* If debugging is active and in step mode, log errors resulting from
* Redis commands. */
if (ldbIsEnabled()) {
ldbLog(sdscatprintf(sdsempty(),"<error> %s",error));
}
lua_newtable(lua);
lua_pushstring(lua,"err");
/* Attempt to figure out where this function was called, if possible */
if(lua_getstack(lua, 1, &dbg) && lua_getinfo(lua, "nSl", &dbg)) {
sds msg = sdscatprintf(sdsempty(), "%s: %d: %s",
dbg.source, dbg.currentline, error);
lua_pushstring(lua, msg);
sdsfree(msg);
} else {
lua_pushstring(lua, error);
}
lua_settable(lua,-3);
}
/* In case the error set into the Lua stack by luaPushError() was generated
* by the non-error-trapping version of redis.pcall(), which is redis.call(),
* this function will raise the Lua error so that the execution of the
* script will be halted. */
static int luaRaiseError(lua_State *lua) {
lua_pushstring(lua,"err");
lua_gettable(lua,-2);
return lua_error(lua);
}
/* Sort the array currently in the stack. We do this to make the output
* of commands like KEYS or SMEMBERS something deterministic when called
* from Lua (to play well with AOf/replication).
*
* The array is sorted using table.sort itself, and assuming all the
* list elements are strings. */
static void luaSortArray(lua_State *lua) {
/* Initial Stack: array */
lua_getglobal(lua,"table");
lua_pushstring(lua,"sort");
lua_gettable(lua,-2); /* Stack: array, table, table.sort */
lua_pushvalue(lua,-3); /* Stack: array, table, table.sort, array */
if (lua_pcall(lua,1,0,0)) {
/* Stack: array, table, error */
/* We are not interested in the error, we assume that the problem is
* that there are 'false' elements inside the array, so we try
* again with a slower function but able to handle this case, that
* is: table.sort(table, __redis__compare_helper) */
lua_pop(lua,1); /* Stack: array, table */
lua_pushstring(lua,"sort"); /* Stack: array, table, sort */
lua_gettable(lua,-2); /* Stack: array, table, table.sort */
lua_pushvalue(lua,-3); /* Stack: array, table, table.sort, array */
lua_getglobal(lua,"__redis__compare_helper");
/* Stack: array, table, table.sort, array, __redis__compare_helper */
lua_call(lua,2,0);
}
/* Stack: array (sorted), table */
lua_pop(lua,1); /* Stack: array (sorted) */
}
/* ---------------------------------------------------------------------------
* Lua reply to Redis reply conversion functions.
* ------------------------------------------------------------------------- */
/* Reply to client 'c' converting the top element in the Lua stack to a
* Redis reply. As a side effect the element is consumed from the stack. */
static void luaReplyToRedisReply(client *c, client* script_client, lua_State *lua) {
int t = lua_type(lua,-1);
if (!lua_checkstack(lua, 4)) {
/* Increase the Lua stack if needed to make sure there is enough room
* to push 4 elements to the stack. On failure, return error.
* Notice that we need, in the worst case, 4 elements because returning a map might
* require push 4 elements to the Lua stack.*/
addReplyErrorFormat(c, "reached lua stack limit");
lua_pop(lua,1); /* pop the element from the stack */
return;
}
switch(t) {
case LUA_TSTRING:
addReplyBulkCBuffer(c,(char*)lua_tostring(lua,-1),lua_strlen(lua,-1));
break;
case LUA_TBOOLEAN:
if (script_client->resp == 2)
addReply(c,lua_toboolean(lua,-1) ? shared.cone :
shared.null[c->resp]);
else
addReplyBool(c,lua_toboolean(lua,-1));
break;
case LUA_TNUMBER:
addReplyLongLong(c,(long long)lua_tonumber(lua,-1));
break;
case LUA_TTABLE:
/* We need to check if it is an array, an error, or a status reply.
* Error are returned as a single element table with 'err' field.
* Status replies are returned as single element table with 'ok'
* field. */
/* Handle error reply. */
/* we took care of the stack size on function start */
lua_pushstring(lua,"err");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TSTRING) {
addReplyErrorFormat(c,"-%s",lua_tostring(lua,-1));
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle status reply. */
lua_pushstring(lua,"ok");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TSTRING) {
sds ok = sdsnew(lua_tostring(lua,-1));
sdsmapchars(ok,"\r\n"," ",2);
addReplySds(c,sdscatprintf(sdsempty(),"+%s\r\n",ok));
sdsfree(ok);
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle double reply. */
lua_pushstring(lua,"double");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TNUMBER) {
addReplyDouble(c,lua_tonumber(lua,-1));
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle big number reply. */
lua_pushstring(lua,"big_number");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TSTRING) {
sds big_num = sdsnewlen(lua_tostring(lua,-1), lua_strlen(lua,-1));
sdsmapchars(big_num,"\r\n"," ",2);
addReplyBigNum(c,big_num,sdslen(big_num));
sdsfree(big_num);
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle verbatim reply. */
lua_pushstring(lua,"verbatim_string");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TTABLE) {
lua_pushstring(lua,"format");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TSTRING){
char* format = (char*)lua_tostring(lua,-1);
lua_pushstring(lua,"string");
lua_gettable(lua,-3);
t = lua_type(lua,-1);
if (t == LUA_TSTRING){
size_t len;
char* str = (char*)lua_tolstring(lua,-1,&len);
addReplyVerbatim(c, str, len, format);
lua_pop(lua,4);
return;
}
lua_pop(lua,1);
}
lua_pop(lua,1);
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle map reply. */
lua_pushstring(lua,"map");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TTABLE) {
int maplen = 0;
void *replylen = addReplyDeferredLen(c);
/* we took care of the stack size on function start */
lua_pushnil(lua); /* Use nil to start iteration. */
while (lua_next(lua,-2)) {
/* Stack now: table, key, value */
lua_pushvalue(lua,-2); /* Dup key before consuming. */
luaReplyToRedisReply(c, script_client, lua); /* Return key. */
luaReplyToRedisReply(c, script_client, lua); /* Return value. */
/* Stack now: table, key. */
maplen++;
}
setDeferredMapLen(c,replylen,maplen);
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle set reply. */
lua_pushstring(lua,"set");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TTABLE) {
int setlen = 0;
void *replylen = addReplyDeferredLen(c);
/* we took care of the stack size on function start */
lua_pushnil(lua); /* Use nil to start iteration. */
while (lua_next(lua,-2)) {
/* Stack now: table, key, true */
lua_pop(lua,1); /* Discard the boolean value. */
lua_pushvalue(lua,-1); /* Dup key before consuming. */
luaReplyToRedisReply(c, script_client, lua); /* Return key. */
/* Stack now: table, key. */
setlen++;
}
setDeferredSetLen(c,replylen,setlen);
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle the array reply. */
void *replylen = addReplyDeferredLen(c);
int j = 1, mbulklen = 0;
while(1) {
/* we took care of the stack size on function start */
lua_pushnumber(lua,j++);
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TNIL) {
lua_pop(lua,1);
break;
}
luaReplyToRedisReply(c, script_client, lua);
mbulklen++;
}
setDeferredArrayLen(c,replylen,mbulklen);
break;
default:
addReplyNull(c);
}
lua_pop(lua,1);
}
/* ---------------------------------------------------------------------------
* Lua redis.* functions implementations.
* ------------------------------------------------------------------------- */
#define LUA_CMD_OBJCACHE_SIZE 32
#define LUA_CMD_OBJCACHE_MAX_LEN 64
static int luaRedisGenericCommand(lua_State *lua, int raise_error) {
int j, argc = lua_gettop(lua);
scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
sds err = NULL;
client* c = rctx->c;
sds reply;
/* Cached across calls. */
static robj **argv = NULL;
static int argv_size = 0;
static robj *cached_objects[LUA_CMD_OBJCACHE_SIZE];
static size_t cached_objects_len[LUA_CMD_OBJCACHE_SIZE];
static int inuse = 0; /* Recursive calls detection. */
/* By using Lua debug hooks it is possible to trigger a recursive call
* to luaRedisGenericCommand(), which normally should never happen.
* To make this function reentrant is futile and makes it slower, but
* we should at least detect such a misuse, and abort. */
if (inuse) {
char *recursion_warning =
"luaRedisGenericCommand() recursive call detected. "
"Are you doing funny stuff with Lua debug hooks?";
serverLog(LL_WARNING,"%s",recursion_warning);
luaPushError(lua,recursion_warning);
return 1;
}
inuse++;
/* Require at least one argument */
if (argc == 0) {
luaPushError(lua,
"Please specify at least one argument for redis.call()");
inuse--;
return raise_error ? luaRaiseError(lua) : 1;
}
/* Build the arguments vector */
if (argv_size < argc) {
argv = zrealloc(argv,sizeof(robj*)*argc);
argv_size = argc;
}
for (j = 0; j < argc; j++) {
char *obj_s;
size_t obj_len;
char dbuf[64];
if (lua_type(lua,j+1) == LUA_TNUMBER) {
/* We can't use lua_tolstring() for number -> string conversion
* since Lua uses a format specifier that loses precision. */
lua_Number num = lua_tonumber(lua,j+1);
obj_len = snprintf(dbuf,sizeof(dbuf),"%.17g",(double)num);
obj_s = dbuf;
} else {
obj_s = (char*)lua_tolstring(lua,j+1,&obj_len);
if (obj_s == NULL) break; /* Not a string. */
}
/* Try to use a cached object. */
if (j < LUA_CMD_OBJCACHE_SIZE && cached_objects[j] &&
cached_objects_len[j] >= obj_len)
{
sds s = cached_objects[j]->ptr;
argv[j] = cached_objects[j];
cached_objects[j] = NULL;
memcpy(s,obj_s,obj_len+1);
sdssetlen(s, obj_len);
} else {
argv[j] = createStringObject(obj_s, obj_len);
}
}
/* Check if one of the arguments passed by the Lua script
* is not a string or an integer (lua_isstring() return true for
* integers as well). */
if (j != argc) {
j--;
while (j >= 0) {
decrRefCount(argv[j]);
j--;
}
luaPushError(lua,
"Lua redis() command arguments must be strings or integers");
inuse--;
return raise_error ? luaRaiseError(lua) : 1;
}
/* Pop all arguments from the stack, we do not need them anymore
* and this way we guaranty we will have room on the stack for the result. */
lua_pop(lua, argc);
/* Log the command if debugging is active. */
if (ldbIsEnabled()) {
sds cmdlog = sdsnew("<redis>");
for (j = 0; j < c->argc; j++) {
if (j == 10) {
cmdlog = sdscatprintf(cmdlog," ... (%d more)",
c->argc-j-1);
break;
} else {
cmdlog = sdscatlen(cmdlog," ",1);
cmdlog = sdscatsds(cmdlog,c->argv[j]->ptr);
}
}
ldbLog(cmdlog);
}
scriptCall(rctx, argv, argc, &err);
if (err) {
luaPushError(lua, err);
sdsfree(err);
goto cleanup;
}
/* Convert the result of the Redis command into a suitable Lua type.
* The first thing we need is to create a single string from the client
* output buffers. */
if (listLength(c->reply) == 0 && (size_t)c->bufpos < c->buf_usable_size) {
/* This is a fast path for the common case of a reply inside the
* client static buffer. Don't create an SDS string but just use
* the client buffer directly. */
c->buf[c->bufpos] = '\0';
reply = c->buf;
c->bufpos = 0;
} else {
reply = sdsnewlen(c->buf,c->bufpos);
c->bufpos = 0;
while(listLength(c->reply)) {
clientReplyBlock *o = listNodeValue(listFirst(c->reply));
reply = sdscatlen(reply,o->buf,o->used);
listDelNode(c->reply,listFirst(c->reply));
}
}
if (raise_error && reply[0] != '-') raise_error = 0;
redisProtocolToLuaType(lua,reply);
/* If the debugger is active, log the reply from Redis. */
if (ldbIsEnabled())
ldbLogRedisReply(reply);
/* Sort the output array if needed, assuming it is a non-null multi bulk
* reply as expected. */
if ((c->cmd->flags & CMD_SORT_FOR_SCRIPT) &&
(rctx->flags & SCRIPT_EVAL_REPLICATION) &&
(reply[0] == '*' && reply[1] != '-')) {
luaSortArray(lua);
}
if (reply != c->buf) sdsfree(reply);
c->reply_bytes = 0;
cleanup:
/* Clean up. Command code may have changed argv/argc so we use the
* argv/argc of the client instead of the local variables. */
for (j = 0; j < c->argc; j++) {
robj *o = c->argv[j];
/* Try to cache the object in the cached_objects array.
* The object must be small, SDS-encoded, and with refcount = 1
* (we must be the only owner) for us to cache it. */
if (j < LUA_CMD_OBJCACHE_SIZE &&
o->refcount == 1 &&
(o->encoding == OBJ_ENCODING_RAW ||
o->encoding == OBJ_ENCODING_EMBSTR) &&
sdslen(o->ptr) <= LUA_CMD_OBJCACHE_MAX_LEN)
{
sds s = o->ptr;
if (cached_objects[j]) decrRefCount(cached_objects[j]);
cached_objects[j] = o;
cached_objects_len[j] = sdsalloc(s);
} else {
decrRefCount(o);
}
}
if (c->argv != argv) {
zfree(c->argv);
argv = NULL;
argv_size = 0;
}
c->user = NULL;
c->argv = NULL;
c->argc = 0;
if (raise_error) {
/* If we are here we should have an error in the stack, in the
* form of a table with an "err" field. Extract the string to
* return the plain error. */
inuse--;
return luaRaiseError(lua);
}
inuse--;
return 1;
}
/* redis.call() */
static int luaRedisCallCommand(lua_State *lua) {
return luaRedisGenericCommand(lua,1);
}
/* redis.pcall() */
static int luaRedisPCallCommand(lua_State *lua) {
return luaRedisGenericCommand(lua,0);
}
/* This adds redis.sha1hex(string) to Lua scripts using the same hashing
* function used for sha1ing lua scripts. */
static int luaRedisSha1hexCommand(lua_State *lua) {
int argc = lua_gettop(lua);
char digest[41];
size_t len;
char *s;
if (argc != 1) {
lua_pushstring(lua, "wrong number of arguments");
return lua_error(lua);
}
s = (char*)lua_tolstring(lua,1,&len);
sha1hex(digest,s,len);
lua_pushstring(lua,digest);
return 1;
}
/* Returns a table with a single field 'field' set to the string value
* passed as argument. This helper function is handy when returning
* a Redis Protocol error or status reply from Lua:
*
* return redis.error_reply("ERR Some Error")
* return redis.status_reply("ERR Some Error")
*/
static int luaRedisReturnSingleFieldTable(lua_State *lua, char *field) {
if (lua_gettop(lua) != 1 || lua_type(lua,-1) != LUA_TSTRING) {
luaPushError(lua, "wrong number or type of arguments");
return 1;
}
lua_newtable(lua);
lua_pushstring(lua, field);
lua_pushvalue(lua, -3);
lua_settable(lua, -3);
return 1;
}
/* redis.error_reply() */
static int luaRedisErrorReplyCommand(lua_State *lua) {
return luaRedisReturnSingleFieldTable(lua,"err");
}
/* redis.status_reply() */
static int luaRedisStatusReplyCommand(lua_State *lua) {
return luaRedisReturnSingleFieldTable(lua,"ok");
}
/* redis.set_repl()
*
* Set the propagation of write commands executed in the context of the
* script to on/off for AOF and slaves. */
static int luaRedisSetReplCommand(lua_State *lua) {
int argc = lua_gettop(lua);
int flags;
scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
if (rctx->flags & SCRIPT_EVAL_REPLICATION) {
lua_pushstring(lua, "You can set the replication behavior only after turning on single commands replication with redis.replicate_commands().");
return lua_error(lua);
} else if (argc != 1) {
lua_pushstring(lua, "redis.set_repl() requires two arguments.");
return lua_error(lua);
}
flags = lua_tonumber(lua,-1);
if ((flags & ~(PROPAGATE_AOF|PROPAGATE_REPL)) != 0) {
lua_pushstring(lua, "Invalid replication flags. Use REPL_AOF, REPL_REPLICA, REPL_ALL or REPL_NONE.");
return lua_error(lua);
}
scriptSetRepl(rctx, flags);
return 0;
}
/* redis.log() */
static int luaLogCommand(lua_State *lua) {
int j, argc = lua_gettop(lua);
int level;
sds log;
if (argc < 2) {
lua_pushstring(lua, "redis.log() requires two arguments or more.");
return lua_error(lua);
} else if (!lua_isnumber(lua,-argc)) {
lua_pushstring(lua, "First argument must be a number (log level).");
return lua_error(lua);
}
level = lua_tonumber(lua,-argc);
if (level < LL_DEBUG || level > LL_WARNING) {
lua_pushstring(lua, "Invalid debug level.");
return lua_error(lua);
}
if (level < server.verbosity) return 0;
/* Glue together all the arguments */
log = sdsempty();
for (j = 1; j < argc; j++) {
size_t len;
char *s;
s = (char*)lua_tolstring(lua,(-argc)+j,&len);
if (s) {
if (j != 1) log = sdscatlen(log," ",1);
log = sdscatlen(log,s,len);
}
}
serverLogRaw(level,log);
sdsfree(log);
return 0;
}
/* redis.setresp() */
static int luaSetResp(lua_State *lua) {
int argc = lua_gettop(lua);
if (argc != 1) {
lua_pushstring(lua, "redis.setresp() requires one argument.");
return lua_error(lua);
}
int resp = lua_tonumber(lua,-argc);
if (resp != 2 && resp != 3) {
lua_pushstring(lua, "RESP version must be 2 or 3.");
return lua_error(lua);
}
scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
scriptSetResp(rctx, resp);
return 0;
}
/* ---------------------------------------------------------------------------
* Lua engine initialization and reset.
* ------------------------------------------------------------------------- */
static void luaLoadLib(lua_State *lua, const char *libname, lua_CFunction luafunc) {
lua_pushcfunction(lua, luafunc);
lua_pushstring(lua, libname);
lua_call(lua, 1, 0);
}
LUALIB_API int (luaopen_cjson) (lua_State *L);
LUALIB_API int (luaopen_struct) (lua_State *L);
LUALIB_API int (luaopen_cmsgpack) (lua_State *L);
LUALIB_API int (luaopen_bit) (lua_State *L);
static void luaLoadLibraries(lua_State *lua) {
luaLoadLib(lua, "", luaopen_base);
luaLoadLib(lua, LUA_TABLIBNAME, luaopen_table);
luaLoadLib(lua, LUA_STRLIBNAME, luaopen_string);
luaLoadLib(lua, LUA_MATHLIBNAME, luaopen_math);
luaLoadLib(lua, LUA_DBLIBNAME, luaopen_debug);
luaLoadLib(lua, "cjson", luaopen_cjson);
luaLoadLib(lua, "struct", luaopen_struct);
luaLoadLib(lua, "cmsgpack", luaopen_cmsgpack);
luaLoadLib(lua, "bit", luaopen_bit);
#if 0 /* Stuff that we don't load currently, for sandboxing concerns. */
luaLoadLib(lua, LUA_LOADLIBNAME, luaopen_package);
luaLoadLib(lua, LUA_OSLIBNAME, luaopen_os);
#endif
}
/* Remove a functions that we don't want to expose to the Redis scripting
* environment. */
static void luaRemoveUnsupportedFunctions(lua_State *lua) {
lua_pushnil(lua);
lua_setglobal(lua,"loadfile");
lua_pushnil(lua);
lua_setglobal(lua,"dofile");
}
/* This function installs metamethods in the global table _G that prevent
* the creation of globals accidentally.
*
* It should be the last to be called in the scripting engine initialization
* sequence, because it may interact with creation of globals.
*
* On Legacy Lua (eval) we need to check 'w ~= \"main\"' otherwise we will not be able
* to create the global 'function <sha> ()' variable. On Lua engine we do not use this trick
* so its not needed. */
void luaEnableGlobalsProtection(lua_State *lua, int is_eval) {
char *s[32];
sds code = sdsempty();
int j = 0;
/* strict.lua from: http://metalua.luaforge.net/src/lib/strict.lua.html.
* Modified to be adapted to Redis. */
s[j++]="local dbg=debug\n";
s[j++]="local mt = {}\n";
s[j++]="setmetatable(_G, mt)\n";
s[j++]="mt.__newindex = function (t, n, v)\n";
s[j++]=" if dbg.getinfo(2) then\n";
s[j++]=" local w = dbg.getinfo(2, \"S\").what\n";
s[j++]= is_eval ? " if w ~= \"main\" and w ~= \"C\" then\n" : " if w ~= \"C\" then\n";
s[j++]=" error(\"Script attempted to create global variable '\"..tostring(n)..\"'\", 2)\n";
s[j++]=" end\n";
s[j++]=" end\n";
s[j++]=" rawset(t, n, v)\n";
s[j++]="end\n";
s[j++]="mt.__index = function (t, n)\n";
s[j++]=" if dbg.getinfo(2) and dbg.getinfo(2, \"S\").what ~= \"C\" then\n";
s[j++]=" error(\"Script attempted to access nonexistent global variable '\"..tostring(n)..\"'\", 2)\n";
s[j++]=" end\n";
s[j++]=" return rawget(t, n)\n";
s[j++]="end\n";
s[j++]="debug = nil\n";
s[j++]=NULL;
for (j = 0; s[j] != NULL; j++) code = sdscatlen(code,s[j],strlen(s[j]));
luaL_loadbuffer(lua,code,sdslen(code),"@enable_strict_lua");
lua_pcall(lua,0,0,0);
sdsfree(code);
}
void luaRegisterRedisAPI(lua_State* lua) {
luaLoadLibraries(lua);
luaRemoveUnsupportedFunctions(lua);
/* Register the redis commands table and fields */
lua_newtable(lua);
/* redis.call */
lua_pushstring(lua,"call");
lua_pushcfunction(lua,luaRedisCallCommand);
lua_settable(lua,-3);
/* redis.pcall */
lua_pushstring(lua,"pcall");
lua_pushcfunction(lua,luaRedisPCallCommand);
lua_settable(lua,-3);
/* redis.log and log levels. */
lua_pushstring(lua,"log");
lua_pushcfunction(lua,luaLogCommand);
lua_settable(lua,-3);
/* redis.setresp */
lua_pushstring(lua,"setresp");
lua_pushcfunction(lua,luaSetResp);
lua_settable(lua,-3);
lua_pushstring(lua,"LOG_DEBUG");
lua_pushnumber(lua,LL_DEBUG);
lua_settable(lua,-3);
lua_pushstring(lua,"LOG_VERBOSE");
lua_pushnumber(lua,LL_VERBOSE);
lua_settable(lua,-3);
lua_pushstring(lua,"LOG_NOTICE");
lua_pushnumber(lua,LL_NOTICE);
lua_settable(lua,-3);
lua_pushstring(lua,"LOG_WARNING");
lua_pushnumber(lua,LL_WARNING);
lua_settable(lua,-3);
/* redis.sha1hex */
lua_pushstring(lua, "sha1hex");
lua_pushcfunction(lua, luaRedisSha1hexCommand);
lua_settable(lua, -3);
/* redis.error_reply and redis.status_reply */
lua_pushstring(lua, "error_reply");
lua_pushcfunction(lua, luaRedisErrorReplyCommand);
lua_settable(lua, -3);
lua_pushstring(lua, "status_reply");
lua_pushcfunction(lua, luaRedisStatusReplyCommand);
lua_settable(lua, -3);
/* redis.set_repl and associated flags. */
lua_pushstring(lua,"set_repl");
lua_pushcfunction(lua,luaRedisSetReplCommand);
lua_settable(lua,-3);
lua_pushstring(lua,"REPL_NONE");
lua_pushnumber(lua,PROPAGATE_NONE);
lua_settable(lua,-3);
lua_pushstring(lua,"REPL_AOF");
lua_pushnumber(lua,PROPAGATE_AOF);
lua_settable(lua,-3);
lua_pushstring(lua,"REPL_SLAVE");
lua_pushnumber(lua,PROPAGATE_REPL);
lua_settable(lua,-3);
lua_pushstring(lua,"REPL_REPLICA");
lua_pushnumber(lua,PROPAGATE_REPL);
lua_settable(lua,-3);
lua_pushstring(lua,"REPL_ALL");
lua_pushnumber(lua,PROPAGATE_AOF|PROPAGATE_REPL);
lua_settable(lua,-3);
/* Finally set the table as 'redis' global var. */
lua_setglobal(lua,"redis");
/* Replace math.random and math.randomseed with our implementations. */
lua_getglobal(lua,"math");
lua_pushstring(lua,"random");
lua_pushcfunction(lua,redis_math_random);
lua_settable(lua,-3);
lua_pushstring(lua,"randomseed");
lua_pushcfunction(lua,redis_math_randomseed);
lua_settable(lua,-3);
lua_setglobal(lua,"math");
}
/* Set an array of Redis String Objects as a Lua array (table) stored into a
* global variable. */
static void luaSetGlobalArray(lua_State *lua, char *var, robj **elev, int elec) {
int j;
lua_newtable(lua);
for (j = 0; j < elec; j++) {
lua_pushlstring(lua,(char*)elev[j]->ptr,sdslen(elev[j]->ptr));
lua_rawseti(lua,-2,j+1);
}
lua_setglobal(lua,var);
}
/* ---------------------------------------------------------------------------
* Redis provided math.random
* ------------------------------------------------------------------------- */
/* We replace math.random() with our implementation that is not affected
* by specific libc random() implementations and will output the same sequence
* (for the same seed) in every arch. */
/* The following implementation is the one shipped with Lua itself but with
* rand() replaced by redisLrand48(). */
static int redis_math_random (lua_State *L) {
/* the `%' avoids the (rare) case of r==1, and is needed also because on
some systems (SunOS!) `rand()' may return a value larger than RAND_MAX */
lua_Number r = (lua_Number)(redisLrand48()%REDIS_LRAND48_MAX) /
(lua_Number)REDIS_LRAND48_MAX;
switch (lua_gettop(L)) { /* check number of arguments */
case 0: { /* no arguments */
lua_pushnumber(L, r); /* Number between 0 and 1 */
break;
}
case 1: { /* only upper limit */
int u = luaL_checkint(L, 1);
luaL_argcheck(L, 1<=u, 1, "interval is empty");
lua_pushnumber(L, floor(r*u)+1); /* int between 1 and `u' */
break;
}
case 2: { /* lower and upper limits */
int l = luaL_checkint(L, 1);
int u = luaL_checkint(L, 2);
luaL_argcheck(L, l<=u, 2, "interval is empty");
lua_pushnumber(L, floor(r*(u-l+1))+l); /* int between `l' and `u' */
break;
}
default: return luaL_error(L, "wrong number of arguments");
}
return 1;
}
static int redis_math_randomseed (lua_State *L) {
redisSrand48(luaL_checkint(L, 1));
return 0;
}
/* This is the Lua script "count" hook that we use to detect scripts timeout. */
static void luaMaskCountHook(lua_State *lua, lua_Debug *ar) {
UNUSED(ar);
scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
if (scriptInterrupt(rctx) == SCRIPT_KILL) {
serverLog(LL_WARNING,"Lua script killed by user with SCRIPT KILL.");
/*
* Set the hook to invoke all the time so the user
         * will not be able to catch the error with pcall and invoke
         * pcall again which will prevent the script from ever been killed
*/
lua_sethook(lua, luaMaskCountHook, LUA_MASKLINE, 0);
lua_pushstring(lua,"Script killed by user with SCRIPT KILL...");
lua_error(lua);
}
}
void luaCallFunction(scriptRunCtx* run_ctx, lua_State *lua, robj** keys, size_t nkeys, robj** args, size_t nargs, int debug_enabled) {
client* c = run_ctx->original_client;
int delhook = 0;
/* We must set it before we set the Lua hook, theoretically the
* Lua hook might be called wheneven we run any Lua instruction
* such as 'luaSetGlobalArray' and we want the run_ctx to be available
* each time the Lua hook is invoked. */
luaSaveOnRegistry(lua, REGISTRY_RUN_CTX_NAME, run_ctx);
if (server.script_time_limit > 0 && !debug_enabled) {
lua_sethook(lua,luaMaskCountHook,LUA_MASKCOUNT,100000);
delhook = 1;
} else if (debug_enabled) {
lua_sethook(lua,luaLdbLineHook,LUA_MASKLINE|LUA_MASKCOUNT,100000);
delhook = 1;
}
/* Populate the argv and keys table accordingly to the arguments that
* EVAL received. */
luaSetGlobalArray(lua,"KEYS",keys,nkeys);
luaSetGlobalArray(lua,"ARGV",args,nargs);
/* At this point whether this script was never seen before or if it was
* already defined, we can call it. We have zero arguments and expect
* a single return value. */
int err = lua_pcall(lua,0,1,-2);
/* Call the Lua garbage collector from time to time to avoid a
* full cycle performed by Lua, which adds too latency.
*
* The call is performed every LUA_GC_CYCLE_PERIOD executed commands
* (and for LUA_GC_CYCLE_PERIOD collection steps) because calling it
* for every command uses too much CPU. */
#define LUA_GC_CYCLE_PERIOD 50
{
static long gc_count = 0;
gc_count++;
if (gc_count == LUA_GC_CYCLE_PERIOD) {
lua_gc(lua,LUA_GCSTEP,LUA_GC_CYCLE_PERIOD);
gc_count = 0;
}
}
if (err) {
addReplyErrorFormat(c,"Error running script (call to %s): %s\n",
run_ctx->funcname, lua_tostring(lua,-1));
lua_pop(lua,1); /* Consume the Lua reply and remove error handler. */
} else {
/* On success convert the Lua return value into Redis protocol, and
* send it to * the client. */
luaReplyToRedisReply(c, run_ctx->c, lua); /* Convert and consume the reply. */
}
/* Perform some cleanup that we need to do both on error and success. */
if (delhook) lua_sethook(lua,NULL,0,0); /* Disable hook */
/* remove run_ctx from registry, its only applicable for the current script. */
luaSaveOnRegistry(lua, REGISTRY_RUN_CTX_NAME, NULL);
}
unsigned long luaMemory(lua_State *lua) {
return lua_gc(lua, LUA_GCCOUNT, 0) * 1024LL;
}
/*
* Copyright (c) 2009-2021, Redis Ltd.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __SCRIPT_LUA_H_
#define __SCRIPT_LUA_H_
/*
* script_lua.c unit provides shared functionality between
* eval.c and function_lua.c. Functionality provided:
*
* * Execute Lua code, assuming that the code is located on
* the top of the Lua stack. In addition, parsing the execution
* result and convert it to the resp and reply ot the client.
*
* * Run Redis commands from within the Lua code (Including
* parsing the reply and create a Lua object out of it).
*
* * Register Redis API to the Lua interpreter. Only shared
* API are registered (API that is only relevant on eval.c
* (like debugging) are registered on eval.c).
*
* Uses script.c for interaction back with Redis.
*/
#include "server.h"
#include "script.h"
#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>
#define REGISTRY_RUN_CTX_NAME "__RUN_CTX__"
void luaRegisterRedisAPI(lua_State* lua);
void luaEnableGlobalsProtection(lua_State *lua, int is_eval);
void luaSaveOnRegistry(lua_State* lua, const char* name, void* ptr);
void* luaGetFromRegistry(lua_State* lua, const char* name);
void luaCallFunction(scriptRunCtx* r_ctx, lua_State *lua, robj** keys, size_t nkeys, robj** args, size_t nargs, int debug_enabled);
unsigned long luaMemory(lua_State *lua);
#endif /* __SCRIPT_LUA_H_ */
...@@ -35,6 +35,7 @@ ...@@ -35,6 +35,7 @@
#include "latency.h" #include "latency.h"
#include "atomicvar.h" #include "atomicvar.h"
#include "mt19937-64.h" #include "mt19937-64.h"
#include "functions.h"
#include <time.h> #include <time.h>
#include <signal.h> #include <signal.h>
...@@ -471,6 +472,31 @@ struct redisCommand scriptSubcommands[] = { ...@@ -471,6 +472,31 @@ struct redisCommand scriptSubcommands[] = {
{NULL}, {NULL},
}; };
struct redisCommand functionSubcommands[] = {
{"create",functionsCreateCommand,-5,
"may-replicate no-script @scripting"},
{"delete",functionsDeleteCommand,3,
"may-replicate no-script @scripting"},
{"kill",functionsKillCommand,2,
"no-script @scripting"},
{"info",functionsInfoCommand,-3,
"no-script @scripting"},
{"list",functionsListCommand,2,
"no-script @scripting"},
{"stats",functionsStatsCommand,2,
"no-script @scripting"},
{"help",functionsHelpCommand,2,
"ok-loading ok-stale @scripting"},
{NULL},
};
struct redisCommand clientSubcommands[] = { struct redisCommand clientSubcommands[] = {
{"caching",clientCommand,3, {"caching",clientCommand,3,
"no-script ok-loading ok-stale @connection"}, "no-script ok-loading ok-stale @connection"},
...@@ -2032,7 +2058,25 @@ struct redisCommand redisCommandTable[] = { ...@@ -2032,7 +2058,25 @@ struct redisCommand redisCommandTable[] = {
"no-auth no-script ok-stale ok-loading fast @connection"}, "no-auth no-script ok-stale ok-loading fast @connection"},
{"failover",failoverCommand,-1, {"failover",failoverCommand,-1,
"admin no-script ok-stale"} "admin no-script ok-stale"},
{"function",NULL,-2,
"",
.subcommands=functionSubcommands},
{"fcall",fcallCommand,-3,
"no-script no-monitor may-replicate no-mandatory-keys @scripting",
{{"read write", /* We pass both read and write because these flag are worst-case-scenario */
KSPEC_BS_INDEX,.bs.index={2},
KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},
functionGetKeys},
{"fcall_ro",fcallCommandReadOnly,-3,
"no-script no-monitor no-mandatory-keys @scripting",
{{"read",
KSPEC_BS_INDEX,.bs.index={2},
KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},
functionGetKeys},
}; };
/*============================ Utility functions ============================ */ /*============================ Utility functions ============================ */
...@@ -2208,6 +2252,11 @@ void dictSdsDestructor(dict *d, void *val) ...@@ -2208,6 +2252,11 @@ void dictSdsDestructor(dict *d, void *val)
sdsfree(val); sdsfree(val);
} }
void *dictSdsDup(dict *d, const void *key) {
UNUSED(d);
return sdsdup((const sds) key);
}
int dictObjKeyCompare(dict *d, const void *key1, int dictObjKeyCompare(dict *d, const void *key1,
const void *key2) const void *key2)
{ {
...@@ -2994,7 +3043,7 @@ void cronUpdateMemoryStats() { ...@@ -2994,7 +3043,7 @@ void cronUpdateMemoryStats() {
/* LUA memory isn't part of zmalloc_used, but it is part of the process RSS, /* LUA memory isn't part of zmalloc_used, but it is part of the process RSS,
* so we must deduct it in order to be able to calculate correct * so we must deduct it in order to be able to calculate correct
* "allocator fragmentation" ratio */ * "allocator fragmentation" ratio */
size_t lua_memory = lua_gc(server.lua,LUA_GCCOUNT,0)*1024LL; size_t lua_memory = evalMemory();
server.cron_malloc_stats.allocator_resident = server.cron_malloc_stats.process_rss - lua_memory; server.cron_malloc_stats.allocator_resident = server.cron_malloc_stats.process_rss - lua_memory;
} }
if (!server.cron_malloc_stats.allocator_active) if (!server.cron_malloc_stats.allocator_active)
...@@ -3516,8 +3565,10 @@ void createSharedObjects(void) { ...@@ -3516,8 +3565,10 @@ void createSharedObjects(void) {
"-NOSCRIPT No matching script. Please use EVAL.\r\n")); "-NOSCRIPT No matching script. Please use EVAL.\r\n"));
shared.loadingerr = createObject(OBJ_STRING,sdsnew( shared.loadingerr = createObject(OBJ_STRING,sdsnew(
"-LOADING Redis is loading the dataset in memory\r\n")); "-LOADING Redis is loading the dataset in memory\r\n"));
shared.slowscripterr = createObject(OBJ_STRING,sdsnew( shared.slowevalerr = createObject(OBJ_STRING,sdsnew(
"-BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.\r\n")); "-BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.\r\n"));
shared.slowscripterr = createObject(OBJ_STRING,sdsnew(
"-BUSY Redis is busy running a script. You can only call FUNCTION KILL or SHUTDOWN NOSAVE.\r\n"));
shared.masterdownerr = createObject(OBJ_STRING,sdsnew( shared.masterdownerr = createObject(OBJ_STRING,sdsnew(
"-MASTERDOWN Link with MASTER is down and replica-serve-stale-data is set to 'no'.\r\n")); "-MASTERDOWN Link with MASTER is down and replica-serve-stale-data is set to 'no'.\r\n"));
shared.bgsaveerr = createObject(OBJ_STRING,sdsnew( shared.bgsaveerr = createObject(OBJ_STRING,sdsnew(
...@@ -4247,7 +4298,7 @@ void initServer(void) { ...@@ -4247,7 +4298,7 @@ void initServer(void) {
server.pubsub_channels = dictCreate(&keylistDictType); server.pubsub_channels = dictCreate(&keylistDictType);
server.pubsub_patterns = dictCreate(&keylistDictType); server.pubsub_patterns = dictCreate(&keylistDictType);
server.cronloops = 0; server.cronloops = 0;
server.in_eval = 0; server.in_script = 0;
server.in_exec = 0; server.in_exec = 0;
server.propagate_in_transaction = 0; server.propagate_in_transaction = 0;
server.client_pause_in_transaction = 0; server.client_pause_in_transaction = 0;
...@@ -4344,6 +4395,7 @@ void initServer(void) { ...@@ -4344,6 +4395,7 @@ void initServer(void) {
if (server.cluster_enabled) clusterInit(); if (server.cluster_enabled) clusterInit();
replicationScriptCacheInit(); replicationScriptCacheInit();
scriptingInit(1); scriptingInit(1);
functionsInit();
slowlogInit(); slowlogInit();
latencyMonitorInit(); latencyMonitorInit();
...@@ -4931,17 +4983,17 @@ void call(client *c, int flags) { ...@@ -4931,17 +4983,17 @@ void call(client *c, int flags) {
/* When EVAL is called loading the AOF we don't want commands called /* When EVAL is called loading the AOF we don't want commands called
* from Lua to go into the slowlog or to populate statistics. */ * from Lua to go into the slowlog or to populate statistics. */
if (server.loading && c->flags & CLIENT_LUA) if (server.loading && c->flags & CLIENT_SCRIPT)
flags &= ~(CMD_CALL_SLOWLOG | CMD_CALL_STATS); flags &= ~(CMD_CALL_SLOWLOG | CMD_CALL_STATS);
/* If the caller is Lua, we want to force the EVAL caller to propagate /* If the caller is Lua, we want to force the EVAL caller to propagate
* the script if the command flag or client flag are forcing the * the script if the command flag or client flag are forcing the
* propagation. */ * propagation. */
if (c->flags & CLIENT_LUA && server.lua_caller) { if (c->flags & CLIENT_SCRIPT && server.script_caller) {
if (c->flags & CLIENT_FORCE_REPL) if (c->flags & CLIENT_FORCE_REPL)
server.lua_caller->flags |= CLIENT_FORCE_REPL; server.script_caller->flags |= CLIENT_FORCE_REPL;
if (c->flags & CLIENT_FORCE_AOF) if (c->flags & CLIENT_FORCE_AOF)
server.lua_caller->flags |= CLIENT_FORCE_AOF; server.script_caller->flags |= CLIENT_FORCE_AOF;
} }
/* Note: the code below uses the real command that was executed /* Note: the code below uses the real command that was executed
...@@ -5070,8 +5122,8 @@ void call(client *c, int flags) { ...@@ -5070,8 +5122,8 @@ void call(client *c, int flags) {
/* If the client has keys tracking enabled for client side caching, /* If the client has keys tracking enabled for client side caching,
* make sure to remember the keys it fetched via this command. */ * make sure to remember the keys it fetched via this command. */
if (c->cmd->flags & CMD_READONLY) { if (c->cmd->flags & CMD_READONLY) {
client *caller = (c->flags & CLIENT_LUA && server.lua_caller) ? client *caller = (c->flags & CLIENT_SCRIPT && server.script_caller) ?
server.lua_caller : c; server.script_caller : c;
if (caller->flags & CLIENT_TRACKING && if (caller->flags & CLIENT_TRACKING &&
!(caller->flags & CLIENT_TRACKING_BCAST)) !(caller->flags & CLIENT_TRACKING_BCAST))
{ {
...@@ -5172,14 +5224,14 @@ void populateCommandMovableKeys(struct redisCommand *cmd) { ...@@ -5172,14 +5224,14 @@ void populateCommandMovableKeys(struct redisCommand *cmd) {
* other operations can be performed by the caller. Otherwise * other operations can be performed by the caller. Otherwise
* if C_ERR is returned the client was destroyed (i.e. after QUIT). */ * if C_ERR is returned the client was destroyed (i.e. after QUIT). */
int processCommand(client *c) { int processCommand(client *c) {
if (!server.lua_timedout) { if (!scriptIsTimedout()) {
/* Both EXEC and EVAL call call() directly so there should be /* Both EXEC and EVAL call call() directly so there should be
* no way in_exec or in_eval or propagate_in_transaction is 1. * no way in_exec or in_eval or propagate_in_transaction is 1.
* That is unless lua_timedout, in which case client may run * That is unless lua_timedout, in which case client may run
* some commands. */ * some commands. */
serverAssert(!server.propagate_in_transaction); serverAssert(!server.propagate_in_transaction);
serverAssert(!server.in_exec); serverAssert(!server.in_exec);
serverAssert(!server.in_eval); serverAssert(!server.in_script);
} }
moduleCallCommandFilters(c); moduleCallCommandFilters(c);
...@@ -5273,8 +5325,8 @@ int processCommand(client *c) { ...@@ -5273,8 +5325,8 @@ int processCommand(client *c) {
* 2) The command has no key arguments. */ * 2) The command has no key arguments. */
if (server.cluster_enabled && if (server.cluster_enabled &&
!(c->flags & CLIENT_MASTER) && !(c->flags & CLIENT_MASTER) &&
!(c->flags & CLIENT_LUA && !(c->flags & CLIENT_SCRIPT &&
server.lua_caller->flags & CLIENT_MASTER) && server.script_caller->flags & CLIENT_MASTER) &&
!(!c->cmd->movablekeys && c->cmd->key_specs_num == 0 && !(!c->cmd->movablekeys && c->cmd->key_specs_num == 0 &&
c->cmd->proc != execCommand)) c->cmd->proc != execCommand))
{ {
...@@ -5309,7 +5361,7 @@ int processCommand(client *c) { ...@@ -5309,7 +5361,7 @@ int processCommand(client *c) {
* the event loop since there is a busy Lua script running in timeout * the event loop since there is a busy Lua script running in timeout
* condition, to avoid mixing the propagation of scripts with the * condition, to avoid mixing the propagation of scripts with the
* propagation of DELs due to eviction. */ * propagation of DELs due to eviction. */
if (server.maxmemory && !server.lua_timedout) { if (server.maxmemory && !scriptIsTimedout()) {
int out_of_memory = (performEvictions() == EVICT_FAIL); int out_of_memory = (performEvictions() == EVICT_FAIL);
/* performEvictions may evict keys, so we need flush pending tracking /* performEvictions may evict keys, so we need flush pending tracking
...@@ -5345,7 +5397,7 @@ int processCommand(client *c) { ...@@ -5345,7 +5397,7 @@ int processCommand(client *c) {
* until first write within script, memory used by lua stack and * until first write within script, memory used by lua stack and
* arguments might interfere. */ * arguments might interfere. */
if (c->cmd->proc == evalCommand || c->cmd->proc == evalShaCommand) { if (c->cmd->proc == evalCommand || c->cmd->proc == evalShaCommand) {
server.lua_oom = out_of_memory; server.script_oom = out_of_memory;
} }
} }
...@@ -5432,7 +5484,7 @@ int processCommand(client *c) { ...@@ -5432,7 +5484,7 @@ int processCommand(client *c) {
* the MULTI plus a few initial commands refused, then the timeout * the MULTI plus a few initial commands refused, then the timeout
* condition resolves, and the bottom-half of the transaction gets * condition resolves, and the bottom-half of the transaction gets
* executed, see Github PR #7022. */ * executed, see Github PR #7022. */
if (server.lua_timedout && if (scriptIsTimedout() &&
c->cmd->proc != authCommand && c->cmd->proc != authCommand &&
c->cmd->proc != helloCommand && c->cmd->proc != helloCommand &&
c->cmd->proc != replconfCommand && c->cmd->proc != replconfCommand &&
...@@ -5447,9 +5499,15 @@ int processCommand(client *c) { ...@@ -5447,9 +5499,15 @@ int processCommand(client *c) {
tolower(((char*)c->argv[1]->ptr)[0]) == 'n') && tolower(((char*)c->argv[1]->ptr)[0]) == 'n') &&
!(c->cmd->proc == scriptCommand && !(c->cmd->proc == scriptCommand &&
c->argc == 2 && c->argc == 2 &&
tolower(((char*)c->argv[1]->ptr)[0]) == 'k')) tolower(((char*)c->argv[1]->ptr)[0]) == 'k') &&
!(c->cmd->proc == functionsKillCommand) &&
!(c->cmd->proc == functionsStatsCommand))
{ {
rejectCommand(c, shared.slowscripterr); if (scriptIsEval()) {
rejectCommand(c, shared.slowevalerr);
} else {
rejectCommand(c, shared.slowscripterr);
}
return C_OK; return C_OK;
} }
...@@ -6280,13 +6338,15 @@ sds genRedisInfoString(const char *section) { ...@@ -6280,13 +6338,15 @@ sds genRedisInfoString(const char *section) {
char peak_hmem[64]; char peak_hmem[64];
char total_system_hmem[64]; char total_system_hmem[64];
char used_memory_lua_hmem[64]; char used_memory_lua_hmem[64];
char used_memory_vm_total_hmem[64];
char used_memory_scripts_hmem[64]; char used_memory_scripts_hmem[64];
char used_memory_rss_hmem[64]; char used_memory_rss_hmem[64];
char maxmemory_hmem[64]; char maxmemory_hmem[64];
size_t zmalloc_used = zmalloc_used_memory(); size_t zmalloc_used = zmalloc_used_memory();
size_t total_system_mem = server.system_memory_size; size_t total_system_mem = server.system_memory_size;
const char *evict_policy = evictPolicyToString(); const char *evict_policy = evictPolicyToString();
long long memory_lua = server.lua ? (long long)lua_gc(server.lua,LUA_GCCOUNT,0)*1024 : 0; long long memory_lua = evalMemory();
long long memory_functions = functionsMemory();
struct redisMemOverhead *mh = getMemoryOverheadData(); struct redisMemOverhead *mh = getMemoryOverheadData();
/* Peak memory is updated from time to time by serverCron() so it /* Peak memory is updated from time to time by serverCron() so it
...@@ -6300,7 +6360,8 @@ sds genRedisInfoString(const char *section) { ...@@ -6300,7 +6360,8 @@ sds genRedisInfoString(const char *section) {
bytesToHuman(peak_hmem,server.stat_peak_memory); bytesToHuman(peak_hmem,server.stat_peak_memory);
bytesToHuman(total_system_hmem,total_system_mem); bytesToHuman(total_system_hmem,total_system_mem);
bytesToHuman(used_memory_lua_hmem,memory_lua); bytesToHuman(used_memory_lua_hmem,memory_lua);
bytesToHuman(used_memory_scripts_hmem,mh->lua_caches); bytesToHuman(used_memory_vm_total_hmem,memory_functions + memory_lua);
bytesToHuman(used_memory_scripts_hmem,mh->lua_caches + mh->functions_caches);
bytesToHuman(used_memory_rss_hmem,server.cron_malloc_stats.process_rss); bytesToHuman(used_memory_rss_hmem,server.cron_malloc_stats.process_rss);
bytesToHuman(maxmemory_hmem,server.maxmemory); bytesToHuman(maxmemory_hmem,server.maxmemory);
...@@ -6323,11 +6384,18 @@ sds genRedisInfoString(const char *section) { ...@@ -6323,11 +6384,18 @@ sds genRedisInfoString(const char *section) {
"allocator_resident:%zu\r\n" "allocator_resident:%zu\r\n"
"total_system_memory:%lu\r\n" "total_system_memory:%lu\r\n"
"total_system_memory_human:%s\r\n" "total_system_memory_human:%s\r\n"
"used_memory_lua:%lld\r\n" "used_memory_lua:%lld\r\n" /* deprecated, renamed to used_memory_vm_eval */
"used_memory_lua_human:%s\r\n" "used_memory_vm_eval:%lld\r\n"
"used_memory_lua_human:%s\r\n" /* deprecated */
"used_memory_scripts_eval:%lld\r\n"
"number_of_cached_scripts:%lu\r\n"
"number_of_functions:%lu\r\n"
"used_memory_vm_functions:%lld\r\n"
"used_memory_vm_total:%lld\r\n"
"used_memory_vm_total_human:%s\r\n"
"used_memory_functions:%lld\r\n"
"used_memory_scripts:%lld\r\n" "used_memory_scripts:%lld\r\n"
"used_memory_scripts_human:%s\r\n" "used_memory_scripts_human:%s\r\n"
"number_of_cached_scripts:%lu\r\n"
"maxmemory:%lld\r\n" "maxmemory:%lld\r\n"
"maxmemory_human:%s\r\n" "maxmemory_human:%s\r\n"
"maxmemory_policy:%s\r\n" "maxmemory_policy:%s\r\n"
...@@ -6366,10 +6434,17 @@ sds genRedisInfoString(const char *section) { ...@@ -6366,10 +6434,17 @@ sds genRedisInfoString(const char *section) {
(unsigned long)total_system_mem, (unsigned long)total_system_mem,
total_system_hmem, total_system_hmem,
memory_lua, memory_lua,
memory_lua,
used_memory_lua_hmem, used_memory_lua_hmem,
(long long) mh->lua_caches, (long long) mh->lua_caches,
dictSize(evalScriptsDict()),
functionsNum(),
memory_functions,
memory_functions + memory_lua,
used_memory_vm_total_hmem,
(long long) mh->functions_caches,
(long long) mh->lua_caches + (long long) mh->functions_caches,
used_memory_scripts_hmem, used_memory_scripts_hmem,
dictSize(server.lua_scripts),
server.maxmemory, server.maxmemory,
maxmemory_hmem, maxmemory_hmem,
evict_policy, evict_policy,
......
...@@ -260,7 +260,7 @@ extern int configOOMScoreAdjValuesDefaults[CONFIG_OOM_COUNT]; ...@@ -260,7 +260,7 @@ extern int configOOMScoreAdjValuesDefaults[CONFIG_OOM_COUNT];
#define CLIENT_CLOSE_AFTER_REPLY (1<<6) /* Close after writing entire reply. */ #define CLIENT_CLOSE_AFTER_REPLY (1<<6) /* Close after writing entire reply. */
#define CLIENT_UNBLOCKED (1<<7) /* This client was unblocked and is stored in #define CLIENT_UNBLOCKED (1<<7) /* This client was unblocked and is stored in
server.unblocked_clients */ server.unblocked_clients */
#define CLIENT_LUA (1<<8) /* This is a non connected client used by Lua */ #define CLIENT_SCRIPT (1<<8) /* This is a non connected client used by Lua */
#define CLIENT_ASKING (1<<9) /* Client issued the ASKING command */ #define CLIENT_ASKING (1<<9) /* Client issued the ASKING command */
#define CLIENT_CLOSE_ASAP (1<<10)/* Close this client ASAP */ #define CLIENT_CLOSE_ASAP (1<<10)/* Close this client ASAP */
#define CLIENT_UNIX_SOCKET (1<<11) /* Client connected via Unix domain socket */ #define CLIENT_UNIX_SOCKET (1<<11) /* Client connected via Unix domain socket */
...@@ -822,6 +822,19 @@ typedef struct redisDb { ...@@ -822,6 +822,19 @@ typedef struct redisDb {
clusterSlotToKeyMapping *slots_to_keys; /* Array of slots to keys. Only used in cluster mode (db 0). */ clusterSlotToKeyMapping *slots_to_keys; /* Array of slots to keys. Only used in cluster mode (db 0). */
} redisDb; } redisDb;
/* forward declaration for functions ctx */
typedef struct functionsCtx functionsCtx;
/* Holding object that need to be populated during
* rdb loading. On loading end it is possible to decide
* whether not to set those objects on their rightful place.
* For example: dbarray need to be set as main database on
* successful loading and dropped on failure. */
typedef struct rdbLoadingCtx {
redisDb* dbarray;
functionsCtx* functions_ctx;
}rdbLoadingCtx;
/* Client MULTI/EXEC state */ /* Client MULTI/EXEC state */
typedef struct multiCmd { typedef struct multiCmd {
robj **argv; robj **argv;
...@@ -1122,7 +1135,7 @@ struct sharedObjectsStruct { ...@@ -1122,7 +1135,7 @@ struct sharedObjectsStruct {
robj *crlf, *ok, *err, *emptybulk, *czero, *cone, *pong, *space, robj *crlf, *ok, *err, *emptybulk, *czero, *cone, *pong, *space,
*queued, *null[4], *nullarray[4], *emptymap[4], *emptyset[4], *queued, *null[4], *nullarray[4], *emptymap[4], *emptyset[4],
*emptyarray, *wrongtypeerr, *nokeyerr, *syntaxerr, *sameobjecterr, *emptyarray, *wrongtypeerr, *nokeyerr, *syntaxerr, *sameobjecterr,
*outofrangeerr, *noscripterr, *loadingerr, *slowscripterr, *bgsaveerr, *outofrangeerr, *noscripterr, *loadingerr, *slowevalerr, *slowscripterr, *bgsaveerr,
*masterdownerr, *roslaveerr, *execaborterr, *noautherr, *noreplicaserr, *masterdownerr, *roslaveerr, *execaborterr, *noautherr, *noreplicaserr,
*busykeyerr, *oomerr, *plus, *messagebulk, *pmessagebulk, *subscribebulk, *busykeyerr, *oomerr, *plus, *messagebulk, *pmessagebulk, *subscribebulk,
*unsubscribebulk, *psubscribebulk, *punsubscribebulk, *del, *unlink, *unsubscribebulk, *psubscribebulk, *punsubscribebulk, *del, *unlink,
...@@ -1203,6 +1216,7 @@ struct redisMemOverhead { ...@@ -1203,6 +1216,7 @@ struct redisMemOverhead {
size_t clients_normal; size_t clients_normal;
size_t aof_buffer; size_t aof_buffer;
size_t lua_caches; size_t lua_caches;
size_t functions_caches;
size_t overhead_total; size_t overhead_total;
size_t dataset; size_t dataset;
size_t total_keys; size_t total_keys;
...@@ -1334,7 +1348,7 @@ struct redisServer { ...@@ -1334,7 +1348,7 @@ struct redisServer {
int sentinel_mode; /* True if this instance is a Sentinel. */ int sentinel_mode; /* True if this instance is a Sentinel. */
size_t initial_memory_usage; /* Bytes used after initialization. */ size_t initial_memory_usage; /* Bytes used after initialization. */
int always_show_logo; /* Show logo even for non-stdout logging. */ int always_show_logo; /* Show logo even for non-stdout logging. */
int in_eval; /* Are we inside EVAL? */ int in_script; /* Are we inside EVAL? */
int in_exec; /* Are we inside EXEC? */ int in_exec; /* Are we inside EXEC? */
int propagate_in_transaction; /* Make sure we don't propagate nested MULTI/EXEC */ int propagate_in_transaction; /* Make sure we don't propagate nested MULTI/EXEC */
char *ignore_warnings; /* Config: warnings that should be ignored. */ char *ignore_warnings; /* Config: warnings that should be ignored. */
...@@ -1719,28 +1733,11 @@ struct redisServer { ...@@ -1719,28 +1733,11 @@ struct redisServer {
is down? */ is down? */
int cluster_config_file_lock_fd; /* cluster config fd, will be flock */ int cluster_config_file_lock_fd; /* cluster config fd, will be flock */
/* Scripting */ /* Scripting */
lua_State *lua; /* The Lua interpreter. We use just one for all clients */ client *script_caller; /* The client running script right now, or NULL */
client *lua_client; /* The "fake client" to query Redis from Lua */ mstime_t script_time_limit; /* Script timeout in milliseconds */
client *lua_caller; /* The client running EVAL right now, or NULL */
char* lua_cur_script; /* SHA1 of the script currently running, or NULL */
dict *lua_scripts; /* A dictionary of SHA1 -> Lua scripts */
unsigned long long lua_scripts_mem; /* Cached scripts' memory + oh */
mstime_t lua_time_limit; /* Script timeout in milliseconds */
monotime lua_time_start; /* monotonic timer to detect timed-out script */
mstime_t lua_time_snapshot; /* Snapshot of mstime when script is started */
int lua_write_dirty; /* True if a write command was called during the
execution of the current script. */
int lua_random_dirty; /* True if a random command was called during the
execution of the current script. */
int lua_replicate_commands; /* True if we are doing single commands repl. */
int lua_multi_emitted;/* True if we already propagated MULTI. */
int lua_repl; /* Script replication flags for redis.set_repl(). */
int lua_timedout; /* True if we reached the time limit for script
execution. */
int lua_kill; /* Kill the script if true. */
int lua_always_replicate_commands; /* Default replication type. */ int lua_always_replicate_commands; /* Default replication type. */
int lua_oom; /* OOM detected when script start? */ int script_oom; /* OOM detected when script start */
int lua_disable_deny_script; /* Allow running commands marked "no-script" inside a script. */ int script_disable_deny_script; /* Allow running commands marked "no-script" inside a script. */
/* Lazy free */ /* Lazy free */
int lazyfree_lazy_eviction; int lazyfree_lazy_eviction;
int lazyfree_lazy_expire; int lazyfree_lazy_expire;
...@@ -2687,6 +2684,7 @@ int sintercardGetKeys(struct redisCommand *cmd,robj **argv, int argc, getKeysRes ...@@ -2687,6 +2684,7 @@ int sintercardGetKeys(struct redisCommand *cmd,robj **argv, int argc, getKeysRes
int zunionInterDiffGetKeys(struct redisCommand *cmd,robj **argv, int argc, getKeysResult *result); int zunionInterDiffGetKeys(struct redisCommand *cmd,robj **argv, int argc, getKeysResult *result);
int zunionInterDiffStoreGetKeys(struct redisCommand *cmd,robj **argv, int argc, getKeysResult *result); int zunionInterDiffStoreGetKeys(struct redisCommand *cmd,robj **argv, int argc, getKeysResult *result);
int evalGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result); int evalGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
int functionGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
int sortGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result); int sortGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
int migrateGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result); int migrateGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
int georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result); int georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
...@@ -2722,8 +2720,17 @@ void scriptingInit(int setup); ...@@ -2722,8 +2720,17 @@ void scriptingInit(int setup);
int ldbRemoveChild(pid_t pid); int ldbRemoveChild(pid_t pid);
void ldbKillForkedSessions(void); void ldbKillForkedSessions(void);
int ldbPendingChildren(void); int ldbPendingChildren(void);
sds luaCreateFunction(client *c, lua_State *lua, robj *body); sds luaCreateFunction(client *c, robj *body);
void luaLdbLineHook(lua_State *lua, lua_Debug *ar);
void freeLuaScriptsAsync(dict *lua_scripts); void freeLuaScriptsAsync(dict *lua_scripts);
int ldbIsEnabled();
void ldbLog(sds entry);
void ldbLogRedisReply(char *reply);
void sha1hex(char *digest, char *script, size_t len);
unsigned long evalMemory();
dict* evalScriptsDict();
unsigned long evalScriptsMemory();
mstime_t evalTimeSnapshot();
/* Blocked clients */ /* Blocked clients */
void processUnblockedClients(void); void processUnblockedClients(void);
...@@ -2769,6 +2776,7 @@ uint64_t dictSdsCaseHash(const void *key); ...@@ -2769,6 +2776,7 @@ uint64_t dictSdsCaseHash(const void *key);
int dictSdsKeyCompare(dict *d, const void *key1, const void *key2); int dictSdsKeyCompare(dict *d, const void *key1, const void *key2);
int dictSdsKeyCaseCompare(dict *d, const void *key1, const void *key2); int dictSdsKeyCaseCompare(dict *d, const void *key1, const void *key2);
void dictSdsDestructor(dict *d, void *val); void dictSdsDestructor(dict *d, void *val);
void *dictSdsDup(dict *d, const void *key);
/* Git SHA1 */ /* Git SHA1 */
char *redisGitSHA1(void); char *redisGitSHA1(void);
......
...@@ -294,7 +294,7 @@ void sortCommandGeneric(client *c, int readonly) { ...@@ -294,7 +294,7 @@ void sortCommandGeneric(client *c, int readonly) {
* scripting and replication. */ * scripting and replication. */
if (dontsort && if (dontsort &&
sortval->type == OBJ_SET && sortval->type == OBJ_SET &&
(storekey || c->flags & CLIENT_LUA)) (storekey || c->flags & CLIENT_SCRIPT))
{ {
/* Force ALPHA sorting */ /* Force ALPHA sorting */
dontsort = 0; dontsort = 0;
......
...@@ -1998,7 +1998,7 @@ void xreadCommand(client *c) { ...@@ -1998,7 +1998,7 @@ void xreadCommand(client *c) {
int moreargs = c->argc-i-1; int moreargs = c->argc-i-1;
char *o = c->argv[i]->ptr; char *o = c->argv[i]->ptr;
if (!strcasecmp(o,"BLOCK") && moreargs) { if (!strcasecmp(o,"BLOCK") && moreargs) {
if (c->flags & CLIENT_LUA) { if (c->flags & CLIENT_SCRIPT) {
/* /*
* Although the CLIENT_DENY_BLOCKING flag should protect from blocking the client * Although the CLIENT_DENY_BLOCKING flag should protect from blocking the client
* on Lua/MULTI/RM_Call we want special treatment for Lua to keep backward compatibility. * on Lua/MULTI/RM_Call we want special treatment for Lua to keep backward compatibility.
......
...@@ -521,6 +521,12 @@ foreach testType {Successful Aborted} { ...@@ -521,6 +521,12 @@ foreach testType {Successful Aborted} {
# Set a key value on replica to check status during loading, on failure and after swapping db # Set a key value on replica to check status during loading, on failure and after swapping db
$replica set mykey myvalue $replica set mykey myvalue
# Set a function value on replica to check status during loading, on failure and after swapping db
$replica function create LUA test {return 'hello1'}
# Set a function value on master to check it reaches the replica when replication ends
$master function create LUA test {return 'hello2'}
# Force the replica to try another full sync (this time it will have matching master replid) # Force the replica to try another full sync (this time it will have matching master replid)
$master multi $master multi
$master client kill type replica $master client kill type replica
...@@ -552,6 +558,9 @@ foreach testType {Successful Aborted} { ...@@ -552,6 +558,9 @@ foreach testType {Successful Aborted} {
# Ensure we still see old values while async_loading is in progress and also not LOADING status # Ensure we still see old values while async_loading is in progress and also not LOADING status
assert_equal [$replica get mykey] "myvalue" assert_equal [$replica get mykey] "myvalue"
# Ensure we still can call old function while async_loading is in progress
assert_equal [$replica fcall test 0] "hello1"
# Make sure we're still async_loading to validate previous assertion # Make sure we're still async_loading to validate previous assertion
assert_equal [s -1 async_loading] 1 assert_equal [s -1 async_loading] 1
...@@ -576,6 +585,9 @@ foreach testType {Successful Aborted} { ...@@ -576,6 +585,9 @@ foreach testType {Successful Aborted} {
# Ensure we see old values from replica # Ensure we see old values from replica
assert_equal [$replica get mykey] "myvalue" assert_equal [$replica get mykey] "myvalue"
# Ensure we still can call old function
assert_equal [$replica fcall test 0] "hello1"
# Make sure amount of replica keys didn't change # Make sure amount of replica keys didn't change
assert_equal [$replica dbsize] 2001 assert_equal [$replica dbsize] 2001
} }
...@@ -595,6 +607,9 @@ foreach testType {Successful Aborted} { ...@@ -595,6 +607,9 @@ foreach testType {Successful Aborted} {
# Ensure we don't see anymore the key that was stored only to replica and also that we don't get LOADING status # Ensure we don't see anymore the key that was stored only to replica and also that we don't get LOADING status
assert_equal [$replica GET mykey] "" assert_equal [$replica GET mykey] ""
# Ensure we got the new function
assert_equal [$replica fcall test 0] "hello2"
# Make sure amount of keys matches master # Make sure amount of keys matches master
assert_equal [$replica dbsize] 1010 assert_equal [$replica dbsize] 1010
} }
...@@ -624,6 +639,10 @@ test {diskless loading short read} { ...@@ -624,6 +639,10 @@ test {diskless loading short read} {
$replica config set dynamic-hz no $replica config set dynamic-hz no
# Try to fill the master with all types of data types / encodings # Try to fill the master with all types of data types / encodings
set start [clock clicks -milliseconds] set start [clock clicks -milliseconds]
# Set a function value to check short read handling on functions
r function create LUA test {return 'hello1'}
for {set k 0} {$k < 3} {incr k} { for {set k 0} {$k < 3} {incr k} {
for {set i 0} {$i < 10} {incr i} { for {set i 0} {$i < 10} {incr i} {
r set "$k int_$i" [expr {int(rand()*10000)}] r set "$k int_$i" [expr {int(rand()*10000)}]
......
start_server {tags {"scripting"}} {
test {FUNCTION - Basic usage} {
r function create LUA test {return 'hello'}
r fcall test 0
} {hello}
test {FUNCTION - Create an already exiting function raise error} {
catch {
r function create LUA test {return 'hello1'}
} e
set _ $e
} {*Function already exists*}
test {FUNCTION - Create function with unexisting engine} {
catch {
r function create bad_engine test {return 'hello1'}
} e
set _ $e
} {*Engine not found*}
test {FUNCTION - Test uncompiled script} {
catch {
r function create LUA test1 {bad script}
} e
set _ $e
} {*Error compiling function*}
test {FUNCTION - test replace argument} {
r function create LUA test REPLACE {return 'hello1'}
r fcall test 0
} {hello1}
test {FUNCTION - test replace argument with function creation failure keeps old function} {
catch {r function create LUA test REPLACE {error}}
r fcall test 0
} {hello1}
test {FUNCTION - test function delete} {
r function delete test
catch {
r fcall test 0
} e
set _ $e
} {*Function not found*}
test {FUNCTION - test description argument} {
r function create LUA test DESCRIPTION {some description} {return 'hello'}
r function list
} {{name test engine LUA description {some description}}}
test {FUNCTION - test info specific function} {
r function info test WITHCODE
} {name test engine LUA description {some description} code {return 'hello'}}
test {FUNCTION - test info without code} {
r function info test
} {name test engine LUA description {some description}}
test {FUNCTION - test info on function that does not exists} {
catch {
r function info bad_function_name
} e
set _ $e
} {*Function does not exists*}
test {FUNCTION - test info with bad number of arguments} {
catch {
r function info test WITHCODE bad_arg
} e
set _ $e
} {*wrong number of arguments*}
test {FUNCTION - test fcall bad arguments} {
catch {
r fcall test bad_arg
} e
set _ $e
} {*Bad number of keys provided*}
test {FUNCTION - test fcall bad number of keys arguments} {
catch {
r fcall test 10 key1
} e
set _ $e
} {*Number of keys can't be greater than number of args*}
test {FUNCTION - test fcall negative number of keys} {
catch {
r fcall test -1 key1
} e
set _ $e
} {*Number of keys can't be negative*}
test {FUNCTION - test function delete on not exiting function} {
catch {
r function delete test1
} e
set _ $e
} {*Function not found*}
test {FUNCTION - test function kill when function is not running} {
catch {
r function kill
} e
set _ $e
} {*No scripts in execution*}
test {FUNCTION - test wrong subcommand} {
catch {
r function bad_subcommand
} e
set _ $e
} {*Unknown subcommand*}
test {FUNCTION - test loading from rdb} {
r debug reload
r fcall test 0
} {hello}
test {FUNCTION - test fcall_ro with write command} {
r function create lua test REPLACE {return redis.call('set', 'x', '1')}
catch { r fcall_ro test 0 } e
set _ $e
} {*Write commands are not allowed from read-only scripts*}
test {FUNCTION - test fcall_ro with read only commands} {
r function create lua test REPLACE {return redis.call('get', 'x')}
r set x 1
r fcall_ro test 0
} {1}
test {FUNCTION - test keys and argv} {
r function create lua test REPLACE {return redis.call('set', KEYS[1], ARGV[1])}
r fcall test 1 x foo
r get x
} {foo}
test {FUNCTION - test command get keys on fcall} {
r COMMAND GETKEYS fcall test 1 x foo
} {x}
test {FUNCTION - test command get keys on fcall_ro} {
r COMMAND GETKEYS fcall_ro test 1 x foo
} {x}
test {FUNCTION - test function kill} {
set rd [redis_deferring_client]
r config set script-time-limit 10
r function create lua test REPLACE {local a = 1 while true do a = a + 1 end}
$rd fcall test 0
after 200
catch {r ping} e
assert_match {BUSY*} $e
assert_match {running_script {name test command {fcall test 0} duration_ms *} engines LUA} [r FUNCTION STATS]
r function kill
after 200 ; # Give some time to Lua to call the hook again...
assert_equal [r ping] "PONG"
}
test {FUNCTION - test script kill not working on function} {
set rd [redis_deferring_client]
r config set script-time-limit 10
r function create lua test REPLACE {local a = 1 while true do a = a + 1 end}
$rd fcall test 0
after 200
catch {r ping} e
assert_match {BUSY*} $e
catch {r script kill} e
assert_match {BUSY*} $e
r function kill
after 200 ; # Give some time to Lua to call the hook again...
assert_equal [r ping] "PONG"
}
test {FUNCTION - test function kill not working on eval} {
set rd [redis_deferring_client]
r config set script-time-limit 10
$rd eval {local a = 1 while true do a = a + 1 end} 0
after 200
catch {r ping} e
assert_match {BUSY*} $e
catch {r function kill} e
assert_match {BUSY*} $e
r script kill
after 200 ; # Give some time to Lua to call the hook again...
assert_equal [r ping] "PONG"
}
}
start_server {tags {"scripting repl"}} {
start_server {} {
test "Connect a replica to the master instance" {
r -1 slaveof [srv 0 host] [srv 0 port]
wait_for_condition 50 100 {
[s -1 role] eq {slave} &&
[string match {*master_link_status:up*} [r -1 info replication]]
} else {
fail "Can't turn the instance into a replica"
}
}
test {FUNCTION - creation is replicated to replica} {
r function create LUA test DESCRIPTION {some description} {return 'hello'}
wait_for_condition 50 100 {
[r -1 function list] eq {{name test engine LUA description {some description}}}
} else {
fail "Failed waiting for function to replicate to replica"
}
}
test {FUNCTION - call on replica} {
r -1 fcall test 0
} {hello}
test {FUNCTION - delete is replicated to replica} {
r function delete test
wait_for_condition 50 100 {
[r -1 function list] eq {}
} else {
fail "Failed waiting for function to replicate to replica"
}
}
test "Disconnecting the replica from master instance" {
r -1 slaveof no one
# creating a function after disconnect to make sure function
# is replicated on rdb phase
r function create LUA test DESCRIPTION {some description} {return 'hello'}
# reconnect the replica
r -1 slaveof [srv 0 host] [srv 0 port]
wait_for_condition 50 100 {
[s -1 role] eq {slave} &&
[string match {*master_link_status:up*} [r -1 info replication]]
} else {
fail "Can't turn the instance into a replica"
}
}
test "FUNCTION - test replication to replica on rdb phase" {
r -1 fcall test 0
} {hello}
test "FUNCTION - test replication to replica on rdb phase info command" {
r -1 function info test WITHCODE
} {name test engine LUA description {some description} code {return 'hello'}}
test "FUNCTION - create on read only replica" {
catch {
r -1 function create LUA test DESCRIPTION {some description} {return 'hello'}
} e
set _ $e
} {*Can not create a function on a read only replica*}
test "FUNCTION - delete on read only replica" {
catch {
r -1 function delete test
} e
set _ $e
} {*Can not delete a function on a read only replica*}
test "FUNCTION - function effect is replicated to replica" {
r function create LUA test REPLACE {return redis.call('set', 'x', '1')}
r fcall test 0
assert {[r get x] eq {1}}
wait_for_condition 50 100 {
[r -1 get x] eq {1}
} else {
fail "Failed waiting function effect to be replicated to replica"
}
}
test "FUNCTION - modify key space of read only replica" {
catch {
r -1 fcall test 0
} e
set _ $e
} {*can't write against a read only replica*}
}
}
\ No newline at end of file
foreach is_eval {0 1} {
if {$is_eval == 1} {
proc run_script {args} {
r eval {*}$args
}
proc run_script_ro {args} {
r eval_ro {*}$args
}
proc run_script_on_connection {args} {
[lindex $args 0] eval {*}[lrange $args 1 end]
}
proc kill_script {args} {
r script kill
}
} else {
proc run_script {args} {
r function create LUA test replace [lindex $args 0]
r fcall test {*}[lrange $args 1 end]
}
proc run_script_ro {args} {
r function create LUA test replace [lindex $args 0]
r fcall_ro test {*}[lrange $args 1 end]
}
proc run_script_on_connection {args} {
set rd [lindex $args 0]
$rd function create LUA test replace [lindex $args 1]
# read the ok reply of function create
$rd read
$rd fcall test {*}[lrange $args 2 end]
}
proc kill_script {args} {
r function kill
}
}
start_server {tags {"scripting"}} { start_server {tags {"scripting"}} {
test {EVAL - Does Lua interpreter replies to our requests?} { test {EVAL - Does Lua interpreter replies to our requests?} {
r eval {return 'hello'} 0 run_script {return 'hello'} 0
} {hello} } {hello}
test {EVAL - Lua integer -> Redis protocol type conversion} { test {EVAL - Lua integer -> Redis protocol type conversion} {
r eval {return 100.5} 0 run_script {return 100.5} 0
} {100} } {100}
test {EVAL - Lua string -> Redis protocol type conversion} { test {EVAL - Lua string -> Redis protocol type conversion} {
r eval {return 'hello world'} 0 run_script {return 'hello world'} 0
} {hello world} } {hello world}
test {EVAL - Lua true boolean -> Redis protocol type conversion} { test {EVAL - Lua true boolean -> Redis protocol type conversion} {
r eval {return true} 0 run_script {return true} 0
} {1} } {1}
test {EVAL - Lua false boolean -> Redis protocol type conversion} { test {EVAL - Lua false boolean -> Redis protocol type conversion} {
r eval {return false} 0 run_script {return false} 0
} {} } {}
test {EVAL - Lua status code reply -> Redis protocol type conversion} { test {EVAL - Lua status code reply -> Redis protocol type conversion} {
r eval {return {ok='fine'}} 0 run_script {return {ok='fine'}} 0
} {fine} } {fine}
test {EVAL - Lua error reply -> Redis protocol type conversion} { test {EVAL - Lua error reply -> Redis protocol type conversion} {
catch { catch {
r eval {return {err='this is an error'}} 0 run_script {return {err='this is an error'}} 0
} e } e
set _ $e set _ $e
} {this is an error} } {this is an error}
test {EVAL - Lua table -> Redis protocol type conversion} { test {EVAL - Lua table -> Redis protocol type conversion} {
r eval {return {1,2,3,'ciao',{1,2}}} 0 run_script {return {1,2,3,'ciao',{1,2}}} 0
} {1 2 3 ciao {1 2}} } {1 2 3 ciao {1 2}}
test {EVAL - Are the KEYS and ARGV arrays populated correctly?} { test {EVAL - Are the KEYS and ARGV arrays populated correctly?} {
r eval {return {KEYS[1],KEYS[2],ARGV[1],ARGV[2]}} 2 a{t} b{t} c{t} d{t} run_script {return {KEYS[1],KEYS[2],ARGV[1],ARGV[2]}} 2 a{t} b{t} c{t} d{t}
} {a{t} b{t} c{t} d{t}} } {a{t} b{t} c{t} d{t}}
test {EVAL - is Lua able to call Redis API?} { test {EVAL - is Lua able to call Redis API?} {
r set mykey myval r set mykey myval
r eval {return redis.call('get',KEYS[1])} 1 mykey run_script {return redis.call('get',KEYS[1])} 1 mykey
} {myval} } {myval}
if {$is_eval eq 1} {
# eval sha is only relevant for is_eval Lua
test {EVALSHA - Can we call a SHA1 if already defined?} { test {EVALSHA - Can we call a SHA1 if already defined?} {
r evalsha fd758d1589d044dd850a6f05d52f2eefd27f033f 1 mykey r evalsha fd758d1589d044dd850a6f05d52f2eefd27f033f 1 mykey
} {myval} } {myval}
...@@ -60,10 +99,11 @@ start_server {tags {"scripting"}} { ...@@ -60,10 +99,11 @@ start_server {tags {"scripting"}} {
catch {r evalsha ffd632c7d33e571e9f24556ebed26c3479a87130 0} e catch {r evalsha ffd632c7d33e571e9f24556ebed26c3479a87130 0} e
set _ $e set _ $e
} {NOSCRIPT*} } {NOSCRIPT*}
} ;# is_eval
test {EVAL - Redis integer -> Lua type conversion} { test {EVAL - Redis integer -> Lua type conversion} {
r set x 0 r set x 0
r eval { run_script {
local foo = redis.pcall('incr',KEYS[1]) local foo = redis.pcall('incr',KEYS[1])
return {type(foo),foo} return {type(foo),foo}
} 1 x } 1 x
...@@ -71,7 +111,7 @@ start_server {tags {"scripting"}} { ...@@ -71,7 +111,7 @@ start_server {tags {"scripting"}} {
test {EVAL - Redis bulk -> Lua type conversion} { test {EVAL - Redis bulk -> Lua type conversion} {
r set mykey myval r set mykey myval
r eval { run_script {
local foo = redis.pcall('get',KEYS[1]) local foo = redis.pcall('get',KEYS[1])
return {type(foo),foo} return {type(foo),foo}
} 1 mykey } 1 mykey
...@@ -82,14 +122,14 @@ start_server {tags {"scripting"}} { ...@@ -82,14 +122,14 @@ start_server {tags {"scripting"}} {
r rpush mylist a r rpush mylist a
r rpush mylist b r rpush mylist b
r rpush mylist c r rpush mylist c
r eval { run_script {
local foo = redis.pcall('lrange',KEYS[1],0,-1) local foo = redis.pcall('lrange',KEYS[1],0,-1)
return {type(foo),foo[1],foo[2],foo[3],# foo} return {type(foo),foo[1],foo[2],foo[3],# foo}
} 1 mylist } 1 mylist
} {table a b c 3} } {table a b c 3}
test {EVAL - Redis status reply -> Lua type conversion} { test {EVAL - Redis status reply -> Lua type conversion} {
r eval { run_script {
local foo = redis.pcall('set',KEYS[1],'myval') local foo = redis.pcall('set',KEYS[1],'myval')
return {type(foo),foo['ok']} return {type(foo),foo['ok']}
} 1 mykey } 1 mykey
...@@ -97,7 +137,7 @@ start_server {tags {"scripting"}} { ...@@ -97,7 +137,7 @@ start_server {tags {"scripting"}} {
test {EVAL - Redis error reply -> Lua type conversion} { test {EVAL - Redis error reply -> Lua type conversion} {
r set mykey myval r set mykey myval
r eval { run_script {
local foo = redis.pcall('incr',KEYS[1]) local foo = redis.pcall('incr',KEYS[1])
return {type(foo),foo['err']} return {type(foo),foo['err']}
} 1 mykey } 1 mykey
...@@ -105,7 +145,7 @@ start_server {tags {"scripting"}} { ...@@ -105,7 +145,7 @@ start_server {tags {"scripting"}} {
test {EVAL - Redis nil bulk reply -> Lua type conversion} { test {EVAL - Redis nil bulk reply -> Lua type conversion} {
r del mykey r del mykey
r eval { run_script {
local foo = redis.pcall('get',KEYS[1]) local foo = redis.pcall('get',KEYS[1])
return {type(foo),foo == false} return {type(foo),foo == false}
} 1 mykey } 1 mykey
...@@ -115,13 +155,13 @@ start_server {tags {"scripting"}} { ...@@ -115,13 +155,13 @@ start_server {tags {"scripting"}} {
r set mykey "this is DB 9" r set mykey "this is DB 9"
r select 10 r select 10
r set mykey "this is DB 10" r set mykey "this is DB 10"
r eval {return redis.pcall('get',KEYS[1])} 1 mykey run_script {return redis.pcall('get',KEYS[1])} 1 mykey
} {this is DB 10} {singledb:skip} } {this is DB 10} {singledb:skip}
test {EVAL - SELECT inside Lua should not affect the caller} { test {EVAL - SELECT inside Lua should not affect the caller} {
# here we DB 10 is selected # here we DB 10 is selected
r set mykey "original value" r set mykey "original value"
r eval {return redis.pcall('select','9')} 0 run_script {return redis.pcall('select','9')} 0
set res [r get mykey] set res [r get mykey]
r select 9 r select 9
set res set res
...@@ -131,7 +171,7 @@ start_server {tags {"scripting"}} { ...@@ -131,7 +171,7 @@ start_server {tags {"scripting"}} {
test {EVAL - Script can't run more than configured time limit} { test {EVAL - Script can't run more than configured time limit} {
r config set lua-time-limit 1 r config set lua-time-limit 1
catch { catch {
r eval { run_script {
local i = 0 local i = 0
while true do i=i+1 end while true do i=i+1 end
} 0 } 0
...@@ -142,71 +182,74 @@ start_server {tags {"scripting"}} { ...@@ -142,71 +182,74 @@ start_server {tags {"scripting"}} {
test {EVAL - Scripts can't run blpop command} { test {EVAL - Scripts can't run blpop command} {
set e {} set e {}
catch {r eval {return redis.pcall('blpop','x',0)} 0} e catch {run_script {return redis.pcall('blpop','x',0)} 0} e
set e set e
} {*not allowed*} } {*not allowed*}
test {EVAL - Scripts can't run brpop command} { test {EVAL - Scripts can't run brpop command} {
set e {} set e {}
catch {r eval {return redis.pcall('brpop','empty_list',0)} 0} e catch {run_script {return redis.pcall('brpop','empty_list',0)} 0} e
set e set e
} {*not allowed*} } {*not allowed*}
test {EVAL - Scripts can't run brpoplpush command} { test {EVAL - Scripts can't run brpoplpush command} {
set e {} set e {}
catch {r eval {return redis.pcall('brpoplpush','empty_list1', 'empty_list2',0)} 0} e catch {run_script {return redis.pcall('brpoplpush','empty_list1', 'empty_list2',0)} 0} e
set e set e
} {*not allowed*} } {*not allowed*}
test {EVAL - Scripts can't run blmove command} { test {EVAL - Scripts can't run blmove command} {
set e {} set e {}
catch {r eval {return redis.pcall('blmove','empty_list1', 'empty_list2', 'LEFT', 'LEFT', 0)} 0} e catch {run_script {return redis.pcall('blmove','empty_list1', 'empty_list2', 'LEFT', 'LEFT', 0)} 0} e
set e set e
} {*not allowed*} } {*not allowed*}
test {EVAL - Scripts can't run bzpopmin command} { test {EVAL - Scripts can't run bzpopmin command} {
set e {} set e {}
catch {r eval {return redis.pcall('bzpopmin','empty_zset', 0)} 0} e catch {run_script {return redis.pcall('bzpopmin','empty_zset', 0)} 0} e
set e set e
} {*not allowed*} } {*not allowed*}
test {EVAL - Scripts can't run bzpopmax command} { test {EVAL - Scripts can't run bzpopmax command} {
set e {} set e {}
catch {r eval {return redis.pcall('bzpopmax','empty_zset', 0)} 0} e catch {run_script {return redis.pcall('bzpopmax','empty_zset', 0)} 0} e
set e set e
} {*not allowed*} } {*not allowed*}
test {EVAL - Scripts can't run XREAD and XREADGROUP with BLOCK option} { test {EVAL - Scripts can't run XREAD and XREADGROUP with BLOCK option} {
r del s r del s
r xgroup create s g $ MKSTREAM r xgroup create s g $ MKSTREAM
set res [r eval {return redis.pcall('xread','STREAMS','s','$')} 1 s] set res [run_script {return redis.pcall('xread','STREAMS','s','$')} 1 s]
assert {$res eq {}} assert {$res eq {}}
assert_error "*xread command is not allowed with BLOCK option from scripts" {r eval {return redis.pcall('xread','BLOCK',0,'STREAMS','s','$')} 1 s} assert_error "*xread command is not allowed with BLOCK option from scripts" {run_script {return redis.pcall('xread','BLOCK',0,'STREAMS','s','$')} 1 s}
set res [r eval {return redis.pcall('xreadgroup','group','g','c','STREAMS','s','>')} 1 s] set res [run_script {return redis.pcall('xreadgroup','group','g','c','STREAMS','s','>')} 1 s]
assert {$res eq {}} assert {$res eq {}}
assert_error "*xreadgroup command is not allowed with BLOCK option from scripts" {r eval {return redis.pcall('xreadgroup','group','g','c','BLOCK',0,'STREAMS','s','>')} 1 s} assert_error "*xreadgroup command is not allowed with BLOCK option from scripts" {run_script {return redis.pcall('xreadgroup','group','g','c','BLOCK',0,'STREAMS','s','>')} 1 s}
} }
if {$is_eval eq 1} {
# only is_eval Lua can not execute randomkey
test {EVAL - Scripts can't run certain commands} { test {EVAL - Scripts can't run certain commands} {
set e {} set e {}
r debug lua-always-replicate-commands 0 r debug lua-always-replicate-commands 0
catch { catch {
r eval "redis.pcall('randomkey'); return redis.pcall('set','x','ciao')" 0 run_script "redis.pcall('randomkey'); return redis.pcall('set','x','ciao')" 0
} e } e
r debug lua-always-replicate-commands 1 r debug lua-always-replicate-commands 1
set e set e
} {*not allowed after*} {needs:debug} } {*not allowed after*} {needs:debug}
} ;# is_eval
test {EVAL - No arguments to redis.call/pcall is considered an error} { test {EVAL - No arguments to redis.call/pcall is considered an error} {
set e {} set e {}
catch {r eval {return redis.call()} 0} e catch {run_script {return redis.call()} 0} e
set e set e
} {*one argument*} } {*one argument*}
test {EVAL - redis.call variant raises a Lua error on Redis cmd error (1)} { test {EVAL - redis.call variant raises a Lua error on Redis cmd error (1)} {
set e {} set e {}
catch { catch {
r eval "redis.call('nosuchcommand')" 0 run_script "redis.call('nosuchcommand')" 0
} e } e
set e set e
} {*Unknown Redis*} } {*Unknown Redis*}
...@@ -214,7 +257,7 @@ start_server {tags {"scripting"}} { ...@@ -214,7 +257,7 @@ start_server {tags {"scripting"}} {
test {EVAL - redis.call variant raises a Lua error on Redis cmd error (1)} { test {EVAL - redis.call variant raises a Lua error on Redis cmd error (1)} {
set e {} set e {}
catch { catch {
r eval "redis.call('get','a','b','c')" 0 run_script "redis.call('get','a','b','c')" 0
} e } e
set e set e
} {*number of args*} } {*number of args*}
...@@ -223,7 +266,7 @@ start_server {tags {"scripting"}} { ...@@ -223,7 +266,7 @@ start_server {tags {"scripting"}} {
set e {} set e {}
r set foo bar r set foo bar
catch { catch {
r eval {redis.call('lpush',KEYS[1],'val')} 1 foo run_script {redis.call('lpush',KEYS[1],'val')} 1 foo
} e } e
set e set e
} {*against a key*} } {*against a key*}
...@@ -232,7 +275,7 @@ start_server {tags {"scripting"}} { ...@@ -232,7 +275,7 @@ start_server {tags {"scripting"}} {
# We must return the table as a string because otherwise # We must return the table as a string because otherwise
# Redis converts floats to ints and we get 0 and 1023 instead # Redis converts floats to ints and we get 0 and 1023 instead
# of 0.0003 and 1023.2 as the parsed output. # of 0.0003 and 1023.2 as the parsed output.
r eval {return run_script {return
table.concat( table.concat(
cjson.decode( cjson.decode(
"[0.0, -5e3, -1, 0.3e-3, 1023.2, 0e10]"), " ") "[0.0, -5e3, -1, 0.3e-3, 1023.2, 0e10]"), " ")
...@@ -240,13 +283,13 @@ start_server {tags {"scripting"}} { ...@@ -240,13 +283,13 @@ start_server {tags {"scripting"}} {
} {0 -5000 -1 0.0003 1023.2 0} } {0 -5000 -1 0.0003 1023.2 0}
test {EVAL - JSON string decoding} { test {EVAL - JSON string decoding} {
r eval {local decoded = cjson.decode('{"keya": "a", "keyb": "b"}') run_script {local decoded = cjson.decode('{"keya": "a", "keyb": "b"}')
return {decoded.keya, decoded.keyb} return {decoded.keya, decoded.keyb}
} 0 } 0
} {a b} } {a b}
test {EVAL - cmsgpack can pack double?} { test {EVAL - cmsgpack can pack double?} {
r eval {local encoded = cmsgpack.pack(0.1) run_script {local encoded = cmsgpack.pack(0.1)
local h = "" local h = ""
for i = 1, #encoded do for i = 1, #encoded do
h = h .. string.format("%02x",string.byte(encoded,i)) h = h .. string.format("%02x",string.byte(encoded,i))
...@@ -256,7 +299,7 @@ start_server {tags {"scripting"}} { ...@@ -256,7 +299,7 @@ start_server {tags {"scripting"}} {
} {cb3fb999999999999a} } {cb3fb999999999999a}
test {EVAL - cmsgpack can pack negative int64?} { test {EVAL - cmsgpack can pack negative int64?} {
r eval {local encoded = cmsgpack.pack(-1099511627776) run_script {local encoded = cmsgpack.pack(-1099511627776)
local h = "" local h = ""
for i = 1, #encoded do for i = 1, #encoded do
h = h .. string.format("%02x",string.byte(encoded,i)) h = h .. string.format("%02x",string.byte(encoded,i))
...@@ -266,7 +309,7 @@ start_server {tags {"scripting"}} { ...@@ -266,7 +309,7 @@ start_server {tags {"scripting"}} {
} {d3ffffff0000000000} } {d3ffffff0000000000}
test {EVAL - cmsgpack can pack and unpack circular references?} { test {EVAL - cmsgpack can pack and unpack circular references?} {
r eval {local a = {x=nil,y=5} run_script {local a = {x=nil,y=5}
local b = {x=a} local b = {x=a}
a['x'] = b a['x'] = b
local encoded = cmsgpack.pack(a) local encoded = cmsgpack.pack(a)
...@@ -298,7 +341,7 @@ start_server {tags {"scripting"}} { ...@@ -298,7 +341,7 @@ start_server {tags {"scripting"}} {
} {82a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a178c0 1 1} } {82a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a17882a17905a17881a178c0 1 1}
test {EVAL - Numerical sanity check from bitop} { test {EVAL - Numerical sanity check from bitop} {
r eval {assert(0x7fffffff == 2147483647, "broken hex literals"); run_script {assert(0x7fffffff == 2147483647, "broken hex literals");
assert(0xffffffff == -1 or 0xffffffff == 2^32-1, assert(0xffffffff == -1 or 0xffffffff == 2^32-1,
"broken hex literals"); "broken hex literals");
assert(tostring(-1) == "-1", "broken tostring()"); assert(tostring(-1) == "-1", "broken tostring()");
...@@ -309,7 +352,7 @@ start_server {tags {"scripting"}} { ...@@ -309,7 +352,7 @@ start_server {tags {"scripting"}} {
} {} } {}
test {EVAL - Verify minimal bitop functionality} { test {EVAL - Verify minimal bitop functionality} {
r eval {assert(bit.tobit(1) == 1); run_script {assert(bit.tobit(1) == 1);
assert(bit.band(1) == 1); assert(bit.band(1) == 1);
assert(bit.bxor(1,2) == 3); assert(bit.bxor(1,2) == 3);
assert(bit.bor(1,2,4,8,16,32,64,128) == 255) assert(bit.bor(1,2,4,8,16,32,64,128) == 255)
...@@ -317,20 +360,22 @@ start_server {tags {"scripting"}} { ...@@ -317,20 +360,22 @@ start_server {tags {"scripting"}} {
} {} } {}
test {EVAL - Able to parse trailing comments} { test {EVAL - Able to parse trailing comments} {
r eval {return 'hello' --trailing comment} 0 run_script {return 'hello' --trailing comment} 0
} {hello} } {hello}
test {EVAL_RO - Successful case} { test {EVAL_RO - Successful case} {
r set foo bar r set foo bar
assert_equal bar [r eval_ro {return redis.call('get', KEYS[1]);} 1 foo] assert_equal bar [run_script_ro {return redis.call('get', KEYS[1]);} 1 foo]
} }
test {EVAL_RO - Cannot run write commands} { test {EVAL_RO - Cannot run write commands} {
r set foo bar r set foo bar
catch {r eval_ro {redis.call('del', KEYS[1]);} 1 foo} e catch {run_script_ro {redis.call('del', KEYS[1]);} 1 foo} e
set e set e
} {*Write commands are not allowed from read-only scripts*} } {*Write commands are not allowed from read-only scripts*}
if {$is_eval eq 1} {
# script command is only relevant for is_eval Lua
test {SCRIPTING FLUSH - is able to clear the scripts cache?} { test {SCRIPTING FLUSH - is able to clear the scripts cache?} {
r set mykey myval r set mykey myval
set v [r evalsha fd758d1589d044dd850a6f05d52f2eefd27f033f 1 mykey] set v [r evalsha fd758d1589d044dd850a6f05d52f2eefd27f033f 1 mykey]
...@@ -361,6 +406,7 @@ start_server {tags {"scripting"}} { ...@@ -361,6 +406,7 @@ start_server {tags {"scripting"}} {
[r evalsha b534286061d4b9e4026607613b95c06c06015ae8 0] [r evalsha b534286061d4b9e4026607613b95c06c06015ae8 0]
} {b534286061d4b9e4026607613b95c06c06015ae8 loaded} } {b534286061d4b9e4026607613b95c06c06015ae8 loaded}
# reply oredering is only relevant for is_eval Lua
test "In the context of Lua the output of random commands gets ordered" { test "In the context of Lua the output of random commands gets ordered" {
r debug lua-always-replicate-commands 0 r debug lua-always-replicate-commands 0
r del myset r del myset
...@@ -387,19 +433,20 @@ start_server {tags {"scripting"}} { ...@@ -387,19 +433,20 @@ start_server {tags {"scripting"}} {
r sadd myset a b c r sadd myset a b c
r eval {return redis.call('sort',KEYS[1],'by','_','get','#','get','_:*')} 1 myset r eval {return redis.call('sort',KEYS[1],'by','_','get','#','get','_:*')} 1 myset
} {a {} b {} c {}} {cluster:skip} } {a {} b {} c {}} {cluster:skip}
} ;# is_eval
test "redis.sha1hex() implementation" { test "redis.sha1hex() implementation" {
list [r eval {return redis.sha1hex('')} 0] \ list [run_script {return redis.sha1hex('')} 0] \
[r eval {return redis.sha1hex('Pizza & Mandolino')} 0] [run_script {return redis.sha1hex('Pizza & Mandolino')} 0]
} {da39a3ee5e6b4b0d3255bfef95601890afd80709 74822d82031af7493c20eefa13bd07ec4fada82f} } {da39a3ee5e6b4b0d3255bfef95601890afd80709 74822d82031af7493c20eefa13bd07ec4fada82f}
test {Globals protection reading an undeclared global variable} { test {Globals protection reading an undeclared global variable} {
catch {r eval {return a} 0} e catch {run_script {return a} 0} e
set e set e
} {*ERR*attempted to access * global*} } {*ERR*attempted to access * global*}
test {Globals protection setting an undeclared global*} { test {Globals protection setting an undeclared global*} {
catch {r eval {a=10} 0} e catch {run_script {a=10} 0} e
set e set e
} {*ERR*attempted to create global*} } {*ERR*attempted to create global*}
...@@ -417,14 +464,16 @@ start_server {tags {"scripting"}} { ...@@ -417,14 +464,16 @@ start_server {tags {"scripting"}} {
} }
r set foo 5 r set foo 5
set res {} set res {}
lappend res [r eval $decr_if_gt 1 foo 2] lappend res [run_script $decr_if_gt 1 foo 2]
lappend res [r eval $decr_if_gt 1 foo 2] lappend res [run_script $decr_if_gt 1 foo 2]
lappend res [r eval $decr_if_gt 1 foo 2] lappend res [run_script $decr_if_gt 1 foo 2]
lappend res [r eval $decr_if_gt 1 foo 2] lappend res [run_script $decr_if_gt 1 foo 2]
lappend res [r eval $decr_if_gt 1 foo 2] lappend res [run_script $decr_if_gt 1 foo 2]
set res set res
} {4 3 2 2 2} } {4 3 2 2 2}
if {$is_eval eq 1} {
# random handling is only relevant for is_eval Lua
test {Scripting engine resets PRNG at every script execution} { test {Scripting engine resets PRNG at every script execution} {
set rand1 [r eval {return tostring(math.random())} 0] set rand1 [r eval {return tostring(math.random())} 0]
set rand2 [r eval {return tostring(math.random())} 0] set rand2 [r eval {return tostring(math.random())} 0]
...@@ -444,13 +493,14 @@ start_server {tags {"scripting"}} { ...@@ -444,13 +493,14 @@ start_server {tags {"scripting"}} {
assert_equal $rand1 $rand2 assert_equal $rand1 $rand2
assert {$rand2 ne $rand3} assert {$rand2 ne $rand3}
} }
} ;# is_eval
test {EVAL does not leak in the Lua stack} { test {EVAL does not leak in the Lua stack} {
r set x 0 r set x 0
# Use a non blocking client to speedup the loop. # Use a non blocking client to speedup the loop.
set rd [redis_deferring_client] set rd [redis_deferring_client]
for {set j 0} {$j < 10000} {incr j} { for {set j 0} {$j < 10000} {incr j} {
$rd eval {return redis.call("incr",KEYS[1])} 1 x run_script_on_connection $rd {return redis.call("incr",KEYS[1])} 1 x
} }
for {set j 0} {$j < 10000} {incr j} { for {set j 0} {$j < 10000} {incr j} {
$rd read $rd read
...@@ -464,9 +514,9 @@ start_server {tags {"scripting"}} { ...@@ -464,9 +514,9 @@ start_server {tags {"scripting"}} {
r flushall r flushall
r config set appendonly yes r config set appendonly yes
r config set aof-use-rdb-preamble no r config set aof-use-rdb-preamble no
r eval {redis.call("set",KEYS[1],"100")} 1 foo run_script {redis.call("set",KEYS[1],"100")} 1 foo
r eval {redis.call("incr",KEYS[1])} 1 foo run_script {redis.call("incr",KEYS[1])} 1 foo
r eval {redis.call("incr",KEYS[1])} 1 foo run_script {redis.call("incr",KEYS[1])} 1 foo
wait_for_condition 50 100 { wait_for_condition 50 100 {
[s aof_rewrite_in_progress] == 0 [s aof_rewrite_in_progress] == 0
} else { } else {
...@@ -481,6 +531,8 @@ start_server {tags {"scripting"}} { ...@@ -481,6 +531,8 @@ start_server {tags {"scripting"}} {
set res set res
} {102} {external:skip} } {102} {external:skip}
if {$is_eval eq 1} {
# script propagation is irrelevant on functions
test {EVAL timeout from AOF} { test {EVAL timeout from AOF} {
# generate a long running script that is propagated to the AOF as script # generate a long running script that is propagated to the AOF as script
# make sure that the script times out during loading # make sure that the script times out during loading
...@@ -528,9 +580,11 @@ start_server {tags {"scripting"}} { ...@@ -528,9 +580,11 @@ start_server {tags {"scripting"}} {
assert {[r mget a{t} b{t} c{t} d{t}] eq {1 2 3 4}} assert {[r mget a{t} b{t} c{t} d{t}] eq {1 2 3 4}}
assert {[r spop myset] eq {}} assert {[r spop myset] eq {}}
} }
} ;# is_eval
test {Call Redis command with many args from Lua (issue #1764)} { test {Call Redis command with many args from Lua (issue #1764)} {
r eval { run_script {
local i local i
local x={} local x={}
redis.call('del','mylist') redis.call('del','mylist')
...@@ -543,7 +597,7 @@ start_server {tags {"scripting"}} { ...@@ -543,7 +597,7 @@ start_server {tags {"scripting"}} {
} {1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100} } {1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100}
test {Number conversion precision test (issue #1118)} { test {Number conversion precision test (issue #1118)} {
r eval { run_script {
local value = 9007199254740991 local value = 9007199254740991
redis.call("set","foo",value) redis.call("set","foo",value)
return redis.call("get","foo") return redis.call("get","foo")
...@@ -551,19 +605,19 @@ start_server {tags {"scripting"}} { ...@@ -551,19 +605,19 @@ start_server {tags {"scripting"}} {
} {9007199254740991} } {9007199254740991}
test {String containing number precision test (regression of issue #1118)} { test {String containing number precision test (regression of issue #1118)} {
r eval { run_script {
redis.call("set", "key", "12039611435714932082") redis.call("set", "key", "12039611435714932082")
return redis.call("get", "key") return redis.call("get", "key")
} 1 key } 1 key
} {12039611435714932082} } {12039611435714932082}
test {Verify negative arg count is error instead of crash (issue #1842)} { test {Verify negative arg count is error instead of crash (issue #1842)} {
catch { r eval { return "hello" } -12 } e catch { run_script { return "hello" } -12 } e
set e set e
} {ERR Number of keys can't be negative} } {ERR Number of keys can't be negative}
test {Correct handling of reused argv (issue #1939)} { test {Correct handling of reused argv (issue #1939)} {
r eval { run_script {
for i = 0, 10 do for i = 0, 10 do
redis.call('SET', 'a{t}', '1') redis.call('SET', 'a{t}', '1')
redis.call('MGET', 'a{t}', 'b{t}', 'c{t}') redis.call('MGET', 'a{t}', 'b{t}', 'c{t}')
...@@ -576,7 +630,7 @@ start_server {tags {"scripting"}} { ...@@ -576,7 +630,7 @@ start_server {tags {"scripting"}} {
test {Functions in the Redis namespace are able to report errors} { test {Functions in the Redis namespace are able to report errors} {
catch { catch {
r eval { run_script {
redis.sha1hex() redis.sha1hex()
} 0 } 0
} e } e
...@@ -594,22 +648,22 @@ start_server {tags {"scripting"}} { ...@@ -594,22 +648,22 @@ start_server {tags {"scripting"}} {
assert_equal $res $expected_dict assert_equal $res $expected_dict
# Test RESP3 client with script in both RESP2 and RESP3 modes # Test RESP3 client with script in both RESP2 and RESP3 modes
set res [r eval {redis.setresp(3); return redis.call('hgetall', KEYS[1])} 1 hash] set res [run_script {redis.setresp(3); return redis.call('hgetall', KEYS[1])} 1 hash]
assert_equal $res $expected_dict assert_equal $res $expected_dict
set res [r eval {redis.setresp(2); return redis.call('hgetall', KEYS[1])} 1 hash] set res [run_script {redis.setresp(2); return redis.call('hgetall', KEYS[1])} 1 hash]
assert_equal $res $expected_list assert_equal $res $expected_list
# Test RESP2 client with script in both RESP2 and RESP3 modes # Test RESP2 client with script in both RESP2 and RESP3 modes
r HELLO 2 r HELLO 2
set res [r eval {redis.setresp(3); return redis.call('hgetall', KEYS[1])} 1 hash] set res [run_script {redis.setresp(3); return redis.call('hgetall', KEYS[1])} 1 hash]
assert_equal $res $expected_list assert_equal $res $expected_list
set res [r eval {redis.setresp(2); return redis.call('hgetall', KEYS[1])} 1 hash] set res [run_script {redis.setresp(2); return redis.call('hgetall', KEYS[1])} 1 hash]
assert_equal $res $expected_list assert_equal $res $expected_list
} }
test {Script return recursive object} { test {Script return recursive object} {
r readraw 1 r readraw 1
set res [r eval {local a = {}; local b = {a}; a[1] = b; return a} 0] set res [run_script {local a = {}; local b = {a}; a[1] = b; return a} 0]
# drain the response # drain the response
while {true} { while {true} {
if {$res == "-ERR reached lua stack limit"} { if {$res == "-ERR reached lua stack limit"} {
...@@ -640,11 +694,11 @@ start_server {tags {"scripting"}} { ...@@ -640,11 +694,11 @@ start_server {tags {"scripting"}} {
test {Timedout read-only scripts can be killed by SCRIPT KILL} { test {Timedout read-only scripts can be killed by SCRIPT KILL} {
set rd [redis_deferring_client] set rd [redis_deferring_client]
r config set lua-time-limit 10 r config set lua-time-limit 10
$rd eval {while true do end} 0 run_script_on_connection $rd {while true do end} 0
after 200 after 200
catch {r ping} e catch {r ping} e
assert_match {BUSY*} $e assert_match {BUSY*} $e
r script kill kill_script
after 200 ; # Give some time to Lua to call the hook again... after 200 ; # Give some time to Lua to call the hook again...
assert_equal [r ping] "PONG" assert_equal [r ping] "PONG"
$rd close $rd close
...@@ -653,7 +707,7 @@ start_server {tags {"scripting"}} { ...@@ -653,7 +707,7 @@ start_server {tags {"scripting"}} {
test {Timedout read-only scripts can be killed by SCRIPT KILL even when use pcall} { test {Timedout read-only scripts can be killed by SCRIPT KILL even when use pcall} {
set rd [redis_deferring_client] set rd [redis_deferring_client]
r config set lua-time-limit 10 r config set lua-time-limit 10
$rd eval {local f = function() while 1 do redis.call('ping') end end while 1 do pcall(f) end} 0 run_script_on_connection $rd {local f = function() while 1 do redis.call('ping') end end while 1 do pcall(f) end} 0
wait_for_condition 50 100 { wait_for_condition 50 100 {
[catch {r ping} e] == 1 [catch {r ping} e] == 1
...@@ -663,7 +717,7 @@ start_server {tags {"scripting"}} { ...@@ -663,7 +717,7 @@ start_server {tags {"scripting"}} {
catch {r ping} e catch {r ping} e
assert_match {BUSY*} $e assert_match {BUSY*} $e
r script kill kill_script
wait_for_condition 50 100 { wait_for_condition 50 100 {
[catch {r ping} e] == 0 [catch {r ping} e] == 0
...@@ -685,8 +739,14 @@ start_server {tags {"scripting"}} { ...@@ -685,8 +739,14 @@ start_server {tags {"scripting"}} {
# senging (in a pipeline): # senging (in a pipeline):
# 1. eval "while 1 do redis.call('ping') end" 0 # 1. eval "while 1 do redis.call('ping') end" 0
# 2. ping # 2. ping
set buf "*3\r\n\$4\r\neval\r\n\$33\r\nwhile 1 do redis.call('ping') end\r\n\$1\r\n0\r\n" if {$is_eval == 1} {
append buf "*1\r\n\$4\r\nping\r\n" set buf "*3\r\n\$4\r\neval\r\n\$33\r\nwhile 1 do redis.call('ping') end\r\n\$1\r\n0\r\n"
append buf "*1\r\n\$4\r\nping\r\n"
} else {
set buf "*6\r\n\$8\r\nfunction\r\n\$6\r\ncreate\r\n\$3\r\nlua\r\n\$4\r\ntest\r\n\$7\r\nreplace\r\n\$33\r\nwhile 1 do redis.call('ping') end\r\n"
append buf "*3\r\n\$5\r\nfcall\r\n\$4\r\ntest\r\n\$1\r\n0\r\n"
append buf "*1\r\n\$4\r\nping\r\n"
}
$rd write $buf $rd write $buf
$rd flush $rd flush
...@@ -698,7 +758,7 @@ start_server {tags {"scripting"}} { ...@@ -698,7 +758,7 @@ start_server {tags {"scripting"}} {
catch {r ping} e catch {r ping} e
assert_match {BUSY*} $e assert_match {BUSY*} $e
r script kill kill_script
wait_for_condition 50 100 { wait_for_condition 50 100 {
[catch {r ping} e] == 0 [catch {r ping} e] == 0
} else { } else {
...@@ -706,6 +766,11 @@ start_server {tags {"scripting"}} { ...@@ -706,6 +766,11 @@ start_server {tags {"scripting"}} {
} }
assert_equal [r ping] "PONG" assert_equal [r ping] "PONG"
if {$is_eval == 0} {
# read the ok reply of function create
assert_match {OK} [$rd read]
}
catch {$rd read} res catch {$rd read} res
assert_match {*killed by user*} $res assert_match {*killed by user*} $res
...@@ -717,18 +782,18 @@ start_server {tags {"scripting"}} { ...@@ -717,18 +782,18 @@ start_server {tags {"scripting"}} {
test {Timedout script link is still usable after Lua returns} { test {Timedout script link is still usable after Lua returns} {
r config set lua-time-limit 10 r config set lua-time-limit 10
r eval {for i=1,100000 do redis.call('ping') end return 'ok'} 0 run_script {for i=1,100000 do redis.call('ping') end return 'ok'} 0
r ping r ping
} {PONG} } {PONG}
test {Timedout scripts that modified data can't be killed by SCRIPT KILL} { test {Timedout scripts that modified data can't be killed by SCRIPT KILL} {
set rd [redis_deferring_client] set rd [redis_deferring_client]
r config set lua-time-limit 10 r config set lua-time-limit 10
$rd eval {redis.call('set',KEYS[1],'y'); while true do end} 1 x run_script_on_connection $rd {redis.call('set',KEYS[1],'y'); while true do end} 1 x
after 200 after 200
catch {r ping} e catch {r ping} e
assert_match {BUSY*} $e assert_match {BUSY*} $e
catch {r script kill} e catch {kill_script} e
assert_match {UNKILLABLE*} $e assert_match {UNKILLABLE*} $e
catch {r ping} e catch {r ping} e
assert_match {BUSY*} $e assert_match {BUSY*} $e
...@@ -761,11 +826,11 @@ foreach cmdrepl {0 1} { ...@@ -761,11 +826,11 @@ foreach cmdrepl {0 1} {
# One with an error, but still executing a command. # One with an error, but still executing a command.
# SHA is: 67164fc43fa971f76fd1aaeeaf60c1c178d25876 # SHA is: 67164fc43fa971f76fd1aaeeaf60c1c178d25876
catch { catch {
r eval {redis.call('incr',KEYS[1]); redis.call('nonexisting')} 1 x run_script {redis.call('incr',KEYS[1]); redis.call('nonexisting')} 1 x
} }
# One command is correct: # One command is correct:
# SHA is: 6f5ade10a69975e903c6d07b10ea44c6382381a5 # SHA is: 6f5ade10a69975e903c6d07b10ea44c6382381a5
r eval {return redis.call('incr',KEYS[1])} 1 x run_script {return redis.call('incr',KEYS[1])} 1 x
} {2} } {2}
test "Connect a replica to the master instance $rt" { test "Connect a replica to the master instance $rt" {
...@@ -778,6 +843,7 @@ foreach cmdrepl {0 1} { ...@@ -778,6 +843,7 @@ foreach cmdrepl {0 1} {
} }
} }
if {$is_eval eq 1} {
test "Now use EVALSHA against the master, with both SHAs $rt" { test "Now use EVALSHA against the master, with both SHAs $rt" {
# The server should replicate successful and unsuccessful # The server should replicate successful and unsuccessful
# commands as EVAL instead of EVALSHA. # commands as EVAL instead of EVALSHA.
...@@ -794,11 +860,12 @@ foreach cmdrepl {0 1} { ...@@ -794,11 +860,12 @@ foreach cmdrepl {0 1} {
fail "Expected 4 in x, but value is '[r -1 get x]'" fail "Expected 4 in x, but value is '[r -1 get x]'"
} }
} }
} ;# is_eval
test "Replication of script multiple pushes to list with BLPOP $rt" { test "Replication of script multiple pushes to list with BLPOP $rt" {
set rd [redis_deferring_client] set rd [redis_deferring_client]
$rd brpop a 0 $rd brpop a 0
r eval { run_script {
redis.call("lpush",KEYS[1],"1"); redis.call("lpush",KEYS[1],"1");
redis.call("lpush",KEYS[1],"2"); redis.call("lpush",KEYS[1],"2");
} 1 a } 1 a
...@@ -812,6 +879,7 @@ foreach cmdrepl {0 1} { ...@@ -812,6 +879,7 @@ foreach cmdrepl {0 1} {
set res set res
} {a 1} } {a 1}
if {$is_eval eq 1} {
test "EVALSHA replication when first call is readonly $rt" { test "EVALSHA replication when first call is readonly $rt" {
r del x r del x
r eval {if tonumber(ARGV[1]) > 0 then redis.call('incr', KEYS[1]) end} 1 x 0 r eval {if tonumber(ARGV[1]) > 0 then redis.call('incr', KEYS[1]) end} 1 x 0
...@@ -823,16 +891,17 @@ foreach cmdrepl {0 1} { ...@@ -823,16 +891,17 @@ foreach cmdrepl {0 1} {
fail "Expected 1 in x, but value is '[r -1 get x]'" fail "Expected 1 in x, but value is '[r -1 get x]'"
} }
} }
} ;# is_eval
test "Lua scripts using SELECT are replicated correctly $rt" { test "Lua scripts using SELECT are replicated correctly $rt" {
r eval { run_script {
redis.call("set","foo1","bar1") redis.call("set","foo1","bar1")
redis.call("select","10") redis.call("select","10")
redis.call("incr","x") redis.call("incr","x")
redis.call("select","11") redis.call("select","11")
redis.call("incr","z") redis.call("incr","z")
} 0 } 0
r eval { run_script {
redis.call("set","foo1","bar1") redis.call("set","foo1","bar1")
redis.call("select","10") redis.call("select","10")
redis.call("incr","x") redis.call("incr","x")
...@@ -861,6 +930,8 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -861,6 +930,8 @@ start_server {tags {"scripting repl external:skip"}} {
} }
} }
if {$is_eval eq 1} {
# replicate_commands is the default on Redis Function
test "Redis.replicate_commands() must be issued before any write" { test "Redis.replicate_commands() must be issued before any write" {
r eval { r eval {
redis.call('set','foo','bar'); redis.call('set','foo','bar');
...@@ -884,11 +955,11 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -884,11 +955,11 @@ start_server {tags {"scripting repl external:skip"}} {
r debug lua-always-replicate-commands 1 r debug lua-always-replicate-commands 1
set e set e
} {*only after turning on*} } {*only after turning on*}
} ;# is_eval
test "Redis.set_repl() don't accept invalid values" { test "Redis.set_repl() don't accept invalid values" {
catch { catch {
r eval { run_script {
redis.replicate_commands();
redis.set_repl(12345); redis.set_repl(12345);
} 0 } 0
} e } e
...@@ -897,8 +968,7 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -897,8 +968,7 @@ start_server {tags {"scripting repl external:skip"}} {
test "Test selective replication of certain Redis commands from Lua" { test "Test selective replication of certain Redis commands from Lua" {
r del a b c d r del a b c d
r eval { run_script {
redis.replicate_commands();
redis.call('set','a','1'); redis.call('set','a','1');
redis.set_repl(redis.REPL_NONE); redis.set_repl(redis.REPL_NONE);
redis.call('set','b','2'); redis.call('set','b','2');
...@@ -924,24 +994,37 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -924,24 +994,37 @@ start_server {tags {"scripting repl external:skip"}} {
} }
test "PRNG is seeded randomly for command replication" { test "PRNG is seeded randomly for command replication" {
set a [ if {$is_eval eq 1} {
r eval { # on is_eval Lua we need to call redis.replicate_commands() to get real randomization
redis.replicate_commands(); set a [
return math.random()*100000; run_script {
} 0 redis.replicate_commands()
] return math.random()*100000;
set b [ } 0
r eval { ]
redis.replicate_commands(); set b [
return math.random()*100000; run_script {
} 0 redis.replicate_commands()
] return math.random()*100000;
} 0
]
} else {
set a [
run_script {
return math.random()*100000;
} 0
]
set b [
run_script {
return math.random()*100000;
} 0
]
}
assert {$a ne $b} assert {$a ne $b}
} }
test "Using side effects is not a problem with command replication" { test "Using side effects is not a problem with command replication" {
r eval { run_script {
redis.replicate_commands();
redis.call('set','time',redis.call('time')[1]) redis.call('set','time',redis.call('time')[1])
} 0 } 0
...@@ -956,6 +1039,7 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -956,6 +1039,7 @@ start_server {tags {"scripting repl external:skip"}} {
} }
} }
if {$is_eval eq 1} {
start_server {tags {"scripting external:skip"}} { start_server {tags {"scripting external:skip"}} {
r script debug sync r script debug sync
r eval {return 'hello'} 0 r eval {return 'hello'} 0
...@@ -984,12 +1068,13 @@ start_server {tags {"scripting needs:debug external:skip"}} { ...@@ -984,12 +1068,13 @@ start_server {tags {"scripting needs:debug external:skip"}} {
r write $cmd r write $cmd
r flush r flush
set ret [r read] set ret [r read]
assert_match {*Unknown Redis command called from Lua script*} $ret assert_match {*Unknown Redis command called from script*} $ret
# make sure the server is still ok # make sure the server is still ok
reconnect reconnect
assert_equal [r ping] {PONG} assert_equal [r ping] {PONG}
} }
} }
} ;# is_eval
start_server {tags {"scripting resp3 needs:debug"}} { start_server {tags {"scripting resp3 needs:debug"}} {
r debug set-disable-deny-scripts 1 r debug set-disable-deny-scripts 1
...@@ -999,7 +1084,7 @@ start_server {tags {"scripting resp3 needs:debug"}} { ...@@ -999,7 +1084,7 @@ start_server {tags {"scripting resp3 needs:debug"}} {
r readraw 1 r readraw 1
test {test resp3 big number protocol parsing} { test {test resp3 big number protocol parsing} {
set ret [r eval "redis.setresp($i);return redis.call('debug', 'protocol', 'bignum')" 0] set ret [run_script "redis.setresp($i);return redis.call('debug', 'protocol', 'bignum')" 0]
if {$client_proto == 2 || $i == 2} { if {$client_proto == 2 || $i == 2} {
# if either Lua or the clien is RESP2 the reply will be RESP2 # if either Lua or the clien is RESP2 the reply will be RESP2
assert_equal $ret {$37} assert_equal $ret {$37}
...@@ -1021,7 +1106,7 @@ start_server {tags {"scripting resp3 needs:debug"}} { ...@@ -1021,7 +1106,7 @@ start_server {tags {"scripting resp3 needs:debug"}} {
} }
test {test resp3 map protocol parsing} { test {test resp3 map protocol parsing} {
set ret [r eval "redis.setresp($i);return redis.call('debug', 'protocol', 'map')" 0] set ret [run_script "redis.setresp($i);return redis.call('debug', 'protocol', 'map')" 0]
if {$client_proto == 2 || $i == 2} { if {$client_proto == 2 || $i == 2} {
# if either Lua or the clien is RESP2 the reply will be RESP2 # if either Lua or the clien is RESP2 the reply will be RESP2
assert_equal $ret {*6} assert_equal $ret {*6}
...@@ -1034,7 +1119,7 @@ start_server {tags {"scripting resp3 needs:debug"}} { ...@@ -1034,7 +1119,7 @@ start_server {tags {"scripting resp3 needs:debug"}} {
} }
test {test resp3 set protocol parsing} { test {test resp3 set protocol parsing} {
set ret [r eval "redis.setresp($i);return redis.call('debug', 'protocol', 'set')" 0] set ret [run_script "redis.setresp($i);return redis.call('debug', 'protocol', 'set')" 0]
if {$client_proto == 2 || $i == 2} { if {$client_proto == 2 || $i == 2} {
# if either Lua or the clien is RESP2 the reply will be RESP2 # if either Lua or the clien is RESP2 the reply will be RESP2
assert_equal $ret {*3} assert_equal $ret {*3}
...@@ -1047,7 +1132,7 @@ start_server {tags {"scripting resp3 needs:debug"}} { ...@@ -1047,7 +1132,7 @@ start_server {tags {"scripting resp3 needs:debug"}} {
} }
test {test resp3 double protocol parsing} { test {test resp3 double protocol parsing} {
set ret [r eval "redis.setresp($i);return redis.call('debug', 'protocol', 'double')" 0] set ret [run_script "redis.setresp($i);return redis.call('debug', 'protocol', 'double')" 0]
if {$client_proto == 2 || $i == 2} { if {$client_proto == 2 || $i == 2} {
# if either Lua or the clien is RESP2 the reply will be RESP2 # if either Lua or the clien is RESP2 the reply will be RESP2
assert_equal $ret {$5} assert_equal $ret {$5}
...@@ -1058,7 +1143,7 @@ start_server {tags {"scripting resp3 needs:debug"}} { ...@@ -1058,7 +1143,7 @@ start_server {tags {"scripting resp3 needs:debug"}} {
} }
test {test resp3 null protocol parsing} { test {test resp3 null protocol parsing} {
set ret [r eval "redis.setresp($i);return redis.call('debug', 'protocol', 'null')" 0] set ret [run_script "redis.setresp($i);return redis.call('debug', 'protocol', 'null')" 0]
if {$client_proto == 2} { if {$client_proto == 2} {
# null is a special case in which a Lua client format does not effect the reply to the client # null is a special case in which a Lua client format does not effect the reply to the client
assert_equal $ret {$-1} assert_equal $ret {$-1}
...@@ -1068,7 +1153,7 @@ start_server {tags {"scripting resp3 needs:debug"}} { ...@@ -1068,7 +1153,7 @@ start_server {tags {"scripting resp3 needs:debug"}} {
} {} } {}
test {test resp3 verbatim protocol parsing} { test {test resp3 verbatim protocol parsing} {
set ret [r eval "redis.setresp($i);return redis.call('debug', 'protocol', 'verbatim')" 0] set ret [run_script "redis.setresp($i);return redis.call('debug', 'protocol', 'verbatim')" 0]
if {$client_proto == 2 || $i == 2} { if {$client_proto == 2 || $i == 2} {
# if either Lua or the clien is RESP2 the reply will be RESP2 # if either Lua or the clien is RESP2 the reply will be RESP2
assert_equal $ret {$25} assert_equal $ret {$25}
...@@ -1082,7 +1167,7 @@ start_server {tags {"scripting resp3 needs:debug"}} { ...@@ -1082,7 +1167,7 @@ start_server {tags {"scripting resp3 needs:debug"}} {
} }
test {test resp3 true protocol parsing} { test {test resp3 true protocol parsing} {
set ret [r eval "redis.setresp($i);return redis.call('debug', 'protocol', 'true')" 0] set ret [run_script "redis.setresp($i);return redis.call('debug', 'protocol', 'true')" 0]
if {$client_proto == 2 || $i == 2} { if {$client_proto == 2 || $i == 2} {
# if either Lua or the clien is RESP2 the reply will be RESP2 # if either Lua or the clien is RESP2 the reply will be RESP2
assert_equal $ret {:1} assert_equal $ret {:1}
...@@ -1092,7 +1177,7 @@ start_server {tags {"scripting resp3 needs:debug"}} { ...@@ -1092,7 +1177,7 @@ start_server {tags {"scripting resp3 needs:debug"}} {
} }
test {test resp3 false protocol parsing} { test {test resp3 false protocol parsing} {
set ret [r eval "redis.setresp($i);return redis.call('debug', 'protocol', 'false')" 0] set ret [run_script "redis.setresp($i);return redis.call('debug', 'protocol', 'false')" 0]
if {$client_proto == 2 || $i == 2} { if {$client_proto == 2 || $i == 2} {
# if either Lua or the clien is RESP2 the reply will be RESP2 # if either Lua or the clien is RESP2 the reply will be RESP2
assert_equal $ret {:0} assert_equal $ret {:0}
...@@ -1109,8 +1194,9 @@ start_server {tags {"scripting resp3 needs:debug"}} { ...@@ -1109,8 +1194,9 @@ start_server {tags {"scripting resp3 needs:debug"}} {
test {test resp3 attribute protocol parsing} { test {test resp3 attribute protocol parsing} {
# attributes are not (yet) expose to the script # attributes are not (yet) expose to the script
# So here we just check the parser handles them and they are ignored. # So here we just check the parser handles them and they are ignored.
r eval "redis.setresp(3);return redis.call('debug', 'protocol', 'attrib')" 0 run_script "redis.setresp(3);return redis.call('debug', 'protocol', 'attrib')" 0
} {Some real reply following the attribute} } {Some real reply following the attribute}
r debug set-disable-deny-scripts 0 r debug set-disable-deny-scripts 0
} }
} ;# foreach is_eval
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment