Unverified Commit 885f6b5c authored by Meir Shpilraien (Spielrein)'s avatar Meir Shpilraien (Spielrein) Committed by GitHub
Browse files

Redis Function Libraries (#10004)

# Redis Function Libraries

This PR implements Redis Functions Libraries as describe on: https://github.com/redis/redis/issues/9906.

Libraries purpose is to provide a better code sharing between functions by allowing to create multiple
functions in a single command. Functions that were created together can safely share code between
each other without worrying about compatibility issues and versioning.

Creating a new library is done using 'FUNCTION LOAD' command (full API is described below)

This PR introduces a new struct called libraryInfo, libraryInfo holds information about a library:
* name - name of the library
* engine - engine used to create the library
* code - library code
* description - library description
* functions - the functions exposed by the library

When Redis gets the `FUNCTION LOAD` command it creates a new empty libraryInfo.
Redis passes the `CODE` to the relevant engine alongside the empty libraryInfo.
As a result, the engine will create one or more functions by calling 'libraryCreateFunction'.
The new funcion will be added to the newly created libraryInfo. So far Everything is happening
locally on the libraryInfo so it is easy to abort the operation (in case of an error) by simply
freeing the libraryInfo. After the library info is fully constructed we start the joining phase by
which we will join the new library to the other libraries currently exist on Redis.
The joining phase make sure there is no function collision and add the library to the
librariesCtx (renamed from functionCtx). LibrariesCtx is used all around the code in the exact
same way as functionCtx was used (with respect to RDB loading, replicatio, ...).
The only difference is that apart from function dictionary (maps function name to functionInfo
object), the librariesCtx contains also a libraries dictionary that maps library name to libraryInfo object.

## New API
### FUNCTION LOAD
`FUNCTION LOAD <ENGINE> <LIBRARY NAME> [REPLACE] [DESCRIPTION <DESCRIPTION>] <CODE>`
Create a new library with the given parameters:
* ENGINE - REPLACE Engine name to use to create the library.
* LIBRARY NAME - The new library name.
* REPLACE - If the library already exists, replace it.
* DESCRIPTION - Library description.
* CODE - Library code.

Return "OK" on success, or error on the following cases:
* Library name already taken and REPLACE was not used
* Name collision with another existing library (even if replace was uses)
* Library registration failed by the engine (usually compilation error)

## Changed API
### FUNCTION LIST
`FUNCTION LIST [LIBRARYNAME <LIBRARY NAME PATTERN>] [WITHCODE]`
Command was modified to also allow getting libraries code (so `FUNCTION INFO` command is no longer
needed and removed). In addition the command gets an option argument, `LIBRARYNAME` allows you to
only get libraries that match the given `LIBRARYNAME` pattern. By default, it returns all libraries.

### INFO MEMORY
Added number of libraries to `INFO MEMORY`

### Commands flags
`DENYOOM` flag was set on `FUNCTION LOAD` and `FUNCTION RESTORE`. We consider those commands
as commands that add new data to the dateset (functions are data) and so we want to disallows
to run those commands on OOM.

## Removed API
* FUNCTION CREATE - Decided on https://github.com/redis/redis/issues/9906
* FUNCTION INFO - Decided on https://github.com/redis/redis/issues/9899

## Lua engine changes
When the Lua engine gets the code given on `FUNCTION LOAD` command, it immediately runs it, we call
this run the loading run. Loading run is not a usual script run, it is not possible to invoke any
Redis command from within the load run.
Instead there is a new API provided by `library` object. The new API's: 
* `redis.log` - behave the same as `redis.log`
* `redis.register_function` - register a new function to the library

The loading run purpose is to register functions using the new `redis.register_function` API.
Any attempt to use any other API will result in an error. In addition, the load run is has a time
limit of 500ms, error is raise on timeout and the entire operation is aborted.

### `redis.register_function`
`redis.register_function(<function_name>, <callback>, [<description>])`
This new API allows users to register a new function that will be linked to the newly created library.
This API can only be called during the load run (see definition above). Any attempt to use it outside
of the load run will result in an error.
The parameters pass to the API are:
* function_name - Function name (must be a Lua string)
* callback - Lua function object that will be called when the function is invokes using fcall/fcall_ro
* description - Function description, optional (must be a Lua string).

### Example
The following example creates a library called `lib` with 2 functions, `f1` and `f1`, returns 1 and 2 respectively:
```
local function f1(keys, args)
    return 1
end

local function f2(keys, args)
    return 2
end

redis.register_function('f1', f1)
redis.register_function('f2', f2)
```

Notice: Unlike `eval`, functions inside a library get the KEYS and ARGV as arguments to the
functions and not as global.

### Technical Details

On the load run we only want the user to be able to call a white list on API's. This way, in
the future, if new API's will be added, the new API's will not be available to the load run
unless specifically added to this white list. We put the while list on the `library` object and
make sure the `library` object is only available to the load run by using [lua_setfenv](https://www.lua.org/manual/5.1/manual.html#lua_setfenv) API. This API allows us to set
the `globals` of a function (and all the function it creates). Before starting the load run we
create a new fresh Lua table (call it `g`) that only contains the `library` API (we make sure
to set global protection on this table just like the general global protection already exists
today), then we use [lua_setfenv](https://www.lua.org/manual/5.1/manual.html#lua_setfenv)
to set `g` as the global table of the load run. After the load run finished we update `g`
metatable and set `__index` and `__newindex` functions to be `_G` (Lua default globals),
we also pop out the `library` object as we do not need it anymore.
This way, any function that was created on the load run (and will be invoke using `fcall`) will
see the default globals as it expected to see them and will not have the `library` API anymore.

An important outcome of this new approach is that now we can achieve a distinct global table
for each library (it is not yet like that but it is very easy to achieve it now). In the future we can
decide to remove global protection because global on different libraries will not collide or we
can chose to give different API to different libraries base on some configuration or input.

Notice that this technique was meant to prevent errors and was not meant to prevent malicious
user from exploit it. For example, the load run can still save the `library` object on some local
variable and then using in `fcall` context. To prevent such a malicious use, the C code also make
sure it is running in the right context and if not raise an error.
parent 568c2e03
proc get_function_code {args} {
return [format "redis.register_function('%s', function(KEYS, ARGV)\n %s \nend)" [lindex $args 0] [lindex $args 1]]
}
start_server {tags {"scripting"}} { start_server {tags {"scripting"}} {
test {FUNCTION - Basic usage} { test {FUNCTION - Basic usage} {
r function create LUA test {return 'hello'} r function load LUA test [get_function_code test {return 'hello'}]
r fcall test 0 r fcall test 0
} {hello} } {hello}
test {FUNCTION - Create an already exiting function raise error} { test {FUNCTION - Create an already exiting library raise error} {
catch { catch {
r function create LUA test {return 'hello1'} r function load LUA test [get_function_code test {return 'hello1'}]
} e } e
set _ $e set _ $e
} {*Function already exists*} } {*already exists*}
test {FUNCTION - Create an already exiting function raise error (case insensitive)} { test {FUNCTION - Create an already exiting library raise error (case insensitive)} {
catch { catch {
r function create LUA TEST {return 'hello1'} r function load LUA TEST [get_function_code test {return 'hello1'}]
} e } e
set _ $e set _ $e
} {*Function already exists*} } {*already exists*}
test {FUNCTION - Create a function with wrong name format} { test {FUNCTION - Create a library with wrong name format} {
catch { catch {
r function create LUA {bad\0foramat} {return 'hello1'} r function load LUA {bad\0foramat} [get_function_code test {return 'hello1'}]
} e } e
set _ $e set _ $e
} {*Function names can only contain letters and numbers*} } {*Library names can only contain letters and numbers*}
test {FUNCTION - Create function with unexisting engine} { test {FUNCTION - Create library with unexisting engine} {
catch { catch {
r function create bad_engine test {return 'hello1'} r function load bad_engine test [get_function_code test {return 'hello1'}]
} e } e
set _ $e set _ $e
} {*Engine not found*} } {*Engine not found*}
test {FUNCTION - Test uncompiled script} { test {FUNCTION - Test uncompiled script} {
catch { catch {
r function create LUA test1 {bad script} r function load LUA test1 {bad script}
} e } e
set _ $e set _ $e
} {*Error compiling function*} } {*Error compiling function*}
test {FUNCTION - test replace argument} { test {FUNCTION - test replace argument} {
r function create LUA test REPLACE {return 'hello1'} r function load LUA test REPLACE [get_function_code test {return 'hello1'}]
r fcall test 0 r fcall test 0
} {hello1} } {hello1}
...@@ -48,7 +52,7 @@ start_server {tags {"scripting"}} { ...@@ -48,7 +52,7 @@ start_server {tags {"scripting"}} {
r fcall TEST 0 r fcall TEST 0
} {hello1} } {hello1}
test {FUNCTION - test replace argument with function creation failure keeps old function} { test {FUNCTION - test replace argument with failure keeps old libraries} {
catch {r function create LUA test REPLACE {error}} catch {r function create LUA test REPLACE {error}}
r fcall test 0 r fcall test 0
} {hello1} } {hello1}
...@@ -62,31 +66,9 @@ start_server {tags {"scripting"}} { ...@@ -62,31 +66,9 @@ start_server {tags {"scripting"}} {
} {*Function not found*} } {*Function not found*}
test {FUNCTION - test description argument} { test {FUNCTION - test description argument} {
r function create LUA test DESCRIPTION {some description} {return 'hello'} r function load LUA test DESCRIPTION {some description} [get_function_code test {return 'hello'}]
r function list r function list
} {{name test engine LUA description {some description}}} } {{library_name test engine LUA description {some description} functions {{name test description {}}}}}
test {FUNCTION - test info specific function} {
r function info test WITHCODE
} {name test engine LUA description {some description} code {return 'hello'}}
test {FUNCTION - test info without code} {
r function info test
} {name test engine LUA description {some description}}
test {FUNCTION - test info on function that does not exists} {
catch {
r function info bad_function_name
} e
set _ $e
} {*Function does not exists*}
test {FUNCTION - test info with bad number of arguments} {
catch {
r function info test WITHCODE bad_arg
} e
set _ $e
} {*wrong number of arguments*}
test {FUNCTION - test fcall bad arguments} { test {FUNCTION - test fcall bad arguments} {
catch { catch {
...@@ -109,12 +91,12 @@ start_server {tags {"scripting"}} { ...@@ -109,12 +91,12 @@ start_server {tags {"scripting"}} {
set _ $e set _ $e
} {*Number of keys can't be negative*} } {*Number of keys can't be negative*}
test {FUNCTION - test function delete on not exiting function} { test {FUNCTION - test delete on not exiting library} {
catch { catch {
r function delete test1 r function delete test1
} e } e
set _ $e set _ $e
} {*Function not found*} } {*Library not found*}
test {FUNCTION - test function kill when function is not running} { test {FUNCTION - test function kill when function is not running} {
catch { catch {
...@@ -140,14 +122,14 @@ start_server {tags {"scripting"}} { ...@@ -140,14 +122,14 @@ start_server {tags {"scripting"}} {
assert_match "*Error trying to load the RDB*" $e assert_match "*Error trying to load the RDB*" $e
r debug reload noflush merge r debug reload noflush merge
r function list r function list
} {{name test engine LUA description {some description}}} {needs:debug} } {{library_name test engine LUA description {some description} functions {{name test description {}}}}} {needs:debug}
test {FUNCTION - test debug reload with nosave and noflush} { test {FUNCTION - test debug reload with nosave and noflush} {
r function delete test r function delete test
r set x 1 r set x 1
r function create LUA test1 DESCRIPTION {some description} {return 'hello'} r function load LUA test1 DESCRIPTION {some description} [get_function_code test1 {return 'hello'}]
r debug reload r debug reload
r function create LUA test2 DESCRIPTION {some description} {return 'hello'} r function load LUA test2 DESCRIPTION {some description} [get_function_code test2 {return 'hello'}]
r debug reload nosave noflush merge r debug reload nosave noflush merge
assert_equal [r fcall test1 0] {hello} assert_equal [r fcall test1 0] {hello}
assert_equal [r fcall test2 0] {hello} assert_equal [r fcall test2 0] {hello}
...@@ -155,21 +137,21 @@ start_server {tags {"scripting"}} { ...@@ -155,21 +137,21 @@ start_server {tags {"scripting"}} {
test {FUNCTION - test flushall and flushdb do not clean functions} { test {FUNCTION - test flushall and flushdb do not clean functions} {
r function flush r function flush
r function create lua test REPLACE {return redis.call('set', 'x', '1')} r function load lua test REPLACE [get_function_code test {return redis.call('set', 'x', '1')}]
r flushall r flushall
r flushdb r flushdb
r function list r function list
} {{name test engine LUA description {}}} } {{library_name test engine LUA description {} functions {{name test description {}}}}}
test {FUNCTION - test function dump and restore} { test {FUNCTION - test function dump and restore} {
r function flush r function flush
r function create lua test description {some description} {return 'hello'} r function load lua test description {some description} [get_function_code test {return 'hello'}]
set e [r function dump] set e [r function dump]
r function delete test r function delete test
assert_match {} [r function list] assert_match {} [r function list]
r function restore $e r function restore $e
r function list r function list
} {{name test engine LUA description {some description}}} } {{library_name test engine LUA description {some description} functions {{name test description {}}}}}
test {FUNCTION - test function dump and restore with flush argument} { test {FUNCTION - test function dump and restore with flush argument} {
set e [r function dump] set e [r function dump]
...@@ -177,17 +159,17 @@ start_server {tags {"scripting"}} { ...@@ -177,17 +159,17 @@ start_server {tags {"scripting"}} {
assert_match {} [r function list] assert_match {} [r function list]
r function restore $e FLUSH r function restore $e FLUSH
r function list r function list
} {{name test engine LUA description {some description}}} } {{library_name test engine LUA description {some description} functions {{name test description {}}}}}
test {FUNCTION - test function dump and restore with append argument} { test {FUNCTION - test function dump and restore with append argument} {
set e [r function dump] set e [r function dump]
r function flush r function flush
assert_match {} [r function list] assert_match {} [r function list]
r function create lua test {return 'hello1'} r function load lua test [get_function_code test {return 'hello1'}]
catch {r function restore $e APPEND} err catch {r function restore $e APPEND} err
assert_match {*already exists*} $err assert_match {*already exists*} $err
r function flush r function flush
r function create lua test1 {return 'hello1'} r function load lua test1 [get_function_code test1 {return 'hello1'}]
r function restore $e APPEND r function restore $e APPEND
assert_match {hello} [r fcall test 0] assert_match {hello} [r fcall test 0]
assert_match {hello1} [r fcall test1 0] assert_match {hello1} [r fcall test1 0]
...@@ -195,11 +177,11 @@ start_server {tags {"scripting"}} { ...@@ -195,11 +177,11 @@ start_server {tags {"scripting"}} {
test {FUNCTION - test function dump and restore with replace argument} { test {FUNCTION - test function dump and restore with replace argument} {
r function flush r function flush
r function create LUA test DESCRIPTION {some description} {return 'hello'} r function load LUA test DESCRIPTION {some description} [get_function_code test {return 'hello'}]
set e [r function dump] set e [r function dump]
r function flush r function flush
assert_match {} [r function list] assert_match {} [r function list]
r function create lua test {return 'hello1'} r function load lua test [get_function_code test {return 'hello1'}]
assert_match {hello1} [r fcall test 0] assert_match {hello1} [r fcall test 0]
r function restore $e REPLACE r function restore $e REPLACE
assert_match {hello} [r fcall test 0] assert_match {hello} [r fcall test 0]
...@@ -207,11 +189,11 @@ start_server {tags {"scripting"}} { ...@@ -207,11 +189,11 @@ start_server {tags {"scripting"}} {
test {FUNCTION - test function restore with bad payload do not drop existing functions} { test {FUNCTION - test function restore with bad payload do not drop existing functions} {
r function flush r function flush
r function create LUA test DESCRIPTION {some description} {return 'hello'} r function load LUA test DESCRIPTION {some description} [get_function_code test {return 'hello'}]
catch {r function restore bad_payload} e catch {r function restore bad_payload} e
assert_match {*payload version or checksum are wrong*} $e assert_match {*payload version or checksum are wrong*} $e
r function list r function list
} {{name test engine LUA description {some description}}} } {{library_name test engine LUA description {some description} functions {{name test description {}}}}}
test {FUNCTION - test function restore with wrong number of arguments} { test {FUNCTION - test function restore with wrong number of arguments} {
catch {r function restore arg1 args2 arg3} e catch {r function restore arg1 args2 arg3} e
...@@ -219,19 +201,19 @@ start_server {tags {"scripting"}} { ...@@ -219,19 +201,19 @@ start_server {tags {"scripting"}} {
} {*wrong number of arguments*} } {*wrong number of arguments*}
test {FUNCTION - test fcall_ro with write command} { test {FUNCTION - test fcall_ro with write command} {
r function create lua test REPLACE {return redis.call('set', 'x', '1')} r function load lua test REPLACE [get_function_code test {return redis.call('set', 'x', '1')}]
catch { r fcall_ro test 0 } e catch { r fcall_ro test 0 } e
set _ $e set _ $e
} {*Write commands are not allowed from read-only scripts*} } {*Write commands are not allowed from read-only scripts*}
test {FUNCTION - test fcall_ro with read only commands} { test {FUNCTION - test fcall_ro with read only commands} {
r function create lua test REPLACE {return redis.call('get', 'x')} r function load lua test REPLACE [get_function_code test {return redis.call('get', 'x')}]
r set x 1 r set x 1
r fcall_ro test 0 r fcall_ro test 0
} {1} } {1}
test {FUNCTION - test keys and argv} { test {FUNCTION - test keys and argv} {
r function create lua test REPLACE {return redis.call('set', KEYS[1], ARGV[1])} r function load lua test REPLACE [get_function_code test {return redis.call('set', KEYS[1], ARGV[1])}]
r fcall test 1 x foo r fcall test 1 x foo
r get x r get x
} {foo} } {foo}
...@@ -247,7 +229,7 @@ start_server {tags {"scripting"}} { ...@@ -247,7 +229,7 @@ start_server {tags {"scripting"}} {
test {FUNCTION - test function kill} { test {FUNCTION - test function kill} {
set rd [redis_deferring_client] set rd [redis_deferring_client]
r config set script-time-limit 10 r config set script-time-limit 10
r function create lua test REPLACE {local a = 1 while true do a = a + 1 end} r function load lua test REPLACE [get_function_code test {local a = 1 while true do a = a + 1 end}]
$rd fcall test 0 $rd fcall test 0
after 200 after 200
catch {r ping} e catch {r ping} e
...@@ -261,7 +243,7 @@ start_server {tags {"scripting"}} { ...@@ -261,7 +243,7 @@ start_server {tags {"scripting"}} {
test {FUNCTION - test script kill not working on function} { test {FUNCTION - test script kill not working on function} {
set rd [redis_deferring_client] set rd [redis_deferring_client]
r config set script-time-limit 10 r config set script-time-limit 10
r function create lua test REPLACE {local a = 1 while true do a = a + 1 end} r function load lua test REPLACE [get_function_code test {local a = 1 while true do a = a + 1 end}]
$rd fcall test 0 $rd fcall test 0
after 200 after 200
catch {r ping} e catch {r ping} e
...@@ -288,18 +270,18 @@ start_server {tags {"scripting"}} { ...@@ -288,18 +270,18 @@ start_server {tags {"scripting"}} {
} }
test {FUNCTION - test function flush} { test {FUNCTION - test function flush} {
r function create lua test REPLACE {local a = 1 while true do a = a + 1 end} r function load lua test REPLACE [get_function_code test {local a = 1 while true do a = a + 1 end}]
assert_match {{name test engine LUA description {}}} [r function list] assert_match {{library_name test engine LUA description {} functions {{name test description {}}}}} [r function list]
r function flush r function flush
assert_match {} [r function list] assert_match {} [r function list]
r function create lua test REPLACE {local a = 1 while true do a = a + 1 end} r function load lua test REPLACE [get_function_code test {local a = 1 while true do a = a + 1 end}]
assert_match {{name test engine LUA description {}}} [r function list] assert_match {{library_name test engine LUA description {} functions {{name test description {}}}}} [r function list]
r function flush async r function flush async
assert_match {} [r function list] assert_match {} [r function list]
r function create lua test REPLACE {local a = 1 while true do a = a + 1 end} r function load lua test REPLACE [get_function_code test {local a = 1 while true do a = a + 1 end}]
assert_match {{name test engine LUA description {}}} [r function list] assert_match {{library_name test engine LUA description {} functions {{name test description {}}}}} [r function list]
r function flush sync r function flush sync
assert_match {} [r function list] assert_match {} [r function list]
} }
...@@ -326,9 +308,9 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -326,9 +308,9 @@ start_server {tags {"scripting repl external:skip"}} {
} }
test {FUNCTION - creation is replicated to replica} { test {FUNCTION - creation is replicated to replica} {
r function create LUA test DESCRIPTION {some description} {return 'hello'} r function load LUA test DESCRIPTION {some description} [get_function_code test {return 'hello'}]
wait_for_condition 50 100 { wait_for_condition 50 100 {
[r -1 function list] eq {{name test engine LUA description {some description}}} [r -1 function list] eq {{library_name test engine LUA description {some description} functions {{name test description {}}}}}
} else { } else {
fail "Failed waiting for function to replicate to replica" fail "Failed waiting for function to replicate to replica"
} }
...@@ -348,10 +330,10 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -348,10 +330,10 @@ start_server {tags {"scripting repl external:skip"}} {
fail "Failed waiting for function to replicate to replica" fail "Failed waiting for function to replicate to replica"
} }
r function restore $e assert_equal [r function restore $e] {OK}
wait_for_condition 50 100 { wait_for_condition 50 100 {
[r -1 function list] eq {{name test engine LUA description {some description}}} [r -1 function list] eq {{library_name test engine LUA description {some description} functions {{name test description {}}}}}
} else { } else {
fail "Failed waiting for function to replicate to replica" fail "Failed waiting for function to replicate to replica"
} }
...@@ -367,9 +349,9 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -367,9 +349,9 @@ start_server {tags {"scripting repl external:skip"}} {
} }
test {FUNCTION - flush is replicated to replica} { test {FUNCTION - flush is replicated to replica} {
r function create LUA test DESCRIPTION {some description} {return 'hello'} r function load LUA test DESCRIPTION {some description} [get_function_code test {return 'hello'}]
wait_for_condition 50 100 { wait_for_condition 50 100 {
[r -1 function list] eq {{name test engine LUA description {some description}}} [r -1 function list] eq {{library_name test engine LUA description {some description} functions {{name test description {}}}}}
} else { } else {
fail "Failed waiting for function to replicate to replica" fail "Failed waiting for function to replicate to replica"
} }
...@@ -385,7 +367,7 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -385,7 +367,7 @@ start_server {tags {"scripting repl external:skip"}} {
r -1 slaveof no one r -1 slaveof no one
# creating a function after disconnect to make sure function # creating a function after disconnect to make sure function
# is replicated on rdb phase # is replicated on rdb phase
r function create LUA test DESCRIPTION {some description} {return 'hello'} r function load LUA test DESCRIPTION {some description} [get_function_code test {return 'hello'}]
# reconnect the replica # reconnect the replica
r -1 slaveof [srv 0 host] [srv 0 port] r -1 slaveof [srv 0 host] [srv 0 port]
...@@ -402,12 +384,12 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -402,12 +384,12 @@ start_server {tags {"scripting repl external:skip"}} {
} {hello} } {hello}
test "FUNCTION - test replication to replica on rdb phase info command" { test "FUNCTION - test replication to replica on rdb phase info command" {
r -1 function info test WITHCODE r -1 function list
} {name test engine LUA description {some description} code {return 'hello'}} } {{library_name test engine LUA description {some description} functions {{name test description {}}}}}
test "FUNCTION - create on read only replica" { test "FUNCTION - create on read only replica" {
catch { catch {
r -1 function create LUA test DESCRIPTION {some description} {return 'hello'} r -1 function load LUA test DESCRIPTION {some description} [get_function_code test {return 'hello'}]
} e } e
set _ $e set _ $e
} {*can't write against a read only replica*} } {*can't write against a read only replica*}
...@@ -420,7 +402,7 @@ start_server {tags {"scripting repl external:skip"}} { ...@@ -420,7 +402,7 @@ start_server {tags {"scripting repl external:skip"}} {
} {*can't write against a read only replica*} } {*can't write against a read only replica*}
test "FUNCTION - function effect is replicated to replica" { test "FUNCTION - function effect is replicated to replica" {
r function create LUA test REPLACE {return redis.call('set', 'x', '1')} r function load LUA test REPLACE [get_function_code test {return redis.call('set', 'x', '1')}]
r fcall test 0 r fcall test 0
assert {[r get x] eq {1}} assert {[r get x] eq {1}}
wait_for_condition 50 100 { wait_for_condition 50 100 {
...@@ -443,12 +425,12 @@ test {FUNCTION can processes create, delete and flush commands in AOF when doing ...@@ -443,12 +425,12 @@ test {FUNCTION can processes create, delete and flush commands in AOF when doing
start_server {} { start_server {} {
r config set appendonly yes r config set appendonly yes
waitForBgrewriteaof r waitForBgrewriteaof r
r FUNCTION CREATE lua test "return 'hello'" r FUNCTION LOAD lua test "redis.register_function('test', function() return 'hello' end)"
r config set slave-read-only yes r config set slave-read-only yes
r slaveof 127.0.0.1 0 r slaveof 127.0.0.1 0
r debug loadaof r debug loadaof
r slaveof no one r slaveof no one
assert_equal [r function list] {{name test engine LUA description {}}} assert_equal [r function list] {{library_name test engine LUA description {} functions {{name test description {}}}}}
r FUNCTION DELETE test r FUNCTION DELETE test
...@@ -457,7 +439,7 @@ test {FUNCTION can processes create, delete and flush commands in AOF when doing ...@@ -457,7 +439,7 @@ test {FUNCTION can processes create, delete and flush commands in AOF when doing
r slaveof no one r slaveof no one
assert_equal [r function list] {} assert_equal [r function list] {}
r FUNCTION CREATE lua test "return 'hello'" r FUNCTION LOAD lua test "redis.register_function('test', function() return 'hello' end)"
r FUNCTION FLUSH r FUNCTION FLUSH
r slaveof 127.0.0.1 0 r slaveof 127.0.0.1 0
...@@ -466,3 +448,420 @@ test {FUNCTION can processes create, delete and flush commands in AOF when doing ...@@ -466,3 +448,420 @@ test {FUNCTION can processes create, delete and flush commands in AOF when doing
assert_equal [r function list] {} assert_equal [r function list] {}
} }
} {} {needs:debug external:skip} } {} {needs:debug external:skip}
start_server {tags {"scripting"}} {
test {LIBRARIES - test shared function can access default globals} {
r function load LUA lib1 {
local function ping()
return redis.call('ping')
end
redis.register_function(
'f1',
function(keys, args)
return ping()
end
)
}
r fcall f1 0
} {PONG}
test {LIBRARIES - usage and code sharing} {
r function load LUA lib1 REPLACE {
local function add1(a)
return a + 1
end
redis.register_function(
'f1',
function(keys, args)
return add1(1)
end,
'f1 description'
)
redis.register_function(
'f2',
function(keys, args)
return add1(2)
end,
'f2 description'
)
}
assert_equal [r fcall f1 0] {2}
assert_equal [r fcall f2 0] {3}
r function list
} {{library_name lib1 engine LUA description {} functions {*}}}
test {LIBRARIES - test registration failure revert the entire load} {
catch {
r function load LUA lib1 replace {
local function add1(a)
return a + 2
end
redis.register_function(
'f1',
function(keys, args)
return add1(1)
end
)
redis.register_function(
'f2',
'not a function'
)
}
} e
assert_match {*second argument to redis.register_function must be a function*} $e
assert_equal [r fcall f1 0] {2}
assert_equal [r fcall f2 0] {3}
}
test {LIBRARIES - test registration function name collision} {
catch {
r function load LUA lib2 replace {
redis.register_function(
'f1',
function(keys, args)
return 1
end
)
}
} e
assert_match {*Function f1 already exists*} $e
assert_equal [r fcall f1 0] {2}
assert_equal [r fcall f2 0] {3}
}
test {LIBRARIES - test registration function name collision on same library} {
catch {
r function load LUA lib2 replace {
redis.register_function(
'f1',
function(keys, args)
return 1
end
)
redis.register_function(
'f1',
function(keys, args)
return 1
end
)
}
} e
set _ $e
} {*Function already exists in the library*}
test {LIBRARIES - test registration with no argument} {
catch {
r function load LUA lib2 replace {
redis.register_function()
}
} e
set _ $e
} {*wrong number of arguments to redis.register_function*}
test {LIBRARIES - test registration with only name} {
catch {
r function load LUA lib2 replace {
redis.register_function('f1')
}
} e
set _ $e
} {*wrong number of arguments to redis.register_function*}
test {LIBRARIES - test registration with to many arguments} {
catch {
r function load LUA lib2 replace {
redis.register_function('f1', function() return 1 end, 'description', 'extra arg')
}
} e
set _ $e
} {*wrong number of arguments to redis.register_function*}
test {LIBRARIES - test registration with no string name} {
catch {
r function load LUA lib2 replace {
redis.register_function(nil, function() return 1 end)
}
} e
set _ $e
} {*first argument to redis.register_function must be a string*}
test {LIBRARIES - test registration with wrong name format} {
catch {
r function load LUA lib2 replace {
redis.register_function('test\0test', function() return 1 end)
}
} e
set _ $e
} {*Function names can only contain letters and numbers and must be at least one character long*}
test {LIBRARIES - test registration with empty name} {
catch {
r function load LUA lib2 replace {
redis.register_function('', function() return 1 end)
}
} e
set _ $e
} {*Function names can only contain letters and numbers and must be at least one character long*}
test {LIBRARIES - math.random from function load} {
catch {
r function load LUA lib2 replace {
return math.random()
}
} e
set _ $e
} {*attempted to access nonexistent global variable 'math'*}
test {LIBRARIES - redis.call from function load} {
catch {
r function load LUA lib2 replace {
return redis.call('ping')
}
} e
set _ $e
} {*attempt to call field 'call' (a nil value)*}
test {LIBRARIES - redis.call from function load} {
catch {
r function load LUA lib2 replace {
return redis.setresp(3)
}
} e
set _ $e
} {*attempt to call field 'setresp' (a nil value)*}
test {LIBRARIES - redis.set_repl from function load} {
catch {
r function load LUA lib2 replace {
return redis.set_repl(redis.REPL_NONE)
}
} e
set _ $e
} {*attempt to call field 'set_repl' (a nil value)*}
test {LIBRARIES - malicious access test} {
# the 'library' API is not exposed inside a
# function context and the 'redis' API is not
# expose on the library registration context.
# But a malicious user might find a way to hack it
# (as demonstrated in this test). This is why we
# have another level of protection on the C
# code itself and we want to test it and verify
# that it works properly.
r function load LUA lib1 replace {
local lib = redis
lib.register_function('f1', function ()
lib.redis = redis
lib.math = math
return {ok='OK'}
end)
lib.register_function('f2', function ()
lib.register_function('f1', function ()
lib.redis = redis
lib.math = math
return {ok='OK'}
end)
end)
}
assert_equal {OK} [r fcall f1 0]
catch {[r function load LUA lib2 {redis.math.random()}]} e
assert_match {*can only be called inside a script invocation*} $e
catch {[r function load LUA lib2 {redis.math.randomseed()}]} e
assert_match {*can only be called inside a script invocation*} $e
catch {[r function load LUA lib2 {redis.redis.call('ping')}]} e
assert_match {*can only be called inside a script invocation*} $e
catch {[r function load LUA lib2 {redis.redis.pcall('ping')}]} e
assert_match {*can only be called inside a script invocation*} $e
catch {[r function load LUA lib2 {redis.redis.setresp(3)}]} e
assert_match {*can only be called inside a script invocation*} $e
catch {[r function load LUA lib2 {redis.redis.set_repl(redis.redis.REPL_NONE)}]} e
assert_match {*can only be called inside a script invocation*} $e
catch {[r fcall f2 0]} e
assert_match {*can only be called on FUNCTION LOAD command*} $e
}
test {LIBRARIES - delete removed all functions on library} {
r function delete lib1
r function list
} {}
test {LIBRARIES - register function inside a function} {
r function load LUA lib {
redis.register_function(
'f1',
function(keys, args)
redis.register_function(
'f2',
function(key, args)
return 2
end
)
return 1
end
)
}
catch {r fcall f1 0} e
set _ $e
} {*attempt to call field 'register_function' (a nil value)*}
test {LIBRARIES - register library with no functions} {
r function flush
catch {
r function load LUA lib {
return 1
}
} e
set _ $e
} {*No functions registered*}
test {LIBRARIES - load timeout} {
catch {
r function load LUA lib {
local a = 1
while 1 do a = a + 1 end
}
} e
set _ $e
} {*FUNCTION LOAD timeout*}
test {LIBRARIES - verify global protection on the load run} {
catch {
r function load LUA lib {
a = 1
}
} e
set _ $e
} {*attempted to create global variable 'a'*}
test {FUNCTION - test function restore with function name collision} {
r function flush
r function load lua lib1 {
local function add1(a)
return a + 1
end
redis.register_function(
'f1',
function(keys, args)
return add1(1)
end
)
redis.register_function(
'f2',
function(keys, args)
return add1(2)
end
)
redis.register_function(
'f3',
function(keys, args)
return add1(3)
end
)
}
set e [r function dump]
r function flush
# load a library with different name but with the same function name
r function load lua lib1 {
redis.register_function(
'f6',
function(keys, args)
return 7
end
)
}
r function load lua lib2 {
local function add1(a)
return a + 1
end
redis.register_function(
'f4',
function(keys, args)
return add1(4)
end
)
redis.register_function(
'f5',
function(keys, args)
return add1(5)
end
)
redis.register_function(
'f3',
function(keys, args)
return add1(3)
end
)
}
catch {r function restore $e} error
assert_match {*Library lib1 already exists*} $error
assert_equal [r fcall f3 0] {4}
assert_equal [r fcall f4 0] {5}
assert_equal [r fcall f5 0] {6}
assert_equal [r fcall f6 0] {7}
catch {r function restore $e replace} error
assert_match {*Function f3 already exists*} $error
assert_equal [r fcall f3 0] {4}
assert_equal [r fcall f4 0] {5}
assert_equal [r fcall f5 0] {6}
assert_equal [r fcall f6 0] {7}
}
test {FUNCTION - test function list with code} {
r function flush
r function load lua library1 {redis.register_function('f6', function(keys, args) return 7 end)}
r function list withcode
} {{library_name library1 engine LUA description {} functions {{name f6 description {}}} library_code {redis.register_function('f6', function(keys, args) return 7 end)}}}
test {FUNCTION - test function list with pattern} {
r function load lua lib1 {redis.register_function('f7', function(keys, args) return 7 end)}
r function list libraryname library*
} {{library_name library1 engine LUA description {} functions {{name f6 description {}}}}}
test {FUNCTION - test function list wrong argument} {
catch {r function list bad_argument} e
set _ $e
} {*Unknown argument bad_argument*}
test {FUNCTION - test function list with bad argument to library name} {
catch {r function list libraryname} e
set _ $e
} {*library name argument was not given*}
test {FUNCTION - test function list withcode multiple times} {
catch {r function list withcode withcode} e
set _ $e
} {*Unknown argument withcode*}
test {FUNCTION - test function list libraryname multiple times} {
catch {r function list withcode libraryname foo libraryname foo} e
set _ $e
} {*Unknown argument libraryname*}
test {FUNCTION - verify OOM on function load and function restore} {
r function flush
r function load lua test replace {redis.register_function('f1', function() return 1 end)}
set payload [r function dump]
r config set maxmemory 1
r function flush
catch {r function load lua test replace {redis.register_function('f1', function() return 1 end)}} e
assert_match {*command not allowed when used memory*} $e
r function flush
catch {r function restore $payload} e
assert_match {*command not allowed when used memory*} $e
r config set maxmemory 0
}
}
...@@ -15,16 +15,16 @@ if {$is_eval == 1} { ...@@ -15,16 +15,16 @@ if {$is_eval == 1} {
} }
} else { } else {
proc run_script {args} { proc run_script {args} {
r function create LUA test replace [lindex $args 0] r function load LUA test replace [format "redis.register_function('test', function(KEYS, ARGV)\n %s \nend)" [lindex $args 0]]
r fcall test {*}[lrange $args 1 end] r fcall test {*}[lrange $args 1 end]
} }
proc run_script_ro {args} { proc run_script_ro {args} {
r function create LUA test replace [lindex $args 0] r function load LUA test replace [format "redis.register_function('test', function(KEYS, ARGV)\n %s \nend)" [lindex $args 0]]
r fcall_ro test {*}[lrange $args 1 end] r fcall_ro test {*}[lrange $args 1 end]
} }
proc run_script_on_connection {args} { proc run_script_on_connection {args} {
set rd [lindex $args 0] set rd [lindex $args 0]
$rd function create LUA test replace [lindex $args 1] $rd function load LUA test replace [format "redis.register_function('test', function(KEYS, ARGV)\n %s \nend)" [lindex $args 1]]
# read the ok reply of function create # read the ok reply of function create
$rd read $rd read
$rd fcall test {*}[lrange $args 2 end] $rd fcall test {*}[lrange $args 2 end]
...@@ -37,7 +37,7 @@ if {$is_eval == 1} { ...@@ -37,7 +37,7 @@ if {$is_eval == 1} {
start_server {tags {"scripting"}} { start_server {tags {"scripting"}} {
test {Script - disallow write on OOM} { test {Script - disallow write on OOM} {
r FUNCTION create lua f1 replace { return redis.call('set', 'x', '1') } r FUNCTION load lua f1 replace { redis.register_function('f1', function() return redis.call('set', 'x', '1') end) }
r config set maxmemory 1 r config set maxmemory 1
...@@ -737,7 +737,7 @@ start_server {tags {"scripting"}} { ...@@ -737,7 +737,7 @@ start_server {tags {"scripting"}} {
set buf "*3\r\n\$4\r\neval\r\n\$33\r\nwhile 1 do redis.call('ping') end\r\n\$1\r\n0\r\n" set buf "*3\r\n\$4\r\neval\r\n\$33\r\nwhile 1 do redis.call('ping') end\r\n\$1\r\n0\r\n"
append buf "*1\r\n\$4\r\nping\r\n" append buf "*1\r\n\$4\r\nping\r\n"
} else { } else {
set buf "*6\r\n\$8\r\nfunction\r\n\$6\r\ncreate\r\n\$3\r\nlua\r\n\$4\r\ntest\r\n\$7\r\nreplace\r\n\$33\r\nwhile 1 do redis.call('ping') end\r\n" set buf "*6\r\n\$8\r\nfunction\r\n\$4\r\nload\r\n\$3\r\nlua\r\n\$4\r\ntest\r\n\$7\r\nreplace\r\n\$81\r\nredis.register_function('test', function() while 1 do redis.call('ping') end end)\r\n"
append buf "*3\r\n\$5\r\nfcall\r\n\$4\r\ntest\r\n\$1\r\n0\r\n" append buf "*3\r\n\$5\r\nfcall\r\n\$4\r\ntest\r\n\$1\r\n0\r\n"
append buf "*1\r\n\$4\r\nping\r\n" append buf "*1\r\n\$4\r\nping\r\n"
} }
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment