Fast autocomplete using Python and Redis
select * from table_name where table_field like 'search%'
Source code can be found at:
ZADDcommand to load items into sorted set. We will specify same score
0.0to each item, so them will sorted lexicographically in the set.
To implement case intensive search we will use some little trick. Before adding each item to the set we will convert it to lowercase and will add original item after
:separator. So when we have want add word
Woodto the set, actually we will add
wood:Wood. In this case we will be able to perform case intensive search and also save original word.
We will use pipelines (part of
redisPython library) to increase loading speed. They will help us to buffer
ZADDcommands and reduce the number of back-and-forth TCP packets between the client and server. They helps us to dramatically increase the loading performance.
ZRANGEBYLEXcommand and will load all strings which starts with query string. So it's will be actual implementation of autocomplete. BTW, superfast autocomplete: we can perform search for string
520679items just in 30-35ms at my laptop.
pip install -r requirements.txt
You need specify connection to
redis, folder with data and temporary
[main] temp_path=./tmp [redis] host=localhost port=6379 password= db=0 [data] main_source=./data/
redisconnection to redis;
data::main_source- folder with data;
main::temp_path- folder for
string_search.py --config .\conf\default.conf --init-data
program will look for all files in this folder. If
.zipfile will be found, it will be uncompressed into temporary folder for further loading.
This operation can took long time, up to 1 minute on slow machines.
alexaquery you should use:
string_search.py --config .\conf\default.conf --search alexa
string_search.py --config .\conf\default.conf --get-length