Friday, May 10, 2013

Redis commands from Postgres

A while ago a client wanted a way to call Redis commands from Postgres, for example to push values to Redis, or perform other Redis housekeeping that can't be done by use of the Redis FDW. I put together a quick Postgres interface using the official hiredis client library, and I have been polishing it up a bit in preparation for my talk at PgCon, and the package is now available for public use.

The basic interface contains functions to provide persistent connection handles, to disconnect, and to call the library functions redisCommand() and redisCommandArgv(). Yesterday, I added functions to push a record to a Redis “table”, and to drop a Redis “table”. I am planning to add a function to push a whole Postgres table or view, but for now this can be achieved by pushing the records - tests yesterday on very modest hardware managed to push 1 million rows in 130 seconds and drop the same table in 45 seconds. Of course, in Redis dropping a table means in effect dropping its component objects, so it's still doing several million operations here.

Of course, Redis doesn't really have tables in the PostgreSQL sense. It has a single global namespace which contains keys, which are strings, and values, which are various sorts of things (strings, integers, lists, hashes, sets, ordered sets). Hence my use of inverted commas above. For this purpose, a Redis table consists of a set of objects whose keys have a common prefix, or whose keys are the elements of a Redis set object. These uses are critical to the effective use of the Redis Foreign Data Wrapper, which I will also be expanding upon in my talk.

I also added some regression  tests which demonstrate the abilities of the interface package. I'll be working on improving the documentation shortly.


5 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Hi, I'm interested in pushing and popping form postgres to redis (most importantly writing). I was looking at the redis_fdw and dont yet see a way to push values to redis yet write features with Postgresql 9.3 are mentioned. (Note didnt see how the texteq comment to push a qual would allow the write). Is there any update on this?

    Many thanks and a big fan!

    ReplyDelete
  3. I should mention the redis command wrapper doesnt seem to be at the given link anymore.

    ReplyDelete
    Replies
    1. It's moved to https://bitbucket.org/IVC-Inc/redis_wrapper

      I haven't had a chance to work on it recently, been focused on jsonb.

      Delete
  4. Thank you for the link! Works great, very cool!

    ReplyDelete