How do I implement Redis keyspace notifications in ElastiCache?

Short description

Redis keyspace notifications allow you to subscribe to PubSub channels. Through the channel, clients receive published events when a Redis command or data alteration occurs. These notifications are useful when an application must respond to changes that occur to the value stored in a particular key or keys. You can also follow dataset changes using Redis keyspace notifications. The keyspace notifications feature is available in versions 2.8.6 and later.

When a command runs that triggers a Redis keyspace notification, the following two events occur that monitoring clients respond to:

• Keyspace notification
• Key-event notification

The keyspace channel receives the name of the event, and the key-event channel receives the name of the key as a message.

Important: The following resolution applies only to Redis (cluster mode disabled) clusters. The resolution doesn't apply to Redis (cluster mode enabled) clusters. For information on keyspace notifications for both types of clusters, see Redis keyspace notifications on the Redis site.

Resolution

Activate Redis keyspace notification

ElastiCache turns off Redis keyspace notifications by default. You can activate keyspace notifications in a custom cache parameter group using the notify-keyspace-events parameter. The notify-keyspace-events value uses multiple parameters to determine which channel to use (keyspace events or key-event events) and the information to post to these channels. For more information, see Redis 2.8.6 added parameters.

1.    Open the ElastiCache console.

2.    To see a list of all available parameter groups, choose Parameter Groups in the navigation pane.

3.    Select the parameter group that you want to modify.
Note: You can't modify a default parameter group. If the parameter groups listed are all default, create a new parameter group by choosing Create Parameter Group, select a Family, and then choose Create.

4.    Choose Edit Parameters.

5.    Scroll to notify-keyspace-events, and then enter AKE in the Value field. For more information on allowed values, see Redis 2.8.6 added parameters.

6.    Choose Save Changes. The changes take effect immediately without restarting the cluster.
Note: Be sure to assign the new modified parameter group to your Redis cluster.

Generate events using redis-cli

If you're using redis-cli, you can run the following command to see how a given command generates events:

#./redis-cli -c -h testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com -p 6379 --csv psubscribe '__key*__:*'

Run any set of commands to create some keyspace events. The following commands set value and expire time as 3 sec for key Redis and subscriber for expired events.

./redis-cli -c -h testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com -p 6379
testdisabled.xm4oz6.ng.0001.use1.cache.amazonaws.com:6379>
testdisabled.xm4oz6.ng.0001.use1.cache.amazonaws.com:6379> SET key1 value1 ex 5
testdisabled.xm4oz6.ng.0001.use1.cache.amazonaws.com:6379>

In a separate session, verify that the events generate. The following is sample output from the events:

# ./redis-cli -h testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com -p 6379 --csv psubscribe '*'
Reading messages... (press Ctrl-C to quit)
"psubscribe","*",1
"pmessage","*","__keyspace@0__:key1","set"
"pmessage","*","__keyevent@0__:set","key1"
"pmessage","*","__keyspace@0__:key1","expire"
"pmessage","*","__keyevent@0__:expire","key1"
"pmessage","*","__keyspace@0__:key1","expired"
"pmessage","*","__keyevent@0__:expired","key1"

In this output example, the first event indicates that you successfully subscribed to a channel. The number 1 represents the number of channels that you currently subscribe to.

The second event is a keyspace notification. The keyspace channel __keyspace@0__:key1 contains the name of the event " set" as a message.

The third event is a key-event notification. The keyevent channel __keyevent@0__:set contains the name of the key " key1" as a message.

After 5 seconds, you can watch the expired message in the output. There are subtle differences in the messages. One message is __keyevent@0__:expire while another message is __keyevent@0__:expired.

Each message read from the PubSub instance is a dictionary with the following keys:

• Type - One of the following: subscription, unsubscribe, psubscribe, punsubscribe, message, pmessage
• Pattern - The pattern that matches the channel of the published message (except for the pmessage type, which does not match in all cases)
• Channel - The channel that the message is posted to or the channel that the message is subscribed to
• Data - Message data

Generate events using Redis-py

1.    Subscribe to notifications. The following example subscribes to all keyspace notifications and prints received events. Use the appropriate script for either Python2 or Python3.

For Python2:

# cat subscribe.py 
import redis
import datetime
import time
r = redis.Redis(host='testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com', port=6379, db=0)
#Subscribing for events matching pattern "__key*__:*"
p = r.pubsub()
p.psubscribe('__key*__:*')
print('Starting message loop')
while True:
    message = p.get_message()
    if message:
        print datetime.datetime.now(), message
    else:
        time.sleep(0.01)

For Python3:

import redis
from datetime import datetime
import time
r = redis.Redis(host = 'testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com', port = 6379, db = 0)
#Subscribing for events matching pattern "__key*__:*
p = r.pubsub()
p.psubscribe('__key*__:*')
print('Starting message loop')
while True:
    message = p.get_message()
    if message:
        print (datetime.now(), message)
    else:
        time.sleep(0.01)

Note: Lower values or the absence of sleep causes the client to constantly poll the channel, potentially overloading the server. Higher values cause more memory usage on the server side. The preceding example sets time.sleep to (0.01).

2.    Set or expire keys.

In a separate session, connect to the cluster and run the createkey.py command. Set the key key2 with value value2 and expire time as 3 seconds.

# cat createkey.py
import redis
  r = redis.Redis(host='testdisabled.xxxxx.ng.0001.use1.cache.amazonaws.com', port=6379, db=0)
  r.set('key2,’value2’)
  r.expire('key2’,3)

Run the subscribe.py command in a separate session. The following is an example output:

# python2 subscribe.py

Starting message loop
2020-08-17 18:16:23.902762 {u'pattern': None, u'type': 'psubscribe', u'channel': '__key*__:*', u'data': 1L}
2020-08-17 18:16:50.375281 {u'pattern': '__key*__:*',
    u'type': 'pmessage', u'channel': '__keyspace@0__:key2', u'data': 'set'}
2020-08-17 18:16:50.375365 {u'pattern': '__key*__:*', u'type': 'pmessage', u'channel': '__keyevent@0__:set', u'data': 'key2'}
2020-08-17 18:16:50.375442 {u'pattern': '__key*__:*',
    u'type': 'pmessage', u'channel': '__keyspace@0__:key2', u'data': 'expire'}
2020-08-17 18:16:50.375499 {u'pattern': '__key*__:*', u'type': 'pmessage', u'channel': '__keyevent@0__:expire', u'data': 'key2'}
2020-08-17 18:16:53.458622 {u'pattern':
    '__key*__:*', u'type': 'pmessage', u'channel': '__keyspace@0__:key2', u'data': 'expired'}
2020-08-17 18:16:53.458856 {u'pattern': '__key*__:*', u'type': 'pmessage', u'channel': '__keyevent@0__:expired', u'data': 'key2'}

Your Redis key expire listener is ready.