redis允许对key设置超时时间,实现过期key的自动淘汰。这篇blog分析下,其自适应(adaptive)的淘汰机制。
redis每隔100ms定时执行的循环(serverCron function)里有如下语句:
655
/*
Expire a few keys per cycle, only if this is a master. 656 * On slaves we wait for DEL operations synthesized by the master 657 * in order to guarantee a strict consistency.
*/
658
if
(server.masterhost == NULL) activeExpireCycle();
正如文中注释所示,只有master执行expire cycle,slave会等候由master传递的DEL消息,保证master-slave在过期值处理上的一致性。(后边代码会看到,redis对过期值的选择是随机抽取的,master-slave完全可能抽取不同的值,因此要求master通过DEL消息实现同步,同时这种expire机制也是不可靠的expire,即key超时后有可能不会被删除)。
activeExpireCycle函数如下:
477
/*
Try to expire a few timed out keys. The algorithm used is adaptive and 478 * will use few CPU cycles if there are few expiring keys, otherwise 479 * it will get more aggressive to avoid that too much memory is used by 480 * keys that can be removed from the keyspace.
*/
481
void
activeExpireCycle(
void
) {
482
int
j;
483
484
for
(j =
0
; j < server.dbnum; j++
) {
485
int
expired;
486
redisDb *db = server.db+
j;
487
488
/*
Continue to expire if at the end of the cycle more than 25% 489 * of the keys were expired.
*/
490
do
{
491
long
num = dictSize(db->
expires);
492
time_t now =
time(NULL);
493
494
expired =
0
;
495
if
(num >
REDIS_EXPIRELOOKUPS_PER_CRON)
496
num =
REDIS_EXPIRELOOKUPS_PER_CRON;
497
while
(num--
) {
498
dictEntry *
de;
499
time_t t;
500
501
if
((de = dictGetRandomKey(db->expires)) == NULL)
break
;
502
t =
(time_t) dictGetEntryVal(de);
503
if
(now >
t) {
504
sds key =
dictGetEntryKey(de);
505
robj *keyobj =
createStringObject(key,sdslen(key));
506
507
propagateExpire(db,keyobj); //将删除操作传播给各个slaves,在此之前,还将del操作记录aof
508
dbDelete(db,keyobj); //这个函数先从db->expires中删除,然后删除db->dict
509
decrRefCount(keyobj);
510
expired++
;
511
server.stat_expiredkeys++
;
512
}
513
}
514
}
while
(expired > REDIS_EXPIRELOOKUPS_PER_CRON/
4
);
515
}
516
}
ExpireCycle每次尝试处理10个key,如果10个key中有>2.5个超时,则继续处理10个key。其用意在于,如果超时的key比例很高,则一次迭代处理很多个,否则等待下次serverCron循环再随机抽取。

