Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incr expired_keys if the expiration time is already expired #1517

Open
wants to merge 2 commits into
base: unstable
Choose a base branch
from

Conversation

RayaCoo
Copy link
Contributor

@RayaCoo RayaCoo commented Jan 7, 2025

This PR modifies two parts of the deleteExpiredKeyFromOverwriteAndPropagate function, which is called when the expiration time has already expired.

  • change Keyspace Event notify from GENERIC-del to EXPIRED-expired.
    Since this delete operation is triggered by EXPIRE, and the flag pass to dbGenericDeleteWithDictIndex is DB_FLAG_KEY_EXPIRED, which is used to notify module, maybe we should align the behavior and send Keyspace Event notify like deleteExpiredKeyAndPropagateWithDictIndex did.
  • As i said befoce, this del operation is triggered by EXPIRE, maybe increasing server.stat_expiredkeys in this case could more accurately reflect the number of keys deleted due to expiration.

Copy link

codecov bot commented Jan 7, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.85%. Comparing base (c0014ef) to head (11348fe).
Report is 5 commits behind head on unstable.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #1517      +/-   ##
============================================
+ Coverage     70.82%   70.85%   +0.03%     
============================================
  Files           120      120              
  Lines         64911    64915       +4     
============================================
+ Hits          45972    45995      +23     
+ Misses        18939    18920      -19     
Files with missing lines Coverage Δ
src/db.c 89.55% <100.00%> (+<0.01%) ⬆️

... and 17 files with indirect coverage changes

@soloestoy
Copy link
Member

The core of this issue is whether we should consider setting an expired time for a key as a delete operation or an expiration.

I believe it should be considered as expiration because the user's intended action is to set an expire time for the key. However, due to some reasons (such as operational delays), the time has already passed. Nonetheless, the key is still deleted due to expiration. Therefore, in terms of statistics and notifications, it should be counted as expiration-based deletion.

Additionally, we have already sent notifications regarding the expiration for the module:

void deleteExpiredKeyFromOverwriteAndPropagate(client *c, robj *keyobj) {
    int deleted = dbGenericDelete(c->db, keyobj, server.lazyfree_lazy_expire, DB_FLAG_KEY_EXPIRED);

int dbGenericDelete(serverDb *db, robj *key, int async, int flags) {
    int dict_index = getKVStoreIndexForKey(key->ptr);
    return dbGenericDeleteWithDictIndex(db, key, async, flags, dict_index);
}

int dbGenericDeleteWithDictIndex(serverDb *db, robj *key, int async, int flags, int dict_index) {
    hashtablePosition pos;
    void **ref = kvstoreHashtableTwoPhasePopFindRef(db->keys, dict_index, key->ptr, &pos);
    if (ref != NULL) {
        robj *val = *ref;
        /* VM_StringDMA may call dbUnshareStringValue which may free val, so we
         * need to incr to retain val */
        incrRefCount(val);
        /* Tells the module that the key has been unlinked from the database. */
        moduleNotifyKeyUnlink(key, val, db->id, flags);

However, there is a change here: the notification has been changed from GENERIC-del to EXPIRED-expired. This might be a potential breaking change, but theoretically, the actual impact should be minimal. Typically, if there's a concern about a key being deleted, the generic, expired, and evicted events would all be subscribed to. @valkey-io/core-team PTAL

@RayaCoo RayaCoo force-pushed the expire-del-statistic branch from 18efebb to aafe746 Compare January 7, 2025 07:18
Copy link
Member

@enjoy-binbin enjoy-binbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall make sense to me.

tests/unit/expire.tcl Outdated Show resolved Hide resolved
tests/unit/expire.tcl Outdated Show resolved Hide resolved
Signed-off-by: Ray Cao <[email protected]>
@madolson
Copy link
Member

madolson commented Jan 7, 2025

This might be a potential breaking change, but theoretically, the actual impact should be minimal.

Yeah, I think at the very least this needs to be marked as a breaking change, since we are changing the event type. Even if the new behavior is more correct, it's such a long standing behavior someone might have coded around it.

@enjoy-binbin enjoy-binbin added breaking-change Indicates a possible backwards incompatible change release-notes This issue should get a line item in the release notes labels Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change Indicates a possible backwards incompatible change release-notes This issue should get a line item in the release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants