fix: avoid potential long term memory growth via uncapped unordered_lru_cache

[PastaPastaPasta]

We use a structure unordered_lru_cache in a number of places for a performant cache. However, some of these instances, we do not set any limit in the number of cached objects. This could be a potential cause for "memory leak" reports. While not a memory leak, setting reasonable limits to these caches may resolve this long term growth, with no to little downside.

unordered_lru_cache usages

src/evo/creditpool.h:110:    unordered_lru_cache<uint256, CCreditPool, StaticSaltedHasher> creditPoolCache GUARDED_BY(cache_mutex) {CreditPoolCacheSize};
src/evo/mnhftx.h:109:    unordered_lru_cache<uint256, Signals, StaticSaltedHasher> mnhfCache GUARDED_BY(cs_cache) {MNHFCacheSize};
src/llmq/blockprocessor.h:48:    mutable std::map<Consensus::LLMQType, unordered_lru_cache<uint256, bool, StaticSaltedHasher>> mapHasMinedCommitmentCache GUARDED_BY(minableCommitmentsCs);
src/llmq/dkgsessionmgr.cpp:188:    static std::map<Consensus::LLMQType, unordered_lru_cache<uint256, int, StaticSaltedHasher>> indexedQuorumsCache GUARDED_BY(cs_indexedQuorumsCache);
src/llmq/instantsend.h:74:    mutable unordered_lru_cache<uint256, CInstantSendLockPtr, StaticSaltedHasher, 10000> islockCache GUARDED_BY(cs_db);
src/llmq/instantsend.h:75:    mutable unordered_lru_cache<uint256, uint256, StaticSaltedHasher, 10000> txidCache GUARDED_BY(cs_db);
src/llmq/instantsend.h:77:    mutable unordered_lru_cache<COutPoint, uint256, SaltedOutpointHasher, 10000> outpointCache GUARDED_BY(cs_db);
src/llmq/quorums.h:249:    mutable std::map<Consensus::LLMQType, unordered_lru_cache<uint256, CQuorumPtr, StaticSaltedHasher>> mapQuorumsCache GUARDED_BY(cs_map_quorums);
src/llmq/quorums.h:251:    mutable std::map<Consensus::LLMQType, unordered_lru_cache<uint256, std::vector<CQuorumCPtr>, StaticSaltedHasher>> scanQuorumsCache GUARDED_BY(cs_scan_quorums);
src/llmq/quorums.h:253:    mutable std::map<Consensus::LLMQType, unordered_lru_cache<uint256, uint256, StaticSaltedHasher>> cleanupQuorumsCache GUARDED_BY(cs_cleanup);
src/llmq/signing.h:119:    mutable unordered_lru_cache<std::pair<Consensus::LLMQType, uint256>, bool, StaticSaltedHasher, 30000> hasSigForIdCache GUARDED_BY(cs_cache);
src/llmq/signing.h:120:    mutable unordered_lru_cache<uint256, bool, StaticSaltedHasher, 30000> hasSigForSessionCache GUARDED_BY(cs_cache);
src/llmq/signing.h:121:    mutable unordered_lru_cache<uint256, bool, StaticSaltedHasher, 30000> hasSigForHashCache GUARDED_BY(cs_cache);
src/llmq/snapshot.h:222:    unordered_lru_cache<uint256, CQuorumSnapshot, StaticSaltedHasher> quorumSnapshotCache GUARDED_BY(snapshotCacheCs);
src/llmq/utils.cpp:103:    static std::map<Consensus::LLMQType, unordered_lru_cache<uint256, std::vector<CDeterministicMNCPtr>, StaticSaltedHasher>> mapQuorumMembers GUARDED_BY(cs_members);
src/llmq/utils.cpp:105:    static std::map<Consensus::LLMQType, unordered_lru_cache<std::pair<uint256, int>, std::vector<CDeterministicMNCPtr>, StaticSaltedHasher>> mapIndexedQuorumMembers GUARDED_BY(cs_indexed_members);
src/llmq/utils.cpp:862:template void InitQuorumsCache<std::map<Consensus::LLMQType, unordered_lru_cache<uint256, bool, StaticSaltedHasher>>>(std::map<Consensus::LLMQType, unordered_lru_cache<uint256, bool, StaticSaltedHasher>>& cache, bool limit_by_connections);
src/llmq/utils.cpp:863:template void InitQuorumsCache<std::map<Consensus::LLMQType, unordered_lru_cache<uint256, std::vector<CQuorumCPtr>, StaticSaltedHasher>>>(std::map<Consensus::LLMQType, unordered_lru_cache<uint256, std::vector<CQuorumCPtr>, StaticSaltedHasher>>& cache, bool limit_by_connections);
src/llmq/utils.cpp:864:template void InitQuorumsCache<std::map<Consensus::LLMQType, unordered_lru_cache<uint256, std::shared_ptr<llmq::CQuorum>, StaticSaltedHasher, 0ul, 0ul>, std::less<Consensus::LLMQType>, std::allocator<std::pair<Consensus::LLMQType const, unordered_lru_cache<uint256, std::shared_ptr<llmq::CQuorum>, StaticSaltedHasher, 0ul, 0ul>>>>>(std::map<Consensus::LLMQType, unordered_lru_cache<uint256, std::shared_ptr<llmq::CQuorum>, StaticSaltedHasher, 0ul, 0ul>, std::less<Consensus::LLMQType>, std::allocator<std::pair<Consensus::LLMQType const, unordered_lru_cache<uint256, std::shared_ptr<llmq::CQuorum>, StaticSaltedHasher, 0ul, 0ul>>>>&cache, bool limit_by_connections);
src/llmq/utils.cpp:865:template void InitQuorumsCache<std::map<Consensus::LLMQType, unordered_lru_cache<uint256, int, StaticSaltedHasher>>>(std::map<Consensus::LLMQType, unordered_lru_cache<uint256, int, StaticSaltedHasher>>& cache, bool limit_by_connections);
src/llmq/utils.cpp:866:template void InitQuorumsCache<std::map<Consensus::LLMQType, unordered_lru_cache<uint256, uint256, StaticSaltedHasher>>>(std::map<Consensus::LLMQType, unordered_lru_cache<uint256, uint256, StaticSaltedHasher>>& cache, bool limit_by_connections);

[knst]

There's no PR at the moment for this issue, because:

• src/evo/creditpool.h:110 - already limited as 1000 - see CreditPoolCacheSize
• src/evo/mnhftx.h:109: - already limited as 1000 - see MNHFCacheSize
• src/llmq/blockprocessor.h:48: - already limited, see llmq::utils::InitQuorumsCache
• src/llmq/dkgsessionmgr.cpp:188 - already limited, see llmq::utils::InitQuorumsCache
• src/llmq/instantsend.h:* - all limited by 10k
• src/llmq/quorums.h:249: - limited by llmq::utils::InitQuorumsCache
• src/llmq/quorums.h:251: - limited by llmq.max_cycles(llmq.keepOldConnections) * (llmq.dkgMiningWindowEnd - llmq.dkgMiningWindowStart))
• src/llmq/quorums.h:253: - limited by llmq::utils::InitQuorumsCache
• src/llmq/signing.h:119: - hard-limit 30k
• src/llmq/signing.h:120: - hard-limit 30k
• src/llmq/signing.h:121: - hard-limit 30k
• src/llmq/snapshot.h:222: - hard-limit 32, see CQuorumSnapshotManager::CQuorumSnapshotManager()
• src/llmq/utils.cpp:103: - limited by llmq::utils::InitQuorumsCache
• src/llmq/utils.cpp:105: - limited by llmq::utils::InitQuorumsCache
• src/llmq/utils.cpp:862: - implementation of llmq::utils::InitQuorumsCache for this some template type
• src/llmq/utils.cpp:863: - implementation of llmq::utils::InitQuorumsCache for this some template type
• src/llmq/utils.cpp:864: - implementation of llmq::utils::InitQuorumsCache for this some template type
• src/llmq/utils.cpp:865 - implementation of llmq::utils::InitQuorumsCache for this some template type
• src/llmq/utils.cpp:866 - implementation of llmq::utils::InitQuorumsCache for this some template type

I will have a look what else can be source of memory-balooning / memory-leak.