BLOB cache read/write/maintenance interface. More...

#include <util/cache/icache.hpp>

Inheritance diagram for ICache:

Classes
struct	SBlobAccessDescr
	BLOB access descriptor. More...

Public Types
enum	EFlags { fBestPerformance = (0 << 0) , fBestReliability = (1 << 0) }
	Flags for the underlying storage. More...

enum	ETimeStampFlags { fNoTimeStamp = 0 , fTimeStampOnCreate = (1 << 0) , fTimeStampOnRead = (1 << 1) , fTrackSubKey = (1 << 2) , fExpireLeastFrequentlyUsed = (1 << 3) , fPurgeOnStartup = (1 << 4) , fCheckExpirationAlways = (1 << 5) }
	ICache keeps timestamps of every cache entry. More...

enum	EKeepVersions { eKeepAll , eDropOlder , eDropAll }
	If to keep already cached versions of the BLOB when storing another version of it (not necessarily a newer one) More...

enum	EBlobVersionValidity { eCurrent , eExpired , eValid = eCurrent }
	BLOB version existence and validity – from the point of view of the underlying cache implementation. More...

typedef int	TBlobVersion

typedef int	TFlags
	Bitwise OR of "EFlags" flags. More...

typedef ETimeStampFlags	ETimeStampPolicy

typedef int	TTimeStampFlags
	Holds a bitwise OR of "ETimeStampFlags". More...

typedef EBlobVersionValidity	EBlobValidity
	Backward-compatibility typedef. More...

typedef TPluginManagerParamTree	TCacheParams
	Key values to search for a cache with given params. More...

Public Member Functions
virtual TFlags	GetFlags ()=0
	Retrieve the effective combination of flags from the underlying storage. More...

virtual void	SetFlags (TFlags flags)=0
	Pass flags to the underlying storage. More...

virtual void	SetTimeStampPolicy (TTimeStampFlags policy, unsigned int timeout, unsigned int max_timeout=0)=0
	Set timestamp update policy. More...

virtual TTimeStampFlags	GetTimeStampPolicy () const =0
	Get timestamp policy. More...

virtual int	GetTimeout () const =0
	Get expiration timeout. More...

virtual bool	IsOpen () const =0

virtual void	SetVersionRetention (EKeepVersions policy)=0
	Set version retention policy. More...

virtual EKeepVersions	GetVersionRetention () const =0
	Get version retention. More...

virtual void	Store (const string &key, TBlobVersion version, const string &subkey, const void *data, size_t size, unsigned int time_to_live=0, const string &owner=kEmptyStr)=0
	Add or replace BLOB. More...

virtual size_t	GetSize (const string &key, TBlobVersion version, const string &subkey)=0
	Check if BLOB exists, return BLOB size. More...

virtual void	GetBlobOwner (const string &key, TBlobVersion version, const string &subkey, string *owner)=0
	Retrieve BLOB owner. More...

virtual bool	Read (const string &key, TBlobVersion version, const string &subkey, void *buf, size_t buf_size)=0
	Fetch the BLOB. More...

virtual IReader *	GetReadStream (const string &key, TBlobVersion version, const string &subkey)=0
	Return sequential stream interface to read BLOB data. More...

virtual IReader *	GetReadStream (const string &key, const string &subkey, TBlobVersion version, EBlobVersionValidity validity)=0
	Request the latest version of a BLOB. More...

virtual void	SetBlobVersionAsCurrent (const string &key, const string &subkey, TBlobVersion version)=0
	Set current valid version for a BLOB. More...

virtual void	GetBlobAccess (const string &key, TBlobVersion version, const string &subkey, SBlobAccessDescr *blob_descr)=0
	Get BLOB access using BlobAccessDescr. More...

virtual IWriter *	GetWriteStream (const string &key, TBlobVersion version, const string &subkey, unsigned int time_to_live=0, const string &owner=kEmptyStr)=0
	Return sequential stream interface to write BLOB data. More...

virtual void	Remove (const string &key, TBlobVersion version, const string &subkey)=0
	Remove specific cache entry. More...

virtual time_t	GetAccessTime (const string &key, TBlobVersion version, const string &subkey)=0
	Return last access time for the specified cache entry. More...

virtual bool	HasBlobs (const string &key, const string &subkey)=0
	Check if any BLOB exists (any version) More...

virtual void	Purge (time_t access_timeout)=0
	Delete all BLOBs older than specified. More...

virtual void	Purge (const string &key, const string &subkey, time_t access_timeout)=0
	Delete BLOBs with access time older than specified. More...

virtual	~ICache ()

virtual bool	SameCacheParams (const TCacheParams *params) const =0

virtual string	GetCacheName (void) const =0

Detailed Description

BLOB cache read/write/maintenance interface.

ICache describes caching service. Any large binary object can be stored in cache and later retrieved. Such cache is a temporary storage and some objects can be purged from cache based on an immediate request, version or access time based replacement (or another implementation specific depreciation rule).

Cache elements are accessed by key-subkey pair, and there can exist more than one version of the "same" (that correspond to the same key-subkey pair) element.

Note: Not all of the cache implementations are actually capable of storing more than one version of the same element.

Definition at line 63 of file icache.hpp.

Member Typedef Documentation

◆ EBlobValidity

typedef EBlobVersionValidity ICache::EBlobValidity

Backward-compatibility typedef.

Deprecated:: Use EBlobVersionValidity instead

See also: EBlobVersionValidity

Definition at line 284 of file icache.hpp.

◆ ETimeStampPolicy

typedef ETimeStampFlags ICache::ETimeStampPolicy

Definition at line 124 of file icache.hpp.

◆ TBlobVersion

typedef int ICache::TBlobVersion

Definition at line 66 of file icache.hpp.

◆ TCacheParams

typedef TPluginManagerParamTree ICache::TCacheParams

Key values to search for a cache with given params.

Used to share cache between readers and writers.

Definition at line 469 of file icache.hpp.

◆ TFlags

typedef int ICache::TFlags

Bitwise OR of "EFlags" flags.

Definition at line 89 of file icache.hpp.

◆ TTimeStampFlags

typedef int ICache::TTimeStampFlags

Holds a bitwise OR of "ETimeStampFlags".

See also: ETimeStampFlags

Definition at line 128 of file icache.hpp.

Member Enumeration Documentation

◆ EBlobVersionValidity

enum ICache::EBlobVersionValidity

BLOB version existence and validity – from the point of view of the underlying cache implementation.

The underlying cache implementation can expire blobs without actually deleting them. Then, the blob can still be read from the cache, and its validity confirmed (or not) against the original data source, and then its version usually validated (or invalidated) in the cache, accordingly.

See also: SetCurrentBlobVersion(), GetReadStream()

Enumerator
eCurrent	The returned BLOB's version is considered valid.
eExpired	Validity of the BLOB version cannot be confirmed.
eValid	Synonym for eCurrent.

Definition at line 274 of file icache.hpp.

◆ EFlags

enum ICache::EFlags

Flags for the underlying storage.

Enumerator

fBestPerformance

The storage to try to get to the best possible overall performance, even if it means sacrificing some reliability.

If combined with 'fBestReliability' flag it at least tries to avoid data corruption. If it is used without 'fBestReliability' flag, then it even allows for a data corruption in pursuit of the best performance.

fBestReliability

Usually, it's not a problem if something fails to get cached sometimes.

So, by default the storage can cut some corners to get more performance by saving on the reliability of caching operations. This flag tells the storage to put more value on the reliability (rather than on its performance) – to the extent allowed by the storage's inherent limitations.

Definition at line 69 of file icache.hpp.

◆ EKeepVersions

enum ICache::EKeepVersions

If to keep already cached versions of the BLOB when storing another version of it (not necessarily a newer one)

See also: Store(), GetWriteStream()

Enumerator
eKeepAll	Do not delete other versions of cache entries.
eDropOlder	Delete the earlier (than the one being stored) versions of the BLOB.
eDropAll	Delete all versions of the BLOB, even those which are newer than the one being stored.

Definition at line 162 of file icache.hpp.

◆ ETimeStampFlags

enum ICache::ETimeStampFlags

ICache keeps timestamps of every cache entry.

This enum defines the policy how it is managed. Different policies can be combined by OR (|)

See also: SetTimeStampPolicy, TTimeStampFlags

Enumerator
fNoTimeStamp	Timestamp management disabled.
fTimeStampOnCreate	Cache element is created with a certain timestamp (default)
fTimeStampOnRead	Timestamp is updated every on every access (read or write)
fTrackSubKey	Timestamp full key-subkey pair. By default only key is taken into account
fExpireLeastFrequentlyUsed	Expire objects older than a certain time frame Example: If object is not accessed within a week it is dropped from the cache.
fPurgeOnStartup	Expired objects should be deleted on cache mount (Open)
fCheckExpirationAlways	Expiration timeout is checked on any access to cache element.

Definition at line 104 of file icache.hpp.

Constructor & Destructor Documentation

◆ ~ICache()

virtual ICache::~ICache ( )

inlinevirtual

Definition at line 465 of file icache.hpp.

Member Function Documentation

◆ GetAccessTime()

virtual time_t ICache::GetAccessTime	(	const string &	key,
		TBlobVersion	version,
		const string &	subkey
	)

pure virtual

Return last access time for the specified cache entry.

Class implementation may want to implement access time based aging scheme for cache managed objects. In this case it needs to track time of every request to BLOB data.

Parameters

key	BLOB identification key
subkey	BLOB identification subkey
version	BLOB version

Returns: last access time

See also: TimeStampUpdatePolicy

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ GetBlobAccess()

virtual void ICache::GetBlobAccess	(	const string &	key,
		TBlobVersion	version,
		const string &	subkey,
		SBlobAccessDescr *	blob_descr
	)

pure virtual

Get BLOB access using BlobAccessDescr.

Method fills blob_descr parameter. If provided buffer has sufficient capacity for BLOB storage, BLOB is saved into the buffer, otherwise IReader is created.

Note: Method supposed to provide fast access to relatively small BLOBs

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CCacheHashedContent::GetHashedContent(), CCacheHashedContent::GetMediatorKeyContent(), CCacheHashedContent::StoreHashedContentByMediator(), and CCacheHashedContent::x_CheckHashContent().

◆ GetBlobOwner()

virtual void ICache::GetBlobOwner	(	const string &	key,
		TBlobVersion	version,
		const string &	subkey,
		string *	owner
	)

pure virtual

Retrieve BLOB owner.

Parameters

owner BLOB owner (as used by method Store)

See also: Store, GetWriteStream

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ GetCacheName()

virtual string ICache::GetCacheName ( void ) const

pure virtual

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ GetFlags()

virtual TFlags ICache::GetFlags ( )

pure virtual

Retrieve the effective combination of flags from the underlying storage.

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ GetReadStream() [1/2]

virtual IReader* ICache::GetReadStream	(	const string &	key,
		const string &	subkey,
		TBlobVersion *	version,
		EBlobVersionValidity *	validity
	)

pure virtual

Request the latest version of a BLOB.

Parameters

[in]	key	BLOB identification key
[in]	subkey	BLOB identification subkey
[out]	version	Version of the returned BLOB
[out]	validity	Validity of the version of the returned BLOB

Returns: Sequential stream interface to read BLOB data. Return NULL if the BLOB data is not found in the cache.

Note: If the BLOB data is not found in the cache, then neither 'version' nor 'validity' values will be set by the method.

See also: SetCurrentBlobVersion()

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ GetReadStream() [2/2]

virtual IReader* ICache::GetReadStream	(	const string &	key,
		TBlobVersion	version,
		const string &	subkey
	)

pure virtual

Return sequential stream interface to read BLOB data.

Parameters

key	BLOB identification key
subkey	BLOB identification subkey
version	BLOB version

Returns: Interface pointer or NULL if BLOB does not exist

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CCacheHashedContent::GetHashedContent(), CCgiApplication::GetSavedRequest(), s_ReadData(), CSGSegmentSmearJob::x_LoadFromNetCache(), CSparseGraph::x_ReadData(), CSparseGraph::x_ReadMap(), and CWigGraph::x_ReadSummary().

◆ GetSize()

virtual size_t ICache::GetSize	(	const string &	key,
		TBlobVersion	version,
		const string &	subkey
	)

pure virtual

Check if BLOB exists, return BLOB size.

Parameters

key	BLOB identification key
subkey	BLOB identification subkey
version	BLOB version

Returns: BLOB size or 0 if it doesn't exist or expired

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CSplitCacheApp::ProcessBlob(), CSplitCacheApp::ProcessEntry(), s_ReadData(), CSparseGraph::x_ReadData(), CSparseGraph::x_ReadMap(), and CWigGraph::x_ReadSummary().

◆ GetTimeout()

virtual int ICache::GetTimeout ( ) const

pure virtual

Get expiration timeout.

Returns: Expiration timeout in seconds

See also: SetTimeStampPolicy

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CCacheCleanerThread::DoJob().

◆ GetTimeStampPolicy()

virtual TTimeStampFlags ICache::GetTimeStampPolicy ( ) const

pure virtual

Get timestamp policy.

Returns: Timestamp policy

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ GetVersionRetention()

virtual EKeepVersions ICache::GetVersionRetention ( ) const

pure virtual

Get version retention.

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ GetWriteStream()

virtual IWriter* ICache::GetWriteStream	(	const string &	key,
		TBlobVersion	version,
		const string &	subkey,
		unsigned int	time_to_live = `0`,
		const string &	owner = `kEmptyStr`
	)

pure virtual

Return sequential stream interface to write BLOB data.

Parameters

key	BLOB identification key
subkey	BLOB identification subkey
version	BLOB version
time_to_live	Individual timeout

Returns: Interface pointer or NULL if BLOB does not exist

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CCgiApplication::SaveRequest(), CCacheHashedContent::StoreHashedContent(), CCacheHashedContent::StoreHashedContentByMediator(), CCacheWriter::WriteSeq_ids(), CBedCoverageGraph::x_SaveData(), CVcfHeatmap::x_SaveData(), CWigGraph::x_SaveSummary(), CSparseGraph::x_WriteData(), and CSGSegmentSmearJob::x_WriteToCache().

◆ HasBlobs()

virtual bool ICache::HasBlobs	(	const string &	key,
		const string &	subkey
	)

pure virtual

Check if any BLOB exists (any version)

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CCacheReader::LoadChunk(), and CSGSegmentSmearJob::x_Execute().

◆ IsOpen()

virtual bool ICache::IsOpen ( ) const

pure virtual

Returns: TRUE if cache is open and operational

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ Purge() [1/2]

virtual void ICache::Purge	(	const string &	key,
		const string &	subkey,
		time_t	access_timeout
	)

pure virtual

Delete BLOBs with access time older than specified.

Function finds all BLOB versions with the specified key and removes the old instances.

Parameters

key	BLOB key
subkey	BLOB identification subkey
access_timeout	Time in seconds. All objects older than this are deleted.

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ Purge() [2/2]

virtual void ICache::Purge ( time_t access_timeout )

pure virtual

Delete all BLOBs older than specified.

Parameters

access_timeout Time in seconds. All objects older than this are deleted.

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CCacheCleanerThread::DoJob().

◆ Read()

virtual bool ICache::Read	(	const string &	key,
		TBlobVersion	version,
		const string &	subkey,
		void *	buf,
		size_t	buf_size
	)

pure virtual

Fetch the BLOB.

Parameters

key	BLOB identification key
subkey	BLOB identification subkey
version	BLOB version
buf	pointer on destination buffer
size	buffer size in bytes (chars)

Returns: FALSE if BLOB doesn't exist or expired

Note: Throws an exception if provided memory buffer is insufficient to read the BLOB

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ Remove()

virtual void ICache::Remove	(	const string &	key,
		TBlobVersion	version,
		const string &	subkey
	)

pure virtual

Remove specific cache entry.

Parameters

key	BLOB identification key
subkey	BLOB identification subkey
version	BLOB version

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CSplitCacheApp::ProcessBlob(), CSplitCacheApp::ProcessEntry(), CCacheHashedContent::StoreHashedContentByMediator(), and CCacheWriter::WriteSeq_ids().

◆ SameCacheParams()

virtual bool ICache::SameCacheParams ( const TCacheParams * params ) const

pure virtual

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ SetBlobVersionAsCurrent()

virtual void ICache::SetBlobVersionAsCurrent	(	const string &	key,
		const string &	subkey,
		TBlobVersion	version
	)

pure virtual

Set current valid version for a BLOB.

Parameters

[in]	key	BLOB identification key
[in]	subkey	BLOB identification subkey
[in]	version	BLOB version which must be considered valid from now on

Note: Throw exception in case of transient errors.; There is an important special case: if the underlying storage is inherently incapable of storing BLOB version info without storing the BLOB itself, and there is currently no blob with the specified key/subkey/version present in the cache, then this function will just do nothing, and it will not throw exception.

See also: GetReadStream()

Implemented in CAsyncWriteCache, CBDB_Cache, and CNetICacheClient.

Referenced by CCacheReader::x_SetBlobVersionAsCurrent().

◆ SetFlags()

virtual void ICache::SetFlags ( TFlags flags )

pure virtual

Pass flags to the underlying storage.

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

◆ SetTimeStampPolicy()

virtual void ICache::SetTimeStampPolicy	(	TTimeStampFlags	policy,
		unsigned int	timeout,
		unsigned int	max_timeout = `0`
	)

pure virtual

Set timestamp update policy.

Parameters

policy	A bitwise combination of "ETimeStampFlags".
timeout	Default expiration timeout for the stored BLOBs.
max_timeout	Maximum value for individually set BLOB timeouts. If "max_timeout" < "timeout", then it 'll be set to "timeout".

Implemented in CSQLITE3_Cache, CBDB_Cache, CNetICacheClient, and CAsyncWriteCache.

Referenced by CICacheCF< CNetICacheClient >::ConfigureTimeStamp().

◆ SetVersionRetention()

virtual void ICache::SetVersionRetention ( EKeepVersions policy )

pure virtual

Set version retention policy.

Parameters

policy Version retention mode

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CICacheCF< CNetICacheClient >::ConfigureICache().

◆ Store()

virtual void ICache::Store	(	const string &	key,
		TBlobVersion	version,
		const string &	subkey,
		const void *	data,
		size_t	size,
		unsigned int	time_to_live = `0`,
		const string &	owner = `kEmptyStr`
	)

pure virtual

Add or replace BLOB.

Parameters

key	BLOB identification key
key	BLOB identification sub-key
version	BLOB version
data	pointer on data buffer
size	data buffer size in bytes (chars)
time_to_live	Individual timeout. Cannot exceed max timeout.

Implemented in CAsyncWriteCache, CSQLITE3_Cache, CBDB_Cache, and CNetICacheClient.

Referenced by CCacheHashedContent::StoreHashedContent(), CCacheHashedContent::StoreHashedContentByMediator(), CCacheHashedContent::StoreMediatorKeyContent(), and CCacheWriter::x_WriteId().

The documentation for this class was generated from the following file:

include/util/cache/icache.hpp

Classes

Public Types

Public Member Functions

Detailed Description

Member Typedef Documentation

◆ EBlobValidity

◆ ETimeStampPolicy

◆ TBlobVersion

◆ TCacheParams

◆ TFlags

◆ TTimeStampFlags

Member Enumeration Documentation

◆ EBlobVersionValidity

◆ EFlags

◆ EKeepVersions

◆ ETimeStampFlags

Constructor & Destructor Documentation

◆ ~ICache()

Member Function Documentation

◆ GetAccessTime()

◆ GetBlobAccess()

◆ GetBlobOwner()

◆ GetCacheName()

◆ GetFlags()

◆ GetReadStream() [1/2]

◆ GetReadStream() [2/2]

◆ GetSize()

◆ GetTimeout()

◆ GetTimeStampPolicy()

◆ GetVersionRetention()

◆ GetWriteStream()

◆ HasBlobs()

◆ IsOpen()

◆ Purge() [1/2]

◆ Purge() [2/2]

◆ Read()

◆ Remove()

◆ SameCacheParams()

◆ SetBlobVersionAsCurrent()

◆ SetFlags()

◆ SetTimeStampPolicy()

◆ SetVersionRetention()

◆ Store()