DiskCache API Reference

The DiskCache Tutorial provides a helpful walkthrough of most methods.

DjangoCache

Read the DjangoCache tutorial for example usage.

class diskcache.DjangoCache(directory, params)

Django-compatible disk and file backed cache.

__init__(directory, params)

Initialize DjangoCache instance.

Parameters:
  • directory (str) – cache directory
  • params (dict) – cache parameters
add(key, value, timeout=<object object>, version=None, read=False, tag=None, retry=True)

Set a value in the cache if the key does not already exist. If timeout is given, that timeout will be used for the key; otherwise the default cache timeout will be used.

Return True if the value was stored, False otherwise.

Parameters:
  • key – key for item
  • value – value for item
  • timeout (float) – seconds until the item expires (default 300 seconds)
  • version (int) – key version number (default None, cache parameter)
  • read (bool) – read value as bytes from file (default False)
  • tag (str) – text to associate with key (default None)
  • retry (bool) – retry if database timeout expires (default True)
Returns:

True if item was added

clear()

Remove all values from the cache at once.

close(**kwargs)

Close the cache connection.

create_tag_index()

Create tag index on cache database.

It is better to initialize cache with tag_index=True than use this.

Raises:Timeout – if database timeout expires
cull()

Cull items from cache until volume is less than size limit.

Returns:count of items removed
decr(key, delta=1, version=None, default=None, retry=True)

Decrement value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent decrement operations will be counted individually.

Unlike Memcached, negative values are supported. Value may be decremented below zero.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters:
  • key – key for item
  • delta (int) – amount to decrement (default 1)
  • version (int) – key version number (default None, cache parameter)
  • default (int) – value if key is missing (default None)
  • retry (bool) – retry if database timeout expires (default True)
Returns:

new value for item on success else None

Raises:

ValueError – if key is not found and default is None

delete(key, version=None, retry=True)

Delete a key from the cache, failing silently.

Parameters:
  • key – key for item
  • version (int) – key version number (default None, cache parameter)
  • retry (bool) – retry if database timeout expires (default True)
Returns:

True if item was deleted

deque(name)

Return Deque with given name in subdirectory.

Parameters:name (str) – subdirectory name for Deque
Returns:Deque with given name
directory

Cache directory.

drop_tag_index()

Drop tag index on cache database.

Raises:Timeout – if database timeout expires
evict(tag)

Remove items with matching tag from cache.

Parameters:tag (str) – tag identifying items
Returns:count of items removed
expire()

Remove expired items from cache.

Returns:count of items removed
get(key, default=None, version=None, read=False, expire_time=False, tag=False, retry=False)

Fetch a given key from the cache. If the key does not exist, return default, which itself defaults to None.

Parameters:
  • key – key for item
  • default – return value if key is missing (default None)
  • version (int) – key version number (default None, cache parameter)
  • read (bool) – if True, return file handle to value (default False)
  • expire_time (float) – if True, return expire_time in tuple (default False)
  • tag – if True, return tag in tuple (default False)
  • retry (bool) – retry if database timeout expires (default False)
Returns:

value for item if key is found else default

get_backend_timeout(timeout=<object object>)

Return seconds to expiration.

Parameters:timeout (float) – seconds until the item expires (default 300 seconds)
has_key(key, version=None)

Returns True if the key is in the cache and has not expired.

Parameters:
  • key – key for item
  • version (int) – key version number (default None, cache parameter)
Returns:

True if key is found

incr(key, delta=1, version=None, default=None, retry=True)

Increment value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent increment operations will be counted individually.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters:
  • key – key for item
  • delta (int) – amount to increment (default 1)
  • version (int) – key version number (default None, cache parameter)
  • default (int) – value if key is missing (default None)
  • retry (bool) – retry if database timeout expires (default True)
Returns:

new value for item on success else None

Raises:

ValueError – if key is not found and default is None

index(name)

Return Index with given name in subdirectory.

Parameters:name (str) – subdirectory name for Index
Returns:Index with given name
pop(key, default=None, version=None, expire_time=False, tag=False, retry=True)

Remove corresponding item for key from cache and return value.

If key is missing, return default.

Operation is atomic. Concurrent operations will be serialized.

Parameters:
  • key – key for item
  • default – return value if key is missing (default None)
  • version (int) – key version number (default None, cache parameter)
  • expire_time (float) – if True, return expire_time in tuple (default False)
  • tag – if True, return tag in tuple (default False)
  • retry (bool) – retry if database timeout expires (default True)
Returns:

value for item if key is found else default

read(key, version=None)

Return file handle corresponding to key from Cache.

Parameters:
  • key – Python key to retrieve
  • version (int) – key version number (default None, cache parameter)
Returns:

file open for reading in binary mode

Raises:

KeyError – if key is not found

set(key, value, timeout=<object object>, version=None, read=False, tag=None, retry=True)

Set a value in the cache. If timeout is given, that timeout will be used for the key; otherwise the default cache timeout will be used.

Parameters:
  • key – key for item
  • value – value for item
  • timeout (float) – seconds until the item expires (default 300 seconds)
  • version (int) – key version number (default None, cache parameter)
  • read (bool) – read value as bytes from file (default False)
  • tag (str) – text to associate with key (default None)
  • retry (bool) – retry if database timeout expires (default True)
Returns:

True if item was set

stats(enable=True, reset=False)

Return cache statistics hits and misses.

Parameters:
  • enable (bool) – enable collecting statistics (default True)
  • reset (bool) – reset hits and misses to 0 (default False)
Returns:

(hits, misses)

FanoutCache

Read the FanoutCache tutorial for example usage.

class diskcache.FanoutCache(directory, shards=8, timeout=0.01, disk=<class 'diskcache.core.Disk'>, **settings)

Cache that shards keys and values.

__contains__(key)

Return True if key matching item is found in cache.

Parameters:key – key for item
Returns:True if key is found
__delitem__(key)

Delete corresponding item for key from cache.

Parameters:key – key for item
Raises:KeyError – if key is not found
__getitem__(key)

Return corresponding value for key from cache.

Parameters:key – key for item
Returns:value for item
Raises:KeyError – if key is not found
__init__(directory, shards=8, timeout=0.01, disk=<class 'diskcache.core.Disk'>, **settings)

Initialize cache instance.

Parameters:
  • directory (str) – cache directory
  • shards (int) – number of shards to distribute writes
  • timeout (float) – SQLite connection timeout
  • diskDisk instance for serialization
  • settings – any of DEFAULT_SETTINGS
__iter__()

Iterate keys in cache including expired items.

__len__()

Count of items in cache including expired items.

__reversed__()

Reverse iterate keys in cache including expired items.

__setitem__(key, value)

Set key and value item in cache.

Parameters:
  • key – key for item
  • value – value for item
add(key, value, expire=None, read=False, tag=None, retry=False)

Add key and value item to cache.

Similar to set, but only add to cache if key not present.

This operation is atomic. Only one concurrent add operation for given key from separate threads or processes will succeed.

When read is True, value should be a file-like object opened for reading in binary mode.

Parameters:
  • key – key for item
  • value – value for item
  • expire (float) – seconds until the key expires (default None, no expiry)
  • read (bool) – read value as bytes from file (default False)
  • tag (str) – text to associate with key (default None)
  • retry (bool) – retry if database timeout expires (default False)
Returns:

True if item was added

check(fix=False)

Check database and file system consistency.

Intended for use in testing and post-mortem error analysis.

While checking the cache table for consistency, a writer lock is held on the database. The lock blocks other cache clients from writing to the database. For caches with many file references, the lock may be held for a long time. For example, local benchmarking shows that a cache with 1,000 file references takes ~60ms to check.

Parameters:fix (bool) – correct inconsistencies
Returns:list of warnings
Raises:Timeout – if database timeout expires
clear()

Remove all items from cache.

Returns:count of items removed
close()

Close database connection.

create_tag_index()

Create tag index on cache database.

It is better to initialize cache with tag_index=True than use this.

Raises:Timeout – if database timeout expires
cull()

Cull items from cache until volume is less than size limit.

Returns:count of items removed
decr(key, delta=1, default=0, retry=False)

Decrement value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent decrement operations will be counted individually.

Unlike Memcached, negative values are supported. Value may be decremented below zero.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters:
  • key – key for item
  • delta (int) – amount to decrement (default 1)
  • default (int) – value if key is missing (default 0)
  • retry (bool) – retry if database timeout expires (default False)
Returns:

new value for item on success else None

Raises:

KeyError – if key is not found and default is None

delete(key, retry=False)

Delete corresponding item for key from cache.

Missing keys are ignored.

Parameters:
  • key – key for item
  • retry (bool) – retry if database timeout expires (default False)
Returns:

True if item was deleted

deque(name)

Return Deque with given name in subdirectory.

>>> cache = FanoutCache('/tmp/diskcache/fanoutcache')
>>> deque = cache.deque('test')
>>> deque.clear()
>>> deque.extend('abc')
>>> deque.popleft()
'a'
>>> deque.pop()
'c'
>>> len(deque)
1
Parameters:name (str) – subdirectory name for Deque
Returns:Deque with given name
directory

Cache directory.

drop_tag_index()

Drop tag index on cache database.

Raises:Timeout – if database timeout expires
evict(tag)

Remove items with matching tag from cache.

Parameters:tag (str) – tag identifying items
Returns:count of items removed
expire()

Remove expired items from cache.

Returns:count of items removed
get(key, default=None, read=False, expire_time=False, tag=False, retry=False)

Retrieve value from cache. If key is missing, return default.

Parameters:
  • key – key for item
  • default – return value if key is missing (default None)
  • read (bool) – if True, return file handle to value (default False)
  • expire_time (float) – if True, return expire_time in tuple (default False)
  • tag – if True, return tag in tuple (default False)
  • retry (bool) – retry if database timeout expires (default False)
Returns:

value for item if key is found else default

incr(key, delta=1, default=0, retry=False)

Increment value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent increment operations will be counted individually.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters:
  • key – key for item
  • delta (int) – amount to increment (default 1)
  • default (int) – value if key is missing (default 0)
  • retry (bool) – retry if database timeout expires (default False)
Returns:

new value for item on success else None

Raises:

KeyError – if key is not found and default is None

index(name)

Return Index with given name in subdirectory.

>>> cache = FanoutCache('/tmp/diskcache/fanoutcache')
>>> index = cache.index('test')
>>> index.clear()
>>> index['abc'] = 123
>>> index['def'] = 456
>>> index['ghi'] = 789
>>> index.popitem()
('ghi', 789)
>>> del index['abc']
>>> len(index)
1
>>> index['def']
456
Parameters:name (str) – subdirectory name for Index
Returns:Index with given name
memoize(name=None, typed=False, expire=None, tag=None)

Memoizing cache decorator.

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

If name is set to None (default), the callable name will be determined automatically.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

>>> from diskcache import FanoutCache
>>> cache = FanoutCache('/tmp/diskcache/fanoutcache')
>>> @cache.memoize(typed=True, expire=1, tag='fib')
... def fibonacci(number):
...     if number == 0:
...         return 0
...     elif number == 1:
...         return 1
...     else:
...         return fibonacci(number - 1) + fibonacci(number - 2)
>>> print(sum(fibonacci(number=value) for value in range(100)))
573147844013817084100

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur. Note the lack of parenthenses after memoize below:

>>> @cache.memoize
... def test():
...     pass
Traceback (most recent call last):
    ...
TypeError: name cannot be callable
Parameters:
  • cache – cache to store callable arguments and return values
  • name (str) – name given for callable (default None, automatic)
  • typed (bool) – cache different types separately (default False)
  • expire (float) – seconds until arguments expire (default None, no expiry)
  • tag (str) – text to associate with arguments (default None)
Returns:

callable decorator

pop(key, default=None, expire_time=False, tag=False, retry=False)

Remove corresponding item for key from cache and return value.

If key is missing, return default.

Operation is atomic. Concurrent operations will be serialized.

Parameters:
  • key – key for item
  • default – return value if key is missing (default None)
  • expire_time (float) – if True, return expire_time in tuple (default False)
  • tag – if True, return tag in tuple (default False)
  • retry (bool) – retry if database timeout expires (default False)
Returns:

value for item if key is found else default

read(key)

Return file handle corresponding to key from cache.

Parameters:key – key for item
Returns:file open for reading in binary mode
Raises:KeyError – if key is not found
reset(key, value=ENOVAL)

Reset key and value item from Settings table.

If value is not given, it is reloaded from the Settings table. Otherwise, the Settings table is updated.

Settings attributes on cache objects are lazy-loaded and read-only. Use reset to update the value.

Settings with the sqlite_ prefix correspond to SQLite pragmas. Updating the value will execute the corresponding PRAGMA statement.

Parameters:
  • key (str) – Settings key for item
  • value – value for item (optional)
Returns:

updated value for item

Raises:

Timeout – if database timeout expires

set(key, value, expire=None, read=False, tag=None, retry=False)

Set key and value item in cache.

When read is True, value should be a file-like object opened for reading in binary mode.

Parameters:
  • key – key for item
  • value – value for item
  • expire (float) – seconds until the key expires (default None, no expiry)
  • read (bool) – read value as raw bytes from file (default False)
  • tag (str) – text to associate with key (default None)
  • retry (bool) – retry if database timeout expires (default False)
Returns:

True if item was set

stats(enable=True, reset=False)

Return cache statistics hits and misses.

Parameters:
  • enable (bool) – enable collecting statistics (default True)
  • reset (bool) – reset hits and misses to 0 (default False)
Returns:

(hits, misses)

volume()

Return estimated total size of cache on disk.

Returns:size in bytes

Cache

Read the Cache tutorial for example usage.

class diskcache.Cache(directory, timeout=60, disk=<class 'diskcache.core.Disk'>, **settings)

Disk and file backed cache.

__contains__(key)

Return True if key matching item is found in cache.

Parameters:key – key matching item
Returns:True if key matching item
__delitem__(key)

Delete corresponding item for key from cache.

Parameters:

key – key matching item

Raises:
  • KeyError – if key is not found
  • Timeout – if database timeout expires
__getitem__(key)

Return corresponding value for key from cache.

Parameters:

key – key matching item

Returns:

corresponding value

Raises:
  • KeyError – if key is not found
  • Timeout – if database timeout expires
__init__(directory, timeout=60, disk=<class 'diskcache.core.Disk'>, **settings)

Initialize cache instance.

Parameters:
  • directory (str) – cache directory
  • timeout (float) – SQLite connection timeout
  • disk – Disk type or subclass for serialization
  • settings – any of DEFAULT_SETTINGS
__iter__()

Iterate keys in cache including expired items.

__len__()

Count of items in cache including expired items.

__reversed__()

Reverse iterate keys in cache including expired items.

__setitem__(key, value, expire=None, read=False, tag=None)

Set key and value item in cache.

When read is True, value should be a file-like object opened for reading in binary mode.

Parameters:
  • key – key for item
  • value – value for item
  • expire (float) – seconds until item expires (default None, no expiry)
  • read (bool) – read value as bytes from file (default False)
  • tag (str) – text to associate with key (default None)
Returns:

True if item was set

Raises:

Timeout – if database timeout expires

add(key, value, expire=None, read=False, tag=None)

Add key and value item to cache.

Similar to set, but only add to cache if key not present.

Operation is atomic. Only one concurrent add operation for a given key will succeed.

When read is True, value should be a file-like object opened for reading in binary mode.

Parameters:
  • key – key for item
  • value – value for item
  • expire (float) – seconds until the key expires (default None, no expiry)
  • read (bool) – read value as bytes from file (default False)
  • tag (str) – text to associate with key (default None)
Returns:

True if item was added

Raises:

Timeout – if database timeout expires

check(fix=False)

Check database and file system consistency.

Intended for use in testing and post-mortem error analysis.

While checking the Cache table for consistency, a writer lock is held on the database. The lock blocks other cache clients from writing to the database. For caches with many file references, the lock may be held for a long time. For example, local benchmarking shows that a cache with 1,000 file references takes ~60ms to check.

Parameters:fix (bool) – correct inconsistencies
Returns:list of warnings
Raises:Timeout – if database timeout expires
clear()

Remove all items from cache.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Returns:count of rows removed
Raises:Timeout – if database timeout expires
close()

Close database connection.

create_tag_index()

Create tag index on cache database.

It is better to initialize cache with tag_index=True than use this.

Raises:Timeout – if database timeout expires
cull()

Cull items from cache until volume is less than size limit.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Returns:count of items removed
Raises:Timeout – if database timeout expires
decr(key, delta=1, default=0)

Decrement value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent decrement operations will be counted individually.

Unlike Memcached, negative values are supported. Value may be decremented below zero.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters:
  • key – key for item
  • delta (int) – amount to decrement (default 1)
  • default (int) – value if key is missing (default 0)
Returns:

new value for item

Raises:
  • KeyError – if key is not found and default is None
  • Timeout – if database timeout expires
delete(key)

Delete corresponding item for key from cache.

Missing keys are ignored.

Parameters:key – key matching item
Returns:True if item was deleted
Raises:Timeout – if database timeout expires
directory

Cache directory.

disk

Disk used for serialization.

drop_tag_index()

Drop tag index on cache database.

Raises:Timeout – if database timeout expires
evict(tag)

Remove items with matching tag from cache.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Parameters:tag (str) – tag identifying items
Returns:count of rows removed
Raises:Timeout – if database timeout expires
expire(now=None)

Remove expired items from cache.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Parameters:now (float) – current time (default None, time.time() used)
Returns:count of items removed
Raises:Timeout – if database timeout expires
get(key, default=None, read=False, expire_time=False, tag=False)

Retrieve value from cache. If key is missing, return default.

Parameters:
  • key – key for item
  • default – value to return if key is missing (default None)
  • read (bool) – if True, return file handle to value (default False)
  • expire_time (bool) – if True, return expire_time in tuple (default False)
  • tag (bool) – if True, return tag in tuple (default False)
Returns:

value for item or default if key not found

Raises:

Timeout – if database timeout expires

incr(key, delta=1, default=0)

Increment value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent increment operations will be counted individually.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters:
  • key – key for item
  • delta (int) – amount to increment (default 1)
  • default (int) – value if key is missing (default None)
Returns:

new value for item

Raises:
  • KeyError – if key is not found and default is None
  • Timeout – if database timeout expires
iterkeys(reverse=False)

Iterate Cache keys in database sort order.

>>> cache = Cache('/tmp/diskcache')
>>> _ = cache.clear()
>>> for key in [4, 1, 3, 0, 2]:
...     cache[key] = key
>>> list(cache.iterkeys())
[0, 1, 2, 3, 4]
>>> list(cache.iterkeys(reverse=True))
[4, 3, 2, 1, 0]
Parameters:reverse (bool) – reverse sort order (default False)
Returns:iterator of Cache keys
pop(key, default=None, expire_time=False, tag=False)

Remove corresponding item for key from cache and return value.

If key is missing, return default.

Operation is atomic. Concurrent operations will be serialized.

Parameters:
  • key – key for item
  • default – value to return if key is missing (default None)
  • expire_time (bool) – if True, return expire_time in tuple (default False)
  • tag (bool) – if True, return tag in tuple (default False)
Returns:

value for item or default if key not found

Raises:

Timeout – if database timeout expires

pull(prefix=None, default=(None, None), side='front', expire_time=False, tag=False)

Pull key and value item pair from side of queue in cache.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

If queue is empty, return default.

Defaults to pulling key and value item pairs from front of queue. Set side to ‘back’ to pull from back of queue. Side must be one of ‘front’ or ‘back’.

Operation is atomic. Concurrent operations will be serialized.

See also Cache.push and Cache.get.

>>> cache = Cache('/tmp/test')
>>> _ = cache.clear()
>>> cache.pull()
(None, None)
>>> for letter in 'abc':
...     print(cache.push(letter))
500000000000000
500000000000001
500000000000002
>>> key, value = cache.pull()
>>> print(key)
500000000000000
>>> value
'a'
>>> _, value = cache.pull(side='back')
>>> value
'c'
>>> cache.push(1234, 'userids')
'userids-500000000000000'
>>> _, value = cache.pull('userids')
>>> value
1234
Parameters:
  • prefix (str) – key prefix (default None, key is integer)
  • default – value to return if key is missing (default (None, None))
  • side (str) – either ‘front’ or ‘back’ (default ‘front’)
  • expire_time (bool) – if True, return expire_time in tuple (default False)
  • tag (bool) – if True, return tag in tuple (default False)
Returns:

key and value item pair or default if queue is empty

Raises:

Timeout – if database timeout expires

push(value, prefix=None, side='back', expire=None, read=False, tag=None)

Push value onto side of queue identified by prefix in cache.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

Defaults to pushing value on back of queue. Set side to ‘front’ to push value on front of queue. Side must be one of ‘back’ or ‘front’.

Operation is atomic. Concurrent operations will be serialized.

When read is True, value should be a file-like object opened for reading in binary mode.

See also Cache.pull.

>>> cache = Cache('/tmp/test')
>>> _ = cache.clear()
>>> print(cache.push('first value'))
500000000000000
>>> cache.get(500000000000000)
'first value'
>>> print(cache.push('second value'))
500000000000001
>>> print(cache.push('third value', side='front'))
499999999999999
>>> cache.push(1234, prefix='userids')
'userids-500000000000000'
Parameters:
  • value – value for item
  • prefix (str) – key prefix (default None, key is integer)
  • side (str) – either ‘back’ or ‘front’ (default ‘back’)
  • expire (float) – seconds until the key expires (default None, no expiry)
  • read (bool) – read value as bytes from file (default False)
  • tag (str) – text to associate with key (default None)
Returns:

key for item in cache

Raises:

Timeout – if database timeout expires

read(key)

Return file handle value corresponding to key from cache.

Parameters:

key – key matching item

Returns:

file open for reading in binary mode

Raises:
  • KeyError – if key is not found
  • Timeout – if database timeout expires
reset(key, value=ENOVAL, update=True)

Reset key and value item from Settings table.

Use reset to update the value of Cache settings correctly. Cache settings are stored in the Settings table of the SQLite database. If update is False then no attempt is made to update the database.

If value is not given, it is reloaded from the Settings table. Otherwise, the Settings table is updated.

Settings with the disk_ prefix correspond to Disk attributes. Updating the value will change the unprefixed attribute on the associated Disk instance.

Settings with the sqlite_ prefix correspond to SQLite pragmas. Updating the value will execute the corresponding PRAGMA statement.

SQLite PRAGMA statements may be executed before the Settings table exists in the database by setting update to False.

Parameters:
  • key (str) – Settings key for item
  • value – value for item (optional)
  • update (bool) – update database Settings table (default True)
Returns:

updated value for item

Raises:

Timeout – if database timeout expires

set(key, value, expire=None, read=False, tag=None)

Set key and value item in cache.

When read is True, value should be a file-like object opened for reading in binary mode.

Parameters:
  • key – key for item
  • value – value for item
  • expire (float) – seconds until item expires (default None, no expiry)
  • read (bool) – read value as bytes from file (default False)
  • tag (str) – text to associate with key (default None)
Returns:

True if item was set

Raises:

Timeout – if database timeout expires

stats(enable=True, reset=False)

Return cache statistics hits and misses.

Parameters:
  • enable (bool) – enable collecting statistics (default True)
  • reset (bool) – reset hits and misses to 0 (default False)
Returns:

(hits, misses)

timeout

SQLite connection timeout value in seconds.

volume()

Return estimated total size of cache on disk.

Returns:size in bytes

Constants

Read the Settings tutorial for details.

diskcache.DEFAULT_SETTINGS
  • statistics (int) default 0 - disabled when 0, enabled when 1.
  • tag_index (int) default 0 - disabled when 0, enabled when 1.
  • eviction_policy (str) default “least-recently-stored” - any of the keys in EVICTION_POLICY as described below.
  • size_limit (int, in bytes) default one gigabyte - approximate size limit of cache.
  • cull_limit (int) default ten - maximum number of items culled during set or add operations.
  • sqlite_auto_vacuum (int) default 1, “FULL” - SQLite auto vacuum pragma.
  • sqlite_cache_size (int, in pages) default 8,192 - SQLite cache size pragma.
  • sqlite_journal_mode (str) default “wal” - SQLite journal mode pragma.
  • sqlite_mmap_size (int, in bytes) default 64 megabytes - SQLite mmap size pragma.
  • sqlite_synchronous (int) default 1, “NORMAL” - SQLite synchronous pragma.
  • disk_min_file_size (int, in bytes) default one kilobyte - values with greater size are stored in files.
  • disk_pickle_protocol (int) default highest Pickle protocol - the Pickle protocol to use for data types that are not natively supported.
diskcache.EVICTION_POLICY
  • least-recently-stored (default) - evict least recently stored keys first.
  • least-recently-used - evict least recently used keys first.
  • least-frequently-used - evict least frequently used keys first.
  • none - never evict keys.

Disk

Read the Disk tutorial for details.

class diskcache.Disk(directory, min_file_size=0, pickle_protocol=0)

Cache key and value serialization for SQLite database and files.

__init__(directory, min_file_size=0, pickle_protocol=0)

Initialize disk instance.

Parameters:
  • directory (str) – directory path
  • min_file_size (int) – minimum size for file use
  • pickle_protocol (int) – pickle protocol for serialization
fetch(mode, filename, value, read)

Convert fields mode, filename, and value from Cache table to value.

Parameters:
  • mode (int) – value mode raw, binary, text, or pickle
  • filename (str) – filename of corresponding value
  • value – database value
  • read (bool) – when True, return an open file handle
Returns:

corresponding Python value

filename(key=UNKNOWN, value=UNKNOWN)

Return filename and full-path tuple for file storage.

Filename will be a randomly generated 28 character hexadecimal string with “.val” suffixed. Two levels of sub-directories will be used to reduce the size of directories. On older filesystems, lookups in directories with many files may be slow.

The default implementation ignores the key and value parameters.

In some scenarios, for example Cache.push, the key or value may not be known when the item is stored in the cache.

Parameters:
  • key – key for item (default UNKNOWN)
  • value – value for item (default UNKNOWN)
get(key, raw)

Convert fields key and raw from Cache table to key.

Parameters:
  • key – database key to convert
  • raw (bool) – flag indicating raw database storage
Returns:

corresponding Python key

hash(key)

Compute portable hash for key.

Parameters:key – key to hash
Returns:hash value
put(key)

Convert key to fields key and raw for Cache table.

Parameters:key – key to convert
Returns:(database key, raw boolean) pair
remove(filename)

Remove a file given by filename.

This method is cross-thread and cross-process safe. If an “error no entry” occurs, it is suppressed.

Parameters:filename (str) – relative path to file
store(value, read, key=UNKNOWN)

Convert value to fields size, mode, filename, and value for Cache table.

Parameters:
  • value – value to convert
  • read (bool) – True when value is file-like object
  • key – key for item (default UNKNOWN)
Returns:

(size, mode, filename, value) tuple for Cache table

Timeout

exception diskcache.Timeout

Database timeout expired.

Deque

class diskcache.Deque(iterable=(), directory=None)

Persistent sequence with double-ended queue semantics.

Double-ended queue is an ordered collection with optimized access at its endpoints.

Items are serialized to disk. Deque may be initialized from directory path where items are stored.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque
Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque += range(5)
>>> list(deque)
[0, 1, 2, 3, 4]
>>> for value in range(5):
...     deque.appendleft(-value)
>>> len(deque)
10
>>> list(deque)
[-4, -3, -2, -1, 0, 0, 1, 2, 3, 4]
>>> deque.pop()
4
>>> deque.popleft()
-4
>>> deque.reverse()
>>> list(deque)
[3, 2, 1, 0, 0, -1, -2, -3]
__delitem__(index)

deque.__delitem__(index) <==> del deque[index]

Delete item in deque at index.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque.extend([None] * 3)
>>> del deque[0]
>>> del deque[1]
>>> del deque[-1]
>>> len(deque)
0
Parameters:index (int) – index of item
Raises:IndexError – if index out of range
__eq__(that)

Return True if and only if deque is equal to that.

__ge__(that)

Return True if and only if deque is greater than or equal to that.

__getitem__(index)

deque.__getitem__(index) <==> deque[index]

Return corresponding item for index in deque.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque.extend('abcde')
>>> deque[0]
'a'
>>> deque[-1]
'e'
>>> deque[2]
'c'
Parameters:index (int) – index of item
Returns:corresponding item
Raises:IndexError – if index out of range
__gt__(that)

Return True if and only if deque is greater than that.

__iadd__(iterable)

deque.__iadd__(iterable) <==> deque += iterable

Extend back side of deque with items from iterable.

__init__(iterable=(), directory=None)

Initialize deque instance.

If directory is None then temporary directory created. The directory will not be automatically removed.

Parameters:
  • iterable – iterable of items to append to deque
  • directory – deque directory (default None)
__iter__() <==> iter(deque)

Return iterator of deque from front to back.

__le__(that)

Return True if and only if deque is less than or equal to that.

__len__() <==> len(deque)

Return length of deque.

__lt__(that)

Return True if and only if deque is less than that.

__ne__(that)

Return True if and only if deque is not equal to that.

__repr__() <==> repr(deque)

Return string with printable representation of deque.

__reversed__() <==> reversed(deque)

Return iterator of deque from back to front.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque.extend('abcd')
>>> iterator = reversed(deque)
>>> next(iterator)
'd'
>>> list(iterator)
['c', 'b', 'a']
__setitem__(index, value)

deque.__setitem__(index, value) <==> deque[index] = value

Store value in deque at index.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque.extend([None] * 3)
>>> deque[0] = 'a'
>>> deque[1] = 'b'
>>> deque[-1] = 'c'
>>> ''.join(deque)
'abc'
Parameters:
  • index (int) – index of value
  • value – value to store
Raises:

IndexError – if index out of range

append(value)

Add value to back of deque.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque.append('a')
>>> deque.append('b')
>>> deque.append('c')
>>> list(deque)
['a', 'b', 'c']
Parameters:value – value to add to back of deque
appendleft(value)

Add value to front of deque.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque.appendleft('a')
>>> deque.appendleft('b')
>>> deque.appendleft('c')
>>> list(deque)
['c', 'b', 'a']
Parameters:value – value to add to front of deque
clear()

Remove all elements from deque.

count(value)

Return number of occurrences of value in deque.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque += [num for num in range(1, 5) for _ in range(num)]
>>> deque.count(0)
0
>>> deque.count(1)
1
>>> deque.count(4)
4
Parameters:value – value to count in deque
directory

Directory path where deque is stored.

extend(iterable)

Extend back side of deque with values from iterable.

Parameters:iterable – iterable of values
extendleft(iterable)

Extend front side of deque with value from iterable.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque.extendleft('abc')
>>> list(deque)
['c', 'b', 'a']
Parameters:iterable – iterable of values
classmethod fromcache(cache, iterable=())

Initialize deque using cache.

>>> cache = Cache('/tmp/diskcache/index')
>>> _ = cache.clear()
>>> deque = Deque.fromcache(cache, [5, 6, 7, 8])
>>> len(deque)
4
>>> 7 in deque
True
>>> deque.popleft()
5
Parameters:
  • cache (Cache) – cache to use
  • iterable – iterable of items
Returns:

initialized Deque

pop()

Remove and return value at back of deque.

If deque is empty then raise IndexError.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque += 'ab'
>>> deque.pop()
'b'
>>> deque.pop()
'a'
>>> deque.pop()
Traceback (most recent call last):
    ...
IndexError: pop from an empty deque
Raises:IndexError – if deque is empty
popleft()

Remove and return value at front of deque.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque += 'ab'
>>> deque.popleft()
'a'
>>> deque.popleft()
'b'
>>> deque.popleft()
Traceback (most recent call last):
    ...
IndexError: pop from an empty deque
remove(value)

Remove first occurrence of value in deque.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque += 'aab'
>>> deque.remove('a')
>>> list(deque)
['a', 'b']
>>> deque.remove('b')
>>> list(deque)
['a']
>>> deque.remove('c')
Traceback (most recent call last):
    ...
ValueError: deque.remove(value): value not in deque
Parameters:value – value to remove
Raises:ValueError – if value not in deque
reverse()

Reverse deque in place.

rotate(steps=1)

Rotate deque right by steps.

If steps is negative then rotate left.

>>> deque = Deque(directory='/tmp/diskcache/deque')
>>> deque.clear()
>>> deque += range(5)
>>> deque.rotate(2)
>>> list(deque)
[3, 4, 0, 1, 2]
>>> deque.rotate(-1)
>>> list(deque)
[4, 0, 1, 2, 3]
Parameters:steps (int) – number of steps to rotate (default 1)

Index

class diskcache.Index(*args, **kwargs)

Persistent mutable mapping with insertion order iteration.

Items are serialized to disk. Index may be initialized from directory path where items are stored.

Hashing protocol is not used. Keys are looked up by their serialized format. See diskcache.Disk for details.

>>> index = Index('/tmp/diskcache/index')
>>> index
Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> index['a']
1
>>> list(index)
['a', 'b', 'c']
>>> len(index)
3
>>> del index['b']
>>> index.popitem()
('c', 3)
__delitem__(key)

index.__delitem__(key) <==> del index[key]

Delete corresponding item for key from index.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.update({'a': 1, 'b': 2})
>>> del index['a']
>>> del index['b']
>>> len(index)
0
>>> del index['c']
Traceback (most recent call last):
    ...
KeyError: 'c'
Parameters:key – key for item
Raises:KeyError – if key is not found
__eq__(other)

index.__eq__(other) <==> index == other

Compare equality for index and other.

Comparison to another index or ordered dictionary is order-sensitive. Comparison to all other mappings is order-insensitive.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> pairs = [('a', 1), ('b', 2), ('c', 3)]
>>> index.update(pairs)
>>> from collections import OrderedDict
>>> od = OrderedDict(pairs)
>>> index == od
True
>>> index == {'c': 3, 'b': 2, 'a': 1}
True
Parameters:other – other mapping in equality comparison
__getitem__(key)

index.__getitem__(key) <==> index[key]

Return corresponding value for key in index.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.update({'a': 1, 'b': 2})
>>> index['a']
1
>>> index['b']
2
>>> index['c']
Traceback (most recent call last):
    ...
KeyError: 'c'
Parameters:key – key for item
Returns:value for item in index with given key
Raises:KeyError – if key is not found
__init__(*args, **kwargs)

Initialize index in directory and update items.

Optional first argument may be string specifying directory where items are stored. When None or not given, temporary directory is created.

>>> index = Index({'a': 1, 'b': 2, 'c': 3})
>>> len(index)
3
>>> directory = index.directory
>>> inventory = Index(directory, d=4)
>>> inventory['b']
2
>>> len(inventory)
4
__iter__() <==> iter(index)

Return iterator of index keys in insertion order.

__len__() <==> len(index)

Return length of index.

__ne__(other)

index.__ne__(other) <==> index != other

Compare inequality for index and other.

Comparison to another index or ordered dictionary is order-sensitive. Comparison to all other mappings is order-insensitive.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> from collections import OrderedDict
>>> od = OrderedDict([('c', 3), ('b', 2), ('a', 1)])
>>> index != od
True
>>> index != {'a': 1, 'b': 2}
True
Parameters:other – other mapping in inequality comparison
__repr__() <==> repr(index)

Return string with printable representation of index.

__reversed__() <==> reversed(index)

Return iterator of index keys in reversed insertion order.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> iterator = reversed(index)
>>> next(iterator)
'c'
>>> list(iterator)
['b', 'a']
__setitem__(key, value)

index.__setitem__(key, value) <==> index[key] = value

Set key and value item in index.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index['a'] = 1
>>> index[0] = None
>>> len(index)
2
Parameters:
  • key – key for item
  • value – value for item
clear()

Remove all items from index.

directory

Directory path where items are stored.

classmethod fromcache(cache, *args, **kwargs)

Initialize index using cache and update items.

>>> cache = Cache('/tmp/diskcache/index')
>>> _ = cache.clear()
>>> index = Index.fromcache(cache, {'a': 1, 'b': 2, 'c': 3})
>>> len(index)
3
>>> 'b' in index
True
>>> index['c']
3
Parameters:
  • cache (Cache) – cache to use
  • args – mapping or sequence of items
  • kwargs – mapping of items
Returns:

initialized Index

items()

Set-like object providing a view of index items.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.update({'a': 1, 'b': 2, 'c': 3})
>>> items_view = index.items()
>>> ('b', 2) in items_view
True
Returns:items view
keys()

Set-like object providing a view of index keys.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.update({'a': 1, 'b': 2, 'c': 3})
>>> keys_view = index.keys()
>>> 'b' in keys_view
True
Returns:keys view
pop(key, default=ENOVAL)

Remove corresponding item for key from index and return value.

If key is missing then return default. If default is ENOVAL then raise KeyError.

>>> index = Index('/tmp/diskcache/index', {'a': 1, 'b': 2})
>>> index.pop('a')
1
>>> index.pop('b')
2
>>> index.pop('c', default=3)
3
>>> index.pop('d')
Traceback (most recent call last):
    ...
KeyError: 'd'
Parameters:
  • key – key for item
  • default – return value if key is missing (default ENOVAL)
Returns:

value for item if key is found else default

Raises:

KeyError – if key is not found and default is ENOVAL

popitem(last=True)

Remove and return item pair.

Item pairs are returned in last-in-first-out (LIFO) order if last is True else first-in-first-out (FIFO) order. LIFO order imitates a stack and FIFO order imitates a queue.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> index.popitem()
('c', 3)
>>> index.popitem(last=False)
('a', 1)
>>> index.popitem()
('b', 2)
>>> index.popitem()
Traceback (most recent call last):
  ...
KeyError
Parameters:last (bool) – pop last item pair (default True)
Returns:key and value item pair
Raises:KeyError – if index is empty
pull(prefix=None, default=(None, None), side='front')

Pull key and value item pair from side of queue in index.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

If queue is empty, return default.

Defaults to pulling key and value item pairs from front of queue. Set side to ‘back’ to pull from back of queue. Side must be one of ‘front’ or ‘back’.

See also Index.push.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> for letter in 'abc':
...     print(index.push(letter))
500000000000000
500000000000001
500000000000002
>>> key, value = index.pull()
>>> print(key)
500000000000000
>>> value
'a'
>>> _, value = index.pull(side='back')
>>> value
'c'
>>> index.pull(prefix='fruit')
(None, None)
Parameters:
  • prefix (str) – key prefix (default None, key is integer)
  • default – value to return if key is missing (default (None, None))
  • side (str) – either ‘front’ or ‘back’ (default ‘front’)
Returns:

key and value item pair or default if queue is empty

push(value, prefix=None, side='back')

Push value onto side of queue in index identified by prefix.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

Defaults to pushing value on back of queue. Set side to ‘front’ to push value on front of queue. Side must be one of ‘back’ or ‘front’.

See also Index.pull.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> print(index.push('apples'))
500000000000000
>>> print(index.push('beans'))
500000000000001
>>> print(index.push('cherries', side='front'))
499999999999999
>>> index[500000000000001]
'beans'
>>> index.push('dates', prefix='fruit')
'fruit-500000000000000'
Parameters:
  • value – value for item
  • prefix (str) – key prefix (default None, key is integer)
  • side (str) – either ‘back’ or ‘front’ (default ‘back’)
Returns:

key for item in cache

setdefault(key, default=None)

Set and get value for key in index using default.

If key is not in index then set corresponding value to default. If key is in index then ignore default and return existing value.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.setdefault('a', 0)
0
>>> index.setdefault('a', 1)
0
Parameters:
  • key – key for item
  • default – value if key is missing (default None)
Returns:

value for item in index with given key

values()

Set-like object providing a view of index values.

>>> index = Index('/tmp/diskcache/index')
>>> index.clear()
>>> index.update({'a': 1, 'b': 2, 'c': 3})
>>> values_view = index.values()
>>> 2 in values_view
True
Returns:values view