DiskCache API Reference

The DiskCache Tutorial provides a helpful walkthrough of most methods.

Cache

Read the Cache tutorial for example usage.

class diskcache.Cache(directory=None, timeout=60, disk=<class 'diskcache.core.Disk'>, **settings)

Disk and file backed cache.

__contains__(key)

Return True if key matching item is found in cache.

Parameters

key – key matching item

Returns

True if key matching item

__delitem__(key, retry=True)

Delete corresponding item for key from cache.

Raises Timeout error when database timeout occurs and retry is False (default True).

Parameters
  • key – key matching item

  • retry (bool) – retry if database timeout occurs (default True)

Raises
  • KeyError – if key is not found

  • Timeout – if database timeout occurs

__getitem__(key)

Return corresponding value for key from cache.

Parameters

key – key matching item

Returns

corresponding value

Raises

KeyError – if key is not found

__init__(directory=None, timeout=60, disk=<class 'diskcache.core.Disk'>, **settings)

Initialize cache instance.

Parameters
  • directory (str) – cache directory

  • timeout (float) – SQLite connection timeout

  • disk – Disk type or subclass for serialization

  • settings – any of DEFAULT_SETTINGS

__iter__()

Iterate keys in cache including expired items.

__len__()

Count of items in cache including expired items.

__reversed__()

Reverse iterate keys in cache including expired items.

__setitem__(key, value)

Set corresponding value for key in cache.

Parameters
  • key – key for item

  • value – value for item

Returns

corresponding value

Raises

KeyError – if key is not found

add(key, value, expire=None, read=False, tag=None, retry=False)

Add key and value item to cache.

Similar to set, but only add to cache if key not present.

Operation is atomic. Only one concurrent add operation for a given key will succeed.

When read is True, value should be a file-like object opened for reading in binary mode.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • key – key for item

  • value – value for item

  • expire (float) – seconds until the key expires (default None, no expiry)

  • read (bool) – read value as bytes from file (default False)

  • tag (str) – text to associate with key (default None)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

True if item was added

Raises

Timeout – if database timeout occurs

check(fix=False, retry=False)

Check database and file system consistency.

Intended for use in testing and post-mortem error analysis.

While checking the Cache table for consistency, a writer lock is held on the database. The lock blocks other cache clients from writing to the database. For caches with many file references, the lock may be held for a long time. For example, local benchmarking shows that a cache with 1,000 file references takes ~60ms to check.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • fix (bool) – correct inconsistencies

  • retry (bool) – retry if database timeout occurs (default False)

Returns

list of warnings

Raises

Timeout – if database timeout occurs

clear(retry=False)

Remove all items from cache.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters

retry (bool) – retry if database timeout occurs (default False)

Returns

count of rows removed

Raises

Timeout – if database timeout occurs

close()

Close database connection.

create_tag_index()

Create tag index on cache database.

It is better to initialize cache with tag_index=True than use this.

Raises

Timeout – if database timeout occurs

cull(retry=False)

Cull items from cache until volume is less than size limit.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters

retry (bool) – retry if database timeout occurs (default False)

Returns

count of items removed

Raises

Timeout – if database timeout occurs

decr(key, delta=1, default=0, retry=False)

Decrement value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent decrement operations will be counted individually.

Unlike Memcached, negative values are supported. Value may be decremented below zero.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • key – key for item

  • delta (int) – amount to decrement (default 1)

  • default (int) – value if key is missing (default 0)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

new value for item

Raises
  • KeyError – if key is not found and default is None

  • Timeout – if database timeout occurs

delete(key, retry=False)

Delete corresponding item for key from cache.

Missing keys are ignored.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • key – key matching item

  • retry (bool) – retry if database timeout occurs (default False)

Returns

True if item was deleted

Raises

Timeout – if database timeout occurs

property directory

Cache directory.

property disk

Disk used for serialization.

drop_tag_index()

Drop tag index on cache database.

Raises

Timeout – if database timeout occurs

evict(tag, retry=False)

Remove items with matching tag from cache.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • tag (str) – tag identifying items

  • retry (bool) – retry if database timeout occurs (default False)

Returns

count of rows removed

Raises

Timeout – if database timeout occurs

expire(now=None, retry=False)

Remove expired items from cache.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • now (float) – current time (default None, time.time() used)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

count of items removed

Raises

Timeout – if database timeout occurs

get(key, default=None, read=False, expire_time=False, tag=False, retry=False)

Retrieve value from cache. If key is missing, return default.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • key – key for item

  • default – value to return if key is missing (default None)

  • read (bool) – if True, return file handle to value (default False)

  • expire_time (bool) – if True, return expire_time in tuple (default False)

  • tag (bool) – if True, return tag in tuple (default False)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

value for item or default if key not found

Raises

Timeout – if database timeout occurs

incr(key, delta=1, default=0, retry=False)

Increment value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent increment operations will be counted individually.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • key – key for item

  • delta (int) – amount to increment (default 1)

  • default (int) – value if key is missing (default 0)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

new value for item

Raises
  • KeyError – if key is not found and default is None

  • Timeout – if database timeout occurs

iterkeys(reverse=False)

Iterate Cache keys in database sort order.

>>> cache = Cache()
>>> for key in [4, 1, 3, 0, 2]:
...     cache[key] = key
>>> list(cache.iterkeys())
[0, 1, 2, 3, 4]
>>> list(cache.iterkeys(reverse=True))
[4, 3, 2, 1, 0]
Parameters

reverse (bool) – reverse sort order (default False)

Returns

iterator of Cache keys

memoize(name=None, typed=False, expire=None, tag=None)

Memoizing cache decorator.

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

If name is set to None (default), the callable name will be determined automatically.

When expire is set to zero, function results will not be set in the cache. Cache lookups still occur, however. Read Case Study: Landing Page Caching for example usage.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

>>> from diskcache import Cache
>>> cache = Cache()
>>> @cache.memoize(expire=1, tag='fib')
... def fibonacci(number):
...     if number == 0:
...         return 0
...     elif number == 1:
...         return 1
...     else:
...         return fibonacci(number - 1) + fibonacci(number - 2)
>>> print(fibonacci(100))
354224848179261915075

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

>>> key = fibonacci.__cache_key__(100)
>>> print(cache[key])
354224848179261915075

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur. Note the lack of parenthenses after memoize below:

>>> @cache.memoize
... def test():
...     pass
Traceback (most recent call last):
    ...
TypeError: name cannot be callable
Parameters
  • cache – cache to store callable arguments and return values

  • name (str) – name given for callable (default None, automatic)

  • typed (bool) – cache different types separately (default False)

  • expire (float) – seconds until arguments expire (default None, no expiry)

  • tag (str) – text to associate with arguments (default None)

Returns

callable decorator

peek(prefix=None, default=(None, None), side='front', expire_time=False, tag=False, retry=False)

Peek at key and value item pair from side of queue in cache.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

If queue is empty, return default.

Defaults to peeking at key and value item pairs from front of queue. Set side to ‘back’ to pull from back of queue. Side must be one of ‘front’ or ‘back’.

Expired items are deleted from cache. Operation is atomic. Concurrent operations will be serialized.

Raises Timeout error when database timeout occurs and retry is False (default).

See also Cache.pull and Cache.push.

>>> cache = Cache()
>>> for letter in 'abc':
...     print(cache.push(letter))
500000000000000
500000000000001
500000000000002
>>> key, value = cache.peek()
>>> print(key)
500000000000000
>>> value
'a'
>>> key, value = cache.peek(side='back')
>>> print(key)
500000000000002
>>> value
'c'
Parameters
  • prefix (str) – key prefix (default None, key is integer)

  • default – value to return if key is missing (default (None, None))

  • side (str) – either ‘front’ or ‘back’ (default ‘front’)

  • expire_time (bool) – if True, return expire_time in tuple (default False)

  • tag (bool) – if True, return tag in tuple (default False)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

key and value item pair or default if queue is empty

Raises

Timeout – if database timeout occurs

peekitem(last=True, expire_time=False, tag=False, retry=False)

Peek at key and value item pair in cache based on iteration order.

Expired items are deleted from cache. Operation is atomic. Concurrent operations will be serialized.

Raises Timeout error when database timeout occurs and retry is False (default).

>>> cache = Cache()
>>> for num, letter in enumerate('abc'):
...     cache[letter] = num
>>> cache.peekitem()
('c', 2)
>>> cache.peekitem(last=False)
('a', 0)
Parameters
  • last (bool) – last item in iteration order (default True)

  • expire_time (bool) – if True, return expire_time in tuple (default False)

  • tag (bool) – if True, return tag in tuple (default False)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

key and value item pair

Raises
  • KeyError – if cache is empty

  • Timeout – if database timeout occurs

pop(key, default=None, expire_time=False, tag=False, retry=False)

Remove corresponding item for key from cache and return value.

If key is missing, return default.

Operation is atomic. Concurrent operations will be serialized.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • key – key for item

  • default – value to return if key is missing (default None)

  • expire_time (bool) – if True, return expire_time in tuple (default False)

  • tag (bool) – if True, return tag in tuple (default False)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

value for item or default if key not found

Raises

Timeout – if database timeout occurs

pull(prefix=None, default=(None, None), side='front', expire_time=False, tag=False, retry=False)

Pull key and value item pair from side of queue in cache.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

If queue is empty, return default.

Defaults to pulling key and value item pairs from front of queue. Set side to ‘back’ to pull from back of queue. Side must be one of ‘front’ or ‘back’.

Operation is atomic. Concurrent operations will be serialized.

Raises Timeout error when database timeout occurs and retry is False (default).

See also Cache.push and Cache.get.

>>> cache = Cache()
>>> cache.pull()
(None, None)
>>> for letter in 'abc':
...     print(cache.push(letter))
500000000000000
500000000000001
500000000000002
>>> key, value = cache.pull()
>>> print(key)
500000000000000
>>> value
'a'
>>> _, value = cache.pull(side='back')
>>> value
'c'
>>> cache.push(1234, 'userids')
'userids-500000000000000'
>>> _, value = cache.pull('userids')
>>> value
1234
Parameters
  • prefix (str) – key prefix (default None, key is integer)

  • default – value to return if key is missing (default (None, None))

  • side (str) – either ‘front’ or ‘back’ (default ‘front’)

  • expire_time (bool) – if True, return expire_time in tuple (default False)

  • tag (bool) – if True, return tag in tuple (default False)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

key and value item pair or default if queue is empty

Raises

Timeout – if database timeout occurs

push(value, prefix=None, side='back', expire=None, read=False, tag=None, retry=False)

Push value onto side of queue identified by prefix in cache.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

Defaults to pushing value on back of queue. Set side to ‘front’ to push value on front of queue. Side must be one of ‘back’ or ‘front’.

Operation is atomic. Concurrent operations will be serialized.

When read is True, value should be a file-like object opened for reading in binary mode.

Raises Timeout error when database timeout occurs and retry is False (default).

See also Cache.pull.

>>> cache = Cache()
>>> print(cache.push('first value'))
500000000000000
>>> cache.get(500000000000000)
'first value'
>>> print(cache.push('second value'))
500000000000001
>>> print(cache.push('third value', side='front'))
499999999999999
>>> cache.push(1234, prefix='userids')
'userids-500000000000000'
Parameters
  • value – value for item

  • prefix (str) – key prefix (default None, key is integer)

  • side (str) – either ‘back’ or ‘front’ (default ‘back’)

  • expire (float) – seconds until the key expires (default None, no expiry)

  • read (bool) – read value as bytes from file (default False)

  • tag (str) – text to associate with key (default None)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

key for item in cache

Raises

Timeout – if database timeout occurs

read(key, retry=False)

Return file handle value corresponding to key from cache.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • key – key matching item

  • retry (bool) – retry if database timeout occurs (default False)

Returns

file open for reading in binary mode

Raises
  • KeyError – if key is not found

  • Timeout – if database timeout occurs

reset(key, value=ENOVAL, update=True)

Reset key and value item from Settings table.

Use reset to update the value of Cache settings correctly. Cache settings are stored in the Settings table of the SQLite database. If update is False then no attempt is made to update the database.

If value is not given, it is reloaded from the Settings table. Otherwise, the Settings table is updated.

Settings with the disk_ prefix correspond to Disk attributes. Updating the value will change the unprefixed attribute on the associated Disk instance.

Settings with the sqlite_ prefix correspond to SQLite pragmas. Updating the value will execute the corresponding PRAGMA statement.

SQLite PRAGMA statements may be executed before the Settings table exists in the database by setting update to False.

Parameters
  • key (str) – Settings key for item

  • value – value for item (optional)

  • update (bool) – update database Settings table (default True)

Returns

updated value for item

Raises

Timeout – if database timeout occurs

set(key, value, expire=None, read=False, tag=None, retry=False)

Set key and value item in cache.

When read is True, value should be a file-like object opened for reading in binary mode.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • key – key for item

  • value – value for item

  • expire (float) – seconds until item expires (default None, no expiry)

  • read (bool) – read value as bytes from file (default False)

  • tag (str) – text to associate with key (default None)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

True if item was set

Raises

Timeout – if database timeout occurs

stats(enable=True, reset=False)

Return cache statistics hits and misses.

Parameters
  • enable (bool) – enable collecting statistics (default True)

  • reset (bool) – reset hits and misses to 0 (default False)

Returns

(hits, misses)

property timeout

SQLite connection timeout value in seconds.

touch(key, expire=None, retry=False)

Touch key in cache and update expire time.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters
  • key – key for item

  • expire (float) – seconds until item expires (default None, no expiry)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

True if key was touched

Raises

Timeout – if database timeout occurs

transact(retry=False)

Context manager to perform a transaction by locking the cache.

While the cache is locked, no other write operation is permitted. Transactions should therefore be as short as possible. Read and write operations performed in a transaction are atomic. Read operations may occur concurrent to a transaction.

Transactions may be nested and may not be shared between threads.

Raises Timeout error when database timeout occurs and retry is False (default).

>>> cache = Cache()
>>> with cache.transact():  # Atomically increment two keys.
...     _ = cache.incr('total', 123.4)
...     _ = cache.incr('count', 1)
>>> with cache.transact():  # Atomically calculate average.
...     average = cache['total'] / cache['count']
>>> average
123.4
Parameters

retry (bool) – retry if database timeout occurs (default False)

Returns

context manager for use in with statement

Raises

Timeout – if database timeout occurs

volume()

Return estimated total size of cache on disk.

Returns

size in bytes

FanoutCache

Read the FanoutCache tutorial for example usage.

class diskcache.FanoutCache(directory=None, shards=8, timeout=0.01, disk=<class 'diskcache.core.Disk'>, **settings)

Cache that shards keys and values.

__contains__(key)

Return True if key matching item is found in cache.

Parameters

key – key for item

Returns

True if key is found

__delitem__(key)

Delete corresponding item for key from cache.

Calls FanoutCache.delete() internally with retry set to True.

Parameters

key – key for item

Raises

KeyError – if key is not found

__getitem__(key)

Return corresponding value for key from cache.

Calls FanoutCache.get() internally with retry set to True.

Parameters

key – key for item

Returns

value for item

Raises

KeyError – if key is not found

__init__(directory=None, shards=8, timeout=0.01, disk=<class 'diskcache.core.Disk'>, **settings)

Initialize cache instance.

Parameters
  • directory (str) – cache directory

  • shards (int) – number of shards to distribute writes

  • timeout (float) – SQLite connection timeout

  • diskDisk instance for serialization

  • settings – any of DEFAULT_SETTINGS

__iter__()

Iterate keys in cache including expired items.

__len__()

Count of items in cache including expired items.

__reversed__()

Reverse iterate keys in cache including expired items.

__setitem__(key, value)

Set key and value item in cache.

Calls FanoutCache.set() internally with retry set to True.

Parameters
  • key – key for item

  • value – value for item

add(key, value, expire=None, read=False, tag=None, retry=False)

Add key and value item to cache.

Similar to set, but only add to cache if key not present.

This operation is atomic. Only one concurrent add operation for given key from separate threads or processes will succeed.

When read is True, value should be a file-like object opened for reading in binary mode.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters
  • key – key for item

  • value – value for item

  • expire (float) – seconds until the key expires (default None, no expiry)

  • read (bool) – read value as bytes from file (default False)

  • tag (str) – text to associate with key (default None)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

True if item was added

cache(name)

Return Cache with given name in subdirectory.

>>> fanout_cache = FanoutCache()
>>> cache = fanout_cache.cache('test')
>>> cache.set('abc', 123)
True
>>> cache.get('abc')
123
>>> len(cache)
1
>>> cache.delete('abc')
True
Parameters

name (str) – subdirectory name for Cache

Returns

Cache with given name

check(fix=False, retry=False)

Check database and file system consistency.

Intended for use in testing and post-mortem error analysis.

While checking the cache table for consistency, a writer lock is held on the database. The lock blocks other cache clients from writing to the database. For caches with many file references, the lock may be held for a long time. For example, local benchmarking shows that a cache with 1,000 file references takes ~60ms to check.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters
  • fix (bool) – correct inconsistencies

  • retry (bool) – retry if database timeout occurs (default False)

Returns

list of warnings

Raises

Timeout – if database timeout occurs

clear(retry=False)

Remove all items from cache.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters

retry (bool) – retry if database timeout occurs (default False)

Returns

count of items removed

close()

Close database connection.

create_tag_index()

Create tag index on cache database.

Better to initialize cache with tag_index=True than use this.

Raises

Timeout – if database timeout occurs

cull(retry=False)

Cull items from cache until volume is less than size limit.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters

retry (bool) – retry if database timeout occurs (default False)

Returns

count of items removed

decr(key, delta=1, default=0, retry=False)

Decrement value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent decrement operations will be counted individually.

Unlike Memcached, negative values are supported. Value may be decremented below zero.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters
  • key – key for item

  • delta (int) – amount to decrement (default 1)

  • default (int) – value if key is missing (default 0)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

new value for item on success else None

Raises

KeyError – if key is not found and default is None

delete(key, retry=False)

Delete corresponding item for key from cache.

Missing keys are ignored.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters
  • key – key for item

  • retry (bool) – retry if database timeout occurs (default False)

Returns

True if item was deleted

deque(name)

Return Deque with given name in subdirectory.

>>> cache = FanoutCache()
>>> deque = cache.deque('test')
>>> deque.extend('abc')
>>> deque.popleft()
'a'
>>> deque.pop()
'c'
>>> len(deque)
1
Parameters

name (str) – subdirectory name for Deque

Returns

Deque with given name

property directory

Cache directory.

drop_tag_index()

Drop tag index on cache database.

Raises

Timeout – if database timeout occurs

evict(tag, retry=False)

Remove items with matching tag from cache.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters
  • tag (str) – tag identifying items

  • retry (bool) – retry if database timeout occurs (default False)

Returns

count of items removed

expire(retry=False)

Remove expired items from cache.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters

retry (bool) – retry if database timeout occurs (default False)

Returns

count of items removed

get(key, default=None, read=False, expire_time=False, tag=False, retry=False)

Retrieve value from cache. If key is missing, return default.

If database timeout occurs then returns default unless retry is set to True (default False).

Parameters
  • key – key for item

  • default – return value if key is missing (default None)

  • read (bool) – if True, return file handle to value (default False)

  • expire_time (float) – if True, return expire_time in tuple (default False)

  • tag – if True, return tag in tuple (default False)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

value for item if key is found else default

incr(key, delta=1, default=0, retry=False)

Increment value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent increment operations will be counted individually.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters
  • key – key for item

  • delta (int) – amount to increment (default 1)

  • default (int) – value if key is missing (default 0)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

new value for item on success else None

Raises

KeyError – if key is not found and default is None

index(name)

Return Index with given name in subdirectory.

>>> cache = FanoutCache()
>>> index = cache.index('test')
>>> index['abc'] = 123
>>> index['def'] = 456
>>> index['ghi'] = 789
>>> index.popitem()
('ghi', 789)
>>> del index['abc']
>>> len(index)
1
>>> index['def']
456
Parameters

name (str) – subdirectory name for Index

Returns

Index with given name

memoize(name=None, typed=False, expire=None, tag=None)

Memoizing cache decorator.

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

If name is set to None (default), the callable name will be determined automatically.

When expire is set to zero, function results will not be set in the cache. Cache lookups still occur, however. Read Case Study: Landing Page Caching for example usage.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

>>> from diskcache import Cache
>>> cache = Cache()
>>> @cache.memoize(expire=1, tag='fib')
... def fibonacci(number):
...     if number == 0:
...         return 0
...     elif number == 1:
...         return 1
...     else:
...         return fibonacci(number - 1) + fibonacci(number - 2)
>>> print(fibonacci(100))
354224848179261915075

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

>>> key = fibonacci.__cache_key__(100)
>>> print(cache[key])
354224848179261915075

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur. Note the lack of parenthenses after memoize below:

>>> @cache.memoize
... def test():
...     pass
Traceback (most recent call last):
    ...
TypeError: name cannot be callable
Parameters
  • cache – cache to store callable arguments and return values

  • name (str) – name given for callable (default None, automatic)

  • typed (bool) – cache different types separately (default False)

  • expire (float) – seconds until arguments expire (default None, no expiry)

  • tag (str) – text to associate with arguments (default None)

Returns

callable decorator

pop(key, default=None, expire_time=False, tag=False, retry=False)

Remove corresponding item for key from cache and return value.

If key is missing, return default.

Operation is atomic. Concurrent operations will be serialized.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters
  • key – key for item

  • default – return value if key is missing (default None)

  • expire_time (float) – if True, return expire_time in tuple (default False)

  • tag – if True, return tag in tuple (default False)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

value for item if key is found else default

read(key)

Return file handle corresponding to key from cache.

Parameters

key – key for item

Returns

file open for reading in binary mode

Raises

KeyError – if key is not found

reset(key, value=ENOVAL)

Reset key and value item from Settings table.

If value is not given, it is reloaded from the Settings table. Otherwise, the Settings table is updated.

Settings attributes on cache objects are lazy-loaded and read-only. Use reset to update the value.

Settings with the sqlite_ prefix correspond to SQLite pragmas. Updating the value will execute the corresponding PRAGMA statement.

Parameters
  • key (str) – Settings key for item

  • value – value for item (optional)

Returns

updated value for item

set(key, value, expire=None, read=False, tag=None, retry=False)

Set key and value item in cache.

When read is True, value should be a file-like object opened for reading in binary mode.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters
  • key – key for item

  • value – value for item

  • expire (float) – seconds until the key expires (default None, no expiry)

  • read (bool) – read value as raw bytes from file (default False)

  • tag (str) – text to associate with key (default None)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

True if item was set

stats(enable=True, reset=False)

Return cache statistics hits and misses.

Parameters
  • enable (bool) – enable collecting statistics (default True)

  • reset (bool) – reset hits and misses to 0 (default False)

Returns

(hits, misses)

touch(key, expire=None, retry=False)

Touch key in cache and update expire time.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters
  • key – key for item

  • expire (float) – seconds until the key expires (default None, no expiry)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

True if key was touched

volume()

Return estimated total size of cache on disk.

Returns

size in bytes

DjangoCache

Read the DjangoCache tutorial for example usage.

class diskcache.DjangoCache(directory, params)

Django-compatible disk and file backed cache.

__init__(directory, params)

Initialize DjangoCache instance.

Parameters
  • directory (str) – cache directory

  • params (dict) – cache parameters

add(key, value, timeout=<object object>, version=None, read=False, tag=None, retry=True)

Set a value in the cache if the key does not already exist. If timeout is given, that timeout will be used for the key; otherwise the default cache timeout will be used.

Return True if the value was stored, False otherwise.

Parameters
  • key – key for item

  • value – value for item

  • timeout (float) – seconds until the item expires (default 300 seconds)

  • version (int) – key version number (default None, cache parameter)

  • read (bool) – read value as bytes from file (default False)

  • tag (str) – text to associate with key (default None)

  • retry (bool) – retry if database timeout occurs (default True)

Returns

True if item was added

cache(name)

Return Cache with given name in subdirectory.

Parameters

name (str) – subdirectory name for Cache

Returns

Cache with given name

clear()

Remove all values from the cache at once.

close(**kwargs)

Close the cache connection.

create_tag_index()

Create tag index on cache database.

Better to initialize cache with tag_index=True than use this.

Raises

Timeout – if database timeout occurs

cull()

Cull items from cache until volume is less than size limit.

Returns

count of items removed

decr(key, delta=1, version=None, default=None, retry=True)

Decrement value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent decrement operations will be counted individually.

Unlike Memcached, negative values are supported. Value may be decremented below zero.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters
  • key – key for item

  • delta (int) – amount to decrement (default 1)

  • version (int) – key version number (default None, cache parameter)

  • default (int) – value if key is missing (default None)

  • retry (bool) – retry if database timeout occurs (default True)

Returns

new value for item on success else None

Raises

ValueError – if key is not found and default is None

delete(key, version=None, retry=True)

Delete a key from the cache, failing silently.

Parameters
  • key – key for item

  • version (int) – key version number (default None, cache parameter)

  • retry (bool) – retry if database timeout occurs (default True)

Returns

True if item was deleted

deque(name)

Return Deque with given name in subdirectory.

Parameters

name (str) – subdirectory name for Deque

Returns

Deque with given name

property directory

Cache directory.

drop_tag_index()

Drop tag index on cache database.

Raises

Timeout – if database timeout occurs

evict(tag)

Remove items with matching tag from cache.

Parameters

tag (str) – tag identifying items

Returns

count of items removed

expire()

Remove expired items from cache.

Returns

count of items removed

get(key, default=None, version=None, read=False, expire_time=False, tag=False, retry=False)

Fetch a given key from the cache. If the key does not exist, return default, which itself defaults to None.

Parameters
  • key – key for item

  • default – return value if key is missing (default None)

  • version (int) – key version number (default None, cache parameter)

  • read (bool) – if True, return file handle to value (default False)

  • expire_time (float) – if True, return expire_time in tuple (default False)

  • tag – if True, return tag in tuple (default False)

  • retry (bool) – retry if database timeout occurs (default False)

Returns

value for item if key is found else default

get_backend_timeout(timeout=<object object>)

Return seconds to expiration.

Parameters

timeout (float) – seconds until the item expires (default 300 seconds)

has_key(key, version=None)

Returns True if the key is in the cache and has not expired.

Parameters
  • key – key for item

  • version (int) – key version number (default None, cache parameter)

Returns

True if key is found

incr(key, delta=1, version=None, default=None, retry=True)

Increment value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent increment operations will be counted individually.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters
  • key – key for item

  • delta (int) – amount to increment (default 1)

  • version (int) – key version number (default None, cache parameter)

  • default (int) – value if key is missing (default None)

  • retry (bool) – retry if database timeout occurs (default True)

Returns

new value for item on success else None

Raises

ValueError – if key is not found and default is None

index(name)

Return Index with given name in subdirectory.

Parameters

name (str) – subdirectory name for Index

Returns

Index with given name

memoize(name=None, timeout=<object object>, version=None, typed=False, tag=None)

Memoizing cache decorator.

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

If name is set to None (default), the callable name will be determined automatically.

When timeout is set to zero, function results will not be set in the cache. Cache lookups still occur, however. Read Case Study: Landing Page Caching for example usage.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur.

Parameters
  • name (str) – name given for callable (default None, automatic)

  • timeout (float) – seconds until the item expires (default 300 seconds)

  • version (int) – key version number (default None, cache parameter)

  • typed (bool) – cache different types separately (default False)

  • tag (str) – text to associate with arguments (default None)

Returns

callable decorator

pop(key, default=None, version=None, expire_time=False, tag=False, retry=True)

Remove corresponding item for key from cache and return value.

If key is missing, return default.

Operation is atomic. Concurrent operations will be serialized.

Parameters
  • key – key for item

  • default – return value if key is missing (default None)

  • version (int) – key version number (default None, cache parameter)

  • expire_time (float) – if True, return expire_time in tuple (default False)

  • tag – if True, return tag in tuple (default False)

  • retry (bool) – retry if database timeout occurs (default True)

Returns

value for item if key is found else default

read(key, version=None)

Return file handle corresponding to key from Cache.

Parameters
  • key – Python key to retrieve

  • version (int) – key version number (default None, cache parameter)

Returns

file open for reading in binary mode

Raises

KeyError – if key is not found

set(key, value, timeout=<object object>, version=None, read=False, tag=None, retry=True)

Set a value in the cache. If timeout is given, that timeout will be used for the key; otherwise the default cache timeout will be used.

Parameters
  • key – key for item

  • value – value for item

  • timeout (float) – seconds until the item expires (default 300 seconds)

  • version (int) – key version number (default None, cache parameter)

  • read (bool) – read value as bytes from file (default False)

  • tag (str) – text to associate with key (default None)

  • retry (bool) – retry if database timeout occurs (default True)

Returns

True if item was set

stats(enable=True, reset=False)

Return cache statistics hits and misses.

Parameters
  • enable (bool) – enable collecting statistics (default True)

  • reset (bool) – reset hits and misses to 0 (default False)

Returns

(hits, misses)

touch(key, timeout=<object object>, version=None, retry=True)

Touch a key in the cache. If timeout is given, that timeout will be used for the key; otherwise the default cache timeout will be used.

Parameters
  • key – key for item

  • timeout (float) – seconds until the item expires (default 300 seconds)

  • version (int) – key version number (default None, cache parameter)

  • retry (bool) – retry if database timeout occurs (default True)

Returns

True if key was touched

Deque

class diskcache.Deque(iterable=(), directory=None)

Persistent sequence with double-ended queue semantics.

Double-ended queue is an ordered collection with optimized access at its endpoints.

Items are serialized to disk. Deque may be initialized from directory path where items are stored.

>>> deque = Deque()
>>> deque += range(5)
>>> list(deque)
[0, 1, 2, 3, 4]
>>> for value in range(5):
...     deque.appendleft(-value)
>>> len(deque)
10
>>> list(deque)
[-4, -3, -2, -1, 0, 0, 1, 2, 3, 4]
>>> deque.pop()
4
>>> deque.popleft()
-4
>>> deque.reverse()
>>> list(deque)
[3, 2, 1, 0, 0, -1, -2, -3]
__delitem__(index)

deque.__delitem__(index) <==> del deque[index]

Delete item in deque at index.

>>> deque = Deque()
>>> deque.extend([None] * 3)
>>> del deque[0]
>>> del deque[1]
>>> del deque[-1]
>>> len(deque)
0
Parameters

index (int) – index of item

Raises

IndexError – if index out of range

__eq__(that)

Return True if and only if deque is equal to that.

__ge__(that)

Return True if and only if deque is greater than or equal to that.

__getitem__(index)

deque.__getitem__(index) <==> deque[index]

Return corresponding item for index in deque.

See also Deque.peekleft and Deque.peek for indexing deque at index 0 or -1.

>>> deque = Deque()
>>> deque.extend('abcde')
>>> deque[1]
'b'
>>> deque[-2]
'd'
Parameters

index (int) – index of item

Returns

corresponding item

Raises

IndexError – if index out of range

__gt__(that)

Return True if and only if deque is greater than that.

__iadd__(iterable)

deque.__iadd__(iterable) <==> deque += iterable

Extend back side of deque with items from iterable.

Parameters

iterable – iterable of items to append to deque

Returns

deque with added items

__init__(iterable=(), directory=None)

Initialize deque instance.

If directory is None then temporary directory created. The directory will not be automatically removed.

Parameters
  • iterable – iterable of items to append to deque

  • directory – deque directory (default None)

__iter__() <==> iter(deque)

Return iterator of deque from front to back.

__le__(that)

Return True if and only if deque is less than or equal to that.

__len__() <==> len(deque)

Return length of deque.

__lt__(that)

Return True if and only if deque is less than that.

__ne__(that)

Return True if and only if deque is not equal to that.

__repr__() <==> repr(deque)

Return string with printable representation of deque.

__reversed__() <==> reversed(deque)

Return iterator of deque from back to front.

>>> deque = Deque()
>>> deque.extend('abcd')
>>> iterator = reversed(deque)
>>> next(iterator)
'd'
>>> list(iterator)
['c', 'b', 'a']
__setitem__(index, value)

deque.__setitem__(index, value) <==> deque[index] = value

Store value in deque at index.

>>> deque = Deque()
>>> deque.extend([None] * 3)
>>> deque[0] = 'a'
>>> deque[1] = 'b'
>>> deque[-1] = 'c'
>>> ''.join(deque)
'abc'
Parameters
  • index (int) – index of value

  • value – value to store

Raises

IndexError – if index out of range

append(value)

Add value to back of deque.

>>> deque = Deque()
>>> deque.append('a')
>>> deque.append('b')
>>> deque.append('c')
>>> list(deque)
['a', 'b', 'c']
Parameters

value – value to add to back of deque

appendleft(value)

Add value to front of deque.

>>> deque = Deque()
>>> deque.appendleft('a')
>>> deque.appendleft('b')
>>> deque.appendleft('c')
>>> list(deque)
['c', 'b', 'a']
Parameters

value – value to add to front of deque

property cache

Cache used by deque.

clear()

Remove all elements from deque.

>>> deque = Deque('abc')
>>> len(deque)
3
>>> deque.clear()
>>> list(deque)
[]
count(value)

Return number of occurrences of value in deque.

>>> deque = Deque()
>>> deque += [num for num in range(1, 5) for _ in range(num)]
>>> deque.count(0)
0
>>> deque.count(1)
1
>>> deque.count(4)
4
Parameters

value – value to count in deque

Returns

count of items equal to value in deque

property directory

Directory path where deque is stored.

extend(iterable)

Extend back side of deque with values from iterable.

Parameters

iterable – iterable of values

extendleft(iterable)

Extend front side of deque with value from iterable.

>>> deque = Deque()
>>> deque.extendleft('abc')
>>> list(deque)
['c', 'b', 'a']
Parameters

iterable – iterable of values

classmethod fromcache(cache, iterable=())

Initialize deque using cache.

>>> cache = Cache()
>>> deque = Deque.fromcache(cache, [5, 6, 7, 8])
>>> deque.cache is cache
True
>>> len(deque)
4
>>> 7 in deque
True
>>> deque.popleft()
5
Parameters
  • cache (Cache) – cache to use

  • iterable – iterable of items

Returns

initialized Deque

peek()

Peek at value at back of deque.

Faster than indexing deque at -1.

If deque is empty then raise IndexError.

>>> deque = Deque()
>>> deque.peek()
Traceback (most recent call last):
    ...
IndexError: peek from an empty deque
>>> deque += 'abc'
>>> deque.peek()
'c'
Returns

value at back of deque

Raises

IndexError – if deque is empty

peekleft()

Peek at value at back of deque.

Faster than indexing deque at 0.

If deque is empty then raise IndexError.

>>> deque = Deque()
>>> deque.peekleft()
Traceback (most recent call last):
    ...
IndexError: peek from an empty deque
>>> deque += 'abc'
>>> deque.peekleft()
'a'
Returns

value at front of deque

Raises

IndexError – if deque is empty

pop()

Remove and return value at back of deque.

If deque is empty then raise IndexError.

>>> deque = Deque()
>>> deque += 'ab'
>>> deque.pop()
'b'
>>> deque.pop()
'a'
>>> deque.pop()
Traceback (most recent call last):
    ...
IndexError: pop from an empty deque
Returns

value at back of deque

Raises

IndexError – if deque is empty

popleft()

Remove and return value at front of deque.

>>> deque = Deque()
>>> deque += 'ab'
>>> deque.popleft()
'a'
>>> deque.popleft()
'b'
>>> deque.popleft()
Traceback (most recent call last):
    ...
IndexError: pop from an empty deque
Returns

value at front of deque

Raises

IndexError – if deque is empty

remove(value)

Remove first occurrence of value in deque.

>>> deque = Deque()
>>> deque += 'aab'
>>> deque.remove('a')
>>> list(deque)
['a', 'b']
>>> deque.remove('b')
>>> list(deque)
['a']
>>> deque.remove('c')
Traceback (most recent call last):
    ...
ValueError: deque.remove(value): value not in deque
Parameters

value – value to remove

Raises

ValueError – if value not in deque

reverse()

Reverse deque in place.

>>> deque = Deque()
>>> deque += 'abc'
>>> deque.reverse()
>>> list(deque)
['c', 'b', 'a']
rotate(steps=1)

Rotate deque right by steps.

If steps is negative then rotate left.

>>> deque = Deque()
>>> deque += range(5)
>>> deque.rotate(2)
>>> list(deque)
[3, 4, 0, 1, 2]
>>> deque.rotate(-1)
>>> list(deque)
[4, 0, 1, 2, 3]
Parameters

steps (int) – number of steps to rotate (default 1)

transact()

Context manager to perform a transaction by locking the deque.

While the deque is locked, no other write operation is permitted. Transactions should therefore be as short as possible. Read and write operations performed in a transaction are atomic. Read operations may occur concurrent to a transaction.

Transactions may be nested and may not be shared between threads.

>>> from diskcache import Deque
>>> deque = Deque()
>>> deque += range(5)
>>> with deque.transact():  # Atomically rotate elements.
...     value = deque.pop()
...     deque.appendleft(value)
>>> list(deque)
[4, 0, 1, 2, 3]
Returns

context manager for use in with statement

Index

class diskcache.Index(*args, **kwargs)

Persistent mutable mapping with insertion order iteration.

Items are serialized to disk. Index may be initialized from directory path where items are stored.

Hashing protocol is not used. Keys are looked up by their serialized format. See diskcache.Disk for details.

>>> index = Index()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> index['a']
1
>>> list(index)
['a', 'b', 'c']
>>> len(index)
3
>>> del index['b']
>>> index.popitem()
('c', 3)
__delitem__(key)

index.__delitem__(key) <==> del index[key]

Delete corresponding item for key from index.

>>> index = Index()
>>> index.update({'a': 1, 'b': 2})
>>> del index['a']
>>> del index['b']
>>> len(index)
0
>>> del index['c']
Traceback (most recent call last):
    ...
KeyError: 'c'
Parameters

key – key for item

Raises

KeyError – if key is not found

__eq__(other)

index.__eq__(other) <==> index == other

Compare equality for index and other.

Comparison to another index or ordered dictionary is order-sensitive. Comparison to all other mappings is order-insensitive.

>>> index = Index()
>>> pairs = [('a', 1), ('b', 2), ('c', 3)]
>>> index.update(pairs)
>>> from collections import OrderedDict
>>> od = OrderedDict(pairs)
>>> index == od
True
>>> index == {'c': 3, 'b': 2, 'a': 1}
True
Parameters

other – other mapping in equality comparison

Returns

True if index equals other

__getitem__(key)

index.__getitem__(key) <==> index[key]

Return corresponding value for key in index.

>>> index = Index()
>>> index.update({'a': 1, 'b': 2})
>>> index['a']
1
>>> index['b']
2
>>> index['c']
Traceback (most recent call last):
    ...
KeyError: 'c'
Parameters

key – key for item

Returns

value for item in index with given key

Raises

KeyError – if key is not found

__init__(*args, **kwargs)

Initialize index in directory and update items.

Optional first argument may be string specifying directory where items are stored. When None or not given, temporary directory is created.

>>> index = Index({'a': 1, 'b': 2, 'c': 3})
>>> len(index)
3
>>> directory = index.directory
>>> inventory = Index(directory, d=4)
>>> inventory['b']
2
>>> len(inventory)
4
__iter__() <==> iter(index)

Return iterator of index keys in insertion order.

__len__() <==> len(index)

Return length of index.

__ne__(other)

index.__ne__(other) <==> index != other

Compare inequality for index and other.

Comparison to another index or ordered dictionary is order-sensitive. Comparison to all other mappings is order-insensitive.

>>> index = Index()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> from collections import OrderedDict
>>> od = OrderedDict([('c', 3), ('b', 2), ('a', 1)])
>>> index != od
True
>>> index != {'a': 1, 'b': 2}
True
Parameters

other – other mapping in inequality comparison

Returns

True if index does not equal other

__repr__() <==> repr(index)

Return string with printable representation of index.

__reversed__() <==> reversed(index)

Return iterator of index keys in reversed insertion order.

>>> index = Index()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> iterator = reversed(index)
>>> next(iterator)
'c'
>>> list(iterator)
['b', 'a']
__setitem__(key, value)

index.__setitem__(key, value) <==> index[key] = value

Set key and value item in index.

>>> index = Index()
>>> index['a'] = 1
>>> index[0] = None
>>> len(index)
2
Parameters
  • key – key for item

  • value – value for item

property cache

Cache used by index.

clear()

Remove all items from index.

>>> index = Index({'a': 0, 'b': 1, 'c': 2})
>>> len(index)
3
>>> index.clear()
>>> dict(index)
{}
property directory

Directory path where items are stored.

classmethod fromcache(cache, *args, **kwargs)

Initialize index using cache and update items.

>>> cache = Cache()
>>> index = Index.fromcache(cache, {'a': 1, 'b': 2, 'c': 3})
>>> index.cache is cache
True
>>> len(index)
3
>>> 'b' in index
True
>>> index['c']
3
Parameters
  • cache (Cache) – cache to use

  • args – mapping or sequence of items

  • kwargs – mapping of items

Returns

initialized Index

items()

Set-like object providing a view of index items.

>>> index = Index()
>>> index.update({'a': 1, 'b': 2, 'c': 3})
>>> items_view = index.items()
>>> ('b', 2) in items_view
True
Returns

items view

keys()

Set-like object providing a view of index keys.

>>> index = Index()
>>> index.update({'a': 1, 'b': 2, 'c': 3})
>>> keys_view = index.keys()
>>> 'b' in keys_view
True
Returns

keys view

memoize(name=None, typed=False)

Memoizing cache decorator.

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

If name is set to None (default), the callable name will be determined automatically.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

>>> from diskcache import Index
>>> mapping = Index()
>>> @mapping.memoize()
... def fibonacci(number):
...     if number == 0:
...         return 0
...     elif number == 1:
...         return 1
...     else:
...         return fibonacci(number - 1) + fibonacci(number - 2)
>>> print(fibonacci(100))
354224848179261915075

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

>>> key = fibonacci.__cache_key__(100)
>>> print(mapping[key])
354224848179261915075

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur. Note the lack of parenthenses after memoize below:

>>> @mapping.memoize
... def test():
...     pass
Traceback (most recent call last):
    ...
TypeError: name cannot be callable
Parameters
  • name (str) – name given for callable (default None, automatic)

  • typed (bool) – cache different types separately (default False)

Returns

callable decorator

peekitem(last=True)

Peek at key and value item pair in index based on iteration order.

>>> index = Index()
>>> for num, letter in enumerate('xyz'):
...     index[letter] = num
>>> index.peekitem()
('z', 2)
>>> index.peekitem(last=False)
('x', 0)
Parameters

last (bool) – last item in iteration order (default True)

Returns

key and value item pair

Raises

KeyError – if cache is empty

pop(key, default=ENOVAL)

Remove corresponding item for key from index and return value.

If key is missing then return default. If default is ENOVAL then raise KeyError.

>>> index = Index({'a': 1, 'b': 2})
>>> index.pop('a')
1
>>> index.pop('b')
2
>>> index.pop('c', default=3)
3
>>> index.pop('d')
Traceback (most recent call last):
    ...
KeyError: 'd'
Parameters
  • key – key for item

  • default – return value if key is missing (default ENOVAL)

Returns

value for item if key is found else default

Raises

KeyError – if key is not found and default is ENOVAL

popitem(last=True)

Remove and return item pair.

Item pairs are returned in last-in-first-out (LIFO) order if last is True else first-in-first-out (FIFO) order. LIFO order imitates a stack and FIFO order imitates a queue.

>>> index = Index()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> index.popitem()
('c', 3)
>>> index.popitem(last=False)
('a', 1)
>>> index.popitem()
('b', 2)
>>> index.popitem()
Traceback (most recent call last):
  ...
KeyError: 'dictionary is empty'
Parameters

last (bool) – pop last item pair (default True)

Returns

key and value item pair

Raises

KeyError – if index is empty

pull(prefix=None, default=(None, None), side='front')

Pull key and value item pair from side of queue in index.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

If queue is empty, return default.

Defaults to pulling key and value item pairs from front of queue. Set side to ‘back’ to pull from back of queue. Side must be one of ‘front’ or ‘back’.

See also Index.push.

>>> index = Index()
>>> for letter in 'abc':
...     print(index.push(letter))
500000000000000
500000000000001
500000000000002
>>> key, value = index.pull()
>>> print(key)
500000000000000
>>> value
'a'
>>> _, value = index.pull(side='back')
>>> value
'c'
>>> index.pull(prefix='fruit')
(None, None)
Parameters
  • prefix (str) – key prefix (default None, key is integer)

  • default – value to return if key is missing (default (None, None))

  • side (str) – either ‘front’ or ‘back’ (default ‘front’)

Returns

key and value item pair or default if queue is empty

push(value, prefix=None, side='back')

Push value onto side of queue in index identified by prefix.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

Defaults to pushing value on back of queue. Set side to ‘front’ to push value on front of queue. Side must be one of ‘back’ or ‘front’.

See also Index.pull.

>>> index = Index()
>>> print(index.push('apples'))
500000000000000
>>> print(index.push('beans'))
500000000000001
>>> print(index.push('cherries', side='front'))
499999999999999
>>> index[500000000000001]
'beans'
>>> index.push('dates', prefix='fruit')
'fruit-500000000000000'
Parameters
  • value – value for item

  • prefix (str) – key prefix (default None, key is integer)

  • side (str) – either ‘back’ or ‘front’ (default ‘back’)

Returns

key for item in cache

setdefault(key, default=None)

Set and get value for key in index using default.

If key is not in index then set corresponding value to default. If key is in index then ignore default and return existing value.

>>> index = Index()
>>> index.setdefault('a', 0)
0
>>> index.setdefault('a', 1)
0
Parameters
  • key – key for item

  • default – value if key is missing (default None)

Returns

value for item in index with given key

transact()

Context manager to perform a transaction by locking the index.

While the index is locked, no other write operation is permitted. Transactions should therefore be as short as possible. Read and write operations performed in a transaction are atomic. Read operations may occur concurrent to a transaction.

Transactions may be nested and may not be shared between threads.

>>> from diskcache import Index
>>> mapping = Index()
>>> with mapping.transact():  # Atomically increment two keys.
...     mapping['total'] = mapping.get('total', 0) + 123.4
...     mapping['count'] = mapping.get('count', 0) + 1
>>> with mapping.transact():  # Atomically calculate average.
...     average = mapping['total'] / mapping['count']
>>> average
123.4
Returns

context manager for use in with statement

values()

Set-like object providing a view of index values.

>>> index = Index()
>>> index.update({'a': 1, 'b': 2, 'c': 3})
>>> values_view = index.values()
>>> 2 in values_view
True
Returns

values view

Recipes

class diskcache.Averager(cache, key, expire=None, tag=None)

Recipe for calculating a running average.

Sometimes known as “online statistics,” the running average maintains the total and count. The average can then be calculated at any time.

>>> import diskcache
>>> cache = diskcache.FanoutCache()
>>> ave = Averager(cache, 'latency')
>>> ave.add(0.080)
>>> ave.add(0.120)
>>> ave.get()
0.1
>>> ave.add(0.160)
>>> ave.pop()
0.12
>>> print(ave.get())
None
add(value)

Add value to average.

get()

Get current average or return None if count equals zero.

pop()

Return current average and delete key.

class diskcache.Lock(cache, key, expire=None, tag=None)

Recipe for cross-process and cross-thread lock.

>>> import diskcache
>>> cache = diskcache.Cache()
>>> lock = Lock(cache, 'report-123')
>>> lock.acquire()
>>> lock.release()
>>> with lock:
...     pass
acquire()

Acquire lock using spin-lock algorithm.

release()

Release lock by deleting key.

class diskcache.RLock(cache, key, expire=None, tag=None)

Recipe for cross-process and cross-thread re-entrant lock.

>>> import diskcache
>>> cache = diskcache.Cache()
>>> rlock = RLock(cache, 'user-123')
>>> rlock.acquire()
>>> rlock.acquire()
>>> rlock.release()
>>> with rlock:
...     pass
>>> rlock.release()
>>> rlock.release()
Traceback (most recent call last):
  ...
AssertionError: cannot release un-acquired lock
acquire()

Acquire lock by incrementing count using spin-lock algorithm.

release()

Release lock by decrementing count.

class diskcache.BoundedSemaphore(cache, key, value=1, expire=None, tag=None)

Recipe for cross-process and cross-thread bounded semaphore.

>>> import diskcache
>>> cache = diskcache.Cache()
>>> semaphore = BoundedSemaphore(cache, 'max-cons', value=2)
>>> semaphore.acquire()
>>> semaphore.acquire()
>>> semaphore.release()
>>> with semaphore:
...     pass
>>> semaphore.release()
>>> semaphore.release()
Traceback (most recent call last):
  ...
AssertionError: cannot release un-acquired semaphore
acquire()

Acquire semaphore by decrementing value using spin-lock algorithm.

release()

Release semaphore by incrementing value.

@diskcache.throttle(cache, count, seconds, name=None, expire=None, tag=None, time_func=<built-in function time>, sleep_func=<built-in function sleep>)

Decorator to throttle calls to function.

>>> import diskcache, time
>>> cache = diskcache.Cache()
>>> count = 0
>>> @throttle(cache, 2, 1)  # 2 calls per 1 second
... def increment():
...     global count
...     count += 1
>>> start = time.time()
>>> while (time.time() - start) <= 2:
...     increment()
>>> count in (6, 7)  # 6 or 7 calls depending on CPU load
True
@diskcache.barrier(cache, lock_factory, name=None, expire=None, tag=None)

Barrier to calling decorated function.

Supports different kinds of locks: Lock, RLock, BoundedSemaphore.

>>> import diskcache, time
>>> cache = diskcache.Cache()
>>> @barrier(cache, Lock)
... def work(num):
...     print('worker started')
...     time.sleep(1)
...     print('worker finished')
>>> import multiprocessing.pool
>>> pool = multiprocessing.pool.ThreadPool(2)
>>> _ = pool.map(work, range(2))
worker started
worker finished
worker started
worker finished
>>> pool.terminate()
@diskcache.memoize_stampede(cache, expire, name=None, typed=False, tag=None, beta=1)

Memoizing cache decorator with cache stampede protection.

Cache stampedes are a type of system overload that can occur when parallel computing systems using memoization come under heavy load. This behaviour is sometimes also called dog-piling, cache miss storm, cache choking, or the thundering herd problem.

The memoization decorator implements cache stampede protection through early recomputation. Early recomputation of function results will occur probabilistically before expiration in a background thread of execution. Early probabilistic recomputation is based on research by Vattani, A.; Chierichetti, F.; Lowenstein, K. (2015), Optimal Probabilistic Cache Stampede Prevention, VLDB, pp. 886-897, ISSN 2150-8097

If name is set to None (default), the callable name will be determined automatically.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

>>> from diskcache import Cache
>>> cache = Cache()
>>> @memoize_stampede(cache, expire=1)
... def fib(number):
...     if number == 0:
...         return 0
...     elif number == 1:
...         return 1
...     else:
...         return fib(number - 1) + fib(number - 2)
>>> print(fib(100))
354224848179261915075

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

>>> key = fib.__cache_key__(100)
>>> del cache[key]

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur.

Parameters
  • cache – cache to store callable arguments and return values

  • expire (float) – seconds until arguments expire

  • name (str) – name given for callable (default None, automatic)

  • typed (bool) – cache different types separately (default False)

  • tag (str) – text to associate with arguments (default None)

Returns

callable decorator

Constants

Read the Settings tutorial for details.

diskcache.DEFAULT_SETTINGS
  • statistics (int) default 0 - disabled when 0, enabled when 1.

  • tag_index (int) default 0 - disabled when 0, enabled when 1.

  • eviction_policy (str) default “least-recently-stored” - any of the keys in EVICTION_POLICY as described below.

  • size_limit (int, in bytes) default one gigabyte - approximate size limit of cache.

  • cull_limit (int) default ten - maximum number of items culled during set or add operations.

  • sqlite_auto_vacuum (int) default 1, “FULL” - SQLite auto vacuum pragma.

  • sqlite_cache_size (int, in pages) default 8,192 - SQLite cache size pragma.

  • sqlite_journal_mode (str) default “wal” - SQLite journal mode pragma.

  • sqlite_mmap_size (int, in bytes) default 64 megabytes - SQLite mmap size pragma.

  • sqlite_synchronous (int) default 1, “NORMAL” - SQLite synchronous pragma.

  • disk_min_file_size (int, in bytes) default 32 kilobytes - values with greater size are stored in files.

  • disk_pickle_protocol (int) default highest Pickle protocol - the Pickle protocol to use for data types that are not natively supported.

diskcache.EVICTION_POLICY
  • least-recently-stored (default) - evict least recently stored keys first.

  • least-recently-used - evict least recently used keys first.

  • least-frequently-used - evict least frequently used keys first.

  • none - never evict keys.

Disk

Read the Disk tutorial for details.

class diskcache.Disk(directory, min_file_size=0, pickle_protocol=0)

Cache key and value serialization for SQLite database and files.

__init__(directory, min_file_size=0, pickle_protocol=0)

Initialize disk instance.

Parameters
  • directory (str) – directory path

  • min_file_size (int) – minimum size for file use

  • pickle_protocol (int) – pickle protocol for serialization

fetch(mode, filename, value, read)

Convert fields mode, filename, and value from Cache table to value.

Parameters
  • mode (int) – value mode raw, binary, text, or pickle

  • filename (str) – filename of corresponding value

  • value – database value

  • read (bool) – when True, return an open file handle

Returns

corresponding Python value

filename(key=UNKNOWN, value=UNKNOWN)

Return filename and full-path tuple for file storage.

Filename will be a randomly generated 28 character hexadecimal string with “.val” suffixed. Two levels of sub-directories will be used to reduce the size of directories. On older filesystems, lookups in directories with many files may be slow.

The default implementation ignores the key and value parameters.

In some scenarios, for example Cache.push, the key or value may not be known when the item is stored in the cache.

Parameters
  • key – key for item (default UNKNOWN)

  • value – value for item (default UNKNOWN)

get(key, raw)

Convert fields key and raw from Cache table to key.

Parameters
  • key – database key to convert

  • raw (bool) – flag indicating raw database storage

Returns

corresponding Python key

hash(key)

Compute portable hash for key.

Parameters

key – key to hash

Returns

hash value

put(key)

Convert key to fields key and raw for Cache table.

Parameters

key – key to convert

Returns

(database key, raw boolean) pair

remove(filename)

Remove a file given by filename.

This method is cross-thread and cross-process safe. If an “error no entry” occurs, it is suppressed.

Parameters

filename (str) – relative path to file

store(value, read, key=UNKNOWN)

Convert value to fields size, mode, filename, and value for Cache table.

Parameters
  • value – value to convert

  • read (bool) – True when value is file-like object

  • key – key for item (default UNKNOWN)

Returns

(size, mode, filename, value) tuple for Cache table

JSONDisk

Read the Disk tutorial for details.

class diskcache.JSONDisk(directory, compress_level=1, **kwargs)

Cache key and value using JSON serialization with zlib compression.

__init__(directory, compress_level=1, **kwargs)

Initialize JSON disk instance.

Keys and values are compressed using the zlib library. The compress_level is an integer from 0 to 9 controlling the level of compression; 1 is fastest and produces the least compression, 9 is slowest and produces the most compression, and 0 is no compression.

Parameters
  • directory (str) – directory path

  • compress_level (int) – zlib compression level (default 1)

  • kwargs – super class arguments

fetch(mode, filename, value, read)

Convert fields mode, filename, and value from Cache table to value.

Parameters
  • mode (int) – value mode raw, binary, text, or pickle

  • filename (str) – filename of corresponding value

  • value – database value

  • read (bool) – when True, return an open file handle

Returns

corresponding Python value

get(key, raw)

Convert fields key and raw from Cache table to key.

Parameters
  • key – database key to convert

  • raw (bool) – flag indicating raw database storage

Returns

corresponding Python key

put(key)

Convert key to fields key and raw for Cache table.

Parameters

key – key to convert

Returns

(database key, raw boolean) pair

store(value, read, key=UNKNOWN)

Convert value to fields size, mode, filename, and value for Cache table.

Parameters
  • value – value to convert

  • read (bool) – True when value is file-like object

  • key – key for item (default UNKNOWN)

Returns

(size, mode, filename, value) tuple for Cache table

Timeout

exception diskcache.Timeout

Database timeout expired.