Runtime Performance Comparison

Because Sorted Containers is implemented in pure-Python, its performance depends directly on the Python runtime. Sorted Containers is primarily developed, tested and benchmarked on CPython 3.6.

Not all runtimes are created equal. The graphs below compare Sorted Containers running on the CPython 3.6, CPython 2.7, and PyPy runtimes. As of Python 3.6 the CPython 3.6 runtime is now faster than the CPython 2.7 runtime. The PyPy runtime displays much more variability due to its JIT-ed nature. Once the just-in-time compiler optimizes the code, performance is often two to ten times faster.

Performance of competing implementations are benchmarked against the CPython 3.6 runtime. An implementation performance comparison is also included with data from popular sorted container packages.

Sorted Containers uses a segmented-list data structure similar to a B-tree limited to two levels of nodes. As part of the implementation, a load factor is used to determine how many values should be stored in each node. This can have a significant impact on performance and a load factor performance comparison is also provided.

Though these benchmarks exercise only one API repeatedly, an effort has also been made to simulate real-world workloads. The simulated workload performance comparison contains examples with comparisons to other implementations, load factors, and runtimes.

Sorted List

Graphs comparing Sorted List performance.

__init__

Initializing with a list of random numbers using SortedList.__init__().

_images/SortedList_runtime-init.png

add

Randomly adding values using SortedList.add().

_images/SortedList_runtime-add.png

contains

Randomly testing membership using SortedList.__contains__().

_images/SortedList_runtime-contains.png

count

Counting objects at random using SortedList.count().

_images/SortedList_runtime-count.png

__delitem__

Deleting objects at random using SortedList.__delitem__().

_images/SortedList_runtime-delitem.png

__getitem__

Retrieving ojbects by index using SortedList.__getitem__().

_images/SortedList_runtime-getitem.png

index

Finding the index of an object using SortedList.index().

_images/SortedList_runtime-index.png

iter

Iterating a SortedList using SortedList.__iter__().

_images/SortedList_runtime-iter.png

pop

Removing the last object using SortedList.pop().

_images/SortedList_runtime-pop.png

remove

Remove an object at random using SortedList.remove().

_images/SortedList_runtime-remove.png

update_large

Updating a SortedList with a large iterable using SortedList.update().

_images/SortedList_runtime-update_large.png

update_small

Updating a SortedList with a small iterable using SortedList.update().

_images/SortedList_runtime-update_small.png

Sorted Dict

Graphs comparing Sorted Dict performance.

__init__

Initializing with a list of pairs of random numbers using SortedDict.__init__().

_images/SortedDict_runtime-init.png

__contains__

Given a key at random, test whether the key is in the dictionary using SortedDict.__contains__().

_images/SortedDict_runtime-contains.png

__getitem__

Given a key at random, retrieve the value using SortedDict.__getitem__().

_images/SortedDict_runtime-getitem.png

__setitem__

Given a key at random, set the value using SortedDict.__setitem__().

_images/SortedDict_runtime-setitem.png

__delitem__

Given a key at random, delete the value using SortedDict.__delitem__().

_images/SortedDict_runtime-delitem.png

iter

Iterate the keys of a SortedDict using SortedDict.__iter__().

_images/SortedDict_runtime-iter.png

setitem_existing

Given an existing key at random, set the value using SortedDict.__setitem__().

_images/SortedDict_runtime-setitem_existing.png

Sorted Set

Graphs comparing Sorted Set performance.

__init__

Initializing with a list of random numbers using SortedSet.__init__().

_images/SortedSet_runtime-init.png

add

Randomly add values using SortedSet.add().

_images/SortedSet_runtime-add.png

contains

Randomly test membership using SortedSet.__contains__().

_images/SortedSet_runtime-contains.png

difference_large

Set difference using SortedSet.difference().

_images/SortedSet_runtime-difference_large.png

difference_medium

Set difference using SortedSet.difference().

_images/SortedSet_runtime-difference_medium.png

difference_small

Set difference using SortedSet.difference().

_images/SortedSet_runtime-difference_small.png

difference_tiny

Set difference using SortedSet.difference().

_images/SortedSet_runtime-difference_tiny.png

difference_update_large

Set difference using SortedSet.difference_update().

_images/SortedSet_runtime-difference_update_large.png

difference_update_medium

Set difference using SortedSet.difference_update().

_images/SortedSet_runtime-difference_update_medium.png

difference_update_small

Set difference using SortedSet.difference_update().

_images/SortedSet_runtime-difference_update_small.png

difference_update_tiny

Set difference using SortedSet.difference_update().

_images/SortedSet_runtime-difference_update_tiny.png

intersection_large

Set intersection using SortedSet.intersection().

_images/SortedSet_runtime-intersection_large.png

intersection_medium

Set intersection using SortedSet.intersection().

_images/SortedSet_runtime-intersection_medium.png

intersection_small

Set intersection using SortedSet.intersection().

_images/SortedSet_runtime-intersection_small.png

intersection_tiny

Set intersection using SortedSet.intersection().

_images/SortedSet_runtime-intersection_tiny.png

intersection_update_large

Set intersection using SortedSet.intersection_update().

_images/SortedSet_runtime-intersection_update_large.png

intersection_update_medium

Set intersection using SortedSet.intersection_update().

_images/SortedSet_runtime-intersection_update_medium.png

intersection_update_small

Set intersection using SortedSet.intersection_update().

_images/SortedSet_runtime-intersection_update_small.png

intersection_update_tiny

Set intersection using SortedSet.intersection_update().

_images/SortedSet_runtime-intersection_update_tiny.png

iter

Iterating a set using iter(SortedSet)().

_images/SortedSet_runtime-iter.png

pop

Remove the last item in a set using SortedSet.pop().

_images/SortedSet_runtime-pop.png

remove

Remove an item at random using SortedSet.remove().

_images/SortedSet_runtime-remove.png

union_large

Set union using SortedSet.union().

_images/SortedSet_runtime-union_large.png

union_medium

Set union using SortedSet.union().

_images/SortedSet_runtime-union_medium.png

union_small

Set union using SortedSet.union().

_images/SortedSet_runtime-union_small.png

union_tiny

Set union using SortedSet.union().

_images/SortedSet_runtime-union_tiny.png

update_large

Set update using SortedSet.update().

_images/SortedSet_runtime-update_large.png

update_medium

Set update using SortedSet.update().

_images/SortedSet_runtime-update_medium.png

update_small

Set update using SortedSet.update().

_images/SortedSet_runtime-update_small.png

update_tiny

Set update using SortedSet.update().

_images/SortedSet_runtime-update_tiny.png

symmetric_difference_large

Set symmetric-difference using SortedSet.symmetric_difference().

_images/SortedSet_runtime-symmetric_difference_large.png

symmetric_difference_medium

Set symmetric-difference using SortedSet.symmetric_difference().

_images/SortedSet_runtime-symmetric_difference_medium.png

symmetric_difference_small

Set symmetric-difference using SortedSet.symmetric_difference().

_images/SortedSet_runtime-symmetric_difference_small.png

symmetric_difference_tiny

Set symmetric-difference using SortedSet.symmetric_difference().

_images/SortedSet_runtime-symmetric_difference_tiny.png

symm_diff_update_large

Set symmetric-difference using SortedSet.symmetric_difference_update().

_images/SortedSet_runtime-symmetric_difference_update_large.png

symm_diff_update_medium

Set symmetric-difference using SortedSet.symmetric_difference_update().

_images/SortedSet_runtime-symmetric_difference_update_medium.png

symm_diff_update_small

Set symmetric-difference using SortedSet.symmetric_difference_update().

_images/SortedSet_runtime-symmetric_difference_update_small.png

symm_diff_update_tiny

Set symmetric-difference using SortedSet.symmetric_difference_update().

_images/SortedSet_runtime-symmetric_difference_update_tiny.png