Python Collections
collections is a module that contains useful container classes.
Contents
Classes
ChainMap
To combine mappings in a manner that preserves values hierarchically, consider using a ChainMap.
The list of mappings is stored internally. Lookups traverse the entire list (in order) until a match is found, while deletions, insertions, and updates only operate on the first mapping.
A simple example would be the Python interpreter's own lookup chain:
import builtins from collections import ChainMap pylookup = ChainMap(locals(), globals(), vars(builtins))
To add a new lowest-priority mapping to an existing ChainMap, update the maps list directly.
from collections import ChainMap inventory = ChainMap(store_inventory, regional_inventory) inventory.maps.append(global_inventory)
To add a new highest-priority mapping to an existing ChainMap, instead create a new object with the new_child() method.
merchandise = inventory.new_child(showroom_inventory)
A ChainMap is much faster than a series of update() calls on a single dictionary, while still offering an ordering of keys that matches such an implementation.
In the context of type annotations, try:
from collections import ChainMap
c: ChainMap[str, int] = ChainMap({"table": 1, "chair": 4})
Counter
To aggregate a collection of duplicated values into a collection of pairs of unique values and the number of occurences of that value in the original collection, use a Counter.
from collections import Counter
animals_in_residence = Counter(cats=4, dogs=8)
# Counter({'cats': 4, 'dogs': 8})
food_orders = Counter(['eggs', 'ham', 'spam'])
# Counter({'eggs': 1, 'ham': 1, 'spam': 1})
favorite_colors = Counter()
for word in ['red', 'blue', 'red', 'green', 'blue', 'blue']:
favorite_colors[word] += 1
# Counter({'blue': 3, 'red': 2, 'green': 1})In all cases, the insertion order is preserved.
If a lookup fails, a Counter returns 0 rather than raising a KeyError.
To combine Counter objects, try:
from collections import Counter
c = Counter(x=2, y=2)
c.update(Counter(x=1, y=1))
# Counter({'x': 3, 'y': 3})On the other hand, to subtract one Counter from another, try:
from collections import Counter
c = Counter(x=2, y=2)
c.subtract(Counter(x=1, y=1))
# Counter({'x': 1, 'y': 1})To delete a pair from a Counter, do not set the corresponding value to 0. Instead delete the attribute (del c['key']).
There are also aggregation methods available. totals() returns the sum of all values' counts. most_common(n) returns the n value and count pairs with the highest counts.
In the context of type annotations, try:
from collections import Counter c: Counter[str] = Counter(['eggs', 'ham', 'spam'])
DefaultDict
A dict with a custom factory function for new values.
Consider the use case of building a str-to-list dict:
d = dict()
for k, v in [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]:
d.setdefault(k, []).append(v)The following is equivalent but faster.
from collections import defaultdict
d = defaultdict(list)
for k, v in [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]:
d[k].append(v)In the context of type annotations, try:
from collections import defaultdict d: defaultdict[str, list[int]] = defaultdict(list)
Deque
A deque is a list with optimized left-hand pop(), append(), and extend() methods. The name is a contraction of 'double-ended queue'.
from collections import deque
d = deque(["ham", "spam", "eggs"])
d.append("lastItem")
i = d.pop() # "lastItem"
d.extend(["lastItem"])
i = d.pop() # "lastItem"
d.appendleft("firstItem")
i = d.popleft() # "firstItem"
d.extendleft(["firstItem"])
i = d.popleft() # "firstItem"To simultaneously pop() and appendleft() (i.e. move the last item into the first position), try using d.rotate(1). To do the inverse (i.e. move the first item into the last position), try rotate(-1).
A deque can be cleared with the clear() method.
To enforce a fixed size on a deque, try:
from collections import deque d = deque([1,2,3,4,5], 3) # deque([3, 4, 5]) d.append(6) # deque([4, 5, 6]) d.appendleft(7) # deque([7, 4, 5])
In the context of type annotations, try:
from collections import deque d: deque[str] = deque(["ham", "spam", "eggs"])
NamedTuple
A tuple with named indices.
# All equivalent definitions
Point = namedtuple('Point', ['x', 'y'])
Point = namedtuple('Point', 'x, y')
Point = namedtuple('Point', 'x y')
# All equivalent instantiations
p = Point(11, 22)
p = Point(x=11, y=22)
p = Point(11, y=22)
p = Point(x=11, 22)
p = Point._make([11, 22])
p = Point(**{'x': 11, 'y': 22})The items can be accessed by index (as usual) but also as attributes (i.e. p.x). The getattr function will also work on a namedtuple (i.e. getattr(p, 'x')).
In the context of type annotations, an additional class is required from typing. See here for details.
OrderedDict
A dict that preserves insertion order. Per implementation detail, the built-in dict already preserves order in Python 3.7+. OrderedDict is supported as a backwards-compatible and future-proof object.
The key differences between dict and OrderedDict are:
dict is optimized for lookups, while OrderedDict is optimized for reordering operations
OrderedDict equality operations check for order
OrderedDict has a different popitem() method; dict does not have a simple equivalent to od.popitem(last=False)
OrderedDict has an extra move_to_end() method
To create a dict that preserves last insertion order, try:
class LastUpdatedOrderedDict(OrderedDict):
def __setitem__(self, key, value):
super().__setitem__(key, value)
self.move_to_end(key)In the context of type annotations, try:
from collections import OrderedDict
od: OrderedDict[str, int] = OrderedDict({"red": 1, "blue": 2)
UserDict
A dict that can be subclassed. The built-in dict can now be subclassed, but UserDict is supported as a backwards-compatible object.
In the context of type annotations, try:
from collections import UserDict
ud: UserDict[str, int] = UserDict({"red": 1, "blue": 2})
UserList
A list that can be subclassed. The built-in list can now be subclassed, but UserList is supported as a backwards-compatible object.
In the context of type annotations, try:
from collections import UserList ul: UserList[int] = UserList([1, 2])
UserString
A str that can be subclassed. The built-in str can now be subclassed, but UserString is supported as a backwards-compatible object.
In the context of type annotations, try:
from collections import UserString
us: UserString = UserString("foo")
Abstract Base Classes
See here for details.
See also
Python collections module documentation
Python Module of the Day article for collections