Introduction and examples

Config and ConfigFile

confattr (config attributes) is a python library to make applications configurable. This library defines the Config class to create attributes which can be changed in a config file. It uses the descriptor protocol to return it’s value when used as an instance attribute.

from confattr import Config

class Car:

	speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')

	def __init__(self) -> None:
		self.speed = 0

	def accelerate(self, value: int) -> None:
		new_speed = self.speed + value
		if new_speed > self.speed_limit:
			raise ValueError('you are going too fast')

		self.speed = new_speed

If you want to access the Config object itself you need to access it as a class attribute:

	def print_config(self) -> None:
		print('{key}: {val}'.format(key=type(self).speed_limit.key, val=self.speed_limit))

You load a config file with a ConfigFile object. You should provide a callback function with set_ui_callback() which informs the user if the config file contains invalid lines. This callback function takes a Message object as argument. You can format it automatically by converting it to a str, e.g. with str(msg). Among other attributes this object also has a notification_level (or lvl for short) which should be used to show messages of different severity in different colors. By default only ERROR messages are reported but you should pass a Config to notification_level when instantiating a ConfigFile object so that the users of your application can change that. When you load a config file with ConfigFile.load() all Config objects which are set in the config file are updated automatically.

It is recognized automatically that the setting traffic-law.speed-limit has an integer value. A value given in a config file is therefore automatically parsed to an integer. If the parsing fails the user interface callback function is called.

if __name__ == '__main__':
	from confattr import ConfigFile, NotificationLevel
	config_file = ConfigFile(appname=__package__,
		notification_level=Config('notification-level', NotificationLevel.ERROR))
	config_file.load()

	# Print error messages which occurred while loading the config file.
	# In this easy example it would have been possible to register the callback
	# before calling load but in the real world the user interface
	# will depend on the values set in the config file.
	# Therefore the messages are stored until a callback is added.
	config_file.set_ui_callback(lambda msg: print(msg))

	c1 = Car()
	print('speed_limit: %s' % c1.speed_limit)

Given the following config file (the location of the config file is determined by ConfigFile.iter_config_paths()):

set traffic-law.speed-limit = 30

The script will give the following output:

speed_limit: 30

You can save the current configuration with ConfigFile.save() if you want to write it to the default location or with ConfigFile.save_file() if you want to specify the path yourself.

	filename = config_file.save()
	print('configuration was written to %s' % filename)

This will write the following file:

# Data types
# ----------
# int:
#   An integer number in python 3 syntax, as decimal (e.g. 42),
#   hexadecimal (e.g. 0x2a), octal (e.g. 0o52) or binary (e.g.
#   0b101010). Leading zeroes are not permitted to avoid confusion
#   with python 2's syntax for octal numbers. It is permissible to
#   group digits with underscores for better readability, e.g.
#   1_000_000.

# notification-level
# ------------------
# one of info, error
set notification-level = error

# traffic-law.speed-limit
# -----------------------
# an int in km/h
set traffic-law.speed-limit = 30

Config file syntax

I have looked at the config files of different applications trying to come up with a syntax as intuitive as possible. Two extremes which have heavily inspired me are the config files of vim and ranger. Quoting and inline comments work like in a POSIX shell (except that there is no difference between single quotes and double quotes) as I am using shlex.split() to split the lines.

Lines starting with a " or # are ignored.

The set command has two different forms. I recommend to not mix them in order to improve readability.

  • set key1=val1 [key2=val2 ...]   (inspired by vimrc)

    set takes an arbitrary number of arguments, each argument sets one setting.

    Has the advantage that several settings can be changed at once. This is useful if you want to bind a set command to a key and process that command with ConfigFile.parse_line() if the key is pressed.

    If the value contains one or more spaces it must be quoted. set greeting='hello world' and set 'greeting=hello world' are equivalent.

  • set key [=] val   (inspired by ranger config)

    set takes two arguments, the key and the value. Optionally a single equals character may be added in between as third argument.

    Has the advantage that key and value are separated by one or more spaces which can improve the readability of a config file.

    If the value contains one or more spaces it must be quoted: set greeting 'hello world' and set greeting = 'hello world' are equivalent.

I recommend to not use spaces in key names so that they don’t need to be wrapped in quotes.

It is possible to include another config file:

  • include filename

    If filename is a relative path it is relative to the directory of the config file it appears in.

    This command is explained in more detail in this example.

Different values for different objects

A Config object always returns the same value, regardless of the owning object it is an attribute of:

from confattr import Config

class Car:
	speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')

c1 = Car()
c2 = Car()

print(c1.speed_limit, c2.speed_limit)
c2.speed_limit = 30  # don't do this, this is misleading!
print(c1.speed_limit, c2.speed_limit)

Output:

50 50
30 30

If you want to have different values for different objects you need to use MultiConfig instead. This requires the owning object to have a special attribute called config_id. All objects which have the same config_id share the same value. All objects which have different config_id can have different values (but don’t need to have different values).

import enum
from confattr import Config, MultiConfig, ConfigId, ConfigFile, NotificationLevel

class Color(enum.Enum):
	RED = 'red'
	YELLOW = 'yellow'
	GREEN = 'green'
	BLUE = 'blue'
	WHITE = 'white'
	BLACK = 'black'


class Car:

	speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')
	color = MultiConfig('car.color', Color.BLACK)

	def __init__(self, config_id: ConfigId) -> None:
		self.config_id = config_id


if __name__ == '__main__':
	config_file = ConfigFile(appname=__package__,
		notification_level=Config('notification-level', NotificationLevel.ERROR))
	config_file.load()

	config_file.set_ui_callback(lambda msg: print(msg))

	cars = []
	for config_id in MultiConfig.config_ids:
		cars.append(Car(config_id))
	cars.append(Car(ConfigId('another-car')))

	for car in cars:
		print('color of %s: %s' % (car.config_id, car.color))

Given the following config file:

set traffic-law.speed-limit = 30

[alices-car]
set car.color = red

[bobs-car]
set car.color = blue

It creates the following output:

color of alices-car: Color.RED
color of bobs-car: Color.BLUE
color of another-car: Color.BLACK

another-car gets the default color black as it is not set in the config file. You can change this default color in the config file by setting it before specifying a config id or after specifying the special config id general (Config.default_config_id). Note how this adds general to MultiConfig.config_ids.

set traffic-law.speed-limit = 30
set car.color = white

[alices-car]
set car.color = red

[bobs-car]
set car.color = blue

Creates the following output:

color of general: Color.WHITE
color of alices-car: Color.RED
color of bobs-car: Color.BLUE
color of another-car: Color.WHITE

MultiConfig.reset

For normal Config instances you can restore a certain state of settings by calling ConfigFile.save(comments=False) (when you have the state that you want to restore later on) and ConfigFile.load() (where you want to restore the saved state).

This is not enough when you are using MultiConfig instances. Consider the following example:

from confattr import MultiConfig, ConfigId, ConfigFile

class Widget:

	greeting = MultiConfig('greeting', 'hello world')

	def __init__(self, name: str) -> None:
		self.config_id = ConfigId(name)

config_file = ConfigFile(appname='example')
config_file.set_ui_callback(lambda msg: print(msg))

w1 = Widget('w1')
assert w1.greeting == 'hello world'

config_file.save(comments=False)
w1.greeting = 'hey you'
assert w1.greeting == 'hey you'

#MultiConfig.reset()   # This is missing
config_file.load()
assert w1.greeting == 'hello world'   # This fails!

The last assert fails because when saving the config no value for w1 has been set yet. It is just falling back to the default value “hello world”. The saved config file is therefore:

set greeting = 'hello world'

After the config was saved the value for w1 is changed to “hey you”. When loading the config the default value is restored to hello world (which makes no difference because it has never been changed) but the value for w1 is not changed because there is no value for w1 in the config file.

The solution is to call MultiConfig.reset() before loading the config.

Include

Consider a backup application which synchronizes one or more directory pairs. The following code might be a menu for it to choose which side should be changed:

from confattr import Config, MultiConfig, ConfigId, ConfigFile, NotificationLevel, Message
import enum
import urwid

CMD_QUIT = 'quit'
CMD_TOGGLE = 'toggle'
CMD_IGNORE = 'ignore'

urwid.command_map['q'] = CMD_QUIT
urwid.command_map[' '] = CMD_TOGGLE
urwid.command_map['i'] = CMD_IGNORE


class Direction(enum.Enum):
	SRC_TO_DST = ' > '
	DST_TO_SRC = ' < '
	IGNORE = ' | '
	TWO_WAY = '<->'

class DirectoryPair:

	path_src = MultiConfig('path.src', '')
	path_dst = MultiConfig('path.dst', '')
	direction = MultiConfig('direction', Direction.SRC_TO_DST)

	def __init__(self, config_id: ConfigId) -> None:
		self.config_id = config_id

	def toggle_direction(self) -> None:
		if self.direction is Direction.SRC_TO_DST:
			self.direction = Direction.DST_TO_SRC
		else:
			self.direction = Direction.SRC_TO_DST

	def ignore(self) -> None:
		self.direction = Direction.IGNORE


class DirectoryPairWidget(urwid.WidgetWrap):  # type: ignore [misc]  # Class cannot subclass "WidgetWrap" (has type "Any") because urwid is not typed yet

	def __init__(self, dirs: DirectoryPair) -> None:
		self.model = dirs
		self.widget_src = urwid.Text(dirs.path_src)
		self.widget_dst = urwid.Text(dirs.path_dst)
		self.widget_direction = urwid.Text('')
		self.update_direction()
		widget = urwid.Columns([self.widget_src, (urwid.PACK, self.widget_direction), self.widget_dst])
		widget = urwid.AttrMap(widget, None, App.ATTR_FOCUS)
		super().__init__(widget)

	def selectable(self) -> bool:
		return True

	def keypress(self, size: 'tuple[int, ...]', key: str) -> 'str|None':
		if not super().keypress(size, key):
			return None

		cmd = self._command_map[key]
		if cmd == CMD_TOGGLE:
			self.model.toggle_direction()
			self.update_direction()
		elif cmd == CMD_IGNORE:
			self.model.ignore()
			self.update_direction()
		else:
			return key

		return None

	def update_direction(self) -> None:
		self.widget_direction.set_text(' %s ' % self.model.direction.value)


class App:

	ATTR_ERROR = NotificationLevel.ERROR.value
	ATTR_INFO = NotificationLevel.INFO.value
	ATTR_FOCUS = 'focus'

	PALETTE = (
		(ATTR_ERROR, 'dark red', 'default'),
		(ATTR_INFO, 'dark blue', 'default'),
		(ATTR_FOCUS, 'default', 'dark blue'),
	)

	notification_level = Config('notification-level', NotificationLevel.ERROR,
		help = {
			NotificationLevel.ERROR: 'show errors in the config file',
			NotificationLevel.INFO: 'additionally show all settings which are changed in the config file',
		}
	)

	def __init__(self) -> None:
		self.config_file = ConfigFile(appname=__package__, notification_level=type(self).notification_level)
		self.config_file.load()
		self.directory_pairs = [DirectoryPair(config_id) for config_id in MultiConfig.config_ids]

		self.body = urwid.ListBox([DirectoryPairWidget(dirs) for dirs in self.directory_pairs])
		self.status_bar = urwid.Pile([])
		self.frame = urwid.Frame(self.body, footer=self.status_bar)
		self.config_file.set_ui_callback(self.on_config_message)

	def run(self) -> None:
		urwid.MainLoop(self.frame, palette=self.PALETTE, input_filter=self.input_filter, unhandled_input=self.unhandled_input, handle_mouse=False).run()


	def on_config_message(self, msg: Message) -> None:
		markup = (msg.notification_level.value, str(msg))
		widget_options_tuple = (urwid.Text(markup), self.status_bar.options('pack'))
		self.status_bar.contents.append(widget_options_tuple)
		self.frame.footer = self.status_bar

	def input_filter(self, keys: 'list[str]', raws: 'list[int]') -> 'list[str]':
		self.status_bar.contents.clear()
		Message.reset()
		return keys

	def unhandled_input(self, key: str) -> bool:
		cmd = urwid.command_map[key]
		if cmd == CMD_QUIT:
			raise urwid.ExitMainLoop()
		self.on_config_message(Message(NotificationLevel.ERROR, 'undefined key: %s' % key))
		return True


if __name__ == '__main__':
	app = App()
	app.run()

Let’s assume there are many more settings how to synchronize a pair of directories than just the direction. You might want to use the same synchronization settings for several directory pairs. You can write these settings to a separate config file and include it for the corresponding directory pairs:

main config file: config
[documents]
set path.src = ~/documents
set path.dst = /media/usb1/documents
include mirror

[music]
set path.src = ~/music
set path.dst = /media/usb1/music
include mirror

[pictures]
set path.src = ~/pictures
set path.dst = /media/usb1/pictures
include two-way
included config file: mirror
set direction = src-to-dst
included config file: two-way
set direction = two-way

This produces the following display:

~/documents              >  /media/usb1/documents  
~/music                  >  /media/usb1/music      
~/pictures              <-> /media/usb1/pictures   
                                                   
                                                   

The config id of the included file starts with the value of the config id that the including file has at the moment of calling include. Otherwise the pattern shown above of reusing a config file for several config ids would not be possible.

If the included file changes the config id the config id is reset to the value it had at the beginning of the include when reaching the end of the included file. Otherwise changing an included file might unexpectedly change the meaning of the main config file or another config file which is included later on.

It is possible to change this default behavior by using include --reset-config-id-before filename or include --no-reset-config-id-after filename.

Generating help

You can generate a help with ConfigFile.write_help() or ConfigFile.get_help().

ConfigFile.get_help() is a wrapper around ConfigFile.write_help(). If you want to print the help to stdout config_file.write_help(HelpWriter(None)) would be more efficient than print(config_file.get_help()). If you want to display the help in a graphical user interface you can implement a custom FormattedWriter which you can pass to ConfigFile.write_help() instead of parsing the output of ConfigFile.get_help().

from confattr import ConfigFile, Config, DictConfig, MultiConfig
from enum import Enum, auto

class Color(Enum):
	RED = auto()
	GREEN = auto()
	BLUE = auto()

Config('answer', 42, unit='', help={
	42: 'The answer to everything',
	23: '''
		The natural number following 22
		and preceding 24
	''',
})
DictConfig('color', dict(foreground=Color.RED, background=Color.GREEN))
MultiConfig('greeting', 'hello world', help='''
	A text that may be displayed
	if your computer is in a good mood.
''')


if __name__ == '__main__':
	config_file = ConfigFile(appname=__package__)
	print(config_file.get_help())

Assuming the above file was contained in a package called exampleapp it would output the following:

The first existing file of the following paths is loaded:
- /home/username/.config/exampleapp/config
- /etc/xdg/exampleapp/config

This can be influenced with the following environment variables:
- XDG_CONFIG_HOME
- XDG_CONFIG_DIRS
- EXAMPLEAPP_CONFIG_PATH
- EXAMPLEAPP_CONFIG_DIRECTORY
- EXAMPLEAPP_CONFIG_NAME

You can also use environment variables to change the values of the
settings listed under `set` command. The corresponding environment
variable name is the name of the setting in all upper case letters
with dots, hypens and spaces replaced by underscores and prefixed with
"EXAMPLEAPP_".

Lines in the config file which start with a `"` or `#` are ignored.

The config file may contain the following commands:

set
===
usage: set key1=val1 [key2=val2 ...]
       set key [=] val

Change the value of a setting.

In the first form set takes an arbitrary number of arguments, each
argument sets one setting. This has the advantage that several
settings can be changed at once. That is useful if you want to bind a
set command to a key and process that command with
ConfigFile.parse_line() if the key is pressed.

In the second form set takes two arguments, the key and the value.
Optionally a single equals character may be added in between as third
argument. This has the advantage that key and value are separated by
one or more spaces which can improve the readability of a config file.

data types:

  int:
    An integer number in python 3 syntax, as decimal (e.g. 42),
    hexadecimal (e.g. 0x2a), octal (e.g. 0o52) or binary (e.g.
    0b101010). Leading zeroes are not permitted to avoid confusion
    with python 2's syntax for octal numbers. It is permissible to
    group digits with underscores for better readability, e.g.
    1_000_000.

  str:
    A text. If it contains spaces it must be wrapped in single or
    double quotes.

application wide settings:

  answer:
    an int

    42: The answer to everything
    23: The natural number following 22 and preceding 24

  color.background:
    one of red, green, blue

  color.foreground:
    one of red, green, blue

settings which can have different values for different objects:
  You can specify the object that a value shall refer to by
  inserting the line `[config-id]` above. `config-id` must be
  replaced by the corresponding identifier for the object.

  greeting:
    a str

    A text that may be displayed if your computer is in a good
    mood.

include
=======
usage: include [--reset-config-id-before | --no-reset-config-id-after]
               path

Load another config file.

This is useful if a config file is getting so big that you want to
split it up or if you want to have different config files for
different use cases which all include the same standard config file to
avoid redundancy or if you want to bind several commands to one key
which executes one command with ConfigFile.parse_line().

By default the loaded config file starts with which ever config id is
currently active. This is useful if you want to use the same values
for several config ids: Write the set commands without a config id to
a separate config file and include this file for every config id where
these settings shall apply.

After the include the config id is reset to the config id which was
active at the beginning of the include because otherwise it might lead
to confusion if the config id is changed in the included config file.

positional arguments:
  path                  The config file to load. Slashes are replaced
                        with the directory separator appropriate for
                        the current operating system.

options:
  --reset-config-id-before
                        Ignore any config id which might be active
                        when starting the include
  --no-reset-config-id-after
                        Treat the included lines as if they were
                        written in the same config file instead of the
                        include command

The help is formatted on two levels:

  1. argparse.HelpFormatter does the merging of lines, wrapping of lines and indentation. It formats the usage and all the command line arguments and options. Unfortunately “All the methods provided by the class are considered an implementation detail” according to it’s doc string. The only safe way to customize this level of formatting is by handing one of the predefined standard classes to the formatter_class parameter of the ConfigFile constructor:

    Additionally I provide another subclass confattr.utils.HelpFormatter which has a few class attributes for customization which I am trying to keep backward compatible. So you can subclass this class and change these attributes. But I cannot guarantee to always support the newest python version.

    If you want any more customization take a look at the source code but be prepared that you may need to change your code with any future python version.

  2. FormattedWriter is intended to do stuff like underlining sections and inserting comment characters at the beginning of lines (when writing help to a config file). This package defines two subclasses: ConfigFileWriter which is used by default in ConfigFile.save() and HelpWriter which is used in ConfigFile.get_help().

    If you want to customize this level of formatting implement your own FormattedWriter and override ConfigFile.get_help() or ConfigFile.save_to_open_file() to use your class.

Custom data types

It is possible to use custom data types. You can use that for example to avoid repeating a part of help which applies to several settings:

import re
from confattr import Config, ConfigFile

class Regex(str):

	type_name = 'regular expression'
	help = '''
	A regular expression in python syntax.
	You can specify flags by starting the regular expression with `(?aiLmsux)`.
	https://docs.python.org/3/library/re.html#regular-expression-syntax
	'''

class Parser:

	re_mount_output = Config('udisksctl.mount-output-pattern', Regex(r'^.*?(?P<mountpath>/(\S+/)*[^/]+?)\.?$'),
		help='a regular expression to parse the output of `udisksctl mount`. Must contain a named group called "mountpath".')
	re_unlock_output = Config('udisksctl.unlock-output-pattern', Regex(r'^.*?(?P<unlockpath>/(\S+/)*[^/]+?)\.?$'),
		help='a regular expression to parse the output of `udisksctl unlock`. Must contain a named group called "unlockpath".')

	def compile_regex(self) -> None:
		'''
		This must be called every time after the config file has been loaded.
		'''
		self.reo_mount_output = re.compile(self.re_mount_output)
		self.reo_unlock_output = re.compile(self.re_unlock_output)

if __name__ == '__main__':
	ConfigFile(appname=__package__).save()

This exports the following config file:

# Data types
# ----------
# regular expression:
#   A regular expression in python syntax. You can specify flags by
#   starting the regular expression with `(?aiLmsux)`.
#   https://docs.python.org/3/library/re.html#regular-expression-syntax

# udisksctl.mount-output-pattern
# ------------------------------
# a regular expression
# a regular expression to parse the output of `udisksctl mount`. Must
# contain a named group called "mountpath".
set udisksctl.mount-output-pattern = '^.*?(?P<mountpath>/(\S+/)*[^/]+?)\.?$'

# udisksctl.unlock-output-pattern
# -------------------------------
# a regular expression
# a regular expression to parse the output of `udisksctl unlock`. Must
# contain a named group called "unlockpath".
set udisksctl.unlock-output-pattern = '^.*?(?P<unlockpath>/(\S+/)*[^/]+?)\.?$'

__str__() must return a string representation suitable for the config file and the constructor must create an equal object if it is passed the return value of __str__(). This is fulfilled by inheriting from str.

type_name is a special str attribute which specifies how the type is called in the config file. If it is missing it is derived from the class name.

help is a special str attribute which contains a description which is printed to the config file. If this is missing a help must be provided via Set.set_help_for_type().

confattr.types defines several such types, including the above definition of Regex.

For more information on the supported data types see Config.

Adding new commands to the config file syntax

You can extend this library by defining new commands which can be used in the config file. All you need to do is subclass ConfigFileCommand and implement the ConfigFileCommand.run() method. Additionally I recommend to provide a doc string explaining how to use the command in the config file. The doc string is used by get_help() which may be used by an in-app help. Optionally you can set ConfigFileCommand.name and ConfigFileCommand.aliases and implement the ConfigFileCommand.save() method.

Alternatively ConfigFileArgparseCommand can be subclassed instead, it aims to make the parsing easier and avoid redundancy in the doc string by using the argparse module. You must implement init_parser() and run_parsed(). You should give a doc string describing what the command does. In contrast to ConfigFileCommand argparse adds usage and the allowed arguments to the output of ConfigFileArgparseCommand.get_help() automatically.

For example you may want to add a new command to bind keys to whatever kind of command. The following example assumes urwid as user interface framework.

import argparse
from collections.abc import Sequence
import urwid
from confattr import ConfigFileArgparseCommand, ConfigFile, Config, NotificationLevel, Message


class Map(ConfigFileArgparseCommand):

	'''
	bind a command to a key
	'''

	def init_parser(self, parser: argparse.ArgumentParser) -> None:
		parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
		parser.add_argument('cmd', help='any urwid command')

	def run_parsed(self, args: argparse.Namespace) -> None:
		urwid.command_map[args.key] = args.cmd
		self.ui_notifier.show_info(f'map {args.key} {args.cmd!r}')


if __name__ == '__main__':
	# config
	choices = Config('choices', ['vanilla', 'strawberry'])
	urwid.command_map['enter'] = 'confirm'
	config_file = ConfigFile(appname=__package__, notification_level=Config('notification-level', NotificationLevel.ERROR))
	config_file.load()

	# show errors in config
	palette = [(NotificationLevel.ERROR.value, 'dark red', 'default')]
	status_bar = urwid.Pile([])
	def on_config_message(msg: Message) -> None:
		markup = (msg.notification_level.value, str(msg))
		widget_options_tuple = (urwid.Text(markup), status_bar.options('pack'))
		status_bar.contents.append(widget_options_tuple)
		if 'frame' in globals():
			frame._invalidate()
	config_file.set_ui_callback(on_config_message)

	def input_filter(keys: 'list[str]', raw: 'list[int]') -> 'list[str]':
		status_bar.contents.clear()
		Message.reset()
		return keys

	# a simple example app showing check boxes and printing the user's choice to stdout
	def key_handler(key: str) -> bool:
		cmd = urwid.command_map[key]
		if cmd == 'confirm':
			raise urwid.ExitMainLoop()
		on_config_message(Message(NotificationLevel.ERROR, f'key {key!r} is not mapped'))
		return True
	checkboxes = [urwid.CheckBox(choice) for choice in choices.value]
	frame = urwid.Frame(urwid.Filler(urwid.Pile(checkboxes)), footer=status_bar)
	urwid.MainLoop(frame, palette=palette, input_filter=input_filter, unhandled_input=key_handler).run()

	for ckb in checkboxes:
		print(f'{ckb.label}: {ckb.state}')

Given the following config file it is possible to move the cursor upward and downward with j and k like in vim:

map j 'cursor down'
map k 'cursor up'
map q 'confirm'

The help for the newly defined command looks like this:

print(Map(ConfigFile(appname=__package__)).get_help())
usage: map key cmd

bind a command to a key

positional arguments:
  key  http://urwid.org/manual/userinput.html#keyboard-input
  cmd  any urwid command

(All subclasses of ConfigFileCommand are saved in ConfigFileCommand.__init_subclass__() and can be retrieved with ConfigFileCommand.get_command_types(). The ConfigFile constructor uses that if commands is not given.)

Writing custom commands to the config file

The previous example has shown how to define new commands so that they can be used in the config file. Let’s continue that example so that calls to the custom command map are written with ConfigFile.save().

All you need to do for that is implementing the ConfigFileCommand.save() method.

should_write_heading is True if there are several commands which implement the save() method.

Experimental support for type checking **kw has been added in mypy 0.981. SaveKwargs depends on typing.TypedDict and therefore is not available before Python 3.8.

import argparse
import typing
if typing.TYPE_CHECKING:
	from typing_extensions import Unpack  # This will hopefully be replaced by the ** syntax proposed in https://peps.python.org/pep-0692/
	from confattr import SaveKwargs

import urwid
from confattr import ConfigFileArgparseCommand, FormattedWriter, ConfigFile, SectionLevel


class Map(ConfigFileArgparseCommand):

	'''
	bind a command to a key
	'''

	def init_parser(self, parser: argparse.ArgumentParser) -> None:
		parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
		parser.add_argument('cmd', help='any urwid command')

	def run_parsed(self, args: argparse.Namespace) -> None:
		urwid.command_map[args.key] = args.cmd

	def save(self, writer: FormattedWriter, **kw: 'Unpack[SaveKwargs]') -> None:
		if self.should_write_heading:
			writer.write_heading(SectionLevel.SECTION, 'Key bindings')

		for key, cmd in sorted(urwid.command_map._command.items(), key=lambda key_cmd: str(key_cmd[1])):
			quoted_key = self.config_file.quote(key)
			quoted_cmd = self.config_file.quote(cmd)
			writer.write_command(f'map {quoted_key} {quoted_cmd}')


if __name__ == '__main__':
	ConfigFile(appname=__package__).save()

However, urwid.command_map contains more commands than the example app uses so writing all of them might be confusing. Therefore let’s add a keyword argument to write only the specified commands:

import argparse
from collections.abc import Sequence
import typing

import urwid
from confattr import ConfigFileArgparseCommand, FormattedWriter, ConfigFile, SectionLevel

if typing.TYPE_CHECKING:
	from typing_extensions import Unpack  # This will hopefully be replaced by the ** syntax proposed in https://peps.python.org/pep-0692/
	from confattr import SaveKwargs

	class MapSaveKwargs(SaveKwargs, total=False):
		urwid_commands: 'Sequence[str]'

class Map(ConfigFileArgparseCommand):

	'''
	bind a command to a key
	'''

	def init_parser(self, parser: argparse.ArgumentParser) -> None:
		parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
		parser.add_argument('cmd', help='any urwid command')

	def run_parsed(self, args: argparse.Namespace) -> None:
		urwid.command_map[args.key] = args.cmd

	def save(self, writer: FormattedWriter, **kw: 'Unpack[MapSaveKwargs]') -> None:
		if self.should_write_heading:
			writer.write_heading(SectionLevel.SECTION, 'Key bindings')

		commands = kw.get('urwid_commands', sorted(urwid.command_map._command.values()))
		for cmd in commands:
			for key in urwid.command_map._command.keys():
				if urwid.command_map[key] == cmd:
					quoted_key = self.config_file.quote(key)
					quoted_cmd = self.config_file.quote(cmd)
					writer.write_command(f'map {quoted_key} {quoted_cmd}')


if __name__ == '__main__':
	urwid_commands = [urwid.CURSOR_UP, urwid.CURSOR_DOWN, urwid.ACTIVATE, 'confirm']
	mapkw: 'MapSaveKwargs' = dict(urwid_commands=urwid_commands)
	kw: 'SaveKwargs' = mapkw
	config_file = ConfigFile(appname=__package__)
	config_file.save(**kw)

This produces the following config file:

# ============
# Key bindings
# ============
map up 'cursor up'
map down 'cursor down'
map ' ' activate
map enter activate

If you don’t care about Python < 3.8 you can import SaveKwargs normally and save a line when calling ConfigFile.save():

kw: SaveKwargs = MapSaveKwargs(urwid_commands=...)
config_file.save(**kw)

Customizing the config file syntax

If you want to make minor changes to the syntax of the config file you can subclass the corresponding command, i.e. Set or Include.

For example if you want to use a key: value syntax you could do the following.

I am setting name to an empty string (i.e. confattr.DEFAULT_COMMAND) to make this the default command which is used if an unknown command is encountered. This makes it possible to use this command without writing out it’s name in the config file.

from confattr import Set, ParseException, ConfigId, FormattedWriter, SectionLevel

import typing
if typing.TYPE_CHECKING:
	from confattr import SaveKwargs
	from typing_extensions import Unpack
	from collections.abc import Sequence


class SimpleSet(Set, replace=True):

	name = ''

	SEP = ':'

	def run(self, cmd: 'Sequence[str]') -> None:
		ln = self.config_file.context_line
		if self.SEP not in ln:
			raise ParseException(f'missing {self.SEP} between key and value')
		key, value = ln.split(self.SEP)
		value = value.lstrip()
		self.parse_key_and_set_value(key, value)

	def save_config_instance(self, writer: FormattedWriter, instance: 'Config[object]', config_id: 'ConfigId|None', **kw: 'Unpack[SaveKwargs]') -> None:
		# this is called by Set.save
		if kw['comments']:
			self.write_config_help(writer, instance)
		value = self.format_value(instance, config_id)
		#value = self.config_file.quote(value)  # not needed because run uses line instead of cmd
		writer.write_command(f'{instance.key}{self.SEP} {value}')


if __name__ == '__main__':
	from confattr import Config, ConfigFile
	color = Config('favorite color', 'white')
	subject = Config('favorite subject', 'math')
	config_file = ConfigFile(appname=__package__)
	config_file.load()
	config_file.set_ui_callback(lambda msg: print(msg))
	print(color.value)
	print(subject.value)

Then a config file might look like this:

favorite color: sky blue
favorite subject: computer science

Please note that it’s still possible to use the include command. If you want to avoid that use

from confattr import ConfigFileCommand, Include
ConfigFileCommand.delete_command_type(Include)

If you want to make bigger changes like using JSON you need to subclass ConfigFile.

Config without classes

If you want to use Config objects without custom classes you can access the value via the Config.value attribute:

from confattr import Config, ConfigFile

backend = Config('urwid.backend', 'auto', allowed_values=('auto', 'raw', 'curses'))

config_file = ConfigFile(appname=__package__)
config_file.load()
config_file.set_ui_callback(lambda msg: print(msg))

print(backend.value)

Given the following config file (the location of the config file is determined by ConfigFile.iter_config_paths()):

set urwid.backend = curses

The script will give the following output:

curses

Testing your application

I recommend doing static type checking with mypy in strict mode and dynamic testing with pytest. tox can run both in a single command and automatically handles virtual environments for you. While you can configure tox to run your tests on several specific python versions you can also simply use py3 which will use whatever python 3 version you have installed. For packaging and publishing your application I recommend flit over the older setuptools because flit is much more intuitive and less error prone.

For dynamic testing you need to consider two things:

  1. Your application must not load a config file from the usual paths so that the tests always have the same outcome no matter which user is running them and on which computer. You can achieve that by setting one of the attributes ConfigFile.config_directory or ConfigFile.config_path or one of the corresponding environment variables APPNAME_CONFIG_DIRECTORY or APPNAME_CONFIG_PATH in the setup of your tests.

    In pytest you can do this with an auto use fixture. tmp_path creates an empty directory for you and monkeypatch cleans up for you after the test is done. If all of your tests are defined in a single file you can define this fixture in that file. Otherwise the definition goes into conftest.py.

    import pytest
    import pathlib
    from confattr import ConfigFile
    
    @pytest.fixture(autouse=True)
    def reset_config(tmp_path: pathlib.Path, monkeypatch: pytest.MonkeyPatch) -> None:
    	monkeypatch.setattr(ConfigFile, 'config_directory', str(tmp_path))
    
  2. Your tests need to change settings in order to test all possibilities but all settings which have been changed in a test must be reset after each test so that the tests always have the same outcome no matter whether they are executed all together or alone.

    Of course you could just save a config file in the setup and load it in the teardown (and don’t forget to call MultiConfig.reset). But keep in mind that you may have many settings and many tests and that they may become more in the future. It is more efficient to let monkeypatch clean up only those settings that you have changed.

    Let’s assume we want to test our car from the first example:

    from sut import Car
    import pytest
    
    def test_car_accelerate(monkeypatch: pytest.MonkeyPatch) -> None:
    	monkeypatch.setattr(Car.speed_limit, 'value', 10)
    
    	c1 = Car()
    	c1.accelerate(5)
    	c1.accelerate(5)
    	with pytest.raises(ValueError):
    		c1.accelerate(5)
    
    	assert c1.speed == 10
    

    If we want to change the value of a MultiConfig setting like in this example for a specific object we would use monkeypatch.setitem() to change MultiConfig.values:

    from sut import Car, Color
    from confattr import ConfigId
    import pytest
    
    def test_car_color(monkeypatch: pytest.MonkeyPatch) -> None:
    	monkeypatch.setattr(Car.color, 'value', Color.WHITE)
    	monkeypatch.setitem(Car.color.values, ConfigId('alices-car'), Color.BLUE)
    	monkeypatch.setitem(Car.color.values, ConfigId('bobs-car'), Color.GREEN)
    
    	alices_car = Car(ConfigId('alices-car'))
    	bobs_car = Car(ConfigId('bobs-car'))
    	another_car = Car(ConfigId('another-car'))
    
    	assert alices_car.color is Color.BLUE
    	assert bobs_car.color is Color.GREEN
    	assert another_car.color is Color.WHITE
    

Environment variables

Settings can be changed via environment variables, too. For example if you have an application called example-app with the following code

from confattr import Config, ConfigFile
greeting = Config('ui.greeting', 'hello world')
ConfigFile(appname=__package__).load()
print(greeting.value)

and you call it like this

EXAMPLE_APP_UI_GREETING='hello environment' example-app

it will print

hello environment

For the exact rules how the names of the environment variables are created are described in ConfigFile.get_env_name().

Environment variables which start with the name of the application but do not match a setting (and are not one those listed below) or have an invalid value are reported as ERROR to the callback registered with ConfigFile.set_ui_callback().

Furthermore this library is influenced by the following environment variables:

  • XDG_CONFIG_HOME defines the base directory relative to which user-specific configuration files should be stored on Linux. [1] [2]

  • XDG_CONFIG_DIRS defines the preference-ordered set of base directories to search for configuration files in addition to the XDG_CONFIG_HOME base directory on Linux. The directories in XDG_CONFIG_DIRS should be separated with a colon. [1] [2]

  • APPNAME_CONFIG_PATH defines the value of ConfigFile.config_path. [2] [3]

  • APPNAME_CONFIG_DIRECTORY defines the value of ConfigFile.config_directory. [2] [3]

  • APPNAME_CONFIG_NAME defines the value of ConfigFile.config_name. [2] [3]