Stop Hardcoding Values in Python Apps - Use YAML Configuration Files Instead

Stop Hardcoding Values in Python Apps - Use YAML Configuration Files Instead

Hardcoded configurations are a terrible idea. Consider this alternative instead

Searching and replacing values in hundreds of Python files is even less fun than you might think. Having a single, central configuration file makes tweaking API URLs, database connections, and others so much easier. Today you’ll learn how to make your Python apps much more robust with YAML configuration files.

So, what is YAML? YAML stands for Yet Another Markup Language. It’s a data serialization language often used when writing configuration files. It’s popular in many programming languages because it’s super easy to read. Other file formats, such as JSON, are also human-readable, but they can’t match the simplicity YAML offers.

Today you’ll learn how to read and write YAML files in Python, and much more. But first, let’s install the thing.

How to Install YAML Python Library

YAML doesn’t ship with Python, so we’ll have to install it. The installation command varies, depending on whether you’re using Pip or Anaconda. Here’s an installation command for both:

Pip:

pip install PyYAML

Anaconda:

conda install -c conda-forge -y pyyaml

And that’s all we need to get started working with YAML files in Python.

How to Read YAML Configuration Files in Python

I’ve created a new folder with two files:

  • app.py - Contains Python code for today.
  • config.yaml - Contains configuration data we don’t want to hardcode in Python scripts.

As for the contents of the YAML file, think of them as key-value pairs. We’ll communicate with a remote REST API today, so the configuration file will contain the API endpoint and local paths where data will be saved:

api_url: https://gorest.co.in/public/v2/users
save_dir: /Users/dradecic/Desktop/users_data
save_file: users.json

The config file is much more readable than JSON because we got rid of all the curly brackets and indentations.

But how can you read this file in Python? Reading a YAML file in Python is best done with the context-manager syntax. The yaml.load() method will parse the contents of the file and store them as a Python dictionary object:

import yaml


with open("config.yaml", "r") as f:
    config = yaml.load(f, Loader=yaml.FullLoader)

print(config)

You can run the above code from the shell - here’s what you will see:

Image 1 - Reading YAML config files in Python (image by author)

Image 1 - Reading YAML config files in Python (image by author)

The configuration is stored as key-value pairs, which means accessing values will be extremely easy. Before we do so, let’s also explore how to write to YAML files from Python.

How to Save YAML Configuration Files in Python

Surround configuration data in triple quotes if you want to save YAML configuration files from Python. The code below shows you how to store two regular key-value pairs and another key that has a list as a value:

import yaml


yml_config = """key: value
another_key: another value
list_of_keys:
- value 1
- value 2
- value 3
"""

with open("configw.yaml", "w") as f:
    f.write(yml_config)

The corresponding YAML file looks like this:

Image 2 - Writing to YAML config files in Python (image by author)

Image 2 - Writing to YAML config files in Python (image by author)

From here, read it as described in the previous section.

How to Use YAML Configuration Files in Python Apps

We’ll now make a small Python script that connects to a remote REST API and downloads the data in JSON format. We already have URLs and paths in config.yaml file - just make sure to update the path to match your operating system.

The code snippet below makes a GET request to the API endpoint and saves the response locally. It also creates the directory structure if it doesn’t exist:

import json
import yaml
import pathlib
import requests


with open("config.yaml", "r") as f:
    config = yaml.load(f, Loader=yaml.FullLoader)


def get_users() -> dict:
    r = requests.get(config["api_url"])
    return r.text


def save_users(users: dict) -> None:
    path = pathlib.Path(config["save_dir"])
    if not path.exists():
        path.mkdir()

    with open(f"{config['save_dir']}/{config['save_file']}", "w") as f:
        json.dump(users, f)


if __name__ == "__main__":
    users = get_users()
    save_users(users=users)

Below you’ll see the contents of the users_data directory:

Image 3 - Saved JSON file (image by author)

Image 3 - Saved JSON file (image by author)

And that’s how you can integrate YAML configuration files into your Python projects. Let’s make a short recap next.

Summary of YAML Configuration Files in Python

Hardcoding values is never a good idea. Sure, it’s easy to change a couple of things in a single Python script, but imagine you had hundreds of them - it can easily become a nightmare, and you’re almost guaranteed to miss a couple of places.

Today you’ve learned how to work with YAML configuration files in Python. The entire setup is as easy as they come, but it’s not a be-all-end-all solution. For example, you can’t use programming logic in YAML files. The only workaround is to include the logic in Python scripts that generate YAML files, but that’s just not convenient.

This shortcoming is easily addressed by using Python files for configuration. Stay tuned if you want to learn more about that.

Stay connected