How to Export and Load Anaconda Environments for Data Science Projects

How to Export and Load Anaconda Environments for Data Science Projects

Sharable Python virtual environments with a single Terminal command

Setting up virtual environments from scratch every time is a tedious and time-consuming process. There’s a way around it. You can use Anaconda to export a Python virtual environment to a YAML file you can then reuse in different projects. You can also share the YAML file with your team, so everyone’s on the same track.

Today you’ll learn how. It’s easier than you think, so let’s get right to it.

Don’t feel like reading? Watch my video instead:


Create and Setup an Anaconda Environment

Anaconda makes it extra easy for data scientists to set up virtual environments. I’m using a scaled-down version called Miniforge, which doesn’t ship with a ton of libraries preinstalled. It doesn’t matter which flavor you’re using.

We’ll start by creating a new environment named test_env based on the latest version of Python 3.9. Open up a Terminal window and type in the following:

conda create --name test_env python=3.9 -y
Image 1 — Creating an Anaconda environment (image by author)

Image 1 — Creating an Anaconda environment (image by author)

The initial setup won’t take long. You’ll see the following message displayed after a couple of seconds:

Image 2 — Creating an Anaconda environment (2) (image by author)

Image 2 — Creating an Anaconda environment (2) (image by author)

Which means the environment is successfully installed. You can activate it with the following command:

conda activate test_env
Image 3 — Activating an Anaconda environment (image by author)

Image 3 — Activating an Anaconda environment (image by author)

You now have everything needed to install a couple of libraries. Let’s stick with the basic ones — Numpy, Pandas, Matplotlib, Scipy, Scikit-Learn, and Jupyter:

conda install -c conda-forge -y numpy pandas matplotlib scipy scikit-learn jupyter jupyterlab
Image 4 — Installing Python libraries (image by author)

Image 4 — Installing Python libraries (image by author)

This could take a while, depending on your Internet speed. Anaconda has to download all listed libraries and their dependencies, so let it do its thing. Once done, you should see something similar displayed:

Image 5 — Installing Python libraries (2) (image by author)

Image 5 — Installing Python libraries (2) (image by author)

That’s how you’d normally approach setting up virtual environments from scratch. Let’s see how you can export it next.

How to Export and Load Anaconda Virtual Environments

Anaconda allows you to export a virtual environment into a YAML file. It’s simple to understand data serialization language often used to create configuration files. YAML is similar to JSON, but without brackets.

Use the following command to export the environment, just remember to modify the export path:

conda env export > \Users\Dario\Desktop\test_env.yaml
Image 6 — Exporting an Anaconda environment (image by author)

Image 6 — Exporting an Anaconda environment (image by author)

You can open the YAML file with any text editor, for example, Visual Studio Code. Here’s what it contains:

Image 7 — Environemnt YAML file (image by author)

Image 7 — Environemnt YAML file (image by author)

The file contains complete instructions on how to set up an identical environment with Anaconda. The only problem is the environment name displayed on the first line. You’ll have to change it as you already have an environment called test_env. Let’s keep things simple and rename it to test_env_2:

Image 8 — Changing the environment name (image by author)

Image 8 — Changing the environment name (image by author)

You can now use the following command to create a virtual environment from the YAML file:

conda env create -f \Users\Dario\Desktop\test_env.yaml
Image 9 — Creating an environment from YAML file (image by author)

Image 9 — Creating an environment from YAML file (image by author)

The command shouldn’t take long to execute, as you already have all the libraries downloaded. Once finished, you’ll see a similar message telling you how to activate the environment:

Image 10 — Creating an environment from YAML file (2) (image by author)

Image 10 — Creating an environment from YAML file (2) (image by author)

Use the displayed command to activate the environment:

conda activate test_env_2
Image 11 — Activating a new Anaconda environment (image by author)

Image 11 — Activating a new Anaconda environment (image by author)

Finally, let’s verify we have the Python libraries installed. Start a Python shell and import everything previously installed — Numpy, Pandas, Scipy, Matplotlib, and Scikit-Learn:

Image 12 — Verifying library installations (image by author)

Image 12 — Verifying library installations (image by author)

It seems like everything works as expected, you have a new environment packed with the dependencies from the YAML file. Easy, right?

Conclusion

And there you have it — reproducible Anaconda environment without breaking a sweat. You can use them to set up base environments you can then tweak according to the project needs. For example, pretty much every data science project requires the libraries we installed today, so why not save yourself a couple of minutes.

Another use case is the ability to share the YAML file with your team, so everyone is on the same track.

Either way, it’s a good-to-know feature of Anaconda.


Stay connected