Frequently Asked Questions

Why the 2 in Brightway2?

Brightway version 1 was a set of programs written during the PhD of Chris Mutel. In the end, there wasn’t that much that could be re-used from this research code, other than the name. Brightway2 is a completely new software framework designed to avoid the fate of Brightway version 1. See also the blog post on technical motivation.

General Python questions

What text editor or IDE should I use?

When you are doing more complex work, it is often nice to work with an intelligent text editor or an integrated development environment (IDE).

Here are some text editors which are often recommended:

Here are some IDEs which are often recommended:

See also the discussion on the Python wiki and a long StackOverflow discussion.

Data management

Which ecoinvent file should I download?

Ecoinvent makes several versions of each system model available:

  • ecoinvent 3.3_xxx_ecoSpold02.7z
  • ecoinvent 3.3_xxx_lci_ecoSpold02.7z
  • ecoinvent 3.3_xxx_lcia_ecoSpold02.7z
  • ecoinvent 3.3_xxx_lcia-cumulated-matrices_xls.7z
  • ecoinvent 3.3_xxx_lci-cumulated-matrices_xls.7z

You want to download and import ecoinvent 3.3_xxx_ecoSpold02.7z. If your import process is taking a long time or a lot of memory, double check to make sure you have the right version.

How do I resolve errors about read-only projects?

Brightway2 uses the fasteners library to create lock files that conflicts in written data. More specifically, when you use projects.set_current to switch to a new project, Brightway will read a bunch of metadata about things like project databases and LCIA methods into memory. It will then try to acquire a lock that gives it the exclusive right to change this metadata in the future. If a second Python kernel switches to the same project, it will load the latest saved version of the metadata, and will then fail to acquire the lock, so will be set to read-only mode (i.e. projects.read_only = True).

This locking was introduced because people were having trouble with multiple people working on the same project and getting conflicts or even corruption in the metadata. The problem is easy to understand - if two kernels load the metadata into memory, and each make a change somewhere, then whoever saves the latest will overwrite the changes of the other.

You can run into problems if a Python kernel is not shut down properly. To be safe, you should always manually shut down each Python shell and each Jupyter notebook. In extreme cases, it may be necessary to manually delete the lock file or restart your computer.

If you are sure that it is safe to write data, then you can trick the projects object into allowing writes - just set projects.read_only = False. But be careful that you only do this in a single Python kernel.

How do I backup my data?

Each project is just a subdirectory (you can get the path using projects.dir), and this can be backed up using any normal tools, including cloud backups like Dropbox.

You can save a snapshot of entire project directory with backup_data_directory, or save a single project with backup_project_directory. Both functions return the filepath of the created archive file.

You can also:

  • Save any data object (like a Database or Method) to a BW2Package using BW2Package.export_objs.
  • Export a database to Excel using bw2io.export.excel.write_lci_activities.

How do I find where my data is saved?

You can find the current project data directory with the command projects.dir; everything will be stored in this folder or a subdirectory. Similarly, you can find the logs directory with the command projects.logs_dir.

Brightway2 uses the appdirs library to select an appropriate, platform-specific path, namely:

  • On Windows: C:\Documents and Settings\<User>\Application Data\Local Settings\pylca\Brightway2
  • On OS X: /Users/<User>/Library/Application Support/Brightway2
  • On Linux: /home/<User>/.local/share/Brightway2

You can specify a custom data directory path by setting the environment variable BRIGHTWAY2_DIR , but this is not recommended for normal use, unless you have multiple installations of brightway2 and want to have a separate data directory for each.

Setting BRIGHTWAY2_DIR in a virtual environment

When you create a virtual environment, it lives in a folder that by default is something like <YOUR_ANACONDA_INSTALL_DIR>/envs/<YOUR_ENV_NAME>. When conda activates or deactivates an environment, it looks for additional scripts in the two subfolders activate.d and deactivate.d within this folder. When virtualenv activates or deactivates an environment, it uses the scripts activate and deactivate in <YOUR_ENV>\bin\.

When using conda, to set the persistent environment variables in the virtual environment, do the following. If using virtualenv, instructions are roughly similar but will need some adaptation.

1.Navigate into your virtual environment folders just doing (both Mac/Linux and Windows):

cd <YOUR_ANACONDA_INSTALL_DIR>/envs/<YOUR_ENV_NAME>

2.Create the two aforementioned folders.

  • For Mac/Linux, type in the terminal:
mkdir -p etc/conda/activate.d
mkdir -p etc/conda/deactivate.d
  • For Windows the command to make folders is slightly different :
mkdir etc\conda\activate.d
mkdir etc\conda\deactivate.d

3.Create scripts in those folders that set and unset the environment variables (in this case BRIGHTWAY2_DIR). The names of the files don’t matter, but the file extensions do.

  • For Mac and Linux, the extension must be .sh files. Inside the folder activate.d create the file whatever_name_you_like.sh and inside it write export BRIGHTWAY2_DIR=/my/custom/directory while in activate.d create whatever_name_you_like.sh and inside write unset BRIGHTWAY2_DIR.
  • For Windows the procedure is exactly the same, you just need to change the file extension from .sh into .bat i.e. instead of whatever_name_you_like.sh use whatever_name_you_like.bat

How can I rename projects?

You can’t. However, you can quickly copy a project to the new name (projects.copy_project("my new name")), and then delete the original (projects.delete_project("old name")).

Data formats

Why are activity dataset keys so confusing? (‘ecoinvent 2.2’, ‘5bbf...’) seems insane!

It is insane, in the sense that it doesn’t make any sense at all to people. Rather, 5bbf2e66f2d75d60726974ac44ab4225 is a computer-generated unique ID. The basic problem is that we need one unique ID for an activity dataset, but there is no ID provided in the ecospold 1 data format. Instead, an activity is uniquely identified by its name, location, category, subcategory, unit, and whether or not it is an infrastructure process! 5bbf2e66f2d75d60726974ac44ab4225 is just an easy way of representing all this information in one string. It is a pain, but there is no good way around it.

Unfortunately, ecospold 2 (the data format used in ecoinvent 3) isn’t more approachable - keys will now look like (‘ecoinvent 3’, ‘fff06f42-6c5f-4aea-b695-93bcaba55fed’). Sorry. At least this time it is ecoinvent generating the unique ID, and not Brightway2.

Why pickle? Serialization X is so much better!

The Python standard library module pickle is the default data storage format for most data. Windows people in particular have slow load times, but also pain in installing things, so adding new dependencies is strongly discouraged.

The pickle module is fast, portable, and built-in. While using compression (such as gzip and bzip2) would reduce the size of the saved files, it also dramatically increases loading and saving times, by a factor of 3 - 30, depending on the test. Overall, the speed of pickle seems to be fine.

The marshal module is faster - 40% faster writing, 25% faster reading - but produces files twice as big, and can change from computer to computer or even when Python is upgraded. The costs and potential risks of marshal overwhelm its speed gains.

Javascript object notation (JSON) is a data for native to javascript which is now widely used for data exchange over the web and between different programming languages. JSON does not match perfectly to python data structures, but the differences are relatively small. JSON is used to store some metadata in Brightway2, such as the user preferences, and the installed LCI databases and LCIA methods. JSON is human readable and editable.

While a JSON module is in the standard library, there is no fast JSON library available for all operating systems and python version; see e.g. anyjson, yajl, and ujson, in addition to the builtin json. Each of these libraries is also not 100% compliant with the JSON spec.

Things like message pack and JSON can’t handle all Python datatypes, and in particular Python allows tuples as dictionary keys, which we use heavily, while others don’t. So, pickle is the default format, even though it is not the hawtness... However, JSON is used as a backup format, as pickle has real drawbacks for archiving.

See also:

Storing Python objects in a SQLite3 database is silly! Why not use X document database?

Where X is one of MongoDB, CouchDB, UnQLite, Vedis, CDB, TinyDB, etc.

This approach may seem strange at first, but is the result of coding, evaluating, and ultimately rejecting several alternatives. Most document databases can’t store all Python objects directly, because they use JSON or some other serialization. We have actually built and tested database backends built on pickle files, JSON files, MongoDB, CodernityDB, and BlitzDB. SQLite3 also has several real advantages:

  • Most importantly, it is included with Python, no new dependencies or installation steps are required.
  • It is famous for being well tested, and is completely cross-platform.
  • It is also more than fast enough. For example, loading every activity from ecoinvent 3+ takes only a few seconds.

Problems

I found a bug! What now?

First, please check the list of Known Issues. However, if your issue isn’t listed, by all means please create a bug report. Here is some good advice on creating a short, self contained, correct example for a bug report.

It is too slow!

If your numerical work after LCA calculations is slow, consider the numexpr and Bottleneck libraries.

I get unicode errors!

Note

All strings should be unicode. In Python 2.7, they have a ‘u’ in front of the string, like u"foo"; in Python 3, all strings are unicode. If you are careful to make sure your data is unicode, you shouldn’t have this problem.

Note

You can specify the encoding of text in your python files as UTF-8 by putting the following as the first line in each file: # -*- coding: utf-8 -*-

A typical error message is:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 426: ordinal not in range(128)

The problem here is that python tries to convert a character from unicode to an encoding which doesn’t support that character. A common default encoding in python 2.X is ascii, which doesn’t support much. You can fix this by changing the default encoding:

import sys
reload(sys)
sys.setdefaultencoding("utf-8")

For understanding the difference between bytestrings and unicode:

First, read What actually changed in the text model between Python 2 and Python 3? - a very understandable and detailed description of what the title says.

Then, see the following resources:

For help in fixing strings:

When upgrading on Windows, I get errors about something called vcvarsall.bat

Note

The Upgrading Brightway2 docs avoid this problem by always using pip with --no-deps.

The problem here is that pip -U install foo will try to upgrade all dependencies of foo. If, for example, scipy is a dependency, and a newer version is available, then pip will try to compile it. Compilation of scipy requires a C compiler, which is why python looks for vcvarsall.bat, which you don’t have.

If you are using something like conda, you should first make sure that all of your libraries are up to date already. Usually they will build the difficult packages so that you don’t have to. In many cases, this should solve the problem, as you will then have the latest version of your dependencies.

If this doesn’t solve the problem, then you have two options:

First, you can tell pip not to update all the dependencies. For example, to get the latest version of foo, you would run:

pip install -U --no-deps foo

Second, you can try to install a C compiler. You can find decent instructions online, as well as discussion on Stack Overflow.

The global warming potential values are different in SimaPro!

The default LCIA characterization factors in Brightway2 come from version 3.2 of the ecoinvent database. For most LCIA methods, these are identical to those found in SimaPro. However, there are important differences for global warming potential:

  1. SimaPro does not include a characterization factors for carbon monoxide, but ecoinvent does. Here is the ecoinvent language:

    Emitted CO is transformed in the atmosphere to CO2 after some time. Not all LCIA methods do consider the global warming potential of CO. Most methods are based on factors published by the IPCC (IPCC 2001). It is assumed that CO2 emissions are calculated with the carbon content of the burned fuels and thus all carbon in the fuel is considered. In ecoinvent CO emissions are subtracted from the theoretical CO2 emissions. Thus a GWP factor is calculated for CO (1.57 kg CO2-eq per kg CO). Otherwise processes with higher CO emissions would benefit from this gap. This is especially important for biomass combustion. Neglecting the formation of CO2 from CO would lead in this case to a negative sum of the global warming potential score.

The value of 1.57 is the ratio of the molecular weights of CO2 and CO.

  1. SimaPro gives biogenic methane a characterization factor of 22 kg CO2-eq, while ecoinvent gives 25, the same value as for other types of methane.

Note

There may be other differences as well - these are the ones we have found.

References:

Why do I get negative results in ecoinvent 3?

Depending on the LCIA method and functional unit, some LCA scores might be negative. Not everything is bad for the environment in every impact category!