Found a regression in Firefox? Give us details with mozregression!

One of the most useful things you can do to help improve the quality for Firefox is to warn developers about regressions and give them all the details that will turn your issue into a useful actionable bug report.

This is actually the raison d’être of Nightly builds: parallelizing the work done by our communities of testers and developpers so as to catch regressions as early as possible and provide a feedback loop to developers in the early stages of development.

Complementary to nighly builds, we also have a nifty Python command line tool called mozregression (with an experimental GUI version but that will be for another post…) that allows a tester to easily find when a regression happened and what code changes are likely to have introduced the regression. This tool is largely the result of the awesome work of Julien Pagès, William Lachance, Heather Arthur and Mike Ling!

How do I install that cool tool?

tl;dr for people that do Python:
mozregression is a python package installable via pip, preferably in a virtualenv.

The installation steps detailed for mozregression on their documentation site are a bit short on details and tell you to install the tool with admin rights on linux/mac (sudo) which is not necessarily a good idea since we don’t want to potentially create conflicts with other python tools installed on your system and that may have different dependencies.

I would instead install mozregression in a virtualenv so as to isolate it from your OS, it works the same, avoids breaking something else on your OS and AFAIK, there is no advantage to having mozregression installed as a global package except of course having it available for all accounts on your system.

The installation steps in a terminal are those:

sudo apt install python-pip virtualenvwrapper
echo 'export WORKON_HOME=~/.virtualenvs' >> ~/.bashrc
source ~/.bashrc
mkvirtualenv mozreg
pip install mozregression
mozregression --write-config

Here are line by line explanations because understanding is always better than copy-pasting and you may want to adapt those steps with your own preferences:

sudo apt install python-pip virtualenvwrapper

» Install the software needed to install python packages (pip) as well as the tool to create virtual environments for Python, the packae names here are for Debian-based distros.

echo 'export WORKON_HOME=~/.virtualenvs' >> ~/.bashrc

» Add to your bash config (adapt the destination if you use another shell) an environment variable indicating where your virtualenvs are stored in your home.

source ~/.bashrc

» Reload your updated .bashrc settings

mkvirtualenv mozreg

» Create a virtualenv called mozreg in ~/.virtualenvs/mozreg/. Your terminal should get the name of this virtualenv prepended to your terminal prompt such as (mozreg) ~ $>, that means you are in this virtualenv now.

pip install mozregression

» The previous step activated the mozreg virtualenv automatically, this step installs mozregression in ~/.virtualenvs/mozreg/lib/python2.7/site-packages/mozregression

mozregression --write-config

» In this final installation step, we are creating a configuration file for mozregression, the defaults are fine and detailed here.

I am set up, how do I use the tool?

Now that mozregression is installed in a virtualenv, first you need to know how to activate/deactivate your virtualenv.

As said previously, if you are already in your virtualenv, your terminal prompt will have (mozreg) mentionned. If you want to exit your virtualenv, type:

deactivate

If you are not in your virtualenv, you can enter it with this command:

workon mozreg

You can now use mozregression following the Quick Start steps in the documentation but basically it boils down to indicating a good and a bad date for Firefox and the bad date is optional, so if you found a regression in Nightly that you know didn’t exist in say Firefox 49, launching mozregression is as easy as:

mozregression --good 49 

Then test the nightly builds that mozregression downloads for you and indicate in the terminal ‘good’ or ‘bad’ for every build you try until the regression is found and you will get a message such as:

INFO: Last good revision: e3cff5e3ae3cc1b20d4861e7934e54c9e407a750
INFO: First bad revision: 5194eeb6ca2d34b5fca72d2a78d6a9ec95a36763
INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=e3cff5e3ae3cc1b20d4861e7934e54c9e407a750&tochange=5194eeb6ca2d34b5fca72d2a78d6a9ec95a36763

INFO: Looks like the following bug has the changes which introduced the regression:
https://bugzilla.mozilla.org/show_bug.cgi?id=1296280

The above text is what you should paste in the bug (for the record, this specific example is copied from bug 1307060), this is the information that will make your bug report actionable by a developer.

In a nutshell

Mozregression is a tool that allows power-users with no development skills to participate actively in the development of Firefox.

Providing such information to developers saves them a lot of time and most importantly, it allows bugzilla triagers to put the right developers in copy of the bug (“Hey Anne, it seems your patch in Bug XXX caused the regression described in this bug, could you have a look please?”)

Being a developer is not the only way you can help Firefox being a better browser, any advanced user able to use a terminal on his machine can participate, so don’t hesitate, file bugs and become a regression range finder!