Training machines to triage Firefox insects

Many bugs, not enough triage

Mozilla receives hundreds of pest reports and feature requests from Opera users every day. Getting bugs towards the right eyes as soon as possible is essential to be able to fix them quickly. This is where annoy triage comes in: till a developer knows an insect exists, they won’ t have the ability to fix it.

Given the top number of bugs filed, it is impractical to make each developer look at every single bug (at the time of creating, we’ d reached bug amount 1536796! ). This is why, on Bugzilla , we group bugs simply by product (e. g. Firefox, Firefox with regard to Android, Thunderbird, etc . ) plus component (a subset of a product, electronic. g. Firefox:: PDF Viewer).

Historically, the product/component project has been mostly done manually simply by volunteers and some developers. Unfortunately, this technique fails to scale, and it is effort that might be better spent elsewhere.

Introducing BugBug

To assist get bugs in front of the right Opera engineers quickly, we developed BugBug, a machine learning tool that will automatically assigns a product and element for each new untriaged bug. Simply by presenting new bugs quickly in order to triage owners, we hope to decrease the particular turnaround time to fix new problems. The tool is based on another technique that we get implemented recently, to differentiate in between bug reports and feature requests. (you actually can read more about this at https://marco-c.github.io/2019/01/18/bugbug.html ).

High-level structures of BugBug training and procedure

Training an auto dvd unit

We have a large instruction set of data for this model: 2 decades worth of bugs which have been evaluated by Mozillians and assigned in order to products and components.

Obviously we can’ t use the frustrate data as-is: any change towards the bug after triage has been finished would be inaccessible to the tool throughout real operation. So , we “ roll back” the bug towards the time it was originally filed. (This sounds easy in practice, but there are a great number of corner cases to take into account! ).

Also, although we have a large number of components, we only really worry about a subset of these. In the past ~2 years, out of 396 components, just 225 components had more than forty-nine bugs filed. Thus, we limit the tool to only look at elements with a number of bugs that is a minimum of 1% of the number of bugs from the largest component.

We all use features collected from the name, the first comment, and the keywords/flags connected with each bug to train an XGBoost model.

High-level overview of the BugBug model

During operation, we all only perform the assignment once the model is assured enough of its choice: at the moment, we are using a 60% self-confidence threshold. With this threshold, we are able to give the right component with a very low fake positive ratio (> 80% accuracy, measured using a validation set of insects that were triaged between December 2018 and March 2019).

BugBug’ s results

Training the model on 2+ years of data (around 100, 500 bugs) takes ~40 minutes on the 6-core machine with 32 GB associated with RAM. The evaluation time is within the order of milliseconds. Considering that the tool does not pause and it is always ready to act, the tool’ s assignment speed is much quicker than manual assignment (which, normally, takes around a week).

Since we deployed BugBug within production at the end of February 2019, we’ ve triaged around 350 insects. The median time for a creator to act on triaged bugs can be 2 days. (9 days will be the average time to act, but it’ s only 4 days whenever we remove outliers. )

BugBug for

Plans for future years

We have plans to make use of machine learning to assist in other software program development processes, for example:

• Identifying duplicate bugs. Regarding bugs which crash Firefox, customers will typically report the same pester several times, in slightly different ways. These types of duplicates are eventually found by triage process, and resolved, yet finding duplicate bugs as quickly as possible offers more information for developers trying to detect a crash.
• Providing extra automated help for developers, like detecting bugs in which “ procedure for reproduce” are missing and inquiring reporters to provide them, or discovering the type of bug (e. g. overall performance, memory usage, crash, and so on).
• Detecting bugs that could be important for a given Firefox release as soon as possible.

Right this moment our tool only assigns elements for Firefox-related products. We would like to increase BugBug to automatically assign elements for other Mozilla products.

We also encourage additional organizations to adopt BugBug. If you use Bugzilla, adopting it will be very easy; otherwise, we’ ll need to add support for the bug tracking system. File a problem on https://github.com/mozilla/bugbug plus we’ ll figure it out there. We are willing to help!

If you liked Training machines to triage Firefox insects by Marco Castelluccio Then you'll love Web Design Agency Miami