on the unfriendliness of bug reporting

[zwol]

There’s been a long discussion on one of the Mozilla mailing lists about how we can do a better job categorizing and triaging incoming bugs. This is my contribution, which I felt deserved a somewhat broader audience.

Attention conservation notice: thousand-word rant about the user friendliness of a bug database.

I, a developer, frequently have trouble figuring out which component I ought to file a bug under, especially when it’s not in one of the areas I work on—which is when it’s most important for me to get it right, so that it comes to the attention of people who do work on that area. It doesn’t help that, for any given bug, I might be supposed to file it under Firefox or under Core, each of which has a dauntingly long list that you have to scroll through via a five-line selection box. (I was hoping that the 3.2 upgrade would at least give me a bigger box, but it didn’t; showing the descriptions is nice but often the descriptions are just as cryptic to me as the names themselves.)

I happened to be complaining about this last night and my SO, who is a web designer, said that she’d completely given up on reporting bugs to us because she had no idea what product and component to select, and the new-bug form gave the impression that if you didn’t get that right you might as well not bother filing the bug at all! —She now has me proxy bug reports for her, which is fine and all but doesn’t scale. And if someone who already knows HTML from CSS from JavaScript and the difference between Firefox and Gecko can’t figure out how to label her bug reports, how can we possibly expect less informed users to?

So I think the user-facing problem—the product+component space is too big and daunting—is more serious than the possibility that QA won’t scale to handle increased triage responsibilities. If MozCo needs to hire some (more?) people whose entire job is to triage bugs, well, I have the impression there’s room in the budget for that.

Now, the component list isn’t the only daunting thing about the new-bug form. I’m looking at it now, and to get to the important part—the description-of-bug box—I have to scroll past an entire screenful of form fields. And I have a huge monitor. And I know that most of the time I can leave all those fields blank. The “bugzilla helper” is even worse; the first thing you see is a giant confusing table of already-filed bugs. I appreciate the attempt to reduce the number of dupes but I’d bet a dollar that lots of users see that and give up without even reading it!

Oh, and halfway down the bugzilla helper it tells you to report broken websites a totally different way. It’s good that we have a guided wizard built into firefox for that—but it’s very bad to let someone get halfway down the page before telling them, no, wait, stop, do something totally different! It’s also bad to demand that the user go try the latest development version before they do anything else, and for the first thing they see when they open the wizard (presumably first time only, but still) to be a minuscule box containing over a thousand words of legalese.

I think we could make life easier for both users and QA by doing some serious user interface redesign on the new bug forms. I’m imagining a directed question-and-answer session with no more than seven (plus or minus two) options at each stage, eventually leading up to one or two freeform entry boxes. The goal of this would be to do a rough-cut categorization that makes less work for human triagers, and at the same time tries to get enough information out of the user that we don’t need to go back for more. It should be clear to people unfamiliar with the terms, but not insulting to experienced users; I think if we do it right it would be easier for everyone than the current form.

For developers it might be good to have a special mode for when you’re filing a bug to describe a specific chunk of work that’s already planned. This could do things like deduce the correct component and suggest reviewers/cc: subscribers from file names or bug numbers mentioned in the description, or even in the patch that you’re attaching as you file the bug.

This is getting pretty far offtopic, but I’d also like to bring up a pet peeve of mine: I don’t think we should ever tell people to stop commenting on a bug. I do realize that when there are widely-cc:ed, widely-commented-on, long-standing, not-gonna-be-fixed-soon bugs, it’s no fun to be on the receiving end of all that mail, and the me-toos can obscure any actual technical discussion that might get the bug solved. Too often, though, I see that turning into hostility to users. A “leave us alone” response to a “hey this is still a problem for me” comment is a great way to ensure that that person never reports a bug again.

I think there ought to be a way for a bug to host a deeply technical conversation about how to fix it and a long train of me-toos at the same time, but I don’t have any great ideas, probably because I read too fast for it to be a serious problem for me.

Comments

[avatraxiom.livejournal.com]

We have some very good ideas between myself and Bugzilla's HCI person on how to fix this, that are too detailed to explain here.

-Max

[zwol.livejournal.com]

Are they written up anywhere? I'd be very interested to see them.

[avatraxiom.livejournal.com]

Unfortunately they're not yet. I had an in-person meeting with our HCI guy and we've done some on-paper sketches. Our HCI guy is really bright, and he has some great ideas about this, and it's pretty much our top UI priority.

-Max

[zwol.livejournal.com]

Well, lemme just say that I'm really glad to hear that you are working on this, and I'd like to hear about it when it's ready for a wider audience.

[avatraxiom.livejournal.com]

Sure, I'll probably blog about it. My Bugzilla-related posts are syndicated to Planet Mozilla, so you'll probably hear about it there. Also, Pyrzak, our HCI guy, has his posts syndicated there, I think.

-Max

We're on the Usability problem

[guy-pyrzak.blogspot.com (from livejournal.com)]

Hi! I'm glad to see this sort of info written out. You'll be happy to hear we're working on this. We've even gotten a group of HCI students to start working on the UI as well! So... if you happen to be in the Pittsburgh PA area, let me know, maybe they could do a usability study with you. Otherwise, stay tuned and you're sure to hear and see much more about the UI improvements we've got in store for Bugzilla!

Re: We're on the Usability problem

[zwol.livejournal.com]

I regret to say I live in San Diego. Just out of curiosity, though, is that CMU's HCI program you're working with?

[elisaana.livejournal.com]

So, you mention this whole non-scalable solution thing, eh? A friend just made the following grumpy post, which perhaps you can file appropriately, please? :)

"I just upgraded to Firefox 3 and I really really wish they would get feedback from people with worse than 20/20 vision before they take them out of beta. I appreciate that they want the Mac version to mesh with the Mac aesthetic, but come on, people! Dark gray background with small darker gray text on all the tabs? Brushed metal is just not very readable. Contrast is good, people. And the text is too small! I don't want icons in the tabs- I want bigger text. Or I want to be able to choose.

"And page zoom is a neat idea. It kind of makes more sense than text zoom. But it's obviously not designed with people who use text zoom regularly in mind. Page zoom ends up pushing things out side-to-side so you need to scroll side-to-side to read text. I'd gotten used to the formatting being all funky and broken when I zoom the text, but I really don't want to have to scroll for every line."

I think someone did suggest how to get a different theme than the gray-on-gray, but is the text zoom thing possible?

Anyway, I hope your suggestions for Bugzilla get some concrete results!

[elisaana.livejournal.com]

Oops, and more here. (Watch out for the C4... but at least they take your project seriously!)

[zwol.livejournal.com]

There's a tickybox in View > Zoom that makes it go back to text-only zoom - it really, really needs to become a per-site mode like the zoom level itself, though. Sometimes zoom-everything looks like ass, and sometimes zoom-just-text looks like ass, and it's not the page's fault either way (except for using unreadably small text in the first place :-P)

I can try to file your friend's bug report but it would help if they were willing to take a screenshot and then mark on it which bits they find hard to read. I don't have a Mac, and even if I did, I'd be guessing ...

[claudiopeedin.livejournal.com]

How about a dark grey or grey. I think most ppl would agree with this. Save our eyes from the strain.

Bug reporting UI & project components

[(Anonymous)]

When I saw the title of this post I thought it might be about project-independent web app UI, one reason people give for switching from Bugzilla to Trac, Jira, FogBugz, and other tools. But it sounds like the root of the UI confusion is the natural cruft and confusion of a long-standing development project that's changed its processes & architecture without a wholesale revamping of its bug reporting. Your wizard-ish suggestion sounds like a good solution, and the process of creating it would probably get other important decisions made along the way (priorities, architecture, workflow, etc.).

-Sumana

Re: Bug reporting UI & project components

[zwol.livejournal.com]

I think it's a bit of both, really - bugzilla's generic UI really isn't very good for bug submitters, some of those deficiencies show up much worse when you have a project this big and complicated, and bugzilla management doesn't always keep up with changes in the rest of the project...

On usability of bug reporting systems generally

[(Anonymous)]

The one thing I hate above all else is having to create an account. Really, if you're serious about quality then you shouldn't give a **** who the data comes from. If I'm frustrated enough with the quality of your software to report a bug and you put obstacles in the way of me doing that then I'm going to go to my fave search engine and find an alternative product instead.

Re: On usability of bug reporting systems generally

[zwol.livejournal.com]

I don't like that either, but I think removing it is going to be a really hard sell in this modern world of spam and more spam.