Going over the wire redux

This is bugging me, and its been bugging me for a while. In nearly three years, we haven’t built up a community of hackers around Firefox, for a myriad of reasons, and now I think we’re in trouble. Of the six people who can actually review in Firefox, four are AWOL, and one doesn’t do a lot of reviews. And I’m on the verge of just walking away indefinitely, since it feels like I’m the only person who cares enough to make it an issue. Things I’ve raised in relatively private contexts have gone unanswered, and I’m growing increasingly cynical about our ability to ship 1.1 early enough to make 1.5 remotely viable this year. At the glacial pace of development we’re currently running at, I don’t see how the ambitious plans for 2.0 are going to be at all viable before late 2006. Maybe I’m wrong, but no one’s bothered to take five minutes and tell me that.

People have brought up hiring, but that’s one of the suggestions that apparently didn’t merit a response, so I’m not going to hold my breath on MoFo staffing someone to work on Firefox and shoulder some of the load. I won’t pretend to understand MoFo’s staffing priorities, I don’t work there, I’m out of the loop in a lot of ways, so I’ll have to assume they have some sort of plan. I’m _hoping_ they have some sort of plan. But until they tell me how we’re going to ship a product with the same size buglist as we had last August, with maybe 1/4 of the core resources, I’m going to be cynical. And while I’m cynical, I’ll be on walkabout, because this isn’t fun anymore.

Update #2: More clarification can be found here.

Edit: Hello Slashdot! Without trying to whore myself, I must say that I’m quite happy with my hosting provider (Dreamhost) right about now, both the “staying alive without a blip” factor and the fact that I won’t come anywhere close to my monthly bandwidth limit even now.

122 Comments

  1. XFree86 guy says:

    The problem with firefox/mozilla now is the same one with XFree86. Th problem is ‘CORE developers’. If you think you are so much a better developers who are different from all the other developers, then your project is in trouble of attracting new hackers.

    There should not be a selected core developers. The developing process should be open and transparent to all the community. The ‘core developers’ should not hide behind a closed door and do their magic inside.

    The solution? Look at what Keith have done to the XFree86? He understand he cannot change the mind of those elitism. So he started the X.org effort and X.org is moving fast with more and more contributions from all over the community.

    People want to contribute, even for huge and messy projects like X11, but the core developer system just closed the door!

    Debian’s 1000+ developers and still growing model is better. Else, you have to find a patching maniac like Linus as the project leader.

    Subversion cannot solve the problem. This is a typical mis-managed organization. The structure itself was rotten.

  2. XFree86 guy says:

    For those of you complaining that it’s hard to get into the “close knit family” of devellopers, or to get permission to edit things – the very fact that you’re sitting here on Mike’s blog debating this is proof that you are wrong.

    We are not wrong. Look at the history (flame war) of the spliting of XFree86. The core team try to keep their elitism status by opening up its developing maillist. After a couple of years, nothing changed. Because they don’t understand or admit that it’s the core team system which is THE problem. mailing list or blog are just tools. Tools cannot fix a wrong system though a good system can use a better tools.

    Anyway, any who are inteseted in finding out what’s wrong with firefox, please read past log of XFree86’s devel mailing list.

  3. Majken says:

    Well then, how do you explain Mike’s presence and credibility here? He went to work, did what he could, and respected the way the team wanted things done. Maybe that’s really the part that’s hard? You think things should be done a different way, and you want to be able to do things that way no matter what? It’s not your project. In the end, someone’s got to run it, and they’ve got to run it the best way they know how. In terms of the product, Ben’s way is working.

    As for the issue of manpower, is that Ben’s fault or MoFo’s? Or is it the fault of the people that want free reign because it’s opensource? At least some of the guys that have requested checkin access have come right out and said they’d use that access to push more stuff through with a less careful review process, then when someone has tried to explain to them “look, we need to do things this way, keep working on bugs and we’ll think about it” they’ve thrown bloody fits. They’re like my parents – they come in saying they want to “help” but by the time they’re done the pots are in the plate cupboard, the kids’ clothes are in my drawers and to this day we can’t find things that were on the coffee table when they got here. By the time we took everything apart and put it back where it belonged we’d done more work than if we’d just done it ourselves in the first place.

    There is something about the words “help” and “free” that give people such a sense of entitlement, especially when used together. Firefox doesn’t need a revolution, firefox IS the revolution and while it needs help, it needs its pots in the pot cupboard and it needs people that can understand that.

  4. Brian Hamlin says:

    This is project burnout – its a management issue. MoFo, get it together NOW.

    Congrats on a product that is shaking the world.

  5. hank says:

    Lots of us want to encourage Firefox development.
    Consider putting out a tip jar? I’m serious, how else can outsiders recognize and encourage developers?

    You’re counting on the pleasure of doing it to reward people? Sure, but — you’re ignoring the HUGE number of us benefiting who have no way to reward/reinforce/thank/support you.

    Employers have quietly started allowing people like me to use Firefox since 1.0 came out, because they’re aware that slight competence using a computer, plus Firefox, is — competence.

    As compared to no competence using a computer plus IE, which requires their spending several staff salaries just cleaning up what Explorer, despite the company firewall and email filters and networked antivirus lets onto the company’s PC network, every single day.

    If people are AWOL, either they’ve been burned out (which allowing a tip jar might help with), or bought off (maybe ditto), or kidnapped (got a ransom note? We’ll help pay) …

    I’d been watching the official FF forums where nothing was very clear except developing confusion. This helps explain why.

    Other than that, if the problem is that Mozilla is more fun to develop, and Firefox is overwhelmed with users but not fun to develop, try rewarding people for moving to Mozilla!

    —> I’d switch to Mozilla, if I knew AdBlock worked in it.<---

  6. Gabriele says:

    “Without trying to whore myself”

    You’re true, few whores get 97$ per work!

  7. Majken says:

    Hank – there is a donate button on the Mozilla site and in the Mozilla store. As for directing praise to a specific developer, you can always email them, or send of an email to the Mozilla Foundation. It’s not huge, but it’s something.

  8. Hacker says:

    I’m a part-time hacker of Firefox and I have to admit I am getting disillusioned as well. Having a minor (one liner) patch that takes over a week to get reviewed is very disheartening and I get the sense that no one cares. Some patches have been sitting there for months – bit-rotted and hacker gone.

    I think more reviewers/sr are needed, and developers trained up to roles – not just ‘figure it out for yourself and you might get invited’. If people are encouraged, they’re more willing to put in extra time and effort.

  9. Mike says:

    I’ll be blogging soon on the plans to make this better. We do need to invest in talent, and not just hope it comes along ready-made. But that involves a serious investment of resources, and a plan. More on this soon.

  10. nomade says:

    Mike, I’m sure not a hacker, but I want to tell you many people help and are around the firefox project. Those people are not known by people like you, because they are not hackers, not famous, they just do a day-to-day job in forums to help newbies with firefox, thunderbird, etc.
    To take the french exemple of Geckozone, it works really well, and it’s not an easy job, even if there is less pressure and if it doesn’t requiere as much (technical) skill as you have.

    This message doesn’t mean “keep enduring the problems“, it just means “please try to solve them, because there are many people involved around the world“. Bye!

  11. Dawk says:

    Time to elicit some corporate $’s & take the bull by the horns!

  12. Tyrone says:

    I’m pissed off at what I read concerning Mozilla/Firefox continuing development. I JUST download Firefox a few days ago. It’s the BEST web browser I ever used! I just LOVE IT! It’s clean, simple, easy to use and faster than IE. I tested the speed against even Opera and it’s faster than Opera! The webpages takes seconds compare to minutes to load. Man, I hope I did not make a mistake and download the BEST browser around only to have it trashed like garbage! Do you realize how many fans there are of Mozilla???? It’s like you are forcing people to switch back to that slow, pity-ful, forever unsecured IE. What are you people thinking about?? What is your MAJOR mal-adjustment???? Mozilla can BEAT IE completely and ANY other browser!!!! You have a faithful following. I REALLY believe you people should seriously re-think this and fully develop the next release of Mozilla suite. Let’s make this the most secured, reliable browser built. Keep giving IE HEADACHES!

  13. […] 12 Marzo 2005

    Segun menciona uno de los desarrolladores de Firefox en su blog, el navegador esta avanzando muy lento en su desarrollo comparado con su crecimiento, ya que a pes […]

  14. […] 3/11/2005 – 6:07 pmMozilla Myths and Clarifications In the week since my now-infamous-and-slashdotted blog post, I’ve had a lot of discussions with a lot of people, […]

  15. Michael says:

    As an OSS developer of a small project I can understand that symptoms of burnout appear after the first adrenalin boost of the initial release. Most OSS projects die after the initial version as the main developer looses interest in the now solved problem. You made your vision materialize and … that’s it, let’s move on to the next cool new technology.

    Maintaining a project and keeping it alive is still much more difficult than starting a new one. Most of the great developers I know don’t like to maintain source code.

    It’s really frustrating that an OSS project like Firefox that got so much media buzz now seems to suffer from these symptoms, too. Come on guys, you have some gold in your hands which took the world by storm. Stand up and face the thread of IE7.

    IBM, Sun or Google: open up your vaults and give some money to Firefox. They can really tickle Big Bill 🙂

  16. unknown says:

    >IBM, Sun or Google: open up your vaults and give some money to Firefox.

    What would IBM, Sun and Google gain by supporting FF?

  17. Dave Bacher says:

    >re-think this and fully develop the next release of Mozilla suite.

    If you are running FireFox, you aren’t running the Mozilla suite. They are retiring the Mozilla Suite, which is a single bloated application which contains the Mozilla Web Browser, the Mozilla Web Composer, Mozilla e-Mail and Chatzilla (I might be missing a component or two, it’s bloated and it’s hard to remember everything).

    There are far better and far more popular web design tools, and in this XML, ASP and ASP.NET age, Composer needs a lot of work to be viable. On the other side of the coin, applications like OpenOffice.org or Microsoft Office save into XHTML or HTML format efficiently, and users seeking just basic web pages are typically far better off using one of those products.

    I use FireFox, and have for a while, as my primary web browser. I was using The Bat, and am now using Thunderbird primarily for e-mail. I haven’t started the Mozilla Suite, although I have 1.7 installed, in over a year.

    >>IBM, Sun or Google: open up your vaults and give some money to Firefox.

    >What would IBM, Sun and Google gain by supporting FF?

    IBM has contributed heavily to the development of Mozilla in the past. Go into BugZilla, search for IBM, and you will find many hits. They maintain commercial ports via Software Choice for OS/2 and for AIX.

    Sun contributes sometimes, and in terms of Java, contributes a ton to the browser. Java is an important feature to many businesses, and is the only secure alternative to ActiveX/XPCOM based plugins Mozilla currently supports.

    And Google, of course, pays the salaries of several key FireFox members, and has generally been very supportive of Mozilla.

    The issue here isn’t money, or paid employees.

    The issue, so people understand it, is that every code change that goes into a secure application like a Web Browser, an ATM machine, the telephone system, etc. has to undergo a review before it is comitted.

    This is to look for blatant security breaches (if (stricmp(password,”letmein”)) goto success) and for less blatant problems (nsprintf(1024, buffer, “%10s”, string)). Problems like this last one are notorious, aren’t adressed well in college, and (most importantly) are easily overlooked even in a code review.

    It’s also to look for blatant bugs, or things that “aren’t right.” Usually you review the prepatch code in parallel with the patched code, using a visual differencing tool, because seeing the changes out of context virtually guarauntees missing something important.

    It’s probably lost here, but expecting one or two people who are programmers themselves to do this task, over months and months and months, is irresponsible of any organization or community.

    Now people are going to chime in “why not let the community do it,” but the community has not proven itself effective at catching these sorts of problems in Linux. In Linux, there is a core of around twelve people catching these problems in the kernel. Individual packages security is rarely audited by anyone external to the package, and even distros such as Redhat Enterprise don’t necessarily audit a significant percentage of the code for security errors.

    On the Microsoft side of things, the situation isn’t much different. You have a handful of people at Microsoft reviewing for security problems, and thousands of people contributing code to their operating system distributions. Just like Linux, they are divided into teams who predominantly work on their chosen package, and just like Linux, the quality of code from the various teams seems to vary considerably.

    The issue is ultimately, you cannot train hundreds of thousands of programmers how to find and correct these errors overnight. And the single voice in the crowd calling out is difficult to find, and difficult to listen to, especially when they are stating things that are unpopular to hear.

    XPCOM and Active-X, for example, work virtually identically. Everything from how components are installed to the level of access they have to the machine are identical. The amount of potential damage is identical. Someone familiar with Security understands that, and puts in controls to try to prevent malicious sites from installing XPCOM components, etc. The common contributor to Mozilla, however, has their head in the sand or puts their hands on their ears and chants lalalalala when people tell them it’s a security risk.

    Simply hiring people blindly doesn’t help Mozilla, and simply adding more reviewers doesn’t help. The reviewers need to be familiar with large portions of the Mozilla code base, a truely formidable task, and the reviewers, likewise, have to be familiar with common security and stability problems, and on top of all this, have to be skilled programmers.

    If you ask any skilled programmer if they would rather spend their days making sure everyone else is doing things right, reading other people’s code, and watching their own hard work get criticized again and again; or if they would rather be writing new code and cotnributing it and have the issue of review be someone else’s problem, I can promise you that the most typical response wouldn’t be wanting to review code for errors, security problems, etc.

  18. unknown says:

    >What would IBM, Sun and Google gain by supporting FF?

    Well it would any Bill – reason enough?

  19. Chris Villanueva says:

    “1.) lowering the barrier for participation
    2.) making clear, easy to find documentation of howto develop on firefox and tasks to be accomplished by clear deadlines that are publicly available…

    should be the top priority if this project is getting off task.”

    I agree with this position. Hopefully, the project will employ an alternative management strategy in terms of delegating duties to others who may not have the greatest technical skills.

    For example, someone like me with 1 year of C, C++ programming experience isn’t ready to contribute productive code, yet. However, I am able to write much of the technical documentation if this information is given to me. I was a Quality Manager for three years for a company of 150 employees and was responsible for all technical documentation (production & quality processes), sales analysis & projections. I would be more than happy to contribute my time. I could produce whatever user/developer/technical documentation you need while simultaneously learning more about the project. As I learn more about Firefox and enhance my programming skills, I could eventually start to contribute. Then, I would have to pass whatever skills assessment/test the Firefox folks want to give me in order to provide my first programming contributions to the project. Someone like me has a vital interest in this project because I sell pre-installed linux desktops. Firefox is one of the reasons people purchase a system from me. I would be willing to help however I can.

    Chris

  20. […] te Blogeintrag von Mike Connor sich auch so sehr in den Medien wiederspiegeln wird wie ein vorheriger, der -wie es scheint- nur allzu gierig von bestimmten Magazinen übernommen wurde. W […]

  21. Gegenoffensive Microsoft

    Nach den Erfolgen des Firefox-Browsers (HNA berichtete) rüstet Microsoft nun zur Gegenoffensive, berichtet Spiegel Online. Während die Redmonder unplanmäßig ihren Internet Explorer in der Version 7 jetzt…

  22. imran says:

    Hi

    With all the design concepts and other dross floating around in my head, I’ve become completely paralyzed on what to do next.

    On one hand, I want to do something and I want to do it right. On the other hand, I don’t know what this “right” is.

    I want to use controllers, but I don’t really know what they are. Right now, I have a bunch of cobbled together pages. I *could* move them into classes, which would probably help factoring them into more controller oriented things, but I want to get it right the first time. And I know I can’t.

    In this mentality, I think I have bitten off more than I can chew. Every small step means more steps when the final design comes into place, but without small steps I cannot get anywhere.

    I need help.

    Imran Hashmi
    http://www.visionstudio.co.uk