Last year, after three and a half years as module owner, and almost seven years of living and breathing Firefox, I stepped down from that position in order to focus on what was then called Weave.  I didn’t write a lot about the decision at the time, mostly because I wanted to focus on building something worth talking about.  But I’m trying to write more, and something I’ve been talking about more lately is why Mozilla is stepping into the services space, so I figured it’d be a good excuse to dust off the old blog.

Admittedly, I was never much of a privacy nerd.  While I wouldn’t give up my SIN for a toque (unlike someone I know…), I never really spent a ton of time thinking about which sites had data about me, or how they used it.  However, the evolution of Facebook, Google, and even Amazon has changed the game, and with it my thinking on the subject.  Much of what we do online today is tracked and used to build a profile of who you are, where you go, and who you know.  Beyond that, there’s all of the information we explicitly put online ourselves, on Facebook, Twitter, Yelp, etc.  Obviously there’s value there to both sides, the ad networks can market more effectively, and users get ads that are more relevant to their interests.  Win-win, right? Well, maybe.

In any exchange, value has to be measured against cost.  A friend deleted his Facebook account a year or so ago because he could not effectively measure either, or make an informed decision about whether the value was worth the cost.  It’s especially hard to reason about cost in this case, because it’s generally not concrete or measurable ahead of time.  But social media and web services can have real-world negative impacts on marriages (and divorces), careers, and even home security.  In the most extreme case, Shi Tao, a Chinese dissident, was imprisioned for 10 years when data from his email account was turned over to the authorities.

And, of course, that’s just what happens without things going wrong.  Sony is right now dealing with the fallout of a massive security breach, where personal data and credit card information for millions of users was obtained.  Facebook made the news last year when apps were leaking user IDs to advertisers, allowing ad networks to directly correlate what they know about you already with who you are on Facebook.  And there’s a long list of security and privacy concerns around more or less any major service you’ve ever used.  I think we can, and should, expect better.

What we’re trying to build with Mozilla Services is a set of products that aim to let you have your cake (privacy) and eat it too (tasty tasty features).  Firefox Sync is our first released product (part of Firefox 4 and up) and uses client side crypto to give users a great user experience without giving Mozilla anything other than encrypted blobs.  The encryption keys are never provided to or stored by Mozilla’s servers, so we simply cannot get to your data.  It’s a great feature, and I’m very excited to build more awesome, privacy-centric services.  It’s a hard problem, both technically and from a UX perspective, but I think it’s important.  That’s why I’m here, and that’s why I’m excited about where we’re going now.  Expect to see/hear more about the next steps soon!

How do I fix this?

New versions of Sync will all work together without any known issues, but if not all of your clients are on the development channels, you may be seeing these errors, depending on your client mix.  If you are using a client that is on one of the development channels, you will need to make sure all of your clients belong to the following list.

• Firefox Sync 1.5b6 (all add-on users, even beta/nightly users, must upgrade to this version)
• Firefox 4 beta 7 or higher (Desktop)
• Firefox 4 beta 1 or higher (Mobile)
• Firefox 4 nightly builds after 20100924 (Desktop & Mobile)
• Firefox Home v1.0.2 or higher

How will most users be affected by this change?

Firefox Sync 1.5 and Firefox 4 b7 will be released at the same time, so users who update via normal channels will get the updated Sync versions automatically, around the same time, so we expect this will be a minimal disruption for those users.  Firefox Home 1.0.2, because it is a read-only client, supports both versions, so those users who are already on 1.0.2 should not need to take any action at this time.

Why is this happening?

As all synced data is encrypted first on the client and then uploaded to the server, Sync defines a format for how data is stored.  This is necessary so that older clients can recognize when they won’t be able to read new records.  Long-time users of Sync will remember that each and every version upgrade until v1.2 required all clients to be updated.  This was easier when Sync was a prototype add-on, but not appropriate for a shipping feature, so around six months ago we did our best to stabilize the storage format, and we haven’t changed compatibility since then.  We have since realized that there was one more minor, but incompatible, change required to the format before Firefox 4, which is the change we’ve just made.

As noted in the release announcement, for this release we’ve moved all of the UI in Firefox to the Tools menu, and out of the status bar in Firefox. As we expected, not everyone is a fan of this move, with the primary concerns being around syncing before leaving a machine and detecting when Sync encounters issues. These are valid concerns, and we will address them as part of the new UI design. As explained in more detail below, we feel the new UI is much closer to how Sync should work, and we intend to move forward with this new UI direction in future development.

We believe that Sync should Just Work, and not need manual intervention or frequent status updates. We did have a lot of work to do around performance and reliability, and having the status bar UI in place meant that we got a lot of feedback when Sync caused problems with performance in the app, so it was kept for much longer than we originally intended.

As we move closer to inclusion in Firefox 4, and the add-on continues to become more performant and stable, we feel it is the right time to move the UI into the background. An important factor in this decision is that we will be tuning Sync to update smaller chunks, more frequently, when you’re actually using a particular device. Currently we default to hourly syncs between multiple computers, which is something that will change very soon. At a greatly-increased frequency, the visual distraction (and the performance overhead of continuous UI updates) was not going to be acceptable, so we needed to make changes. That said, no first attempt is perfect, and we still have work to do on the concerns noted above.

We do understand that many users have come to rely on forcing syncs to happen, in order to make the experience of switching between devices better, especially as we do not sync before exiting the application. This is far from an ideal user experience, so our goal is to make this unnecessary, with the changes mentioned above. We’ll be posting more details on these changes in a couple of weeks, after the Mozilla Summit next week.

As for detecting problems, error notifications haven’t changed, and will still be displayed in the status bar as they always have been. We intend to expand these notifications to cover some cases identified by your feedback (i.e. delayed login due to master password, autoconnect being disabled for an extended period of time, etc). While our design goals are for Sync to live in the background, when user intervention is required we will continue to show UI in the main browser window.

Thanks to everyone for your feedback so far, we’ll be doing a better job of posting updates to the dev mailing list (http://groups.google.com/group/mozilla-labs-weave-dev is the current list) so people can get involved with earlier feedback or even jumping into the development process. We also have the development channel builds at https://services.mozilla.com/sync/install.php for anyone interested in following the tip of development.

The lesson that hits home every day is that while we should always think about the present when designing something, we should also keep an eye on the long view, and take the short-term costs where the long term savings warrant.  Software development timescales are not the same as subway lines and bridges, and it doesn’t always pay to take the long view, but we should always be conscious of what we’re trading off.  We especially need to be aware of this in Labs, as we have to balance the competing desires of going faster and getting features into products.  It’s not easy, and we won’t always make the right bets, but we should always try.

OH: It’s not personal, it’s just mconnor

At first I laughed at this, but it (and a succession of other things) prompted me to write something that’s considerably overdue.  Around nine months ago, it was confirmed that I’ve been living with Asperger Syndrome, which explains a great many things about how I act, how I treat people, why I fail, and why I succeed.  I’ve managed to succeed much more than I’ve failed, through a combination of skill, hard work, and a lot of luck, much of that while living in the world’s biggest fishbowl.  But along the way I’ve hurt feelings, caused anger, and brought a lot of frustration and tension into the lives of many people I respect and care about.  Almost all of it was unintentional, and unconscious, and often the responses, especially the angry ones, have been bewildering and upsetting.  But at least now I know why.

Living as an undiagnosed aspie isn’t especially awesome.  I don’t read people well, without distinct effort and focus.  I especially don’t do well in large social situations (a key factor in some of my infamous episodes), but I just thought I wasn’t especially well-adjusted.  I tend to not self-censor, and say things without really thinking them through.   A really big one is that I tend to argue points strongly, and express opinions in very certain and definitive terms.  (I’m still working on that one…)  Hardest of all, I can come off as anti-social and cranky, but those who know me best know I’m a big teddy bear.

On the other hand, there are some aspects that are really helpful.  I can grasp really complex and chaotic systems (like how software gets shipped).  I often notice details and patterns that others don’t.  I can handle doing grindy tasks for long periods of time, without it driving me crazy.  I can focus on things to the exclusion of others to get things done under pressure (that one cuts both ways, to be fair).  I don’t think I’d be where I am today without some of those traits.

Since I’ve been diagnosed, I’ve also shifted my day to day focus to Labs, especially Weave, which was a transition into a less rough-and-tumble environment than the Firefox critical path.  This has been a challenge for me, but it’s also been a learning experience.  Being blunt and aggressive is something I’ve had to tone down, and working with a lot of people who aren’t used to me has taught me that I need to modulate my approach.  For anyone interested, there’s a few key things that matter in living with AS in a tricky world:

1. I need to slow down my responses, and be much more intentional about what I say and how I say it.  I will throw things out there, and I may not even mean what I say to be definitive, but that’s how people take it anyway.
2. I need to listen more than I talk.  Something I learned from John Lilly a while back is “argue as if you were right, listen as if you were wrong” but that only works if there’s an established trust relationship, and I don’t always apply my best judgement.
3. I need to be honest with myself about my limitations and my mistakes.  Everyone makes mistakes, and that’s okay.  But I need to accept and own the problems I create, and I need to work to not repeat them.

Ultimately, AS is a blessing and a curse.  I like my brain, I like who I am 98% of the time.  The other 2% sucks, and while I’m working on that, change is slow when it’s brain wiring.  In the meantime, I want to work even better within Mozilla, so I would like anyone reading this, who interacts with me, to keep a few simple guidelines in mind:

1. WYSIWYG – I put my cards on the table right up front, as much as I can.  If you’re reading ulterior motives into my behaviour, you’re probably getting it wrong, but please feel free to call me on it, I may not realize it.
2. If I upset you, I almost certainly didn’t mean it, and telling me would be very helpful for me to continue to learn.  I know it’s hard to call people on social gaffes, but I will thank you for it (even if it’s hard to hear).
3. Not understanding is hard, so I’ll often ask lots of questions if I don’t understand.  That often comes across as arguing with someone’s choices, which is unfortunate and damaging.  Again, call me on it if you think I’m doing it.
4. I probably shouldn’t ever talk to press without a grownup around.  Just sayin’.

I suspect some of the people reading this post also have AS.  Jeff Atwood has written about AS on Coding Horror, it was in Wired a long time ago, and it’s not exactly news.  But I have learned a lot from having it confirmed, and from learning about how it impacts, and how I can adapt.  I’ve quit drinking (it’s just not a good idea), I’m in a healthy relationship now, and I’m constantly working to get better.  It’s a process, and often a frustrating one at that, but I am determined to make the best of it.  I would encourage anyone who identifies with what I’ve written to seek out a diagnosis.  It doesn’t mean you have to change who you are, but it gives you the opportunity to change how you affect others, and if you’re anything like me, that’s what really matters most.

Overall, the intent is to give the sub-module owners more responsibility for driving their area forward in line with our overall goals.  This means identifying pain points (technical and user-facing) and building a plan to solve those problems will be the responsibility of the sub-module owners.  Obviously the technical and product leaders will continue to work with these individuals, but it will be the responsibility of sub-module owners to drive this work forward.  Dietrich Ayala‘s ownership of Places over the last two years is a fantastic example of how this should work in practice.  In line with this change, I’ve named new owners for a number of sub-modules.

Another important change I intend to make is to the role of a module-wide peer, since the traditional role is somewhat unclear in a decentralized model.  The closest we have in Mozilla to the idea of cross-module oversight is Brendan, and he doesn’t have any official peers.  Given this, I contemplated doing away with the old concept of peers entirely, and treat the reviewer list as peers in the despot/policy sense.  However, I have concluded that it would be very beneficial to have peers to help with the technical and product oversight role, especially guiding and mentoring people who are learning how to scale up.   To this end, I’ve asked Vlad and Gavin to act as peers under this new model.  This is not just a technical role, but has responsibilities related to product direction and stewardship, which we’ll evolve over time as we get used to this model.

Beyond that, we’re going to add five new (and long overdue) reviewers to various areas (and add/merge some areas).  Ehsan Akhgari, Ryan Flint, Dao Gottwald, Jim Mathies, and Johnathan Nightingale will join the reviewers group with various areas under their care.  They’ve all been playing key roles over the last months and years, and it’s high time we start rewarding them in the traditional Mozilla manner: more work and more responsibility!  Please join me in welcoming everyone to their new roles.

I’ve already updated the official review docs to reflect these changes, so as of now, this is official.  Please let me know of any concerns via email or in the comments.

With the formation of Mozilla Labs, we’ve been better able to try things that we couldn’t really explore as part of our normal ship process.  That’s been great to see, especially as we have explored new and compelling features and interaction models, but now we need to figure out the right way of bringing the best pieces of these projects to all of our users.  Learning how to integrate R&D with the rest of our development process is going to take time and effort, but the goal is to establish a repeatable and transparent process that we can continue to use as we grow Mozilla Labs and the various Mozilla products.

This of course doesn’t mean we’ll take something from every project, or even that every project has to result in a core feature to be successful.  The primary goal of Labs is to explore and innovate and try things.  Sometimes that will be successful, other times it won’t.  Sometimes we will learn things that inspire a different approach entirely, and that is an important success state.  But when we create something compelling, we need to be prepared to learn lessons and adopt the best pieces in ways that make sense.

As our first pass, we have decided to focus on incorporating some aspects of three projects in particular: Prism, Personas, and Ubiquity.  The key defining characteristic of these three projects is that we’ve spent enough time exploring those spaces that we feel confident in identifying and uplifting the most useful pieces.  Users should expect to see these new features in the next Firefox release after 3.1.

Some preliminary work has been done on identifying key elements of all three projects, and we will continue to refine these plans in order to get a good jump on things as Firefox 3.1 finishes.  You can find these first passes here:

Personas
Ubiquity
Prism

We absolutely crave feedback, so please feel free comment here, send mail or comment on the Talk pages on the wiki if you have questions or concerns.  We hope to get to a crisper and more tightly-scoped set of plans over the next month or so, and we’ll continue to point out when there are more changes that we’d like feedback on.

The Pencil Project is pretty awesome, and has a ton of potential.  This took me two minutes to build.

Much has been made of the issues related to fsync, and this is where the distro connection comes in especially handy.  We’ve already received a firm committment from Red Hat/Fedora, Ubuntu, and Novell/OpenSUSE to ship the mitigation patch from the bug, if we do not otherwise need to do an RC2 and thus have a chance to take it in Firefox 3 proper.  We’re going to continue to reach out to the rest of the distros shipping Firefox to roll in the patch.  This means we can ship sooner, while still ensuring the vast majority of Linux users get the patch.  I’d like to thank Alexander Sack, Chris Aillon, and Michael Wolf for being highly responsive to our requests.

Its pretty fantastic to look back at those five years, from the uncertainty of the Foundation startup, through the huge buzz around Firefox 1.0 and the launch, the growth and maturity of the organization through the challenges of shipping follow-on releases, all the way up until today, where we expect to ship the first release candidate for Firefox 3.  I can say honestly that its the best release we’ve ever done, and I’m excited about getting it to 170 million people as soon as possible.

What’s most interesting to me is that we’ve now grown enough that we’re no longer aiming for It Just Works.  We’ve done that already, so now we’re aiming for the holy grail of Does What I Mean.  Its a much higher bar, but that’s where we need to go next.  We need to do it on mobile, we need to do it on the desktop, and we need to figure out how help people do it everywhere.  And that is my new Five Year Plan.

I hope you’ll all come along for the ride.