Bandcamp Email Bug – May 2018

We recently discovered and fixed a bug that inadvertently included certain users’ email addresses in the HTML of some Bandcamp pages. When present, the email address was not visible on the page, but did appear in the HTML.

No other personal information was included and there was no breach of our security systems, so you do not need to take any action to secure your Bandcamp account.

Although we cannot determine which specific accounts may have been impacted, if you created a Bandcamp account before March 20, 2018 and visited a Bandcamp site between March 20, 2018 and May 7, 2018 while logged-in, there’s a good chance your email was affected by the bug.

This should not have happened, and we sincerely apologize. The security of our users’ information is a top priority for us, and we are reviewing our development and security practices to ensure that something like this doesn’t happen again.

About the Bug

For the software developers out there (both professional and armchair), here is a detailed technical description of the bug:

In March, we rolled out an updated version of our “fan onboarding” flow — the introductory screens a new user sees immediately after signing up. As part of this work, we introduced a new “onboarding” object into our web controller code, which is the server code we use to generate pages on bandcamp.com. The onboarding object is a short-lived bucket of values associated with the user viewing the page, used by our page rendering logic to determine which onboarding steps the user has already completed.

A subsequent change added the user’s email address to this object. This alone is not dangerous or unusual, and allowed us to render an additional UI element. However, instead of adding the email value where the onboarding object is created, we added it elsewhere in the controller code, overriding the original value. This seemed safe in context, but combined with other decisions, it became dangerous:

  1. The onboarding object, at first glance always unique per request, was instead sometimes a reference to a shared object containing default values.
  2. This shared object was intended to be read-only, but its values could be modified. This meant that when overriding the email value, we might inadvertently modify the shared object.

The result was a race condition: when processing a page requested by a logged-in user, we would sometimes store that user’s email value in the shared object, where it might be picked up for page rendering in independent, parallel requests (our request handling environment uses multiple threads). Whether or not a user’s email showed up in someone else’s page depended on the precise timing of parallel requests on a given rendering app, and the types of users making those requests. To make matters worse, we optimistically wrote the onboarding data into the page even when it wasn’t needed for the current user. This increased the number of pages potentially affected.

Once we understood the problem, the immediate fix was simple — we modified the code to duplicate the shared object for every request. This eliminated the cross-request issue.

What We’ve Learned

There are several useful engineering lessons here. First, arbitrarily overriding values in a complex object can be dangerous, especially if it’s done far from the code where the object is created. Instead, if we had modified the object initialization to support an email value, it would have been immediately obvious that the email shouldn’t apply in some cases.

Second, read-only objects shared across multiple threads should be frozen or have appropriate access permissions set at the language level, even if it appears they are never modified in code. If the shared object in question here had been frozen, we would have caught the problem during development.

Third, we should be more careful not to render data and HTML we don’t need for the current page. This is just good practice in any case, as unused elements increase the page size and slow network transfers and page rendering.

Finally, and most important, we need to do a better job of reviewing code changes which involve the output of personal information.

Protecting the personal information of Bandcamp’s users is a top priority of our software engineering team. Our failure to do so in this case is a reminder of our blind spots as engineers, and our responsibility to continuously improve our development practices. We hope that our sharing the details of this bug and our response is useful to the software development community and our users.

Shawn Grunberger, Co-founder & CTO