Analysis of the Alexa Top 1M sites (June 2017)

Prior to the release of the Mozilla Observatory a year ago, I ran a scan of the Alexa Top 1M websites. Despite being available for years, the usage rates of modern defensive security technologies was frustratingly low. A lack of tooling combined with poor and scattered documentation had led to there being little awareness around countermeasures such as Content Security Policy (CSP), HTTP Strict Transport Security (HSTS), and Subresource Integrity (SRI).

A few months after the Observatory's release — and 1.5M Observatory scans later — I reassessed the Top 1M websites. The situation appeared as if it was beginning to improve, with the use of HSTS and CSP up by approximately 50%. But were those improvements simply low-hanging fruit, or has the situation continued to improve over the following months?

The pace of improvement across the web appears to be continuing at an astounding rate. Although a 36% increase in the number of sites that support HTTPS might seem small, the absolute numbers are quite large — it represents over 119,000 websites.

Not only that, but 93,000 of those websites have chosen to be HTTPS by default, with 18,000 of them forbidding any HTTP access at all through the use of HTTP Strict Transport Security.

The sharp jump in the rate of Content Security Policy (CSP) usage is similarly surprising. It can be difficult to implement for a new website, and often requires extensive rearchitecting to retrofit to an existing site, as most of the Alexa Top 1M sites are. Between increasingly improving documentation, advances in CSP3 such as 'strict-dynamic', and CSP policy generators such as the Mozilla Laboratory, it appears that we might be turning a corner on CSP usage around the web.

Observatory Grading

Despite this progress, the vast majority of large websites around the web continue to not use Content Security Policy and Subresource Integrity. As these technologies — when properly used — can nearly eliminate huge classes of attacks against sites and their users, they are given a significant amount of weight in Observatory scans.

As a result of their low usage rates amongst established websites, they typically receive failing grades from the Observatory. Nevertheless, I continue to see improvements across the board:

As 969,924 scans were successfully completed in the last survey, a decrease in failing grades by 2.8% implies that over 27,000 of the largest sites in the world have improved from a failing grade in the last eight months alone.

In fact, my research indicates that over 50,000 websites around the web have directly used the Mozilla Observatory to improve their grades, indicated by scanning their website, making an improvement, and then scanning their website again. Of these 50,000 websites, over 2,500 have improved all the way from a failing grade to an A or A+ grade.

When I first built the Observatory a year ago at Mozilla, I had never imagined that it would see such widespread use. 2.8M scans across 1.55M unique domains later, it seems to have made a significant difference across the internet. I feel incredibly lucky to work at a company like Mozilla that has provided me with a unique opportunity to work on a tool designed solely to make internet a better place.

Please share the Mozilla Observatory so that the web can continue to see improvements over the years to come!

Footnotes:

1. Allows 'unsafe-inline' in neither script-src nor style-src
2. Allows 'unsafe-inline' in style-src only
3. Amongst sites that set cookies
4. Disallows foreign origins from reading the domain's contents within user's context
5. Redirects from HTTP to HTTPS on the same domain, which allows HSTS to be set
6. Redirects from HTTP to HTTPS, regardless of the final domain
7. As listed in the Chromium preload list
8. max-age set to at least six months
9. Percentage is of sites that load scripts from a foreign origin
10. Percentage is of sites that load scripts
11. CSP frame-ancestors directive is allowed in lieu of an XFO header
12. Strong CSP policy forbidding 'unsafe-inline' is allowed in lieu of an XXSSP header

— Tuesday, June 13 2017[Category: Security] [Permalink]

Recording MTGO in 4K with OBS

One of the perennial complaints about MTGO streams and recordings is how difficult the cards are to read. And it's no surprise — pretty much any program would struggle with the requirements that Magic imposes. It has to combine sometimes obscene amounts of text with an eye-wateringly small rendering area of less than a dozen square centimeters.

This problem is exacerbated by the fact that most MTGO streamers record at the standard resolution of 1080p. There are simply not enough pixels available to legibly render a font in such a small area at such a low resolution. Here is an example of what I mean:

Left: 1080p Channel Fireball recording
Middle: 4K recording, smallest hand size
Right: 4K recording, standard hand size

This is not meant to pick on Channel Fireball. Their content is impeccable, but much of it is hard to read unless you know exactly what is going on. To be clear, it's not just Channel Fireball; even professional content put out by Wizards of the Coast suffers from similar issues. Unless you follow Standard, I suspect you'll have a hard time telling me what these cards from the recent Magic Online Championship do:

Is 4K really that big of a deal?

YouTube displays recordings in a variety of resolutions, from 144p all the way up to 2160p (4K). It may not seem as if there is a big difference between 1080p and 2160p, but remember that the “1080” in “1080p” only refers to the number of vertical pixels. In terms of overall pixels, there is a pretty vast gulf between 1080p and 4K:

As you can see, 4K gives you 400% more pixels to render legible text in the exact same amount of screen space. That's why the Deathrite Shaman on the right can clearly display its entire text box despite taking up the exact same amount of space as the Deathrite Shaman on the left.

Cranking OBS to 11

Before I began recording, I simply assumed that 1080p recordings were a matter of inertia. Everybody recorded in 1080p, so what was the point in trying to bump the resolution up to 4K? After all, 4K means more CPU usage, larger files, and slower downloads. Why bother when nobody else was doing it?

It turns out my assumptions were way, way wrong. It turns out that it's actually really difficult to record in 4K while playing MTGO. My machine isn't top-of-the-line, but it's nothing to scoff at: a 2013 Macbook Pro with quad 2.6GHz i7 CPUs and 16GB of memory.

I assumed that all I'd have to do was tell OBS to record at the native “Retina” resolution (3360x2100) and I could go on my merry way. What happened when I did that?

PAIN. Telling OBS to record at 4K in real time took so much CPU that my machine was rendered completely useless. I couldn't actually play MTGO, because each click took over 15 seconds to register.

I then tried telling OBS to use one of its faster CPU settings (ultrafast), but the image quality came out very poor, with lots of noise and other encoding artifacts:

Path to 4K

This process left with a whole new respect and understanding for the professionals who do these recordings. It's simply impossible to have:

1. High quality 4K recordings
2. Low CPU usage
3. Managably-sized video files that you can upload directly to YouTube

I realized I'd have to compromise and do my recordings in two steps:

1. Record with at high quality and low CPU usage, but large video sizes
2. Re-encode post-recording to generate high-quality videos with low file sizes

This is certainly more work, and takes a lot more time. But it comes with some benefits:

1. The recording takes very little CPU usage, instead of causing the typical OBS lag
2. The re-encoding takes 8-12 hours, but the settings used result in YouTube quickly generating all the other resolutions post-upload
3. My screen's resolution (3360x2100) is actually lower than 4K (3840x2160), but the re-encoding lets me upscale to 4K

The Technical Details

So how did I do it? Here are my OBS video settings:

1. Recording Format: mp4
2. Encoder: x264
3. Rescale Output: unchecked (native resolution)
4. Rate Control: CRF
5. CRF: 12
6. Keyframe Interval: 0 (auto)
7. CPU Usage Preset: superfast
8. Tune: stillimage
9. Variable Framerate (VFR): checked

These settings generate very high quality recordings that average about 1GB for every ten minutes of recording. Lowering the CRF value leads to higher quality files at the cost of increased CPU usage, and 12 was the highest quality my machine could handle. If you find these settings too aggressive, bump CRF to a higher number.

Once I am finished recording, I have an automated job that upscales and re-encodes with ffmpeg, using the optimal YouTube video settings:

$ ffmpeg -i input.mp4 \
  -c:v libx264 \
  -crf 21 \
  -tune stillimage \
  -bf 2 \
  -c:a copy \
  -pix_fmt yuv420p \
  -flags +cgop \
  -sws_flags lanczos \
  -movflags faststart \
  -vf scale=-1:2160 \
  output.mp4

You don't need to know what all those settings do. Suffice to say, the generated files are perfect for YouTube and have no perceptible loss in quality despite being 75% smaller.

How big of a difference does 4K make in practice? All I can say is to take a gander at one of my recent video settings and witness the 4K difference for yourself.

— Tuesday, March 07 2017[Category: Magic] [Permalink]

Understanding CORS

RTFM… just kidding! There is no manual for the CORS (Cross-Origin Resource Sharing) specification. I really had you going there, didn't I?

Don't worry, it's not your fault. After all, here is what a Google search provides:

Each of these sites contains a wealth of information about CORS, and each of them is far over the head of your average developer. Given the frequent questions that I receive from confused and frightened developers trying to understand these documents, I thought it might be helpful to boil CORS down into a couple simple examples.

Q. If I have static content that depends neither upon cookies nor user-specific URLs and/or parameters and I want to share my site's content with the web, what should I do?

A.

Access-Control-Allow-Origin: *

Q. Well, that is great and all. But what if I want to let a foreign website interact with my site, as a logged-in user, allowing them to do anything they could as if they were on my site? I swear that I understand the risks that this entails and that I really trust this other site to not make any security mistakes such as falling victim to a cross-site scripting (XSS) attack.

A.

Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, HEAD, OPTIONS, POST, PUT
Access-Control-Allow-Origin: https://example.com
Access-Control-Expose-Headers: X-Poop-Emoji
Access-Control-Max-Age: 300

Where these headers mean the following:

• Access-Control-Allow-Credentials means that the user's cookies (such as their session cookies) will be sent with the request
• Access-Control-Allow-Origin is the whitelisted origin sent in the Origin header by the browser and not * nor blindly reflected

And these optional headers mean the following:

• Access-Control-Allow-Methods is the list of allowed HTTP methods beyond GET, HEAD, and POST
• Access-Control-Expose-Headers allows example.com to read the contents of the X-Poop-Emoji header (💩, obviously)
• Access-Control-Max-Age allows example.com to make these requests without preflights for the next 300 seconds

Again, please be aware that you need to be very careful with Access-Control-Allow-Credentials. Even if you think you're safe by only allowing idempotent methods such as GET, that might be enough to steal an anti-CSRF token and let attackers go to town with CSRF attacks.

If you need additional documentation about other features in CORS, I highly recommend the frustratingly hard to locate CORS for Developers document by Brad Hill.

— Tuesday, November 29 2016[Category: Security] [Permalink]

Analysis of the Alexa Top 1M sites (October 2016)

Last April, I ran a scan of the Alexa Top 1M websites using the Mozilla Observatory. The results were dire, indicating a broad lack of awareness around modern security technologies such as Content Security Policy, Strict Transport Security, Subresource Integrity, and others.

But that was six months ago. With the Mozilla Observatory being publicly released almost two months ago, I was curious as to whether significant improvement had been made around the internet. After all, in those two months, the Observatory has scanned approximately 1.3M sites, totalling over 2.5M scans.

With that in mind, I ran a new scan of the Alexa Top 1M at the end of October, and here is what I found:

I'll admit, I was a bit taken aback by the overall improvement across the top million sites, especially as some of these security technologies are almost a decade old.

When we did our initial scan of the top million six months ago, a stunning 97.6% of websites were given a failing grade from the Observatory. Have those results changed since then, given the improvements above?

While a decrease of 1.5% in failing grades might seem like only a small improvement, the latest Observatory scan contained 962,011 successful scans. With each percentage point representing nearly ten thousand sites, a drop from 97.6% to 96.09% represents approximately fifteen thousand top websites making significant improvements in their security.

I'm excited for the possibility of seeing further improvements as additional surveys are completed. Please share the Mozilla Observatory and help to make the internet a safer and more secure place for everyone!

Footnotes:

1. Allows 'unsafe-inline' in neither script-src nor style-src
2. Allows 'unsafe-inline' in style-src only
3. Amongst sites that set cookies
4. Disallows foreign origins from reading the domain's contents within user's context
5. Redirects from HTTP to HTTPS on the same domain, which allows HSTS to be set
6. Redirects from HTTP to HTTPS, regardless of the final domain
7. As listed in the Chromium preload list
8. max-age set to at least six months
9. Percentage is of sites that load scripts from a foreign origin
10. Percentage is of sites that load scripts
11. CSP frame-ancestors directive is allowed in lieu of an XFO header
12. Strong CSP policy forbidding 'unsafe-inline' is allowed in lieu of an XXSSP header

— Monday, November 14 2016[Category: Security] [Permalink]