1. We have a QA initiative where we work with the community more closely than ever to get your bug reports into the hands of the developers. Please use this forum for LIVE Server bug reporting. Do follow the format below, because it will help us out greatly in responding. If you do not, it's possible your bug report will be misinterpreted, or worse, lost!

    Read BEFORE submitting your first bug: Reporting Bugs… QA 101 Document
    • Search for your bug before posting in order to avoid duplicate reports.
    • Only reply to an existing thread if you have additional information for the reported bug. ALL extraneous commentary will be deleted to avoid cluttering the reports.
    • Keep your bug report short and factual.
    • There is no need to submit crash logs. Crash data we require is automatically logged.
    Bug Report Template
    1. Title:
    2. Reproduction Rate:
    3. Blocker?
    4. Details:
    5. Steps to Reproduce:
    6. User Specs:
    To get started, use /bug in-game (/devbug if on QA) to auto-create this template. It will even auto-fill some of the required information and open the browser for you. Then take the information that was just saved to your system's clipboard and paste it into a new QA forum post. Thank you bug hunters!
Dismiss Notice
This Section is READ ONLY - All Posts Are Archived

Main Wordpress website does not have GZIP content compression enabled

Discussion in 'Critical Issues (Blockers, Performance, Crashes)' started by John Markus, Oct 27, 2018.

  1. John Markus

    John Markus Avatar

    Messages:
    243
    Likes Received:
    277
    Trophy Points:
    18
    Gender:
    Male
    Location:
    Tokyo, Japan
    Title: Main Wordpress website does not have GZIP content compression enabled

    Apparenly your new wordpress template does not have GZIP content compression enabled.
    Enabling this will reduce the web site traffic by more than 75% (store page will have 95% reduction), and will greatly improve web page load speed.

    You can use online tester below to see if the page is GZIP compressed:
    https://checkgzipcompression.com/

    Doing content compression is one-liner (or use a cacheing plugin in the wordpress).
    PHP:
    // Start Output Buffering, specify gzip compression handler
    ob_start('ob_gzhandler');
    It would be always a good idea to see what the PageSpeed says on a web site before release:
    https://developers.google.com/speed...roudoftheavatar.com/?page_id=9085&tab=desktop
     
    Last edited: Oct 28, 2018
    tphilipp likes this.
  2. Chris

    Chris Tech Lord Moderator SOTA Developer

    Messages:
    2,331
    Likes Received:
    26,635
    Trophy Points:
    190
    Gender:
    Male
  3. John Markus

    John Markus Avatar

    Messages:
    243
    Likes Received:
    277
    Trophy Points:
    18
    Gender:
    Male
    Location:
    Tokyo, Japan
    You can overcome the BREACH exploit (which relies on response data size changing when right guessed value is injected into the website) by:
    1. Randomly change the gzip compression level between 1 thru 9 (easy!)
    2. Add random padding to the response data to change compressed response length (easy!)
    3. Sanitize input properly, and stop showing user injected data on web pages unless really necessary (harder!)
    4. Do not place valuable (monetizable) data in response content (do you ?)
    5. Disable compression only on pages that have sensitive data (easy if you plan it right)
    Facebook does utilize adding random padding, and they do enable compression.

    P.S.: CRIME no longer works in modern browsers (post Sept. 2012)
     
    Last edited: Oct 29, 2018
    tphilipp likes this.
  4. tphilipp

    tphilipp SotA Dev Moderator SOTA Developer

    Messages:
    526
    Likes Received:
    1,702
    Trophy Points:
    63
    To elaborate Chris' response, and to give you an idea of why we intentionally removed gzip compression (b/c we actually used to do that, and we used to do that for all requests one level higher up on the load balancer, instead of using php's output buffer options):

    - you are correct that CRIME is now "fixed" (as in "you need a modern browser", we don't control that, though), and that the derivative BREACH can be made somewhat/sufficiently secure with padding, and your detailed list shows one approach to secure this
    - however, in a team where multiple people might work on this code, half of those points are hard or impossible to enforce, as not everyone sees the bigger picture; simply put, IMHO it won't work as it's way too error prone to make a mistake, which is too subtle to even immediately notice
    - the padding part is easy, iff your padding randomness is fine; still, it adds pointless complexity which again can easily be missed once by whoever will work or take over this code
    - those classes of attacks still work with properly sanitized input, so the first part of your point 3. is not important here; this stuff is purely based on user injected data - and that is a given by design, as the site intentionally allows post comments, for example
    - there are more attacks based on size-measuring principles like the newer HEIST, lifting the measurement to the TCP level and doing it entirely in the browser, which facilitate execution of those even more; not sure what else we'll see based on this

    So, let's see at the potential benefit in our real world scenario here, to see if it would justify this extra work. Loading the main Updates section of the site (as of today) comes with:

    - 73KB of html
    - 444KB of static text files (fonts, css and js)
    - 4.3MB of images

    All in all this makes up roughly 5MB of data. The image data would barely be reduced by gzip compression, so we end up with potentially reducing 500KB down to maybe 100KB. So out of 5MB, we would save 400KB. So this would save 7% of the traffic. This simply doesn't justify potentially sacrificing security, even if done right, b/c someone in a year from now takes over some of the code and doesn't see the full picture and removes the padding, or it breaks along the way, etc..

    You are right that the store page comes with a lot more of preloaded text and would reduce even more. And I agree, it would make a difference there that would be potentially noticeable. However, the store will be changed, in order to not only delay-load images (e.g. when you open a popup), but also that more text will be loaded on demand. This will save on traffic, as nobody will click through and read every single item, anyways. Once this is done, we'll end up in a similar scenario as described above.

    That said, I might reintroduce compression for fonts, css, and other static files (that aren't already compressed by themselves, like images), as those should be safe. This means that in my example above, the 444KB would be good candidates for compression, so we'll get maybe an overall reduction of 5-6% of traffic. Problem here is that those are partly (but could potentially be all) served from a CDN, and that one simply doesn't support that transparently. So the (speed-)performance optimization already applied here, to use a CDN for static stuff, is conflicting with the optimization to compress. Yeah, theoretically we could precompress the files and serve them from CDN with the correct headers, however this then would not work on browsers that don't support or request/want data to be compressed.
     
    Jaesun, John Markus and oplek like this.
  5. John Markus

    John Markus Avatar

    Messages:
    243
    Likes Received:
    277
    Trophy Points:
    18
    Gender:
    Male
    Location:
    Tokyo, Japan
    "pointless complexity" is a cute way of saying "It is not complex at all, however I have not spent time to think about it".
    • This could be countered by making gzip compression work only if padding data is present (padding data should have easily detectable prefix / suffix (or use other methods that eliminates searches), or make output handler replace placeholder with random data).
    • Failure to embed the padding (or the placeholder) in the template will result in gzip compression not being turned on, and will be quite foolproof (especially if you name the placeholder in such a way that it contains warnings).
    • You can add another safely layer by adding a placeholder for disabling gzip (getting this placeholder casually copied will not harm the system, just the optional compression not being turned on).
    Talks about images and static text files is more of a red herring; if they are static, they should be made properly cachable.
    All that matters with content compression are the recurring Cache-Controlled texts that are reloaded on every access (compressing store page from 1.6MB down to 70KB is pretty big optimization).

    That being said, I like your ongoing effort on making the store pages load lighter (they seem to be especially heavy on Friday sale times).
     
    Last edited: Oct 30, 2018
    tphilipp likes this.
  6. tphilipp

    tphilipp SotA Dev Moderator SOTA Developer

    Messages:
    526
    Likes Received:
    1,702
    Trophy Points:
    63
    I'm surprised that you claim that I haven't spent time thinking about it. If that would be true, we would still compress it all, and I wouldn't be able to discuss those attacks, here. However, we intentionally disabled compression that we used to run in 2013 b/c of those possible exploits. Compared to other important work, I still think this is pointless complexity. Maybe my words have been chosen badly, I didn't intend to make it sound dismissive. If that was the case, then I apologize. I appreciate you reporting things like this and that we have discussions like this. I guess better words would've been "pointless

    I was hoping that my long post would shed some light on the decisions behind it. Every time I push our non-tech-savvy post-authors to rethink their image choices (as they literally don't care sometimes about what they upload, e.g. a 2mb picture which they then display as an icon... *sigh*) we already save more than what the gzip compression would ever do in our specific case.

    About your 3 points:
    1) if I would do it that way, I would compress 73KB from a total of 5MB from my example, and nothing else of what would be compressible, so I would not even save 1%. The bigger gain would come from the static files, which aren't driven by the php output buffers.
    2) & 3) correct, nothing wrong with that

    Anyways, as said, store optimization is incoming, the store needs this and other work, anyways. There, I fully agree with you, there the compression would make a noticeable difference at the moment. That said, I'm not sure why they are heavier during sales time, b/c there is nothing limiting it or any other bottleneck: the server cluster can scale horizontally on demand, so I'm not sure what might cause what you are seeing. I'll look into this all.
     
    Jaesun, John Markus and Feeyo like this.
  7. Feeyo

    Feeyo Avatar

    Messages:
    1,624
    Likes Received:
    2,399
    Trophy Points:
    113
    Location:
    Aelasar’s Forest
    Just put a varnish system in front of the webserver (nginx) with memcached and you are good to.
     
    John Markus likes this.
  8. tphilipp

    tphilipp SotA Dev Moderator SOTA Developer

    Messages:
    526
    Likes Received:
    1,702
    Trophy Points:
    63
    I love varnish (might be biased though b/c I'm a BSD guy, as is phk the author of varnish, haha). That said, I'm not convinced it would make a big difference, here - the amount of cacheable data we serve directly is fairly small, and the rest comes from a CDN. Still, comes time I'd love to try and see. Let's see how we can grow our community by 2 magnitudes, and hell yeah, it would pay off. :)
     
    Feeyo and John Markus like this.
  9. John Markus

    John Markus Avatar

    Messages:
    243
    Likes Received:
    277
    Trophy Points:
    18
    Gender:
    Male
    Location:
    Tokyo, Japan
    I am not here to argue with you (you are the boss), however you will compress 73KB from total of 73KB (down to 14KB).

    Counting all the resources will matter if you want to optimize for first-time visitor, however we are dedicated customers and will visit the website frequently enough to be able to leverage the caches (I looked at the raw traffic just in case, and only the wordpress page is loaded on the second access; none of the static texts, fonts and images are reloaded. So again, red herring).

    I suppose "Update" page may not by the most suitable for explaining compression advantages (and I do not think they need to be enabled on every pages if it does not fit in with current website designs).
     
    tphilipp likes this.
  10. tphilipp

    tphilipp SotA Dev Moderator SOTA Developer

    Messages:
    526
    Likes Received:
    1,702
    Trophy Points:
    63
    You have a good point about the first time visitors, and I fully agree, this is crucial and ideally the size loaded is as tiny as possible to make it as snappy as possible. This for example led to the slideshow on the new landing page to have a few lines of js that wait for an image to be loaded before it queues the next one up. As another example, the static images (not the ones uploaded by post authors) are made sure to be tiny, while keeping some minimum level of quality (e.g. the .pngs - for when we need transparency - are quantized and color reduced so they can be paletted, and I doubt they can be compressed further).

    So, what I'm trying to say with examples like this (or the one example I mentioned about about our post-authors often embedding badly sized images), is that I'm totally in agreement with trying to get the size down as much as possible. The 5MB size from my example for basically a blogroll is IMHO too bloated. The 1.9MB of html for the store page is also crazy. That will be addressed by just not sending stuff that isn't needed.

    The main question for me is what can be done realistically that would lead to some solid improvement, given the time and workload at hand. I unfortunately don't have the time to fully concentrate on the site and have to cut corners, as there are so many other tasks that need my attention also (site and other things like the server clusters, internal networking, ...). I hate to cut corners on security, though. Once I work myself out from under the pile of plenty of other pressing things, there surely is time for improvements like this.
     
    Jaesun and John Markus like this.
  11. John Markus

    John Markus Avatar

    Messages:
    243
    Likes Received:
    277
    Trophy Points:
    18
    Gender:
    Male
    Location:
    Tokyo, Japan
    Oh, I understand. I would not be up at 4 AM if I was not doing final touches at hardening a website that is scheduled to launch today :(
     
  12. tphilipp

    tphilipp SotA Dev Moderator SOTA Developer

    Messages:
    526
    Likes Received:
    1,702
    Trophy Points:
    63
    Hang in there, and good luck that everything works out. I hear you, I can sooo relate!!
     
    John Markus likes this.
  13. Echondas

    Echondas Bug Brigade – Bug Hunter

    Messages:
    1,815
    Likes Received:
    2,329
    Trophy Points:
    125
    Gender:
    Male
    Location:
    NY
    tphilipp and John Markus like this.
  14. tphilipp

    tphilipp SotA Dev Moderator SOTA Developer

    Messages:
    526
    Likes Received:
    1,702
    Trophy Points:
    63
    I'm not familiar with f5, but yes, they would most likely be vulnerable to those kind of attacks, b/c those attacks aren't against specific software but exploit the very design of how compression works. However, it's only dangerous if the attacker can control the output (e.g. having a search string echoed back to him).

    The thing that is exploited here is that when compression is in play, it allows the attacker to match for certain patterns in the transferred data without even having to be able to read it, by simply observing the size changes of the stream. Basically, if he injects some text that already exists in the stream, the compressed text would be smaller than compared to having injected something random.
     
    John Markus and Echondas like this.
  15. Echondas

    Echondas Bug Brigade – Bug Hunter

    Messages:
    1,815
    Likes Received:
    2,329
    Trophy Points:
    125
    Gender:
    Male
    Location:
    NY
    Makes sense. Thanks for jumping into this subject - I think several of us techie folks enjoyed reading about this.
     
    John Markus and tphilipp like this.