Some Turbulence

You’ve probably noticed that the ride has been a bit bumpy on WP.com the past week with a widely covered DDoS attack last week and then another interruption a few hours ago from an unrelated network maintenance issue. Our core mission is enabling the world to publish, and anything that affects the availability of your site or dashboard impedes that.

Anything less than 100% availability drives us to work harder. Rest assured that behind-the-scenes here at Automattic we’re learning as much as we can from these incidents so we’re even better prepared and more able to adapt next time something unexpected comes our way. (We weather DDoS attacks every day on WP.com and 99.9% of them have no user impact.)

In a previous post I wrote about downtime I ended with the sentence, “For those following along at home, we’re now powered by 152 physical processors, 511 gigabytes of memory (RAM), 174 hard disks with several terabytes of storage, and we’re adding new servers constantly.”

That last part about “adding new servers” was especially true, for WP.com we’re now at 8,921 CPU cores across 2,475 physical processors, 8,200 gigabytes of memory (RAM). We’ve changed how we’ve done storage, but now that layer includes on its own 1.3 terabytes of RAM, 1.3 petabytes of storage, and 8.9 terabytes of solid state disks. (Plus Amazon S3, which we use as backup to our internal systems.) Also, we’re adding new servers constantly. :)

Keep blogging, and we’ll keep doing our best to make sure you never have to think or worry about this stuff.


Missing out on the latest WordPress.com developments? Enter your email below to receive future announcements direct to your inbox. An email confirmation will be sent before you will start receiving notifications - please check your spam folder if you don't receive this.

Join 17,282,372 other followers

Matt

364 Comments

Comments are closed.

  1. Treasureyourhealth

    Thanks for that heads up! I love that I have finally turned to blogging. Still learning how to maneuver WP, but am thrilled with all
    the possibilities. And, remember, keep taking those deep breaths! (*_~)

    Like

  2. diogeneseassociates

    Y’know what? Considering the cost and quality of the WordPress service, a little downtime due to the odd hiccup is a small price to pay.

    Thanks for keeping us informed and thanks for providing such a great service.

    Like

  3. dreamsburnred

    A little downtime for a great service isnt a issue. WordPress.com has and always will be my blog host.

    Like

  4. cookingoodies

    I hadnt even noticed ! WordPress is wonderful!

    Like

  5. mindblowingfood

    Thanks for making such a brilliant tool to connect people and let us share our passions!

    Like

  6. dontpokeme

    Please show the negative..You are a wonderful service-host but…we know there are glitches that we all feel…

    Like

  7. Kerrina

    Thank you for all your hard work!

    Like

  8. Craig

    Kudos to Automattic … because server up-time and fending off various DDoS attacks is not automatic. This blogger thanks you.

    Like

  9. terrydarc

    Thanks for the great job, WP! We all appreciate your taking the time/effort to keep us going and for informing us what was going on. I hope that information continues in the future.

    On the attack itself: sounds like a concerted effort by a large group of computers (this is not the sort of thing a disgruntled techie launches from his/her PC) but a concerted effort to control the msg. As shown in the Mideast, the 21st century lives free (and dies) by the message getting out. My sincere thanks.
    -Terry

    Like

  10. alligramz

    Ya Done Good, Guys! Thank you for watching out for us!

    Like

  11. hljourney

    Thank you so very much for all that you do behind the scenes to make everything work correctly!! I greatly appreciate it!

    Like

  12. gatherer4

    Hey, I just started with WordPress and I have no complaints. It’s nice to know there are people out there working on your behalf. Thank you, Matt.

    Like

  13. Harold

    Thanks for the work behind the curtains! And a peek every once in a while!

    Like

  14. Taufik

    several times down here in Indonesia . . . but its OK
    You’ve done a great work guys :)

    Like

  15. HannibalTC

    Keep up the good work!

    Like

  16. markgorman

    Cool. You are the guys. In all my spheres of commercial interest I demand WordPress as the blogging platform and it’s ability to integrate into websites is legendary. Much love.

    Like

  17. italianhandful

    So appreciate the update and info. Thanks. Adagio

    Like

  18. Pingback: That’s A Lot Of Hardware
  19. silvercannon

    Thanks! You all do great work and I appreciate having this available.

    Like

  20. Steve

    I most certainly appreciate all that WordPress does to allow worldwide publishing to be within reach of the average person. I run several WP websites, and while an occasional hiccup can be taxing during a time of heavy uploading or site modification, you folks always get it handled quickly and efficiently, so I cannot complain (especially since the price is most assuredly right)! You have earned my strong loyalty over the course of the past two-plus years, which has led to several other author/publishers signing on at my highest recommendation. Thank you!

    Like

  21. bccmee

    This is my favorite blog provider and even if there are glitches, you *communicate* and you *care*!

    Like

  22. thf2

    I appreciate you guys!! Keep pressin’!!

    Like

  23. debbyfriselladesigns

    Thank you for keeping us posted on the details…I love WordPress just for that reason!!!

    Like

  24. What's for tea tonight dear?

    We do appreciate your efforts to keep the show on the road, all the more so when you’re up front if there are problems. Keep up the good work!

    Like

  25. My Camera, My Friend

    Thank you for letting us know what’s going on.

    Like

  26. chandrapzm

    Thank you for all the WP teams have done. I really appreciate it. :)

    Like

  27. Ћэ §åŀэςmãη

    Wow, Keep up with keeping WP safe!

    Like

  28. Success Coaching Group

    If all businesses were so transparent and forthcoming…ahhhh, I can dream :-)
    Thanks for the updates!

    Like

  29. Martin Scherer

    Bastards

    Anything we can do to help?

    Like

  30. undeniablymaybe

    You guys are the best, which is why, when I recently made the decision to start a real blog again after so many years away, I chose WP. Thanks, keep up the good work.

    Like

  31. andro51

    Thank you for keeping us all up to speed on things, you’re doing a fantastic job by the way…

    Like

  32. tknologlvr

    Thanks for all you guys and gals do and for keeping us informed.

    Like

  33. Trang Đài

    Love you WordPress! <3

    Like

  34. fullphaser

    There’s really no need to apologize for being knocked out by a DDoS attack, in those situations the best you can do is try desperately to retune network switches and packet filters and ride the wave.

    Like

  35. kbelle1

    Thanks for your dedication. Looking forward to smoother sailing!

    Like

  36. Mira Faraday

    Thank you guys and gals, for what you do for all of us. We greatly appreciate your hard work and dedication that all of us benefit from each day.
    Yes, when things go down we all may panic. We as well know ourselves when our own work or home gadgets go down frustration occurs too.
    My hat is off to each of you for all that you do to keep us happy! Have a wonderful day when everything is fixed, the you can go home. ( teasing)
    Many smiles across the miles to let you know your appreciated.

    Like

  37. Achmad Fauzi

    Thanks for Your best job … :D

    Like

  38. John Ryan Recabar

    the mark of an excellent service is that when you do not even noticed you are being served because it is seamless as seamless can go. that’s why i chose wordpress.

    thanks, wp.

    Like

  39. metalodyssey

    You dudes ROCK! This stuff happens… and we carry on bloggin’! :)

    Like

  40. eClinik

    Wish I could have the same firepower in my basement. ;-)

    Like

  41. LDMartin1959

    I guess I don’t get it. It doesn’t make sense to me that anyone would DDoS a blog site. Unless ya’ll really, really, really pissed someone off.

    Like

  42. technogran1

    Well done all at WP keeping everything running smoothly. I also have to confess I never noticed any problem.

    Like

  43. Margaret

    Just want to thank you all for your dedicated hard work to make WordPress the very best place to post all of our words, words, and more words. … Your expertise shows and puts everyone at ease… Thanks again.

    Like

  44. elcampeador

    Hat is off to you folks. I’d assist if I could, unfortunately some wise guy just encircled my abode with kryptonite. I can’t get to a phone booth..

    Like

  45. horse11

    I Love wp as you folks are humble and willing to communicate and share!!

    Like

  46. elteee

    Thank you for taking the time to explain what the turbulence was for those who felt it. Even though I don’t really understand what you were saying (it’s like you’re speaking another language!) it’s so comforting to know that you and your team are there to help make our ride as smooth as you possibly can. It’s a GREAT ride! Thanks for all you do. It’s very much appreciated!

    Like

  47. excogitator

    Never mind buddy, WordPress still rocks!!!

    Appreciate your efforts and honesty.

    Like

  48. mymysticmuse

    WP is awesome. I’m grateful for the space you’ve created for us here Many blessings to you all.

    Like

  49. Lucky Teacher

    I started my blog on another site but grew too frustrated within two weeks. So glad I then found WordPress. The features WordPress offers make that other service seem even lamer. Even more so, I appreciate the ease with which I can find blogs on similar subjects. THANK YOU WORDPRESS.

    Like

  50. Rahul Bakale

    Thanks for this update and for your services; we appreciate this openness in communication. It is good to know what WordPress offers us in terms of the hardware.

    Like

  51. Abigail

    Didn’t have a single problem, so obviously your work paid off. Not to mention I’m getting more visits than ever. :)

    Like

  52. Walter Allievi

    keep up the good work!!!

    Like

  53. Alexander Kormilitsyn

    hi! can’t imagine those figures you are writing about. petabytes… must be something exorbital! thanks for sharing, as it’s always interesting to know the details of what been doing behind. wishing you all the best in your efforts!

    Like

  54. Ian Gardner

    GOOD SHOW! Keep it up. :-)

    Like

  55. Lady Marilyn Kay Dennis

    I only understood a very small part of all that, but the little that I did understand provokes the comment “Wow!” Thank you for keeping things running smoothly.

    Like

  56. hanin

    thanks, you guys let me sit back and relax while you banging your head over those turbulence. way more than free service I expect, then…

    Like

  57. Matthew Lowes

    Dittos. Really appreciate WordPress!!!

    Like

  58. apollodorosh

    Keep it up, so we can keep going :-D

    Like

  59. Admin

    Türkiye’den Teşekkürler. [Thanks from Turkey.]

    Like

  60. prank1989

    Matt,

    Thanks alot Matt for your this blog post. Great work your doing. I wish wordpress will be at the top of the edge. Keep it up dude. ;)

    Like

  61. bagnidilucca

    Thank you for the explanation. I thought I must have been doing something wrong.

    Like

  62. mels63galaxie

    it all looks great to me, and I am just a budding blogger. keep up the good work and thanks for keeping us informed in language that I can understand!

    Like

  63. Joey Chong

    Thank you for solving all problems and technical issues WordPress!!
    Can’t tell you how reckless I felt without my blog..
    But I know it will be up and running in no time =)

    Like

  64. Admin

    Hey Matt,
    thanks for giving wordpress users such a huge space!

    Like

  65. r3dg3

    Thanks, wordpress! You rock!

    Like

  66. kweide

    Well done

    Like

  67. Shailesh

    Thanks for updating us. I am using WP since long time. Presently I am working on a blog -Environment, Energy, Sustainability and Technology. WP helped me a lot in this initiative.

    Thanks!

    Like

  68. gonecycling

    It’s all greatly appreciated – thanks for keeping us up-and-writing.

    Like

  69. nita36

    Thank you for your efforts to keep everything going well. I appreciate your effort as well.
    from Nita

    Like

  70. jeauxmoer

    This is exceptional! Those new phrases and hardware specs leave an old techie like me speechless.

    Like

  71. cinnamon_dragon

    WoredPress is the best. Keep on going! I would be soooooo sad, if there would be no wordpress :(

    Like

  72. menchu4901

    I now understand what those odd behaviours were a few times I was doing site admin, but nothin’ to worry me as I know WordPress handles things well. Thanks guys! Keep up the good work.
    PS: With all that RAM, core processors, servers, I think it must all make up a small nation in size. :< )

    Like

  73. Follygirl

    Thank you!!!
    Greetings from Germany

    Like

  74. Debbie Bates

    Thanks WP. Our bloggy world reflects the Big Blue Marble in general- World keeps getting bigger; world has some bad days; tomorrow is another day; WP fixes big problems faster. Thank you.

    Like

  75. R.

    I LOVE WordPress. And I always will!!!

    Like

  76. BornToInspire Barbara Sherry Rose, PhD

    Matt, You and all of your team are doing a superior, outstanding job! Your dedication and commitment to excellence is so evident, and the care you have for all who blog on WordPress shines right through. Just keep going – the world needs you!

    Like

  77. Cicci

    No one can be perfect all the time.. ;-)

    Like

  78. ahmad

    thanks matt…

    Like

  79. Nyoman Wiastra

    Yeah I do believe on what you are doing on WP!

    Like

  80. illage3

    Oh well, these things happen. I am glad that you are doing something about the issues, and hope to see more themes being implemented. It must take a lot of time and money to have all those servers up and running. Keep up the good work.

    Like

  81. nobusysignal

    Kudos WordPress.

    Like

  82. Ethan Banks

    We love you, WordPress & Automattic.

    Like

  83. Olga SE

    Yes, I did notice that and frankly speaking I started to worry. But after reading this post I feel reassured. Thank you for sharing your difficulties with us. It’s so nice to realize that my small blog is being taken care of.

    Like

  84. kyriako

    Very good job… thanks for the excellent quality hosting..

    Like

  85. Maxi

    There is no substitute for hard work … and no one works harder than WordPress.

    Thank you.

    Like

  86. ScientologyParent

    Love the insight into your infrastructure! And I can appreciate what it takes to keep online through DDoS attacks – so annoying.

    But thanks for keeping it all together. We appreciate it!

    Like

  87. Hosk

    I noticed it a couple of times but you guys fixed it so quickly I wasn’t sure if I imagined it.

    Like

  88. Marissa Jean

    I’m so happy I understood most of that! Thank goodness wordpress knows what it’s doing, tumblr is constantly going down…

    Like

  89. fridal12

    You are fantastic!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    Like

  90. arzki

    That’s lot of hardware. Good thing you have some redundant systems :) From a guy coming from technical background, I know what I am talking about. Cheers!

    Like

  91. Chen Li Yong

    It’s great to know that our blog are in such a very good hand. And for worrying our blog 24/7, we will thinking to expand our blog’s content 24/7 too. Thanks for the great work. I really appreciate it.

    Like

  92. metaglossia

    I just wish to join my voice to the others in encouraging WP to keep up the good work and remain in the lead in the the blogging world. You have shown beyond all doubts that you have just what it takes to face all sorts of difficulties. Keep on and thanks.

    Like

  93. Charlousie

    Sounds good! ;)

    Like

  94. Hans

    Keep the good work up. I even didnt notice it that you were suffering from a DDoS

    Like

  95. জামাল উদ্দিন

    Thanks for keeping us up-to-date .
    WP Rocks ! Keep it up .

    Like

  96. JG

    Keeping those numbers in mind, you have some serious horsepower supporting WP and I don’t want sound ungrateful for the service we receive but I could imagine that in order to support your power house you need some huge generators to keep all those processors. I do hope that some of the energy consumed is coming from renewable energy sources … does it ?

    Like

  97. Carlos Acosta

    Thanks for the info, impressive numbers…

    Like

  98. horsewhispertje

    Thank You!

    Like

  99. sylviahubbard1

    man that sounds like a lot of computers. i knew you guys were on the job. I wasn’t worried… much!

    Like

  100. neelyssimpson

    Thanks for all the hard work you do!

    Like

Follow

Get every new post delivered to your Inbox.

Join 17,282,372 other followers

%d bloggers like this: