Why our websites crashed

As you are probably aware, the Advanced Custom Fields websites experienced catastrophic server issues this week resulting in major downtime and serious headaches.

Please accept our apologies for any frustration caused and know that we are doing everything we can to get our service’s back online.

What Happened

Our problems began on Wednesday morning when we experienced an increase in traffic to our Connect API website. This site is responsible for providing JSON update information, activating licenses and downloading plugin files. It is unclear at this stage how much the traffic increased by, as our web host are not providing us with much information – which is something we will be pursuing to better understand.

In short, the traffic hitting our Connect API was too much for the server and it crashed. We tried everything at our disposal; Upgrading CPU and Memory resources, endlessly contacting Hosting Support, and making changes to our websites. Nothing seemed to work, so we temporarily disabled the Connect API website in order to allow the remaining sites to load.

What Now

With the Connect API temporarily disabled, we are not able to process requests for activating a license, receiving update information or downloading updates via WP admin. The good news is that we have spun up a new site for handling download requests, so you can now access the plugin files via your online account as per normal.

We are currently in limbo waiting (very) patiently for a reply from our web host in regards to placing a “HTTP request limit” on our Connect API website which will help avoid downtime from too many requests. Until we hear back from them, there is not much else we can do.

Update Friday 6th July. We have currently placed a limit to the incoming connections to our Connect API website. This means that not all “update pings” will be successful, but the server will stay online.

What Next

Would you believe it if I told you we were ready to launch the all new ACF PRO version 5.7 that very same Wednesday morning? Well, we were! The timing of this server issue could not have been worse…

Or perhaps, it couldn’t have been better? If there is anything good to take away from this, it is that once we can confidently fix this server issue, we will avoid any downtime when 5.7 is released.

Looking forward, we want to first get our Connect API website back online, ensure all our websites and services are stable, then release ACF PRO 5.7.

Lastly, I would like to say a big thank-you to all our amazing customers. I have replied to a lot of support tickets today and am humbled by everyone’s understanding and support!