So the census website gave us all something to laugh or cringe or rage at in some measure over the last 24 hours. If you are seeing this, you probably saw I had some things to say about it. Some in jest. Some less so. But what happened?

There are stories out this morning that the site was hacked. First of all, if I could ask the ABS to never be so stupid as to use that phrase again, that would be great. When people already have privacy concerns with the data being gathered using a generic chronically misunderstood term like “hacked” just makes you look terrible and the data insecure. Of course people have always had privacy concerns about the census, but when you are now asking for identifying information they are all the more valid.

A little more reading indicates the “hacking” was really a series of Denial of Service (DOS or Distributed Denial of Service – DDOS) attacks. At least as far as the information I am seeing. When you say “hacked” a lot of people envisage some teenage techie savant, or foreign government spy breaking into the system and copying everything to an elaborate USB stick (in fact, that is the definition). But that is not what a Denial of Service attack is. A DOS is simply an attempt to make a system so busy or overloaded it can’t be used by anyone. Just like the first 2 minutes after Ticketmaster puts Beyonce tickets on sale. Or a thousand extra people turning up to catch your train. Suddenly there is no room for you. It just gets too busy. The distinct difference is that any data stored by the website previously is generally no less secure after or during a DOS attack than it was otherwise. This is why just saying “hacked” is a terrible idea.

So, what did happen?

Well, I wasn’t there in the room, so I don’t know. What I do know is that they claim the site was taken down as a security measure after the DOS attacks. Not unreasonable. But why  then was I able to finally get a page at 11pm. Was it brought back up after only a couple of hours? And why did that page then tell me to try again in 15 minutes?

I think they were just too busy. Much too busy. Did they get hit by a DOS? Yeah, could well be. If so, I don’t know that they coped very well. Where did that DOS come from? I don’t know… who has an interest in that not going well? Though, of course, dickhead anarchists are dickhead anarchists sometimes.

Here is my problem…

“In the lead-up to census night, the ABS spent hundreds of thousands of dollars on load testing and said its servers could handle 1 million forms per hour.”

ABC Online (http://www.abc.net.au/news/2016-08-10/australian-bureau-of-statistics-says-census-website-hacked/7712216)

This seems to be true. A quick look at the government tenders website last night indicated to me probably over half a million dollars was spent on load testing the census site. But here is the thing about testing: What if you find something you don’t know how to fix? Or you only find 99% of the issues?

I still haven’t seen the whole site working (for obvious reasons), so it’s a very initial guess. But, does it look to me like the site was on the most efficient technology and deployed the most scalable way? No, not really. But they certainly tried to scale. Did they learn anything from the Obamacare rollout or even from ClickFrenzy or any of the other mass site failures that have been seen very often here and sometimes abroad? I don’t know. I would have hoped so. The result suggests not enough at least.

I also have a problem with the figure from the quote above. 1 million forms per hour. That’s 278 per second. Seems like a lot. Indeed, it is. It’s not really enough, though, is it?

Numbers I saw indicated they expected up to 16 million census responses. That seems high. There are only about 9 million households in Australia. Lets say that has grown a bit and there are 10 million. And we are all “supposed” (not technically required) to fill in our census Tuesday night. Lets say a few get in early, during the day or whatever. Maybe 2 million. Maybe a million will mail in the form. That leaves 7 million to try and fill out their forms probably in the 3 hours between 6 and 9pm (don’t want to leave it too late!) on the specified night. Yeah, well over 2 million an hour. Maybe they are more spread out… but we would probably still see a bunch of attempts around 7-8 o’clock. You might expect a peak around that time of 3 million/hour? I think that’s reasonable. So, 1 million an hour is actually quite inadequate. Furthermore, as people experience failures and come back to try again “15 minutes later” that simply increases load for those already trying those 15 minutes later.

There may have been attacks. If so, your data is probably as safe as it ever was. But, it doesn’t seem like they were really prepared for an operation on this scale. No, let me be clear. This is no reason to defund or attack the ABS. There are maybe people there who made some dumb decisions, but maybe they got the best advice available from the closest things to experts available. And maybe they were just wrong, or inadequate for the scale of the task. Regardless, the ABS still performs a critical role in the management of this country and crippling them will not help them do that job better.

Anyway, it’s the morning after census night. Mine hasn’t been lodged (I will try in 4 days – https://twitter.com/rossmack/status/763018667059650564) and who knows what really went on, but that’s my 2 cents. I guess we will find out more in the coming days.