There was never any real doubt that the US Government continues to hoover up online data from around the world. But it has now been proven after researchers found a backup of their bulk data harvesting stored on the internet, where anyone could access it.
Cybersecurity experts Upguard have revealed that they discovered a trove of billions of social media posts and other personal data which had been collected by the Pentagon and stored without any password or other security in place, on the internet.
Billions of personal files exposed
They were discovered on September 6th this year by UpGuard’s Chris Vickery. He identified three AWS S3 buckets which were not secured and on examining the content found them to be jammed full of social media posts and website comments.
The majority of the data appears to have originated in the Middle East, Central and South-East Asia and much of the content is not in English. But some of the data did include content from US citizens and other US allies. UpGuard detailed one folder which contained posts from the Canadian online forum Connect2Edmonton, many of which were expressing anger or concerns about Donald Trump.
According to UpGuard, the most likely reason the data was unprotected was an internal issue known as a ‘misconfiguration’. They estimate that ‘misconfigurations’ can account for up to 90% of data security breaches. It is not clear how long this data had been unprotected, but any Amazon Web Services S3 user could access it and this service is free to sign up for.
Why was this data of interest?
Chris Vickery, who has informed the Pentagon of his discovery and ensured the files are now secure before making this information public, asked some very pertinent questions about his discovery in his report on the matter. “Why, for instance, were each of these posts collected? What triggered their inclusion in these repositories?”
Needless to say, no-one at either the Pentagon of the US Department of Defense was willing to comment, but Centcom, which collected the data, predictably tried to downplay the leak. “It is not collected nor processed for any intelligence purposes,” they said in a statement.
“All of the information is readily available public information related to our activities and obtained through commercial off-the-shelf programs in accordance with U.S. Code and Department of Defense policy in a consistent manner.”
When pushed on why this data had been collected, the only response offered was “to support public information gathering, measurement and engagement of our online programs.” A non-answer which they refused to elaborate on.
A timely revelation
The revelation is a timely one at a moment when the debate over the renewal of the Foreign Intelligence Surveillance Act (FISA) and particularly the controversial section 702, continues to hot up. The sheer scale and scope of this data, plus the fact that it contained information from US citizens is bound to chime with legislators still sat on the fence on this issue.
That will come as little surprise to those who understand the huge volume of data the US Government holds on both its own citizens and those of the rest of the world. But what is a shock, is how insecure that data has proved to be.
When governments argue about the importance of bulk data collection, they press the national security argument strongly and tend to gloss over the potential risk they are placing their citizens under by collecting and storing all this data.
Experts have long cautioned that bulk data collection is a huge privacy risk precisely because this data is open to exploitation from hackers and others. In this instance, they wouldn’t have had to work very hard. The data was open to anyone who signed up for a free Amazon AWS Account.
While this data may not have been the most sensitive that the US Government holds, it is still impossible to know who, if anyone, has accessed it already. Their lack of diligence in ensuring it was properly secured is massively worrying given everything else they hold about us.
Little wonder then that most people are so sceptical about the benefits of bulk data collection and so wary of the risks. Little wonder too that VPN use is on the rise in the US and around the world and predicted to shoot up in the next four or five years.
US citizens no longer trust their governments to keep their data safe and so are taking steps to protect it themselves. And on the basis of this latest revelation, they are absolutely right to be doing so!