Extraordinary Popular Delusions

"Men… think in herds … they only recover their senses slowly, and one by one."

Tag: nsa

Through a PRISM, darkly

I’m currently working my way through the many excellent presentations given at the 2013 CCC Congress (you can look through the videos yourself here). This talk, given by Kurt Opsahl, Senior Staff Attorney at the Electronic Frontier Foundation is a really good introduction to the US legal framework around the NSA’s activities. Once you’ve watched this, the next thing on your list should be Jacob Appelbaum’s To Protect and Infect, Part 2, which explains some of the most recent stories to emerge from the Snowden document cache.


UK MPs debate oversight of the security services

Earlier today, MPs took part in a three hour debate on oversight of the security services. Video of today’s three hour debate is now available here, and it’s well worth viewing:

31.10.13 Westminster Hall debate on oversight of the security services

Of particular note are the exchanges between members of the Intellifenge and Security Committee (ISC) and Parliamentary colleagues, which reveal that no scrutiny of Prism or Tempora took place in that committee before Edward Snowden’s disclosures put the existence of those programmes into the public domain. It is not at all clear that members of the committee knew what GCHQ was up to until the Guardian drew their attention to it.

A full transcript of the debate should be available soon (here) and I’ll highlight some of the key passages when it is.

Update (4/11)

I promised to identify the sections of the debate which tackled the degree of information open to the ISC, particularly about the PRISM and Tempora programmes. The first came about in a question from Tom Watson to George Howarth, a member of the ISC:

Mr George Howarth (Knowsley) (Lab):

Let me demonstrate that by reference to the issue that the hon. Gentleman has talked about at some length, and legitimately so. I am talking about the Prism programme—what the UK’s involvement in it was and so on. Not once during his speech, unless I missed it, did he refer to the fact that the Intelligence and Security Committee, which he considers to be inadequate, has already looked at the Prism programme and what our own agencies’, and particularly GCHQ’s, involvement in and knowledge of that was. We issued a statement—an interim statement, I might add—in July. In the course of that statement, which has not been referred to so far, we arrived at some important conclusions. The first one was:

“It has been alleged that GCHQ circumvented UK law by using the NSA’s PRISM programme to access the content of private communications. From the evidence we have seen, we have concluded that this is unfounded.”

For obvious reasons, it is impossible for me to go into detail about all the evidence that we were able to look at, but we did look in detail at very important pieces of information and we were able also to look at what authorisations were involved in the process of accessing the information, particularly the communications within it. The law has not been broken.

Mr Watson: I am reassured by my right hon. Friend’s thoroughness in the investigation. Was July the first time that the Committee had examined Prism, and was that after the Guardian revelations? [Laughter.]

Mr Howarth: It was after the Guardian revelations. The hon. Member for Cambridge seems to think that that is funny. Actually, he would still be sitting here today if we had not gone and looked at this matter after the allegations emerged. He would be accusing us of being inadequate in our responsibilities.

So, the ISC did not examine GCHQ’s involvement in PRISM before information about the programme’s existence reached the public domain. That could mean that the committee didn’t know about it, or knew about it and chose not to concern itself with it. George Howarth was pressed on the issue of whether the ISC knew about the programme by Rory Stewart – and his answer is incredibly evasive.

Rory Stewart: Will the right hon. Gentleman clarify why the Committee did not look into Prism before The Guardian published its allegations?

Mr Howarth: Let me answer the hon. Gentleman very carefully; I hope that he will forgive me for being none too specific in my answer. Part of our responsibility, which did not just emerge after the revelations about Prism, is to look at what the agencies do, what their capacities are and how they use those capacities. It is a continuous process. We have in the head of GCHQ. We take evidence. We probe what it is doing and what it is capable of doing. Therefore, it is not that we did not have any concerns or any interest in what GCHQ was capable of. That is an ongoing process, but inevitably, when something new emerges, it is appropriate that, as a Committee, we look into it.

I have answered the hon. Gentleman’s question perhaps not as accurately as he would have liked, but—I am not being evasive when I say this—if I went any further, I would be going into detail that at this stage I do not think is relevant.

The issue was later put to the chair of the committee, Sir Michael Rifkind, who refused to answer the question:

Mr Meacher: Will the right hon. and learned Gentleman explain why the Committee did not find out about the Tempora programme when it began to operate?

Sir Malcolm Rifkind: The right hon. Gentleman does not have the faintest idea whether the Committee was aware of programmes of any kind. We are given classified information, and the whole point of an independent Committee having access to top secret information, whatever that is, is that we do not announce what such information is. If he can devise a system whereby secret information can be made available to all law-abiding British citizens, without its being simultaneously made available to the rest of the world, I am interested in hearing about it, but I do not think that he is likely to meet that requirement.

Also of note was the question put by Julian Huppert to the Under-Secretary of State James Brokenshire – but answered by Michael Rifkind:

Dr Huppert: The Minister makes the extremely good point that it is “past operations” that can be looked at, and there are constraints on what the ISC can look at; it does not have a completely free rein on operational matters. What happens if an operation lasts for many, many years? At what stage is there any sort of scrutiny of that?

James Brokenshire: To be fair to the hon. Gentleman, he took part in the consideration of the Justice and Security Act 2013, although he did not make then a number of the points that he has made this afternoon. However, we need to be very careful to ensure that scrutiny does not seek to cut across into direct, ongoing operational activity. I am quite sure that, given the robustness of the new powers that the ISC itself will hold, that consideration is very much in the forefront of the minds of the Committee members.

Sir Malcolm Rifkind: In response to the perfectly reasonable issue raised by the hon. Member for Cambridge (Dr Huppert), I must say that this point was seized on by the ISC itself. We have completed discussions with the Government, the results of which will appear in a memorandum of understanding that will be published and include details of how these matters will be dealt with. That will ensure that that consideration cannot be used as an improper way of preventing the ISC from obtaining access to operations that—by any normal, common-sense approach—could be considered as completed.

Finally, as a reminder of the quality of rhetoric that tends to prevail when issues are not subjected to proper scrutiny:

Mr Adam Holloway (Gravesham) (Con): If in the last few weeks, we had lost a city to nuclear terrorism or there had been a gigantic mass casualty, I wonder whether the hon. Gentleman’s constituents would see Edward Snowden as a trendy, cool whistleblower or as a traitor.

Fourth European Parliament hearing on surveillance: special whistleblower edition

Monday’s fourth #EPInquiry hearing was relatively well-reported, largely because Edward Snowden supplied a statement, delivered to the inquiry by the Government Accountability Project’s Jesselyn Radack.

Audio of the full hearing is available here, thanks to Henrik Alexandersson, who has also posted the audio of the previous three hearings.

The speakers were Marc Rotenberg (EPIC), Catherine Crump (ACLU), Thomas Drake (NSA whistleblower), J. Kirk Wiebe (NSA whistleblower), Annie Machon (MI5 whistleblower), Jesselyn Radack (Government Accountability Project) and John Devitt (Transparency International). Video of the following presentations has been made available by the Government Accountability Project:

Jesselyn Radack

Thomas Drake

J. Kirk Wiebe

The next hearing is tomorrow, Thursday 3rd October and one of the subjects up for discussion will be GCHQ’s aggressive actions against the Belgian national telecoms company, Belgacom – whose clients include the European Parliament. Unfortunately, GCHQ’s director has declined the opportunity to justify himself in front of the Committee.

Missed my posts on the first three #EPInquiry hearings? Find them here (one, two, three).

Update (3/9)

Full video of the hearing is now available:

A few #EPInquiry-related updates

Next #EPInqury hearing tomorrow

In my last few posts, I’ve been tracking the European Parliament Inquiry into surveillance in and by EU member states (first hearing, second hearing).

Tomorrow (Tuesday), the Civil Liberties Committee (LIBE) holds its third hearing, which it trails as follows:

There are five sessions foreseen in the programme focusing on “Allegations of NSA tapping into the SWIFT data used in the TFTP programme”, “Exchange of views with US Administration”, “Feedback of the meeting of the EU-US Transatlantic group of experts on data protection of 19/20 September 2013″, “Exchange of views with US Civil Society (part I)” and “Presentation of the study on the US surveillance programmes and their impact on EU citizens’ privacy”.

The study referred to on US surveillance programmes and their impact on EU citizens’ privacy, prepared by Caspar Bowden is available here.
 The hearing will be broadcast live from 8am UK Time.

More on The Athens Affair

Jacob Appelbaum’s presentation to the first #EPInquiry hearing used an incident in Greece in 2004-5 as a potential example of NSA interference abroad which is not subject to any meaningful limits whatsoever:

the NSA is not bound by European laws and they don’t care what your laws say. So when you say it would be proportionate and balanced to wiretap people for the purposes of terrorism, you are also tacitly endorsing the NSA to wiretap everyone in your country without any judicial process or any proportionality whatsoever.

This is what happened in Greece with the Athens affair, almost certainly – we don’t know it was the NSA, but it was an actor with sufficient capabilities. They were able to wiretap the Prime Minister as well as Members of Parliament. It also moves the risk from a world that was military to one where someone operates a computer and they’re your last line of defence between your Prime Minister being wiretapped or not.

In the case of the Vodafone incident in Greece, the person in charge of that telephone switch was found hanged to death in his apartment. And the reason is he wasn’t trained to do these things or defend an entire nation in that way. So it[NSA impunity] changes the balance of power in a very serious fashion.

Most of the reporting on the Athens Affair in the English-language media appeared in 2007 when the news initially broke. Greekemmy has now updated the story at WikiLeaks-press.org with information on the evidence turned up by a subsequent public inquiry in 2010-11. This inquiry identified the US Embassy in Athens as the agency responsible for the interception. An announcement of a criminal investigation into US embassy employees followed, but this seems to have been quietly dropped.

European Parliament holds second surveillance inquiry hearing

Following on from my last post, I’m just catching up with the second hearing of the European Parliament’s Civil Liberties Committee into surveillance in and by EU countries. This was held on Thursday 12th September and, like the first hearing, was divided into two sessions.

The first, private, session saw MEPs briefed on the results of a meeting between EU and US data protection experts back in July. There were two strands to the EU’s response to PRISM in mid-June; one was the public inquiry arranged by the European Parliament and the other was the ad hoc working group formed by the Council Presidency and Commission doing the reporting in this closed session.

The second session included a briefing from the Chair of the Article 29 Working Party, Jacob Kohnstamm, on the impact of surveillance on privacy and US-EU Data Protection Agreements. Audio of this second session has been released on the EU website  – although it’s not the most user friendly interface I’ve ever encountered.

Documents from the meeting are also available here.  Of these, Kohnstamm’s letter to EU Commissioner Viviane Reding forms the basis of his presentation to the Inquiry and is certainly worth looking at.

It also needs to be clarified if these American intelligence programs are in line with European and international law. This includes the International Covenant on Civil and Political Rights, which lays down the right to privacy in a general way. More importantly, the necessity and proportionality of these programs according to the Council of Europe Convention 108 needs to be further assessed. WP29 therefore considers it is likely that the current practice of apparent large-scale collection and accessing of personal data of non-US persons is not covered by the Council of Europe Cybercrime Convention. This is particularly relevant in light of the on-going discussion within the Council of Europe Cybercrime Convention Committee (T-CY) on the preparations for an additional protocol meant to facilitate trans-border data flows in this field.

Documents relating to the first #EPInquiry hearing have also been released.

The next #EPInquiry hearing is scheduled for 24th September:

There are five sessions foreseen in the programme focusing on “Allegations of NSA tapping into the SWIFT data used in the TFTP programme”, “Exchange of views with US Administration”, “Feedback of the meeting of the EU-US Transatlantic group of experts on data protection of 19/20 September 2013”, “Exchange of views with US Civil Society (part I)” and “Presentation of the study on the US surveillance programmes and their impact on EU citizens’ privacy”.

Update (19/9)

Kohnstamm does not understate the importance of the Snowden revelations (this from the audio clip):

Based on the reports… it is highly likely that the fundamental rights of human beings have indeed been infringed on… The fundamental trust between government and citizens is at stake.

He also makes clear that the surveillance activities of EU member states will also need to be assessed for their compliance with international law and EU standards, which may themselves need to change to offer better protection for individuals’ privacy.

Beware spooks bearing gifts

There’s much in yesterday’s batch of Snowden revelations that still needs to be explained fully – this blog post by Matthew Green offers the most useful analysis I’ve seen so far.

In the meantime, this paragraph from the New York Times’ version of the story (as tweeted by Trevor Timm) caught my eye:

Even agency programs ostensibly intended to guard American communications are sometimes used to weaken protections. The N.S.A.’s Commercial Solutions Center, for instance, invites the makers of encryption technologies to present their products to the agency with the goal of improving American cybersecurity. But a top-secret N.S.A. document suggests that the agency’s hacking division uses that same program to develop and “leverage sensitive, cooperative relationships with specific industry partners” to insert vulnerabilities into Internet security products.

This caught my eye because it reminded me that, just this summer MI5 and GCHQ offered a “cyber-health check” to all FTSE 350 companies as a prelude to “an in-depth discussion with each company’s audit firm about areas in which a company may be particularly vulnerable.” In response to this announcement, John Colley, managing director of (ISC)², a membership body for information security professionals, questioned whether the methodology of the “health check” – asking company chairs, rather than technicians, to fill out a questionnaire – was likely to be draw out a well-informed response:

Logically, infosecurity professionals are better placed to provide such information as they are dealing with security issues on a day-today basis, they have knowledge of the exact security measures in place within their organisation and insight into areas where more investment is needed as they closely monitor the evolving threat landscape, and so are more likely to provide the relevant and accurate data.

Colley went on to note that it was not clear if audits were mandatory and sounded a note of caution over what might happen to data the authorities went over the heads of security professionals to obtain:

It is also unclear as to what the GCHQ and MI5 will do with the information revealed by these cyber-audits.  In this age of state sponsored cyber-attacks and PRISM, there are great sensitivities surrounding governments’ objectives for accessing data.

The “cyber-health check” is just one of a number of initiatives central government has recently launched in the area of cyber-crime, several of which are aimed at private companies. Some of this activity may be well intentioned, no doubt, but we also know from yesterday’s reports that GCHQ have a specific programme that focuses on compromising VPNs, the means by which many large companies enable employees to securely access their systems from outside the office:

By 2010, the Edgehill program, the British counterencryption effort, was unscrambling VPN traffic for 30 targets and had set a goal of an additional 300.

Ongoing revelations about Anglo-American attempts to undermine the fabric of online security make it difficult to assume good faith in this area. It is certainly interesting that the initial approach of the “cyber-health check” is being made to senior corporate positions, rather than those in the best position to weigh up the potential risks of such an approach.  Ultimately, if security of information is a selling point for any FTSE 350 company, they might be well advised to be wary of spooks bearing questionnaires and promises of audits.

Update (9/9)

This post started with a link to Matthew Green’s excellent discussion of the latest NSA revelations. Today it has emerged that the author has come under pressure to remove his post from the servers of his employer (Johns Hopkins University). The mirrored version of the post on university servers has in fact been removed.  It is not clear from where the impetus for this move originated, but Green has said that “this isn’t my dean’s fault.”

While there is no reason to suspect that Matthew Green’s post will disappear from Blogger, it is sensible to take precautions. The first link in the previous paragraph will take you to an archived version of the post.

Update II (10/9)

The move from John Hopkins became a textbook example of the Streisland effect – and it does not look like direct external pressure was involved. Ars Technica provides a comprehensive account here.

Update III (24/9)

Australia’s Security Intelligence Organisation (ASIO) is taking a different approach (“Unlike the UK government’s cyber security evaluation centre, the ACSC’s offer to the private sector will not focus on vetting technology equipment”), inviting private business to co-locate within their new headquarters.

A senior analyst at the Australian Strategic Policy Institute, Dr Tobias Feakin, welcomed the move to integrate private firms into the new cyber operations centre, but said companies would have to be “willing to share data with government, otherwise momentum will be lost and they won’t keep their focus on such efforts”.

#Miranda: some additional notes on reliability, legality and security

The past few days have turned up some articles that shed further light on the subject of David Miranda’s detention (which I have previously blogged on here and here).

The reliability of Oliver Robbins

Ryan Chittum, a writer for the Columbia Journalism Review, was cited in Oliver Robbins’ witness statement. In another piece for CJR, Chittum takes issue with the way his writing was used by Robbins and demonstrates how selective quoting meant that the original sense of his piece was lost:

Here’s Robbins:

In an article published on the same day by the Columbia Journalism Review (“Guardian bombshells in an escalating battle against journalism”) Ryan Chittum wrote that the claimant “was serving as a human passenger pigeon, shuttling encrypted files on USB drives between filmmaker Laura Poitras and Greenwald”.

And here’s what I actually wrote:

Miranda was serving as a human passenger pigeon, shuttling encrypted files on USB drives between filmmaker Laura Poitras and Greenwald because, as the whole world now knows, the Internet is fully bugged by the US and UK governments.

Chittum’s conclusion on the reliability of Oliver Robbins’ statement is worth noting:

If it were just a clipped quote, there wouldn’t be much to protest here. But that kind of thing raises questions about what else in Robbins’s testimony isn’t all there. It turns out that Robbins uses selective quotes, specious reasoning, questionable numbers, and flat-out disingenuous claims to make his case that journalists merely possessing secrets was a grave danger to the United Kingdom.

UN Special Rapporteurs question the legality of Miranda’s detention

The Guardian reports that two UN Special Rapporteurs, Frank La Rue (who holds the UN’s free expression brief) and Ben Emmerson (human rights and counter-terrorism) have written to David Cameron to request further information on the grounds for David Miranda’s detention under Schedule 7 powers which, as Ben Emmerson notes, are currently the subject of challenge in the European Court of Human Rights.

This follows a similar move from the Council of Europe, whose Secretary General Thorbjorn Jagland wrote to Home Secretary Theresa May a few days after David Miranda was detained, questioning whether UK actions might have a “chilling effect” on journalists’ freedom of expression, as guaranteed in Article 10 of the European Convention on Human Rights,


One of the more important practical conclusions to be drawn from my analysis of the UK Government’s witness statements in Home Office v Miranda, one I maybe should have drawn out more clearly, is that – as far as we can tell – encryption works. Despite the presumably rather large resources UK authorities have dedicated to this problem, they have only been able to decrypt, and read the contents of, the encrypted file they had the password for.

Related to this, and prompted by another series of Washington Post articles sourced by Edward Snowden, Bruce Schneier wrote a very interesting article for Wired this week on what the NSA (probably) can and can’t do.

So learning to use TrueCrypt is a worthwhile use of your time. For those wondering where to start, the tutorial on the TrueCrypt website tries to ensure that you understand the process before taking any major steps. Alternatively, attending a CryptoParty – like this one proposed for Mozfest in London next month – may be useful if you want to discuss the process with someone face to face. Journalists working with extraordinarily sensitive data may want to bear this in mind too.

Update (6/9)

If you’re wondering where yesterday’s Snowden stories in the Guardian, New York Times and ProPublica leave my statements above, this post will explain more.

Update II (7/9)

Glenn Greenwald discussed David Miranda’s detention and what the UK Government had to say about it on yesterday’s edition of Democracy Now. Here’s what Glenn said regarding the UK witness statements:

He hasn’t gotten any of his belongings back. And one of the things that happened is that the U.K. government just outright lied about what took place that day. They claimed he was carrying a password that allowed them access to 58,000 classified documents. He was not carrying any password that allowed them access to any documents. They actually filed an affidavit the same day they made that claim, saying—asking the court to let them continue to keep his belongings on the ground that all of the material he was carrying was heavily encrypted, that they couldn’t break the encryption, and they only got access to 75 of the documents that he was carrying, most of which are probably ones related to his school work and personal use. But, of course, media outlet has just uncritically repeated what the U.K. government had said, as though it were true. It wasn’t true; it was a pack of lies. But even if it were true, the idea that you’re going to detain somebody under a terrorism law who you think is working with journalists is incredibly menacing, as menacing as anything the U.K. government denounces when other countries do it.

Thanks to those in the comments here and on twitter who alerted me to this interview.

#Miranda: Where is the UK Government getting its numbers from?

A few days ago I blogged on hints Glenn Greenwald made about witness testimony the UK Government was due to give in court about its grounds for continuing examination of electronic material confiscated from David Miranda.

In that blog, I suggested that if the UK Government really had only managed to decrypt “something like 75 documents”, it cast their assertions about the number of documents Miranda was carrying in a rather different light. Many news organisations have taken the “58,000 documents” figure as fact. But what is it really based on?

The court hearing was heard yesterday afternoon and, at its conclusion, Government lawyers released the testimony of Oliver Robbins, a senior civil servant who has held intelligence related positions in the Cabinet Office under the present and last governments. His is the securocrat’s voice par excellence.

At the outset, it should be noted that Robbins’ testimony isn’t the court filing Greenwald was referring to in the comment that prompted my last blog. That, it transpires, was a separate statement by Detective Superintendent Caroline Goode, from the Metropolitan Police’s Counter-Terrorism Command. Goode’s statement has not been released in full, but sections from it have been reported in the press. The fullest account of Goode’s statement, from which many of the others are drawn, is this Reuters piece.

Let’s look at what we know of Goode’s reported statement first.

Caroline Goode’s evidence

Use of TrueCrypt

Detective Superintendent Goode said that the information on the external hard drive was encrypted by a system called “True Crypt [sic],” which she said “renders the material extremely difficult to access.”

This is useful information. First of all, note the use of the word “access” to mean “access in readable form” and that Goode’s comments relate to just one of the devices taken from Miranda.

TrueCrypt is widely used encryption software that is free to use and download; many of those reading this blog will be familiar with its features. For those who aren’t, the TrueCrypt homepage describes what this software does (I’ve preserved the hyperlinks to more detailed resources on the Truecrypt website for those who want to read further):

Main features:

  • Creates a virtual encrypted disk within a file and mounts it as a real disk.

  • Encrypts an entire partition or storage device such as USB flash drive or hard drive.

  • Encrypts a partition or drive where Windows is installed (pre-boot authentication)

  •            (…)
  • Provides plausible deniability, in case an adversary forces you to reveal the password: Hidden volume (steganography) and hidden operating system.

Knowing what TrueCrypt does is useful because it gives us a good basis on which to assess the validity of subsequent statements. Note that TrueCrypt encrypts entire hard drives, or portions of them, rather than individual files. An area of a hard drive that has been encrypted with TrueCrypt is very much like a container you can drop files into. You need a password to open the container before you can access the files within it. This container is often called a TrueCrypt file but it can also be called a TrueCrypt volume.

60 GB of data and only a third of it “accessed”

Goode said the hard drive contained around 60 gigabytes of data, “of which only 20 have been accessed to date.” She said that she had been advised that the hard drive contains “approximately 58,000 UK documents which are highly classified in nature, to the highest level.”

Note first of all that Goode is still discussing only one of David Miranda’s electronic devices – an external hard drive . She then notes that only a 20GB portion of that external hard drive has been “accessed” – which either means that the remaining 40GB data is inaccessible (presumably because it is contained within one or more encrypted TrueCrypt volumes), or that the police simply haven’t got around to examining them. Given that Goode’s colleagues have now had access to that external hard drive for nearly two weeks, the former possibility is presumably the more likely of the two.

Incidentally, there is nothing in Goode’s statement to say that we’re dealing with a 60GB hard drive. The external hard drive could just as well be one of larger capacity holding only 60GB of data.

Finally, Goode “has been advised” about what the hard drive as a whole contains. This is not knowledge that she has determined herself, independently, from access to those 20GB of data. It seems odd that Goode’s reported statement about the content of the drive, including the 40GB of data she has not been able to “access”, does not rely to any extent on the 20GB she has.

“Only 75 documents have been reconstructed

Goode said the process to decode the material was complex and that “so far only 75 documents have been reconstructed since the property was initially received.”

This is the statement that Glenn hinted at earlier this week.

“Reconstructed” is a strange word for Goode to use. The most natural interpretation is to see “reconstructed” as a synonym for “decrypted” or “put into a form that can be read”, although this doesn’t really fit in with the idea of a “complex” process. They may not have the technical nous of Edward Snowden, but I assume that Counter Terrorism Command are familiar with the process of mounting an encrypted TrueCrypt volume and typing in a password.

So what else could Goode mean here? It’s easy to exclude a few possibilities: even if the Met and GCHQ were trying very hard to open an encrypted volume by brute force, they wouldn’t be able to individually decrypt the files within it one by one.

What Goode could mean is that analysts have been able to recover deleted files from unallocated space on the hard drive (space that isn’t being used for data now, but may have been in the past). That, at least, is more of a fit for the idea of a “complex process.”

Let’s leave the vagueness about where the files came from to one side for the moment.  Are there any other insights we can draw from Goode’s statement?

The first thing to note is that 75 documents out of an estimated total of 58,000 is an absolutely tiny proportion. It is difficult to see how such a minute sample could give a true indication of the entire collection of material held unless one or more of those decrypted files served as a kind of index to the whole. Indeed, if the files have been reconstructed from unallocated space – meaning they had previously been deleted – then they may tell you even less about what is currently on the drive.

There’s a further ambiguity when Goode talks about “the property” – is she referring to the external hard drive here, or Miranda’s confiscated belongings as a whole?  If the latter is the case, then it is by no means certain that the “accessed” 20GB portion of the external hard drive contains any documents at all – those 75 could have been obtained from elsewhere.

If we take the opposing view and suppose that Goode’s “the property” means only the external hard drive discussed previously, then those 75 documents came from the “accessible” 20GB portion of the external hard drive or were recovered from unallocated space. Caroline Goode’s evidence could just as easily mean one of these scenarios as the other: it is remarkable for the range of possibilities it does not exclude.

Summary of Caroline Goode’s evidence

Caroline Goode’s evidence suggests that David Miranda’s hard drive contains a TrueCrypt volume or volumes of a total size of 40GB that UK police have no access to. The 20GB encrypted portion of Miranda’s external hard drive that the police have been able to access contains, at most, 75 files. It is possible that some – or even all – of those files came from other devices, or from unallocated space on the same device.

Goode’s statements about the remainder of the documents do not seem to be based on insights gained from the 75. This would tend to support Glenn Greenwald’s assertion that UK police have not been able to access anything sensitive. It certainly does not clarify how the total figure of 58,000 documents the Home Office has asserted is on Miranda’s external hard drive has been arrived at.

Oliver Robbins’ evidence

What follows is a close analysis of Oliver Robbins’ testimony – and I do think it deserves to be looked at very closely indeed. There is much in Robbins’ statement that deserves detailed analysis but, for the purposes of this blog post, I will restrict my attention to Robbins’ comments on the UK Government’s access to, and analysis of, the Miranda data.

Indefinite room for ambiguity.

[in justifying why the Government needs “continuing access” to the material seized from Miranda] … no information that has so far been analysed by Her Majesty’s Government (“HMG”) has identified a journalist source or has contained any items prepared by a journalist with a view to publication. The information that has been accessed consists entirely of misappropriated material in the form of approximately 58,000 highly classified intelligence documents. [para 6]

The first thing to note here is that Robbins’ use of the word “accessed” is different from Goode’s. As we saw above, when Goode talks about data “accessed” she means data that can be accessed in readable form. Robbins’ use of the word is broader because his witness statement is making an argument about the Government’s need for “continuing access” [para 5] to all the material seized from Miranda, including that which has not been decrypted. Robbins’ use of “access” therefore more closely corresponds to the idea of physical access to the  devices themselves. This is confusing.

Robbins goes on to talk about a subset of  the information that has been “analysed.” We are not told whether this means analysis of encrypted information, but given that he goes on to make statements as to the content of this information, it is likely to be the case that this information can be read in some form. What Robbins says about this analysed material is that none of it “has identified a journalist source” and neither does it contain “items prepared by a journalist with a view to publication.”

Of course, Robbins’ purpose here is to reject the idea that the Miranda material contains anything that should be withheld from examination, but It’s worth noting that the category of data which meets those two stipulations of his is quite a wide one: it includes shopping lists, youtube videos of cats and many other items of limited relevance to national security.

What Robbins says next is interesting: he moves straight from a limited description of a small subset of data to make a claim about the entirety of the Miranda material (“that has been accessed”). Putting to one side for the moment the ambiguity about whether Robbins is really talking about Goode’s external hard drive here or the Miranda devices in total, It is not at all clear on what he is basing this rather striking claim.

Let’s think about this situation in a different context. Imagine if you had a bookcase that, apart from a couple of volumes, consisted only of books with unopened pages. What Robbins says would be like asserting that all the books in the bookcase are illustrated, purely on the basis that, of the two books you can examine without a penknife, neither was printed in London or inscribed with the owner’s name. It is certainly a claim that can be made, but not one that deserves to be taken particularly seriously.

Wait, so it’s not your assertion after all?

I am advised that the data recovered from the claimant is almost certain to contain some of the material passed by Mr Snowden to Ms Poitras and Mr Greenwald. Much of the material is encrypted. However, among the unencrypted documents recovered from the claimant was a piece of paper that included the password for decrypting one of the encrypted files on the external hard drive recovered from the claimant. I have been briefed that the authorities have therefore been able to examine the data contained in this file. They have been able to determine that the external hard drive contains approximately 58,000 highly classified UK intelligence documents. Work continues to access the content of the other files on the hard drive and the USB sticks. [para 13]

There’s a lot in this paragraph, so let’s take it line by line. The first sentence seems to answer the question posed in the previous section: Robbins’ assertion about the content of the Miranda data is second hand after all (“I am advised”).  It is also indefinite (“almost certain”) which seems to contradict the conclusive phrasing (“the data that has been accessed… consists entirely of”) of the previous paragraph.

Once again, this is confusing – so let’s try to resolve the contradiction. Is it possible that, when Robbins talks about “the data that has been accessed” in paragraph 6 he is slipping between the broad interpretation of the word “accessed” he has used in his previous sentences and the narrower sense – that of data that can be read and analysed – used by Caroline Goode? It’s much easier, after all, to be definite about the content of documents you’re able to read than ones you cannot.

I’m not sure this works either. Goode testified that the material “accessed” in the sense that it could be “analysed” amounted to a 20GB portion of an external hard drive, which may contain all, or maybe only some, of a total of 75 documents. To say this consists “entirely of misappropriated material in the form of approximately 58,000 highly classified intelligence documents” is just a nonsense.  Robbins must therefore be using the word “accessed” in his usual sense and what he says is inconsistent with his previous paragraph.

Does the rest of paragraph 13 make things any clearer? Certainly, the next three sentences are straightforward. We know that “much of the information” carried by Miranda was encrypted and that Caroline Goode and her colleagues were able to decrypt one encrypted file on the external hard drive. By Goode’s own account, she and her colleagues were able to examine the data contained within this file. These sentences are consistent both with Robbins’ own statement and those of others.

What follows is much more troublesome. “They [the authorities] have been able to determine that the external hard drive contains approximately 58,000 highly classified UK intelligence documents.” The analysis of Goode’s statement shows that she and and her colleagues could not derive the presence of “58,000… documents” from what she found – and she didn’t claim to have done.

But have I missed something here? Could it be that Robbins’ “they” isn’t referring to Goode and her police colleagues at all? Could he be referring to different “authorities” altogether? Might they be the same authorities who “advised” both Robbins and  Goode of “58,000 documents” figure and on whom both rely?  I think that is likely and, although a casual reader may feel that the two sentences below bear a logical connection, in fact they do not:

I have been briefed that the authorities have therefore been able to examine the data contained in this file. They have been able to determine that the external hard drive contains approximately 58,000 highly classified UK intelligence documents.

In my opinion, this comes close to being a misleading statement. Oliver Robbins could equally well have expressed himself as follows:

I have been briefed that the authorities have therefore been able to examine the shopping lists and pictures of cats contained in this file. Independently of this, others have been able to determine that the external hard drive contains approximately 58,000 highly classified UK intelligence documents.

GCHQ’s assessment

And what of that troublesome “58,000… documents” claim? The source for Robbins’ second authority becomes clearer in his next paragraph:

On the basis of GCHQ assessments, the totality of UK intelligence documents that would potentially have been accessible to Mr Snowden while we was working at the NSA is consistent with the volume of documents which we know to be on the external hard drive. [para 14]

This appears to be the best candidate for what the “58,000 documents” figure is actually based on. But what does it amount to? Let’s turn to “the volume of documents which we know to be on the external hard drive” first.

What we know about the external hard drive is that it is divided into at least two encrypted files, one of 20GB which the police are able to access and a further encrypted file (maybe more than one) of 40GB size. Because the police have access to the decrypted 20GB file, they can make an assessment about the number of documents within it (a maximum of 75). All that can be said about the other file(s) is that they have a total size of 40GB.

An encrypted file’s size is not dependent on the amount of data it contains.  A 10GB encrypted file could contain 10kb data or 6 GB data – unless you can decrypt the file, you have no way of telling which is the case.

As such, GCHQ’s statement is almost meaningless. You could say that the maximum volume of documents an encrypted file could contain is 40GB – but that’s something you could say of any 40GB encrypted file. GCHQ’s assertion about “the volume of contents which we know to be on the external hard drive” appears to play on an ambiguity in the word volume (one can talk about a volume of documents, but it’s also a synonym for an encrypted file) in order to hide that it has no basis in fact.

In essence, what GCHQ seems to be saying here is that what it assesses to be “the totality of UK intelligence documents… potentially accessible to Mr Snowden” would fit on a 40 GB hard drive. That logic, if applied widely, could lead to an awful lot of Schedule 7 detentions at our airports and it’s an assessment made entirely independently of the Miranda data.

So, where does that leave the “58,000 documents” figure? Nowhere good. It looks like nothing more than a worst-case scenario GCHQ based on guesswork but presented as indubitable fact.


Neither of the witness statements presented by the UK Government in Home Office v Miranda are adequately precise about the matters they raise.  Cryptographers have developed a vocabulary that is adequate to expressing these subjects with clarity – when they talk about “plain text” and “cypher text”, others understand what they mean. In contrast, when Caroline Goode and Oliver Robbins use terms like “access” and “analysis” in their statements, there is significant ambiguity in what they mean. This ambiguity leaves real potential for confusion; it also presents unacceptable opportunities for others to be misled.

I am concerned by the extent of the ambiguity in the statements presented in Home Office v Miranda. The UK Government has represented itself in language that is so vague that it may not have a case at all, yet it has presented its case in the strongest way possible – and has been accepted as such, without much demur, in much of the media.

I think it’s worth taking a moment to reflect on this. If a group of witness statements took a similar approach to legal issues as these have to technical ones, if they had eschewed technical terms in favour of ambiguous natural language and took advantage of that fact to obfuscate as these have, I think those imaginary witness statements would have received a much more critical reception.  I am concerned that our courtrooms and our newsrooms may not be equipped to cut through some of this confusion and dubious statements may be allowed to stand without receiving proper scrutiny. It is not difficult to see how parties could take advantage of this, if they wished to do so.


While I know what TrueCrypt is, I am by no means a technical expert. My intention in this piece is to show how ambiguous the UK Government’s statements are, rather than put together a definitive account of what happened – I’m not sure that’s even possible on the evidence available.

The Q&As that follow below are an outlet for some of the fun speculative stuff I couldn’t justify putting in this post.

If there’s something you think I’ve got wrong in this piece, I’d be very interested to hear about it. Please email me or leave a comment below.


Have Greenwald, Miranda and Poitras been guilty of “very poor judgement in their security arrangements”?

Travelling with a password written on a piece of paper isn’t great. Transiting through Heathrow may have been inadvisable. But, if – as seems very possible – nothing of significance has been  compromised you have to say that, on the face it it, not really.

Given that the Cabinet Office expressed its worries to the Guardian in terms of their ability to protect information from cyber attack, I think it’s relatively clear why the Government would like to cast doubt on others’ security practices if possible.

Is the 20GB encrypted file on the external hard drive a dummy volume intended to be surrendered without cost?

The thought has crossed my mind: it would certainly make it easier to explain why David Miranda was found in possession of an encryption key in a UK transit area. I am not sure it is possible to say for sure on the evidence of the statements presented, but I think this falls within the range of possibilities.

Is it possible that one of the 75 files the police have is an index to the rest?

It is possible – and if the case would make the “58,000 documents” figure much more credible – but I think on the balance of probabilities it is unlikely.

Were GCHQ just plucking a number out of the air with that “58,000 documents” thing?

Not entirely. One possibility is that they’ve plucked a number out of the Guardian.

On 2 August, the Guardian printed a fascinating feature article that is based partly on GCHQ’s internal “GCWiki”, making reference to this and many other GCHQ documents. That, and the discussions we know the Cabinet Office have had with the Guardian may have formed the starting point for GCHQ’s worst-case estimate.

Are you sure? They must know what Snowden has!

If the NSA doesn’t know what Snowden has, there’s no reason why GCHQ should.

Oh come on. if we’ve learned anything from the Snowden files it’s that GCHQ and the NSA have other ways of acquiring this kind of information.

Of course. Whether surveillance information is admissible in court is another matter, though, and one we should probably leave to David Miranda’s capable legal team.

Have the media been negligent in reporting the “58,000 documents” figure as fact?



Update (2/9)

This post proved to be quite a popular one, with 7250 page views yesterday alone. It also provoked quite a bit of discussion – I’d like to thank all of those whose contributions prompted me to make the following additions to my Q&A section.

Do you think Miranda was using a hidden volume?

It’s certainly a possibility and the first (pre-publication) draft of this post did in fact make that suggestion. Why did I leave it out? Because while the facts in Goode and Robbins’ statements do not exclude the possibility of a hidden volume, they also do not exclude a number of other possibilities. There’s nothing in the statements analysed to rule out the possibility that, for instance, police found a 20GB .tc file and a 40GB .tc file on that external hard drive but can only open the former.

Of course, this is yet another example of how the two witness statements are not adequately precise.

Why do you rule out the possibility that one of the files police have been able to access is an index to the rest?

I don’t rule it out, I say that – on the balance of probabilities – it is unlikely. Some of the reasons why I continue to think this are covered in this storify. Other very relevant points have been made in the comments section below.

Which media sources have used the 58,000 documents claim?

That’s an easy question to answer. A very cursory examination of articles published on this subject will reveal sources which take the “58,000 documents” claims as fact without even mentioning that they originated from a government witness statement (one, two, three, four).  The number of sources which note the origins of the claim  without subjecting it to any critical assessment is even higher. Critical scrutiny of the Government claims has in fact been strikingly absent, until now.

Has anyone else cast doubt on the Government’s story?

They have  – although, as far as I am aware, mine is the only account which goes through the Government witness statements in detail. Links which I could have included in my original post include this piece from Alan Rusbridger and Friday’s statement from David Miranda’s legal team.


Buried in the comments: Greenwald, Miranda, Clegg and an indefinite number of documents.

After a Snowden-imposed absence of a few days Glenn Greenwald posted a new blog early this morning. Of the items in the blog proper, I can definitely recommend David Carr’s NYT piece on journalists waging the US Government’s war against journalists for them. Unfortunately, the same has largely been true in the UK – in part due to wholly unadmirable, parochial concerns like the ones John Naughton points to here.

But there are a couple of interesting points hidden in the comments that also deserve to be drawn out.

Nick Clegg and the reasons for Miranda’s detention

The issue of whether the detention of David Miranda under Schedule 7 of the Terrorism Act 2000 was lawful has been the subject of much excellent legal blogging. Pieces I have found particularly useful include those by Jack of Kent, Head of Legal and Adam Wagner. Daniel Isenberg’s roundup of these posts and others is very useful. And on the wider implications of Schedule 7, Tim Hardy’s article  is also well worth a look.

For David Miranda’s nine hour detention at Heathrow to have been lawful, he had to have been detained for the purposes of determining whether he was a ‘terrorist’, under the terms of the Act. Police do not need a reason to suspect someone is a terrorist to use Schedule 7 against them, but those powers must only be used to determine whether in fact they are a terrorist or helping a terrorist. As law and plain language often take divergent paths, there is a debate about how broadly ‘terrorism’ should understood under the terms of the Act – but there isn’t any doubt that uses of Schedule 7 must be justified in this way.

Last Friday the Guardian published a piece by Nick Clegg which merits little comment other than to note how it was edited post-publication. Hidden in the comments to Glenn’s latest piece is an archived copy of the Clegg article as originally published, complete with the now-deleted sentence at the start of paragraph six:

The intent behind detaining Miranda was the same: to retrieve or destroy classified information.

A footnote on the currently available version of the article reads as follows:

• This article was amended on 23 August 2013 after a request from the deputy prime minister’s office based on legal reasons. The footnote was amended on 25 August 2013 to give greater clarity.

Now, I Am Not A Lawyer – or even a legal blogger – but this particular amendment “for legal reasons” doesn’t increase my confidence that Schedule 7 was used in an appropriate way in David Miranda’s case. Just as concerning is that those in positions of power  – not least those who have posed in support of civil liberties in the past – in practice understand, or care, little about what the restrictions on their powers really are. To the extent that, on a point of law that is the talking point of the week, they don’t notice they’ve overstepped the mark until someone pulls them up on it.

Is the UK Government in possession of decrypted Snowden files?

Given that the UK Government, both in overt statements and in freudian slips like that above, has justified its actions in terms of protecting the public from the disclosure of documents of the utmost sensitivity,  I think also it’s worth taking a look at the factual coherence of those statements, regardless of whether they have legal weight or not.

David Miranda was detained at Heathrow for nine hours. During that time, according to his lawyers’ letter prior to legal action (see para 57):

Our client was required to answer numerous questions and to divulge the confidential passwords to his personal computer, telephone and encrypted storage devices.

Note that it is illegal to withhold encryption passwords from police in the UK.

In public comments and legal statements, the Home Office have asserted that Miranda was carrying “tens of thousands of documents… highly sensitive material.” Major media outlets have reported this as fact.

In light of all this, two responses from Glenn Greenwald (first, second) in the comments section of his latest piece are worth noting:

[UK police] haven’t been able to get access to those documents, as they acknowledged today.

In their court filing. I don’t know the exact numbers, but they said they were only able to access something like 75 documents of the tens of thousands they claim he was carrying – and I’d be willing to bet those 75 they claimed they access have absolutely nothing to do with NSA.

A few points to make here – foremost among them that I hope that the Home Office legal submission Glenn refers to makes it into the public domain soon. Secondly, it would make sense that, if indeed David Miranda were carrying journalistic material, he did not also carry the relevant encryption key(s). That would be sensible.

But, that being so, how can the Home Office assert so confidently that Miranda was carrying “thousands of documents”? Unless police have been able to access the file system on one of the devices Miranda was carrying while not being able to access the files themselves, this doesn’t really add up.


For those not aware of them, services like News Sniffer (for some UK publications) and Newsdiffs (US) track the changes in previously-published articles. It turns out that the Clegg article and its subsequent correction coincided with the Guardian changing its main URL, so – in one of those strange internet quirks – it was missed by News Sniffer.  Thanks to @semanticist and @johnleach for drawing that to my attention.

Update II (5/9)

David Allen Green was kind enough to reference this post at Jack of Kent.