Extraordinary Popular Delusions

"Men… think in herds … they only recover their senses slowly, and one by one."

Tag: gchq

GCHQ’s “light oversight regime” as situation comedy

One of the more memorable quotes to have been featured in the Snowden reporting to date for me is the unnamed senior legal official from GCHQ’s briefing note that “We have a light oversight regime compared with the US”.

For an illustration of how this works, I highly recommend reading Intelligence Services Commissioner Sir Mark Waller’s evidence to the Home Affairs Select Committee from last Tuesday. The Committee had to fight to get Waller to appear before it and, seeing what transpired, you can understand why. All the comparisons to Yes, Minister are entirely justified (although, as you may gather from his evidence, Mark Waller is in fact a retired senior judge rather than a career mandarin).

A representative sample:

Sir Mark Waller: …I just thought it was absolutely wrong to publish my report without going down to GCHQ in order to see whether there was anything in the allegation that was being made [in the Snowden reporting]. The allegation that was being made at that time was that GCHQ were taking no notice of UK law. They were doing it all through America and they were behaving unlawfully.

Chair: You went down to GCHQ.

Sir Mark Waller: Yes.

Chair: You went to see who there?

Sir Mark Waller: I saw the second head of the agency, in fact.

Chair: How did you satisfy yourself? It seems, from your comment, that what you did was you had a discussion with them, you heard what they had to say and you have accepted what they had to say.

Sir Mark Waller: Certainly.

Chair: Is that it?

Sir Mark Waller: Certainly.

Chair: Just a discussion?

Sir Mark Waller: Certainly.

Chair: Nothing else?

Sir Mark Waller: Certainly.

Chair: That is the way you were satisfied that there was no circumventing on UK law. You went to see them. You sat round a table. You had a discussion-

Sir Mark Waller: You have to remember that I had done a year and a half’s inspection. I have a very good idea as to what the ethos of this agency is.

Chair: Of course.

Sir Mark Waller: They know perfectly well that they have to make out their case and the legality of their cases and so on and I have absolutely, clearly, accepted that-

Chair: Of course. How many times have you visited GCHQ in the three years and two months that you have been the Commissioner?

Sir Mark Waller: Three years and two months. Well, again, each visit in 2012 is in the report. Effectively, I do two inspections a year.

Chair: So you have been about six times?

Sir Mark Waller: Yes.

Chair: Six times in three years?

Sir Mark Waller: Yes.


Liberty and Others v GCHQ

The legal challenges made by Liberty, Privacy International, Amnesty International, the ACLU and others in the wake of Edward Snowden’s revelations had their first hearing in the Investigatory Powers Tribunal today. The IPT is the tribunal set up under the Regulation of Investigatory Powers Act (RIPA).  It does not usually meet in public, so the announcement below is a bit of a souvenir.


This is the first of two groups of challenges against GCHQ’s interception and information sharing practices. The other is an appeal direct to the ECtHR (Big Brother Watch v United Kingdom), which the Strasbourg court has decided to fast track.

Today’s hearing was a directions hearing, which means that none of the substantive claims were argued, but questions as to approach were tackled and dates were set. The full hearing has been scheduled for 14-18 July this year – which is rather earlier than the ECtHR will hear their case, even though they’ve decided to fast track. The July hearing will be open to the public, although it sounds like there may also be sections of argument that are closed (more on that below).

dramatis personae

There are three separate groups of claimants: Amnesty International (represented by Kirsty Brimelow of Doughty Street Chambers), Privacy International and Bytes For All (Ben Jaffey of Blackstone Chambers) and Liberty and the ACLU (Matthew Ryder of Matrix Chambers). As far as I am aware, the only groups to have made their initial documentation public are Privacy International and Bytes for All. Privacy International’s claim deals with two main issues: the extent to which information sharing is regulated under RIPA (lets’s broadly call that issue PRISM) and the legality of mass surveillance (that’s Tempora).

The first issue dealt with was Amnesty joining the proceedings. Today’s hearing isn’t quite the first time Snowden’s revelations have been brought before the IPT (even in public). On 30 January, Abdel Hakim Belhaj and Fatima Boudchar were granted a limited injunction against the use of any legally privileged information that may have been acquired by surveillance (the court did not rule on whether any surveillance had in fact happened). The violation of legal privilege in breach of article 6 of the ECHR appears to be part of Amnesty’s argument in this case too, so there was some discussion as to what should be discussed purely in relation to the Belhaj case and what should be included in July’s hearing.

“This tribunal is unique in being able to proceed on assumed facts”

The bulk of the morning hearing saw attempts to reach agreement on the hypothetical premises on which the argument could proceed. Part of the difficulty here is that the UK government is still adopting a strict ‘neither confirm nor deny’ policy when it comes to Tempora – to the extent of not even being willing to confirm or deny how the word is pronounced.  It became evident over the course of the morning that the government would have preferred to restrict the court to an assessment of whether the RIPA framework itself was in accordance with ECHR rather than adjudicating whether particular alleged actions would be legal under RIPA itself or the Human Rights Act.

That approach was decisively rejected (“surely if you’re not allowed to do it at all, we can say so?”) so we will be hearing arguments about whether Tempora activity would be lawful – although the points at issue will be presented as “claimants allegations” rather than “agreed premises”.

In the absence of authoritative advice to the contrary, by the way, Mr Justice Burton decided that the IPT would go with the ‘Latin’ rather than ‘Japanese’ pronunciation of tempora. That means an emphasis on the first, rather than the second syllable.

Metadata and communications data

An interesting question that came up was whether communications data and metadata is synonymous – as it transpired, this was brought up by Matthew Ryder as a result of David Omand asserting that there was a difference (listen back to the LSE debate to hear for yourself). It seems that the government has responded to the effect that there is no meaningful difference between the two terms.


The afternoon session confirmed dates for the main hearing in July and then returned to the main theme of the morning, this time in detailed discussion about how the main issues of the case should be framed. Should the government be able to limit discussion to an assessment of the compatibility of its legal framework with the ECHR or should the question be whether the alleged practices themselves are compatible with the law? Is it possible the alleged practices might not be wholly authorised by RIPA, making the first option too narrow?

The argument on these issues was quite dense: at one point, it appeared as though the government was saying that, if the alleged activities took place, they could only have been authorised by RIPA, but that was not conceded formally. The final formulation is still to be confirmed, but it looks like it will represent a bit of a compromise for both sides.

Neither confirm nor deny

As mentioned earlier, the UK government will still neither confirm nor deny that the Tempora programme exists, despite the amount of information now in the public domain. (PRISM is a bit of a different matter, because its existence has already been acknowledged on the other side of the Atlantic). On the basis of some of Ben Jaffey’s submissions today, it looks like this stance will be challenged in July, particularly if – as seems likely – the government moves to hold a closed session after the open one.

UK MPs debate oversight of the security services

Earlier today, MPs took part in a three hour debate on oversight of the security services. Video of today’s three hour debate is now available here, and it’s well worth viewing:

31.10.13 Westminster Hall debate on oversight of the security services

Of particular note are the exchanges between members of the Intellifenge and Security Committee (ISC) and Parliamentary colleagues, which reveal that no scrutiny of Prism or Tempora took place in that committee before Edward Snowden’s disclosures put the existence of those programmes into the public domain. It is not at all clear that members of the committee knew what GCHQ was up to until the Guardian drew their attention to it.

A full transcript of the debate should be available soon (here) and I’ll highlight some of the key passages when it is.

Update (4/11)

I promised to identify the sections of the debate which tackled the degree of information open to the ISC, particularly about the PRISM and Tempora programmes. The first came about in a question from Tom Watson to George Howarth, a member of the ISC:

Mr George Howarth (Knowsley) (Lab):

Let me demonstrate that by reference to the issue that the hon. Gentleman has talked about at some length, and legitimately so. I am talking about the Prism programme—what the UK’s involvement in it was and so on. Not once during his speech, unless I missed it, did he refer to the fact that the Intelligence and Security Committee, which he considers to be inadequate, has already looked at the Prism programme and what our own agencies’, and particularly GCHQ’s, involvement in and knowledge of that was. We issued a statement—an interim statement, I might add—in July. In the course of that statement, which has not been referred to so far, we arrived at some important conclusions. The first one was:

“It has been alleged that GCHQ circumvented UK law by using the NSA’s PRISM programme to access the content of private communications. From the evidence we have seen, we have concluded that this is unfounded.”

For obvious reasons, it is impossible for me to go into detail about all the evidence that we were able to look at, but we did look in detail at very important pieces of information and we were able also to look at what authorisations were involved in the process of accessing the information, particularly the communications within it. The law has not been broken.

Mr Watson: I am reassured by my right hon. Friend’s thoroughness in the investigation. Was July the first time that the Committee had examined Prism, and was that after the Guardian revelations? [Laughter.]

Mr Howarth: It was after the Guardian revelations. The hon. Member for Cambridge seems to think that that is funny. Actually, he would still be sitting here today if we had not gone and looked at this matter after the allegations emerged. He would be accusing us of being inadequate in our responsibilities.

So, the ISC did not examine GCHQ’s involvement in PRISM before information about the programme’s existence reached the public domain. That could mean that the committee didn’t know about it, or knew about it and chose not to concern itself with it. George Howarth was pressed on the issue of whether the ISC knew about the programme by Rory Stewart – and his answer is incredibly evasive.

Rory Stewart: Will the right hon. Gentleman clarify why the Committee did not look into Prism before The Guardian published its allegations?

Mr Howarth: Let me answer the hon. Gentleman very carefully; I hope that he will forgive me for being none too specific in my answer. Part of our responsibility, which did not just emerge after the revelations about Prism, is to look at what the agencies do, what their capacities are and how they use those capacities. It is a continuous process. We have in the head of GCHQ. We take evidence. We probe what it is doing and what it is capable of doing. Therefore, it is not that we did not have any concerns or any interest in what GCHQ was capable of. That is an ongoing process, but inevitably, when something new emerges, it is appropriate that, as a Committee, we look into it.

I have answered the hon. Gentleman’s question perhaps not as accurately as he would have liked, but—I am not being evasive when I say this—if I went any further, I would be going into detail that at this stage I do not think is relevant.

The issue was later put to the chair of the committee, Sir Michael Rifkind, who refused to answer the question:

Mr Meacher: Will the right hon. and learned Gentleman explain why the Committee did not find out about the Tempora programme when it began to operate?

Sir Malcolm Rifkind: The right hon. Gentleman does not have the faintest idea whether the Committee was aware of programmes of any kind. We are given classified information, and the whole point of an independent Committee having access to top secret information, whatever that is, is that we do not announce what such information is. If he can devise a system whereby secret information can be made available to all law-abiding British citizens, without its being simultaneously made available to the rest of the world, I am interested in hearing about it, but I do not think that he is likely to meet that requirement.

Also of note was the question put by Julian Huppert to the Under-Secretary of State James Brokenshire – but answered by Michael Rifkind:

Dr Huppert: The Minister makes the extremely good point that it is “past operations” that can be looked at, and there are constraints on what the ISC can look at; it does not have a completely free rein on operational matters. What happens if an operation lasts for many, many years? At what stage is there any sort of scrutiny of that?

James Brokenshire: To be fair to the hon. Gentleman, he took part in the consideration of the Justice and Security Act 2013, although he did not make then a number of the points that he has made this afternoon. However, we need to be very careful to ensure that scrutiny does not seek to cut across into direct, ongoing operational activity. I am quite sure that, given the robustness of the new powers that the ISC itself will hold, that consideration is very much in the forefront of the minds of the Committee members.

Sir Malcolm Rifkind: In response to the perfectly reasonable issue raised by the hon. Member for Cambridge (Dr Huppert), I must say that this point was seized on by the ISC itself. We have completed discussions with the Government, the results of which will appear in a memorandum of understanding that will be published and include details of how these matters will be dealt with. That will ensure that that consideration cannot be used as an improper way of preventing the ISC from obtaining access to operations that—by any normal, common-sense approach—could be considered as completed.

Finally, as a reminder of the quality of rhetoric that tends to prevail when issues are not subjected to proper scrutiny:

Mr Adam Holloway (Gravesham) (Con): If in the last few weeks, we had lost a city to nuclear terrorism or there had been a gigantic mass casualty, I wonder whether the hon. Gentleman’s constituents would see Edward Snowden as a trendy, cool whistleblower or as a traitor.

Fourth European Parliament hearing on surveillance: special whistleblower edition

Monday’s fourth #EPInquiry hearing was relatively well-reported, largely because Edward Snowden supplied a statement, delivered to the inquiry by the Government Accountability Project’s Jesselyn Radack.

Audio of the full hearing is available here, thanks to Henrik Alexandersson, who has also posted the audio of the previous three hearings.

The speakers were Marc Rotenberg (EPIC), Catherine Crump (ACLU), Thomas Drake (NSA whistleblower), J. Kirk Wiebe (NSA whistleblower), Annie Machon (MI5 whistleblower), Jesselyn Radack (Government Accountability Project) and John Devitt (Transparency International). Video of the following presentations has been made available by the Government Accountability Project:

Jesselyn Radack

Thomas Drake

J. Kirk Wiebe

The next hearing is tomorrow, Thursday 3rd October and one of the subjects up for discussion will be GCHQ’s aggressive actions against the Belgian national telecoms company, Belgacom – whose clients include the European Parliament. Unfortunately, GCHQ’s director has declined the opportunity to justify himself in front of the Committee.

Missed my posts on the first three #EPInquiry hearings? Find them here (one, two, three).

Update (3/9)

Full video of the hearing is now available:

European Parliament holds third surveillance hearing

Unlike previous hearings (one, two), I’ve been able to follow this one as it proceeded (and even, eventually, found a machine that will relay the European Parliament’s  livestream properly). This storified account of the hearing will be updated throughout the day.

[View the story “#EPInquiry hearing three” on Storify]

It is worth noting that the full-day hearing concludes with an acknowledgement that responses given by members of the EU-US Transatlantic group of experts on data protection and the speakers from EuroPol and SWIFT were unsatisfactory – those relate to the first two sessions recounted at that storify link. Caspar Bowden’s presentation in the fifth session is well worth your time too.

The next #EPInquiry hearing is scheduled for Monday 30 October and will include another set of interviews with representatives from US civil society (I think the ACLU this time around) and a second session on whistleblower protections. The hearing after that, on  Thursday 3 October will look into GCHQ’s compromise of the mobile network Belgian national telecoms company, Belgacom, which coincidentally provides services to EU institutions. UK representatives have been invited to this hearing, although it remains to be seen whether any will turn up.

Update (2/10)

GCHQ have declined the opportunity to justify their actions at the fifth #EPInquiry hearing tomorrow.


European Parliament holds second surveillance inquiry hearing

Following on from my last post, I’m just catching up with the second hearing of the European Parliament’s Civil Liberties Committee into surveillance in and by EU countries. This was held on Thursday 12th September and, like the first hearing, was divided into two sessions.

The first, private, session saw MEPs briefed on the results of a meeting between EU and US data protection experts back in July. There were two strands to the EU’s response to PRISM in mid-June; one was the public inquiry arranged by the European Parliament and the other was the ad hoc working group formed by the Council Presidency and Commission doing the reporting in this closed session.

The second session included a briefing from the Chair of the Article 29 Working Party, Jacob Kohnstamm, on the impact of surveillance on privacy and US-EU Data Protection Agreements. Audio of this second session has been released on the EU website  – although it’s not the most user friendly interface I’ve ever encountered.

Documents from the meeting are also available here.  Of these, Kohnstamm’s letter to EU Commissioner Viviane Reding forms the basis of his presentation to the Inquiry and is certainly worth looking at.

It also needs to be clarified if these American intelligence programs are in line with European and international law. This includes the International Covenant on Civil and Political Rights, which lays down the right to privacy in a general way. More importantly, the necessity and proportionality of these programs according to the Council of Europe Convention 108 needs to be further assessed. WP29 therefore considers it is likely that the current practice of apparent large-scale collection and accessing of personal data of non-US persons is not covered by the Council of Europe Cybercrime Convention. This is particularly relevant in light of the on-going discussion within the Council of Europe Cybercrime Convention Committee (T-CY) on the preparations for an additional protocol meant to facilitate trans-border data flows in this field.

Documents relating to the first #EPInquiry hearing have also been released.

The next #EPInquiry hearing is scheduled for 24th September:

There are five sessions foreseen in the programme focusing on “Allegations of NSA tapping into the SWIFT data used in the TFTP programme”, “Exchange of views with US Administration”, “Feedback of the meeting of the EU-US Transatlantic group of experts on data protection of 19/20 September 2013”, “Exchange of views with US Civil Society (part I)” and “Presentation of the study on the US surveillance programmes and their impact on EU citizens’ privacy”.

Update (19/9)

Kohnstamm does not understate the importance of the Snowden revelations (this from the audio clip):

Based on the reports… it is highly likely that the fundamental rights of human beings have indeed been infringed on… The fundamental trust between government and citizens is at stake.

He also makes clear that the surveillance activities of EU member states will also need to be assessed for their compliance with international law and EU standards, which may themselves need to change to offer better protection for individuals’ privacy.

Beware spooks bearing gifts

There’s much in yesterday’s batch of Snowden revelations that still needs to be explained fully – this blog post by Matthew Green offers the most useful analysis I’ve seen so far.

In the meantime, this paragraph from the New York Times’ version of the story (as tweeted by Trevor Timm) caught my eye:

Even agency programs ostensibly intended to guard American communications are sometimes used to weaken protections. The N.S.A.’s Commercial Solutions Center, for instance, invites the makers of encryption technologies to present their products to the agency with the goal of improving American cybersecurity. But a top-secret N.S.A. document suggests that the agency’s hacking division uses that same program to develop and “leverage sensitive, cooperative relationships with specific industry partners” to insert vulnerabilities into Internet security products.

This caught my eye because it reminded me that, just this summer MI5 and GCHQ offered a “cyber-health check” to all FTSE 350 companies as a prelude to “an in-depth discussion with each company’s audit firm about areas in which a company may be particularly vulnerable.” In response to this announcement, John Colley, managing director of (ISC)², a membership body for information security professionals, questioned whether the methodology of the “health check” – asking company chairs, rather than technicians, to fill out a questionnaire – was likely to be draw out a well-informed response:

Logically, infosecurity professionals are better placed to provide such information as they are dealing with security issues on a day-today basis, they have knowledge of the exact security measures in place within their organisation and insight into areas where more investment is needed as they closely monitor the evolving threat landscape, and so are more likely to provide the relevant and accurate data.

Colley went on to note that it was not clear if audits were mandatory and sounded a note of caution over what might happen to data the authorities went over the heads of security professionals to obtain:

It is also unclear as to what the GCHQ and MI5 will do with the information revealed by these cyber-audits.  In this age of state sponsored cyber-attacks and PRISM, there are great sensitivities surrounding governments’ objectives for accessing data.

The “cyber-health check” is just one of a number of initiatives central government has recently launched in the area of cyber-crime, several of which are aimed at private companies. Some of this activity may be well intentioned, no doubt, but we also know from yesterday’s reports that GCHQ have a specific programme that focuses on compromising VPNs, the means by which many large companies enable employees to securely access their systems from outside the office:

By 2010, the Edgehill program, the British counterencryption effort, was unscrambling VPN traffic for 30 targets and had set a goal of an additional 300.

Ongoing revelations about Anglo-American attempts to undermine the fabric of online security make it difficult to assume good faith in this area. It is certainly interesting that the initial approach of the “cyber-health check” is being made to senior corporate positions, rather than those in the best position to weigh up the potential risks of such an approach.  Ultimately, if security of information is a selling point for any FTSE 350 company, they might be well advised to be wary of spooks bearing questionnaires and promises of audits.

Update (9/9)

This post started with a link to Matthew Green’s excellent discussion of the latest NSA revelations. Today it has emerged that the author has come under pressure to remove his post from the servers of his employer (Johns Hopkins University). The mirrored version of the post on university servers has in fact been removed.  It is not clear from where the impetus for this move originated, but Green has said that “this isn’t my dean’s fault.”

While there is no reason to suspect that Matthew Green’s post will disappear from Blogger, it is sensible to take precautions. The first link in the previous paragraph will take you to an archived version of the post.

Update II (10/9)

The move from John Hopkins became a textbook example of the Streisland effect – and it does not look like direct external pressure was involved. Ars Technica provides a comprehensive account here.

Update III (24/9)

Australia’s Security Intelligence Organisation (ASIO) is taking a different approach (“Unlike the UK government’s cyber security evaluation centre, the ACSC’s offer to the private sector will not focus on vetting technology equipment”), inviting private business to co-locate within their new headquarters.

A senior analyst at the Australian Strategic Policy Institute, Dr Tobias Feakin, welcomed the move to integrate private firms into the new cyber operations centre, but said companies would have to be “willing to share data with government, otherwise momentum will be lost and they won’t keep their focus on such efforts”.

#Miranda: Where is the UK Government getting its numbers from?

A few days ago I blogged on hints Glenn Greenwald made about witness testimony the UK Government was due to give in court about its grounds for continuing examination of electronic material confiscated from David Miranda.

In that blog, I suggested that if the UK Government really had only managed to decrypt “something like 75 documents”, it cast their assertions about the number of documents Miranda was carrying in a rather different light. Many news organisations have taken the “58,000 documents” figure as fact. But what is it really based on?

The court hearing was heard yesterday afternoon and, at its conclusion, Government lawyers released the testimony of Oliver Robbins, a senior civil servant who has held intelligence related positions in the Cabinet Office under the present and last governments. His is the securocrat’s voice par excellence.

At the outset, it should be noted that Robbins’ testimony isn’t the court filing Greenwald was referring to in the comment that prompted my last blog. That, it transpires, was a separate statement by Detective Superintendent Caroline Goode, from the Metropolitan Police’s Counter-Terrorism Command. Goode’s statement has not been released in full, but sections from it have been reported in the press. The fullest account of Goode’s statement, from which many of the others are drawn, is this Reuters piece.

Let’s look at what we know of Goode’s reported statement first.

Caroline Goode’s evidence

Use of TrueCrypt

Detective Superintendent Goode said that the information on the external hard drive was encrypted by a system called “True Crypt [sic],” which she said “renders the material extremely difficult to access.”

This is useful information. First of all, note the use of the word “access” to mean “access in readable form” and that Goode’s comments relate to just one of the devices taken from Miranda.

TrueCrypt is widely used encryption software that is free to use and download; many of those reading this blog will be familiar with its features. For those who aren’t, the TrueCrypt homepage describes what this software does (I’ve preserved the hyperlinks to more detailed resources on the Truecrypt website for those who want to read further):

Main features:

  • Creates a virtual encrypted disk within a file and mounts it as a real disk.

  • Encrypts an entire partition or storage device such as USB flash drive or hard drive.

  • Encrypts a partition or drive where Windows is installed (pre-boot authentication)

  •            (…)
  • Provides plausible deniability, in case an adversary forces you to reveal the password: Hidden volume (steganography) and hidden operating system.

Knowing what TrueCrypt does is useful because it gives us a good basis on which to assess the validity of subsequent statements. Note that TrueCrypt encrypts entire hard drives, or portions of them, rather than individual files. An area of a hard drive that has been encrypted with TrueCrypt is very much like a container you can drop files into. You need a password to open the container before you can access the files within it. This container is often called a TrueCrypt file but it can also be called a TrueCrypt volume.

60 GB of data and only a third of it “accessed”

Goode said the hard drive contained around 60 gigabytes of data, “of which only 20 have been accessed to date.” She said that she had been advised that the hard drive contains “approximately 58,000 UK documents which are highly classified in nature, to the highest level.”

Note first of all that Goode is still discussing only one of David Miranda’s electronic devices – an external hard drive . She then notes that only a 20GB portion of that external hard drive has been “accessed” – which either means that the remaining 40GB data is inaccessible (presumably because it is contained within one or more encrypted TrueCrypt volumes), or that the police simply haven’t got around to examining them. Given that Goode’s colleagues have now had access to that external hard drive for nearly two weeks, the former possibility is presumably the more likely of the two.

Incidentally, there is nothing in Goode’s statement to say that we’re dealing with a 60GB hard drive. The external hard drive could just as well be one of larger capacity holding only 60GB of data.

Finally, Goode “has been advised” about what the hard drive as a whole contains. This is not knowledge that she has determined herself, independently, from access to those 20GB of data. It seems odd that Goode’s reported statement about the content of the drive, including the 40GB of data she has not been able to “access”, does not rely to any extent on the 20GB she has.

“Only 75 documents have been reconstructed

Goode said the process to decode the material was complex and that “so far only 75 documents have been reconstructed since the property was initially received.”

This is the statement that Glenn hinted at earlier this week.

“Reconstructed” is a strange word for Goode to use. The most natural interpretation is to see “reconstructed” as a synonym for “decrypted” or “put into a form that can be read”, although this doesn’t really fit in with the idea of a “complex” process. They may not have the technical nous of Edward Snowden, but I assume that Counter Terrorism Command are familiar with the process of mounting an encrypted TrueCrypt volume and typing in a password.

So what else could Goode mean here? It’s easy to exclude a few possibilities: even if the Met and GCHQ were trying very hard to open an encrypted volume by brute force, they wouldn’t be able to individually decrypt the files within it one by one.

What Goode could mean is that analysts have been able to recover deleted files from unallocated space on the hard drive (space that isn’t being used for data now, but may have been in the past). That, at least, is more of a fit for the idea of a “complex process.”

Let’s leave the vagueness about where the files came from to one side for the moment.  Are there any other insights we can draw from Goode’s statement?

The first thing to note is that 75 documents out of an estimated total of 58,000 is an absolutely tiny proportion. It is difficult to see how such a minute sample could give a true indication of the entire collection of material held unless one or more of those decrypted files served as a kind of index to the whole. Indeed, if the files have been reconstructed from unallocated space – meaning they had previously been deleted – then they may tell you even less about what is currently on the drive.

There’s a further ambiguity when Goode talks about “the property” – is she referring to the external hard drive here, or Miranda’s confiscated belongings as a whole?  If the latter is the case, then it is by no means certain that the “accessed” 20GB portion of the external hard drive contains any documents at all – those 75 could have been obtained from elsewhere.

If we take the opposing view and suppose that Goode’s “the property” means only the external hard drive discussed previously, then those 75 documents came from the “accessible” 20GB portion of the external hard drive or were recovered from unallocated space. Caroline Goode’s evidence could just as easily mean one of these scenarios as the other: it is remarkable for the range of possibilities it does not exclude.

Summary of Caroline Goode’s evidence

Caroline Goode’s evidence suggests that David Miranda’s hard drive contains a TrueCrypt volume or volumes of a total size of 40GB that UK police have no access to. The 20GB encrypted portion of Miranda’s external hard drive that the police have been able to access contains, at most, 75 files. It is possible that some – or even all – of those files came from other devices, or from unallocated space on the same device.

Goode’s statements about the remainder of the documents do not seem to be based on insights gained from the 75. This would tend to support Glenn Greenwald’s assertion that UK police have not been able to access anything sensitive. It certainly does not clarify how the total figure of 58,000 documents the Home Office has asserted is on Miranda’s external hard drive has been arrived at.

Oliver Robbins’ evidence

What follows is a close analysis of Oliver Robbins’ testimony – and I do think it deserves to be looked at very closely indeed. There is much in Robbins’ statement that deserves detailed analysis but, for the purposes of this blog post, I will restrict my attention to Robbins’ comments on the UK Government’s access to, and analysis of, the Miranda data.

Indefinite room for ambiguity.

[in justifying why the Government needs “continuing access” to the material seized from Miranda] … no information that has so far been analysed by Her Majesty’s Government (“HMG”) has identified a journalist source or has contained any items prepared by a journalist with a view to publication. The information that has been accessed consists entirely of misappropriated material in the form of approximately 58,000 highly classified intelligence documents. [para 6]

The first thing to note here is that Robbins’ use of the word “accessed” is different from Goode’s. As we saw above, when Goode talks about data “accessed” she means data that can be accessed in readable form. Robbins’ use of the word is broader because his witness statement is making an argument about the Government’s need for “continuing access” [para 5] to all the material seized from Miranda, including that which has not been decrypted. Robbins’ use of “access” therefore more closely corresponds to the idea of physical access to the  devices themselves. This is confusing.

Robbins goes on to talk about a subset of  the information that has been “analysed.” We are not told whether this means analysis of encrypted information, but given that he goes on to make statements as to the content of this information, it is likely to be the case that this information can be read in some form. What Robbins says about this analysed material is that none of it “has identified a journalist source” and neither does it contain “items prepared by a journalist with a view to publication.”

Of course, Robbins’ purpose here is to reject the idea that the Miranda material contains anything that should be withheld from examination, but It’s worth noting that the category of data which meets those two stipulations of his is quite a wide one: it includes shopping lists, youtube videos of cats and many other items of limited relevance to national security.

What Robbins says next is interesting: he moves straight from a limited description of a small subset of data to make a claim about the entirety of the Miranda material (“that has been accessed”). Putting to one side for the moment the ambiguity about whether Robbins is really talking about Goode’s external hard drive here or the Miranda devices in total, It is not at all clear on what he is basing this rather striking claim.

Let’s think about this situation in a different context. Imagine if you had a bookcase that, apart from a couple of volumes, consisted only of books with unopened pages. What Robbins says would be like asserting that all the books in the bookcase are illustrated, purely on the basis that, of the two books you can examine without a penknife, neither was printed in London or inscribed with the owner’s name. It is certainly a claim that can be made, but not one that deserves to be taken particularly seriously.

Wait, so it’s not your assertion after all?

I am advised that the data recovered from the claimant is almost certain to contain some of the material passed by Mr Snowden to Ms Poitras and Mr Greenwald. Much of the material is encrypted. However, among the unencrypted documents recovered from the claimant was a piece of paper that included the password for decrypting one of the encrypted files on the external hard drive recovered from the claimant. I have been briefed that the authorities have therefore been able to examine the data contained in this file. They have been able to determine that the external hard drive contains approximately 58,000 highly classified UK intelligence documents. Work continues to access the content of the other files on the hard drive and the USB sticks. [para 13]

There’s a lot in this paragraph, so let’s take it line by line. The first sentence seems to answer the question posed in the previous section: Robbins’ assertion about the content of the Miranda data is second hand after all (“I am advised”).  It is also indefinite (“almost certain”) which seems to contradict the conclusive phrasing (“the data that has been accessed… consists entirely of”) of the previous paragraph.

Once again, this is confusing – so let’s try to resolve the contradiction. Is it possible that, when Robbins talks about “the data that has been accessed” in paragraph 6 he is slipping between the broad interpretation of the word “accessed” he has used in his previous sentences and the narrower sense – that of data that can be read and analysed – used by Caroline Goode? It’s much easier, after all, to be definite about the content of documents you’re able to read than ones you cannot.

I’m not sure this works either. Goode testified that the material “accessed” in the sense that it could be “analysed” amounted to a 20GB portion of an external hard drive, which may contain all, or maybe only some, of a total of 75 documents. To say this consists “entirely of misappropriated material in the form of approximately 58,000 highly classified intelligence documents” is just a nonsense.  Robbins must therefore be using the word “accessed” in his usual sense and what he says is inconsistent with his previous paragraph.

Does the rest of paragraph 13 make things any clearer? Certainly, the next three sentences are straightforward. We know that “much of the information” carried by Miranda was encrypted and that Caroline Goode and her colleagues were able to decrypt one encrypted file on the external hard drive. By Goode’s own account, she and her colleagues were able to examine the data contained within this file. These sentences are consistent both with Robbins’ own statement and those of others.

What follows is much more troublesome. “They [the authorities] have been able to determine that the external hard drive contains approximately 58,000 highly classified UK intelligence documents.” The analysis of Goode’s statement shows that she and and her colleagues could not derive the presence of “58,000… documents” from what she found – and she didn’t claim to have done.

But have I missed something here? Could it be that Robbins’ “they” isn’t referring to Goode and her police colleagues at all? Could he be referring to different “authorities” altogether? Might they be the same authorities who “advised” both Robbins and  Goode of “58,000 documents” figure and on whom both rely?  I think that is likely and, although a casual reader may feel that the two sentences below bear a logical connection, in fact they do not:

I have been briefed that the authorities have therefore been able to examine the data contained in this file. They have been able to determine that the external hard drive contains approximately 58,000 highly classified UK intelligence documents.

In my opinion, this comes close to being a misleading statement. Oliver Robbins could equally well have expressed himself as follows:

I have been briefed that the authorities have therefore been able to examine the shopping lists and pictures of cats contained in this file. Independently of this, others have been able to determine that the external hard drive contains approximately 58,000 highly classified UK intelligence documents.

GCHQ’s assessment

And what of that troublesome “58,000… documents” claim? The source for Robbins’ second authority becomes clearer in his next paragraph:

On the basis of GCHQ assessments, the totality of UK intelligence documents that would potentially have been accessible to Mr Snowden while we was working at the NSA is consistent with the volume of documents which we know to be on the external hard drive. [para 14]

This appears to be the best candidate for what the “58,000 documents” figure is actually based on. But what does it amount to? Let’s turn to “the volume of documents which we know to be on the external hard drive” first.

What we know about the external hard drive is that it is divided into at least two encrypted files, one of 20GB which the police are able to access and a further encrypted file (maybe more than one) of 40GB size. Because the police have access to the decrypted 20GB file, they can make an assessment about the number of documents within it (a maximum of 75). All that can be said about the other file(s) is that they have a total size of 40GB.

An encrypted file’s size is not dependent on the amount of data it contains.  A 10GB encrypted file could contain 10kb data or 6 GB data – unless you can decrypt the file, you have no way of telling which is the case.

As such, GCHQ’s statement is almost meaningless. You could say that the maximum volume of documents an encrypted file could contain is 40GB – but that’s something you could say of any 40GB encrypted file. GCHQ’s assertion about “the volume of contents which we know to be on the external hard drive” appears to play on an ambiguity in the word volume (one can talk about a volume of documents, but it’s also a synonym for an encrypted file) in order to hide that it has no basis in fact.

In essence, what GCHQ seems to be saying here is that what it assesses to be “the totality of UK intelligence documents… potentially accessible to Mr Snowden” would fit on a 40 GB hard drive. That logic, if applied widely, could lead to an awful lot of Schedule 7 detentions at our airports and it’s an assessment made entirely independently of the Miranda data.

So, where does that leave the “58,000 documents” figure? Nowhere good. It looks like nothing more than a worst-case scenario GCHQ based on guesswork but presented as indubitable fact.


Neither of the witness statements presented by the UK Government in Home Office v Miranda are adequately precise about the matters they raise.  Cryptographers have developed a vocabulary that is adequate to expressing these subjects with clarity – when they talk about “plain text” and “cypher text”, others understand what they mean. In contrast, when Caroline Goode and Oliver Robbins use terms like “access” and “analysis” in their statements, there is significant ambiguity in what they mean. This ambiguity leaves real potential for confusion; it also presents unacceptable opportunities for others to be misled.

I am concerned by the extent of the ambiguity in the statements presented in Home Office v Miranda. The UK Government has represented itself in language that is so vague that it may not have a case at all, yet it has presented its case in the strongest way possible – and has been accepted as such, without much demur, in much of the media.

I think it’s worth taking a moment to reflect on this. If a group of witness statements took a similar approach to legal issues as these have to technical ones, if they had eschewed technical terms in favour of ambiguous natural language and took advantage of that fact to obfuscate as these have, I think those imaginary witness statements would have received a much more critical reception.  I am concerned that our courtrooms and our newsrooms may not be equipped to cut through some of this confusion and dubious statements may be allowed to stand without receiving proper scrutiny. It is not difficult to see how parties could take advantage of this, if they wished to do so.


While I know what TrueCrypt is, I am by no means a technical expert. My intention in this piece is to show how ambiguous the UK Government’s statements are, rather than put together a definitive account of what happened – I’m not sure that’s even possible on the evidence available.

The Q&As that follow below are an outlet for some of the fun speculative stuff I couldn’t justify putting in this post.

If there’s something you think I’ve got wrong in this piece, I’d be very interested to hear about it. Please email me or leave a comment below.


Have Greenwald, Miranda and Poitras been guilty of “very poor judgement in their security arrangements”?

Travelling with a password written on a piece of paper isn’t great. Transiting through Heathrow may have been inadvisable. But, if – as seems very possible – nothing of significance has been  compromised you have to say that, on the face it it, not really.

Given that the Cabinet Office expressed its worries to the Guardian in terms of their ability to protect information from cyber attack, I think it’s relatively clear why the Government would like to cast doubt on others’ security practices if possible.

Is the 20GB encrypted file on the external hard drive a dummy volume intended to be surrendered without cost?

The thought has crossed my mind: it would certainly make it easier to explain why David Miranda was found in possession of an encryption key in a UK transit area. I am not sure it is possible to say for sure on the evidence of the statements presented, but I think this falls within the range of possibilities.

Is it possible that one of the 75 files the police have is an index to the rest?

It is possible – and if the case would make the “58,000 documents” figure much more credible – but I think on the balance of probabilities it is unlikely.

Were GCHQ just plucking a number out of the air with that “58,000 documents” thing?

Not entirely. One possibility is that they’ve plucked a number out of the Guardian.

On 2 August, the Guardian printed a fascinating feature article that is based partly on GCHQ’s internal “GCWiki”, making reference to this and many other GCHQ documents. That, and the discussions we know the Cabinet Office have had with the Guardian may have formed the starting point for GCHQ’s worst-case estimate.

Are you sure? They must know what Snowden has!

If the NSA doesn’t know what Snowden has, there’s no reason why GCHQ should.

Oh come on. if we’ve learned anything from the Snowden files it’s that GCHQ and the NSA have other ways of acquiring this kind of information.

Of course. Whether surveillance information is admissible in court is another matter, though, and one we should probably leave to David Miranda’s capable legal team.

Have the media been negligent in reporting the “58,000 documents” figure as fact?



Update (2/9)

This post proved to be quite a popular one, with 7250 page views yesterday alone. It also provoked quite a bit of discussion – I’d like to thank all of those whose contributions prompted me to make the following additions to my Q&A section.

Do you think Miranda was using a hidden volume?

It’s certainly a possibility and the first (pre-publication) draft of this post did in fact make that suggestion. Why did I leave it out? Because while the facts in Goode and Robbins’ statements do not exclude the possibility of a hidden volume, they also do not exclude a number of other possibilities. There’s nothing in the statements analysed to rule out the possibility that, for instance, police found a 20GB .tc file and a 40GB .tc file on that external hard drive but can only open the former.

Of course, this is yet another example of how the two witness statements are not adequately precise.

Why do you rule out the possibility that one of the files police have been able to access is an index to the rest?

I don’t rule it out, I say that – on the balance of probabilities – it is unlikely. Some of the reasons why I continue to think this are covered in this storify. Other very relevant points have been made in the comments section below.

Which media sources have used the 58,000 documents claim?

That’s an easy question to answer. A very cursory examination of articles published on this subject will reveal sources which take the “58,000 documents” claims as fact without even mentioning that they originated from a government witness statement (one, two, three, four).  The number of sources which note the origins of the claim  without subjecting it to any critical assessment is even higher. Critical scrutiny of the Government claims has in fact been strikingly absent, until now.

Has anyone else cast doubt on the Government’s story?

They have  – although, as far as I am aware, mine is the only account which goes through the Government witness statements in detail. Links which I could have included in my original post include this piece from Alan Rusbridger and Friday’s statement from David Miranda’s legal team.


Quotes of the day

The fact is that a lot of the arguments over this could give succour to the [Assad] regime.

Craig Oliver, David Cameron’s Director of Communications, via ITV, before tonight’s vote

It is very clear tonight that, while the House has not passed a motion, it is clear to me that the British parliament, reflecting the views of the British people, does not want to see British military action. I get that and the government will act accordingly.

David Cameron’s reaction to the vote, confirming that he would not use the royal prerogative and take action against Parliament’s wishes

One senior Whitehall figure talked about how the UK had “handed back its deputy sheriff badge” to the United States and would pay a heavy price in esteem and cooperation in the future with the US.

via Channel 4

The abrupt halt in British momentum towards military action left the diplomatic choreography in chaos and US officials “livid” with the British, according to Western diplomatic sources at the United Nations in New York.

via Business Insider

What happened in the House of Commons tonight was significant. It is not often that I feel that our Parliamentary system does much to provide a check on executive powers and adequately represent public feeling, but this evening – in voting against the Government’s motion on military action in Syria – it has done so.

It is a rare enough event for a British Prime Minister to lose a three-line-whipped Parliamentary vote – in theory, it’s an event that can bring down a government – but to lose a vote on an issue of foreign policy is almost unprecedented.

There will be a great deal of comment in the coming week about what this vote means, but a few things are clear now. First, and most obviously, it is clear that the Afghanistan and especially Iraq Wars have had a significant and lasting effect on our politics. Opposition to military action is much more widespread than it used to be – in fact, that much was already evident in opinion polling on UK action in Libya.

More than that, though, there is a widespread scepticism about official cases for war, intelligence dossiers, improvised legal arguments… in short, all the official paraphernalia introduced by Blair to bolster public support for wars of dubious legality.  What we’re seeing is a tearing away of the mystique of the state and the magical thinking of “national security” – and that is to be welcomed. It’s also a useful reminder that, despite lukewarm response to the Snowden revelations about GCHQ’s industrial-scale surveillance, things are changing in the UK. In at least some respects, people are more sceptical these days.

Secondly, tonight’s vote has constitutional significance. Parliament voted against the executive’s plans for the use of force and Cameron has agreed that “the government will act accordingly” – that is, he has agreed that prerogative powers will not be used to initiate military action regardless of what Parliament thinks. It is now inconceivable that, should a situation like this arise again, Parliamentary approval would not be sought. (One caveat: it’s not entirely clear that British non-involvement in military action against Syria extends to the non-involvement of British military bases abroad. It may be significant that tonight Deputy PM Nick Clegg did not rule out US use of the UK’s base in Cyprus).  Still, in the haphazard and inadequate way the British state develops, this counts as a constitutional moment of some significance. Remarkably, it’s actually a move in the right direction.

Finally, tonight’s vote marks a divergence between the foreign policy of the United States and the usual determination of the UK to entertain it at all costs. While it is not clear whether the UK’s withdrawal from action in Syria will make much difference to Obama’s plans (the New York Times and the London Times have differing takes on this), the absence of one of the regular former imperial fig leaves for US unilateralism can’t help but make the latter seem more exposed. Given that so much of the Whitehall conception of the UK’s “national interest” seems to rely on being some kind of dodgy subcontractor for the US, it’s really extraordinary that this has happened. It would be nice to be able to think of this as the start of something bigger… but I’ll try to keep those hopes buttoned down for now.

Update (30/8)

The Parliamentary vote did, in fact, make a great deal of difference to Obama’s plans.