Archive for the ‘maps’ Category
I’m not quite as sceptical as some about this. However, it’s not clear to me how this differs from the sort of thing UNOSAT does all the time – here’s their analysis of imagery over Abyei, the key border area between North and South Sudan. Actually it looks like the “Enough Project” is going to be using UNOSAT imagery itself, going by UNOSAT’s own website.
If you follow the link you’ll see that they have more than reasonable capability (50cm resolution) and that they routinely observe the presence of refugees/displaced persons and returnees, construction, and the like. There’s obvious relevance to an effort to monitor potential conflict along the border, especially as oil prospecting is an issue. You can’t easily hide oil exploration from a satellite that can resolve objects 50cm across.
However, the downside is that the UNOSAT report is comparing images over a two-year period. I would suspect that they will need much more frequent passes to be operationally responsive, which is where the costs get interesting.
Also, I’ve just been over to the website and it’s a bit of an unstructured clickaround. What I’ve always liked about MySociety sites is that they all have a function – FixMyStreet reports things in your street that need fixing, WDTK issues Freedom of Information Act requests, TWFY looks up information on MPs, TheStraightChoice logged what candidates promised and said about each other during their campaigns. DemocracyClub, for example, worked because as soon as you logged in it gave you something to do and some feedback about doing it, and then it hassled you to do something more. It had structure.
Notoriously, if you don’t give volunteers something to do as soon as they show up, they’ll wander off. It is nowhere easier to wander off than on the Internet. And so there’s a button to twitbookspace it and a donation link. There isn’t, however, a to-do list or, say, a list of pairs of images that need comparing.
This paper in PLoS One is fascinating (if heavily blogged already). Basically, BT let some researchers from MIT, Cornell, UCL, and their own R&D division have an anonymised slice through their call-detail record (CDR) pile, the database from which phone bills are calculated. The scientists filtered out all the numbers that only made or accepted calls, in order to get rid of the call centres and spammers, and drew the rest as a massive directed multi-graph network. The conclusions are fascinating; in human terms, Wales isn’t a meaningful unit, and neither is England. Scotland, however, forms a well defined sub-graph.
Instead, Wales splits into three geographic tiers with very little interconnection. These regions don’t respect the border at all – not surprisingly, the northern tier is completely integrated with Liverpool and Manchester and the central tier with the West Midlands. South Wales is clearly identified, with a sharply defined border along the water between it and the West Country. There’s also a well-defined western border to Yorkshire, and interestingly also between the West and South Ridings but not between them and the North Riding. Essex is an extension of London, but Kent is distinct. So is Norfolk.
In fact, England isn’t really identifiable on the maps: surprisingly, the administrative units that fit best to the BT data are the EU regions much hated by ‘kippers. More broadly, if it’s got a recognisable accent, it’s a recognisable presence on the graph – although the big exception is Yorkshire. There’s even a territory for people with no recognisable accent, a sort of motorway crescent to the west of London which is described as a “tech corridor” – in fact, if you were to draw all the Formula One teams’ workshops on the map, they would essentially all fall within it, as would Vodafone, O2, Cable & Wireless, and 3UK’s headquarters, Aldermaston, Eidos, Surrey Satellite Tech, chunks of BAE and Thales, and Electronic Arts UK, so perhaps they have a point. In the end, though, this potentially interesting zone – Ballardia? – gets lumped in with the Cameroonian central-southwest.
So the government thinks this is clever. They also think it constitutes a “searchable online database”. It is not searchable, nor is it a database. It is a collection of links to department web sites, some of which actually lead to useful documents, some of which lead to utterly pointless intermediary pages, some of which lead to documents in a sensible format, some of which lead to documents in pointlessly wrong formats, and some of which lead to PDF files. It provides no clue how often this data will be released or when or where. The URIs sometimes suggest that they might be predictable, sometimes they are just random alphanumeric sequences. Basically, what he said.
Meanwhile, very few of these documents have made it onto data.gov.uk, the government’s data web site (pro-tip: the hint is in the name) which provides all that stuff out of the box. This is not just disappointing – this is actively regressive. Is it official policy to break data.gov.uk?
Anyway, I’ve been fiddling with NetworkX, the network-graph library for Python from Los Alamos National Laboratory. Sadly it doesn’t have a method
networkx.earth_shattering_kaboom(). I’ve eventually decided that the visualisation paradigm I wanted was looking me in the eye all along – kc claffy‘s Skitter graph, used by CAIDA to map the Internet’s peering architecture.
The algorithm is fairly simple – nodes are located in terms of polar coordinates, on a circular chart. In the original, the concept is that you are observing from directly above the north or south pole. This gives you two dimensions – angle, or in other words, how far around the circle you are, and radius, your location on the line from the centre to the edge. claffy et al used the longitude of each Autonomous System’s WHOIS technical contact address for their angles, and the inverse of each node’s linkdegree for the radius. Linkdegree is a metric of how deeply connected any given object in the network is; taking the inverse (i.e 1/linkdegree) meant that the more of it you have, the more central you are.
My plan is to define the centre as the prime minister, and to plot the ministries at the distance from him given by the weighting I’d already given them – basically, the prime minister is 1 and the rest are progressively less starting with Treasury and working down – and an arbitrary angle. I’m going to sort them by weight, so that importance falls in a clockwise direction, for purely aesthetic reasons. Then, I’ll plot the lobbies. As they are the unknown factors, they all start with the same, small node weighting. Then add the edges – the links – which will have weights given by the weight of the ministry involved divided by the number of outside participants at that meeting, so a one-on-one is the ideal case.
When we come to draw the graph, the lobbies will be plotted with the mean angle of the ministries they have meetings with, and the inverse of their linkdegree, with the node size scaled by its traffic. Traffic in this case basically means how many meetings it had. Therefore, it should be possible to see both how effective the lobbying was, from the node’s position, and how much effort was expended, from its size. The edges will be coloured by date, so as to make change over time visible. If it works, I’ll also provide some time series things – unfortunately, if the release frequency is quarterly, as it may be, this won’t be very useful.
Anyway, as always, to-do no.1 is to finish the web scraping – the Internet’s dishes. And think of a snappy name.
So I scraped the government meetings data and rescraped it as one-edge-per-row. And then, obviously enough, I tidied it up in a spreadsheet and threw it at ManyEyes as a proof-of-concept. Unfortunately, IBM’s otherwise great web site is broken, so although it will preview the network diagram, it fails to actually publish it to the web. Oh well, ticket opened, etc.
Anyway, I was able to demonstrate the thing to Daniel Davies on my laptop, on the bar of the Nelson’s Retreat pub in Old Street. This impressed him excessively. Specifically, we were interested by an odd outlier on the chart. Before I get into that, though, here are some preliminary findings.
1 – Clegg’s Diary
At first sight, Nick Clegg appears to be unexpectedly influential. His calender included meetings with NATO, the World Bank, the Metropolitan Police, the Gates Foundation, and oddly enough, Lord Robertson of Port Ellen. Not only that, he had one-to-one meetings with all of them. However, he also got The Elders (i.e. retired politicos playing at shop) and the leader of the Canadian opposition, one Michael Ignatieff, Esq. God help us, is Clegg turning out to be a Decent?
2 – Dave from PR’s surprisingly dull world
The Prime Minister, no less, meets with some remarkably dull people. In fact, he met quite a lot of people who you’d expect to be left to flunkies while leaving quite a lot of important people to Nick Clegg. He did get BP, Shell, Pfizer, Rupert Murdoch, the TUC general secretary, and Ratan Tata (twice!) as one-on-ones, but he also met a surprising number of minor worthies from Cornwall and vacuous photocalls with people from Facebook.
3 – Francis Maude, evil genius of the coalition
Secretary of State for the Cabinet Office and Paymaster-General, Francis Maude MP, is the surprise hit, as far as I can make out. He seems to have a special responsibility for anything that smacks of privatisation – therefore, the monetary value of meeting him is probably high. Of course, if your evil genius is Francis Mediocritus, you’ve got problems. No wonder we’re in such a mess. All these points are also true of Oliver Letwin.
4 – Communication and Strategy Management Ltd
This is our far outlier. Some of the least significant people on the chart appear to be government whips, which is obviously an artefact of the data set. The data release does not cover intra-governmental or parliamentary meetings, nor does it cover diplomatic activity. Whips, of course, are a key institution in the political system. Given their special role with regard to both the government and parliament, it’s not surprising that they appear to be sheltered from external lobbying – access to the Whips’ Office would be such a powerful and occult influence that it must be held closely.
So what on earth is Communication and Strategy Management Ltd., a company which had one-on-one access to the Government Chief Whip, the Rt. Hon. Patrick McLoughlin MP, and which according to Companies House was founded on the 11th of April? It has no web site or perceptible public presence. It is located in what looks like a private house, here, not far from Stratford upon Avon:
Evidently the hub of political influence, but those are the facts. The directors are Elizabeth Ann Murphy and Richard Anthony Cubitt Murphy*, ignoring a company-formation agent who was a director for one day when setting up the company. It’s not as if C&SM Ltd is a constituent of McLoughlin’s – he’s MP for the Derbyshire Dales. Actually, either the directors are related or else there was a cockup, as Murphy’s name on the books was amended from Bromley the day after the company was formed and both were born in 1963. The Companies House filing* doesn’t give any other information – accounts aren’t due for a while – except that the one share issued is held by Norman Younger, who is a partner in the company formation service that was used.
Anyway, the next stop is to learn how this works and put up a nice little dashboard page to help watch the lobbysphere. I’d be happier doing something with python – such as nodebox – but the diagram is already too big to be useful without interactivity, and you can’t stick a NodeBox window in a web page. I’ve got the search terms for the data as an RSS feed from data.gov.uk, so it should just be a matter of adding more URIs as departments release their data.
*Not the Richard Murphy, who is too young.
*WebCheck – it’s not an ugly website, it’s a way of life…
Keep the shit and the drinking water separate, and you’ve gone most of the way from an average life expectancy of 35 to one of 75. Boris Johnson, famously, decided that replacing London’s water mains was a minor issue that could be thrown out as a sop to the roads lobby.
So here’s the Borisfeed. It monitors Transport for London traffic data, BBC travel alerts, an automated Google News search, and posts to Flickr, filters everything but burst water mains, and excretes them as stinking RSS. This is only the beginning; I’m planning to keep statistics as well, and perhaps pull everything together in a little page of Boris Johnson, with the inevitable IBM ManyEyes charts. (For some reason I nearly wrote MakingEyes.) Perhaps I could even have it push out updates through Twitter or some XMPP thing.
Via Airminded, find your local V2 rocket strike. London, Antwerp, Tel Aviv, Jerusalem, Riyadh, and Tehran have what in common? That’s right, it’s the list of cities that have been subjected to attack from space.
Then, why not go here and look up how big a hole it made? Someone’s photographed and flickr’d a whole set of London County Council damage assessment maps.
My local strike is now a small, never-used park on one side of the street and a pretty grim council estate on the other. But damage in this corner of London was limited compared to further down the Holloway Road. Oddly, there seems to be a correlation between the degree of damage and the London Profiler crime rate; the area south of Torrington Way, which has a sky-high crime rate, was pretty much flattened. (Sadly, the LCC maps aren’t geotagged, so making up a KML overlay would be annoyingly difficult.)
Question – is it that these areas were rebuilt as council housing and filled with the poor, or that the architecture caused the crime? After all, they were hardly peachy suburbia before being destroyed. Strange, though, to think that Wernher von Braun partly decided where tonight’s post-pub kebab stabbing is likely to happen.
This really is getting strange. The Tories look worryingly convinced of the wisdom of a plan to build a gigantic airport in the North Sea, split between two separate islands, because you never need to change the runway a plane is going to depart from…right? At the same time, the Government is considering a gigantic tidal power scheme in the Bristol Channel. It’s like French engineering civil servants seized control in a bloodless coup.
In fact it’s not; they would at least think they were being rational, but surely not even the promoters of this weird rush to create Big Dumb Objects all over the shop can believe this.
On the one hand, you’ve got the Tories, who are trying to convince themselves that they can find £40 billion, before inevitable cost overruns, to create a operationally crippled airport 53 miles from central London and only 101 miles from the nearest point of Dutch territory, dependent for land transport on spare capacity on the CTRL and on the 6 (I think) Crossrail and 2 LTS train paths an hour slated for the Southend/Shoeburyness route, and for road access on pure handwaving.
BorisWatch deserves some kind of medal for their reporting here; they successfully derived the actual location of the project by following Boris’s boat trip in real time on ShipAIS, a ship-tracking ham radio site, and then prepared a handy Google Map, which is where I got the measurements from.
How often, I wonder, would Borisport be fogged in? Even with CATIIIA/B autoland it’s a serious constraint, and enough of it will stop ground operations even if you can still get in. And then there’s all those heat-seeking gulls to worry about; they hunt in packs! The air traffic control issues are pretty gnarly, too – departures conflicting with arrivals into LHR, LCY and LGW.
Further, they want to be seen as “green” whilst also creating another Heathrow-and-a-half. But why? What is it with this obsession with airports in the Thames estuary? As always, the key to the present lies in the past. Here’s the Hansard transcript of the debate on the Maplin Development Bill back in 1973. Three things come to mind – first of all, weren’t MPs great back then? Of course, there is the usual parish pumpery, Bufton Tuftonism and tiresome faff, but there’s also a lot of well-informed intelligent debate, and in the end the government lost!
Second, all the problems are still the same. This is because they are mostly what the Soviet general staff called the permanently-operating factors – terrain, human terrain, infrastructure. Third, there’s a fascinating bit of the social history of ideas here. We join the debate with Douglas Jay MP on his feet, following up an excellent showing (or shoeing) from Tony Crosland…
Mr. Jay: What was the pressure exerted on the Roskill Commission to omit Stansted from its short list? The Times told us on 4th March 1969 that its inclusion would have been “emotive”. At the same date the Financial Times said that its omission was “diplomatic”. The British Airports Authority and the Board of Trade assumed that it was bound to be on the short list. The British Airports Authority was even told that it need not ask the commission to put it on because it was certain to be included. Yet it was omitted, and the commission’s work was handicapped from the start. 700 Thus handicapped, in my opinion the Roskill Commission did its very best. Faced with the resulting choice between Foulness and a South Midlands site for which there is a good deal to be said, it came down decisively against Foulness and in favour of Cublington.
Then we had another curious alliance between landed interests in Bedfordshire and Buckinghamshire opposed to Cublington and commercial interests anxious to develop Foulness—
Mr. Norman Tebbit (Epping): Before the right hon. Gentleman leaves the point about Stansted, in fairness to my predecessor in this House I ought to say that he was one of those opposed to the Stansted project. I would never think of him as being in the pockets of wealthy landowners or any set of that kind. It happens that I disagree with him on this issue as on many others, but it is right to be fair to him. Incidentally, I have a house on the approach to Stansted too.
Mr. Jay: I never suggested that. I was recalling what happened. According to The Times of 5th April 1971, the group resisting Cublington spent £50,000 “to persuade the Roskill Commission that the airport should be built at Foulness and not at Cublington”—” not just that it should not be built at Cublington but that it should be built at Foulness.
After the Roskill Commission’s report, this group spent a great deal more, and the same article in The Times said that the pro-Foulness propaganda groups together spent “at least £700,000” to convince the public and Parliament that Foulness was the right solution.
At this point Sir John Howard enters the argument. According to the article in The Times that I have quoted, he was head of a civil engineering firm and, incidentally, a former chairman of the National Union of Conservative and Unionist Associations, though no doubt that is irrelevant. He happened to live near Thurleigh in Bedfordshire and he founded the Thames Estuary Development Company to promote the Maplin project. The Times says that Sir John “first lighted on Foulness during the fight against Stansted, in which he was closely involved.”
He “lighted” on Foulness as it were by chance. His consortium, backed also by RTZ, John Mowlem and Shell, spent more than £500,000 in supporting the Foulness case. Much of the driving force in all this thus came not from people impressed with the merits of Foulness but from those who wanted to keep the airport away from other sites.
Here I return to the speech of the hon. Member for Southend, East. What was the opinion of more than 150,000 people living in the Southend area about this? That is for them and their representatives to say, and I am sure that we shall hear the hon. Member for Essex, South-East (Sir Bernard Braine)—
Sir Bernard Braine: I hope that the right hon. Gentleman will be accurate. There are 310,000 people living in the three constituencies bounded by the Thames and the Crouch who are affected by this proposal.
Mr. Jay: I always believe in understatements because they strengthen one’s case. The hon. Gentleman has strengthened my case further. What were the opinions of those 300,000 persons—far more than live within 20 miles round Stansted, perhaps three times as many? I am sure that the hon. Member for Southend, East will not question this as a fact. But I understand that with the support of the leader of the Southend Corporation the corporation took a share in Sir John Howard’s consortium, and the town clerk of Southend, according to The Times, became a director of it. Whether that was the best way of handling these matters, I have no doubt that all those concerned thought that they were acting in the best interests of Southend.
Sir S. McAdden: The right hon. Gentleman asked what were the opinions of the people of Southend. They were never consulted. This was a decision of the council to invest £100,000 of the ratepayers’ money in Tedco. The council thought that it would make £6 million. Instead, it has lost the lot.
Mr. Jay: It is what I have always suspected to be the truth. I stated it rather diffidently, but the hon Member for Southend, East has confirmed it. From the point of view of this House, the opinion of the Roskill Commission on Maplin is worth a good deal more than 702 that of this consortium formed in the way that I have described.
I am afraid that what emerges from the story is that both the selection of Maplin and the omission of Stansted have been influenced far too much by the money spent on the commercial publicity and far too little by serious consideration of the public interest.
I see Tebbit was already as much of an arse as he later became, too. Permanently operating factors in the human terrain.
More seriously, I’m fascinated by the fact that the whole idea of Maplin/Foulness/Sheppey/Marinair/Borisport pushed by three different Conservative administrations originates with a gaggle of Tory squires trying to win a planning row in some completely different bit of the country. I wonder if Sir John Howard ever seriously meant it? Or did it just get out of hand? The Tories always will be the party of the Landed Interest, just as when their first response to the great crash of 2008 was to look for handouts to their property-shark contingent; another permanently operating factor.
Meanwhile, over the wall, the Government has aimed squarely for a soggy compromise. My own views on Heathrow expansion are heterodox and unpopular. Here goes: I don’t particularly mind if aviation makes up 29% of the 2050 CO2 target, so long as we get there. Nobody sets out to emit CO2 – it’s waste, and when did you last hear of someone saying “Thank God our widget production line produces so many widget flakes we have to dispose of”? Converting stuff into more valuable stuff is what it’s all about, and any production of valueless stuff makes us poorer.
I’m with James Hansen on this one – it’s the coal-fired power stations, stupid, and the buildings. If we can’t fix the cars and buildings and power generation, it doesn’t matter a fucking jot what we do about aviation. Because, after all, buildings are easy, power and cars are getting easier, aeroplanes are hard. We’re not far now; look at this hub-drive electric motor project at Michelin. Solar and wind are now the leading sources of new electrical power.
And, if there must be expansion, it ought to be at an existing airport because of the ATC issues. And if we’re going to be expanding an existing airport, well, it may as well be the one the airlines want to use. Further, it’s good to maintain the various conventions that limit activity at Heathrow – I was surprised to see that mixed-mode operation accounted for almost a third of the expected capacity increase. And yes, I did hold this view when I lived there.
And if we’re doing this, we ought also to do other things, like building a north-south high-speed rail route and better public transport in general – saving oil and CO2 emissions for things that we can’t yet substitute. Like insisting on change to the European ATC system, which could save 10% or more of the air fuel requirement without pouring concrete or sacrificing anything at all. Like air-source heat pumps and insulation, or…well, enter your favourite project here.
Unfortunately, the government has no credibility on this. Neither does it have any credibility on the eventual target for movements at LHR anyway – they always burst the target, which isn’t included in an act of parliament and therefore is pretty meaningless. And their efforts to balance the Heathrow decisions are crap – a high speed rail “hub” at LHR? On a line from where to where? Great Western electrification is good, but this sounds like a piece of recreational investment that might seriously harm the prospects of building a proper LGV network.
And the responsible minister is Geoff. Fucking. Hoon. Of all people. Aren’t you in jail? Aren’t you dead yet? (I suppose that does not die which can eternal lie.) And so, I conclude, I’d better oppose it anyway. It’s the only way to be safe.
Meanwhile, across the way, the Tories want to “examine” high speed rail. Woo. More talk. And, ah, build a forty billion quid airport in the sea, whilst keeping Heathrow open as well (good luck with the 70-odd mile transfer!). As someone said:
Our government is pitiful, whoever you vote for.
They surely can’t mean this; back in 1969, the Foulness scheme was a political manoeuvre, a Straussian statement. I suspect its resurrection is something similar.
What are they trying to hide? Is this an effort to kibosh offshore wind development? Are Dave from PR, Gideon and Boris climate change deniers? Or what?
Quite a score for our reader “Ajay”, who I think is the first to spot that the Mumbai terrorist attack bears a very close resemblance to the coup plot in Frederick Forsyth’s The Dogs of War, which makes it the third and possibly fourth case of someone actually using Forsyth’s book as a practical handbook. The exact number depends on whether you believe the story that Forsyth actually took part in planning a coup in Equatorial Guinea in 1977 which didn’t go ahead, and recycled the work he did on it as a novel. Forsyth now semi-confesses to this, but this may be self-publicity from a man who was, after all, sacked from the BBC for making up the news.
Certainly, however, the so-called “Wonga Coup” team in Equatorial Guinea read the book, Mike Hoare’s “Froth Blowers’ Society” attack on the Seychelles apparently issued a copy to every participant, and now this. How does, say, Curzio Malaparte’s Theory of the Coup d’Etat compare to that? Forsyth can probably claim that more people have died as a direct result of his book than any other book not written by an economist.
In fact it’s closer than you might think; the Grauniad, whose coverage of the whole incident was excellent, has a neat map you might want to consult. Apparently, two of the Zodiacs used were found at the north end of Back Bay, on the west/left hand side of the map; this suggests the attack plan was very close indeed to Forsyth’s. There are two groups of targets, and each group is fairly close to a beach on that side of the peninsula, even though some of them are closer to the east (harbour) side. But doing it this way saves navigating around the headland and keeps away from the main port, where you could expect a police presence.
Very roughly, it’s about 2,050 feet from each of two landing spots near the targets to a point exactly half-way across the entrance of the bay, so you’d know when to set course – this is exactly how the mercenaries in TDOW set up their attack, launching further out to sea in a big group and using the ship as a mark to lay their course to the jumping-off point, which they identify by a transit between two landmarks. Of course, there are plenty of buildings they could line up to identify this waypoint laterally (the Wankhede Stadium looks like a candidate).
Politically, this implies that the “Deccan Mujahideen” weren’t from Deccan at all – otherwise, as someone pointed out, they’d just have taken the train in. Clearly they needed to cross a border, or else the ship and the Zodiacs would have been just more moving parts. This also suggests that they couldn’t rely on getting arms in India. I wonder what they did with the ship? One option would be to have her sink; another for her to sail quietly on, although the chances of getting away wouldn’t be great.
There was worrying reporting that a Pakistani merchant ship had been stopped by the Indian navy but fortunately, if you like your Ganges without plutonium, a search of the ship revealed nothing suspicious.
There’s another question – this wasn’t designed as a suicide attack. Suicide attackers have no need of false papers, cash, and certainly not credit cards:
A bag found in the Taj Mahal hotel contained 400 rounds of ammunition, grenades, identity cards, rations, $1,000 (£650) in cash and international credit cards, indicating a meticulously planned operation.
That certainly sounds like the equipment of someone who at least wanted to keep the option of escape open, and of course we have little idea how many people landed. It was quite possibly a suicidal mission, but that’s not the same thing. The special horror here was that the violence was dispersed and prolonged; it happened all over the place, and it kept happening.
This of course carries some information as to what kind of group carried it out. Clearly, they weren’t the sort of people who you recruit because all you need is someone to carry the bomb. They had to take independent action, and they had to sustain their will over an extended period of time. Good relations between India and Pakistan don’t really provide much net information; when things are bad, you’d expect terrorism, and when things are good there are people who want them to be bad again.
Meanwhile, the Dogs of War parallel holds in another way – the mercenaries’ exit strategy is the weakest bit of the plot, and had it been put into action the endgame would probably have been a lot like the last day or so in Mumbai, with the coupsters being gradually picked off around the presidential palace as they ran out of time, ammunition and ideas.
So I’m trying to make a little Google Map overlay, that will reload data from a remote server every time the page it contains loads. Obviously, I read the documentation, look at examples, sign up for a Google API key. But it’s not working, and I’m getting error messages that the API key is “for a different website” and that I can go get another one. I now have three, which the docs say I shouldn’t. Any ideas?