Archive for the ‘Internet’ Category


The National Rail website has borrowed the sensible URIs from Although it does seem a bit of a miss that there’s no shortener like – say –, as those are by definition quite long strings.

The website that identifies the satellite a dish is pointing at, or tells you where to point the dish for a given satellite. Includes a mobile augmented-reality app!

The Vatican has IPv6, and they peer with major Italian ISPs.

Blackwater, after the party. Curiously the entire piece doesn’t mention that the UAE military operates Mirage 2000s and such things, but it still makes it fairly clear that Erik Prince’s new job in Dubai is basically providing foreign goons to beat up the subcontinental construction workers if they make any trouble.

An interview about Piggipedia.

You know there’s been a revolution when people enjoy a talk by Alex Callinicos.

Daniel Davies is notorious for making more than full use of a joke once he gets hold of it. I think this is the original source (perhaps even the Urquell) of his line that Black Swan author Nassim Nicholas Taleb must be furious at what Hollywood did to his book in development.

Well, I watched the movie on a plane the other day. It was that or The Social Network – I was planning on a week in Silicon Valley surrounded by tightly wound super-ambitious geeks, so I get enough of that at work. As it happens, there are a couple of good lessons about risk in Black Swan. Perhaps Taleb shouldn’t be so touchy about it after all.

1. Tail-risk is real

Just as Nassim Taleb said in the book, no matter how good your planning, you can’t hedge everything and you will tend to underestimate the weirder and wilder ends of the distribution. One day, something not just bad but incalculably weird, something you never expected you didn’t expect, will come rapid-roping into your back garden and piss in the pool. Of course, it’s likely to happen on stage on the first night at the worst possible moment. You’ll have to be ready, but you can only be ready in a general sense. Get your trigger movements right – far better to be calling an ambulance and plunging into the fray than locked in the bunker with a PR agent and a large amount of toilet paper. Act right in the crisis and much will be forgiven.

Inevitably, if you want a clue, look at the things you try to repress and deny and don’t believe could ever happen. That’s why you deny them.

2. That said, you’ve got to put up with it

All precautions must be seen in the light of the scale of the threat. Too much security is as dangerous as too little (this may be more Schneier than Taleb). Without a certain amount of optimism bias and risk tolerance, you’ll never get anything done. In fact, you’ll end being terrified of your shadow. (And why did you choose the word shadow, with its, ah, many meanings, Mr. Garrovell?) Your colleagues may well wish they had your job, but that’s no reason to kill yourself. In fact, after a certain level of neurosis is passed, self-protection shades over into self-sabotage – delivering just what you imagine your enemies want, whether they be real or imagined.

3. Don’t draw conclusions based on regional accents

Black Swan is the only movie I can think of in which New Yorkers see an outsider – a Californian – as being unimaginably evil, sophisticated, cool, and cunning. In fact, this was the plot detail that kept coming back to me. Wall Street and City investors in dozens of regional mortgage lenders that turned into financial neutron bombs imagined they were smarter than the offcomed’uns.

Remember this post from 2006, and especially this one from a year later on the next big miscarriage of justice? Well, look what just happened. It’s far worse than even I thought – the police were well aware that there were serious problems with the Landslide case as early as February, 2003. Specifically, the old National Crime Squad seems to have been extremely gung-ho about the whole project while the regional police forces were much more sceptical. Later the whole thing was slung to the Child Exploitation and Online Protection Centre, one of the weird sort-of police agencies that proliferated in the late Blair years. Meanwhile, a suspect has succeeded in claiming damages. Both cases show various police forces in a very bad light indeed – the US postal inspector in the first suddenly retired to look after his “sick wife” when his evidence was challenged, while as for Hertfordshire Police:

Despite this, the officer, Detective Constable Brian Hopkins, pressed three charges of possession of indecent images of children. Mr Justice Mackay said he cut a “rather pathetic figure” in the witness box, having initially claimed he could not give evidence because of a psychiatric condition….The judge found that Mr Hopkins, who has since left policing, not only had “no honest belief in the possession charges when he caused them to be brought against [Mr Clifford]”, but did so “to protect his own position”.

Feel the fremdscham, baby!

Meanwhile, my bank card has been compromised. So I was in San Francisco of a Sunday, walking around the Tenderloin looking for a cash point that wasn’t looking back at me with mischief in its eye. Preferably one attached to a bank. I eventually walked up as far as Van Ness and found a Wells Fargo branch. It wouldn’t give me any money, nor would the Bank of America. So I ended up phoning the bank at extortionate roaming rates, standing on the forecourt with a small encampment of the homeless. Thinking that I had less US currency to my name than they did, I struggled through the IVR thickets, confirmed my salary hadn’t somehow vanished, and got into a queue to report that the fraud-detection robots had zapped me. I stayed on the line until AT&T dropped the call after 12 minutes. The phone started whining; it’s like a little jet fighter. You can do a lot of cool things with it, but it’s best not to go too far from the refuelling tanker or you’re screwed. Back to the hotel. I tried to call them on Skype, but AT&T’s WLAN was too bad to hear the IVRs. I plugged in the phone, called again, explained that I didn’t want to report the card stolen but rather the opposite, and sat in 23 minutes of queues. Curses…curses…24 hour fraud algorithms…not 24 hour staffing, though…why not call me?…banks…banks…banks…!

And then I got through. And the fraud investigators told me that the police had found my card in a list of cloned cards offered for sale on the black market. In the circumstances, they hadn’t called me or given out any information for fear of giving away the secret, as the investigation was still going on. Oh…right. They listed some transactions, agreed to let me withdraw up to £100 a day in cash and honour direct debits, and left the Visa facility frozen. They refused to say anything about where or when the security breach might have occurred, although I think the detail about the Visa card might be significant. Call us when you get back to the UK – and by the way, here’s the direct number.

The whole incident had just been annoying up to that point, but this changed the game. I was left with a whole load of surplus indignation on my hands past its use-by date. It cluttered up my room at the Phoenix like a chunk of un-Californian, clanking machinery. I suppose I could spend it on the thieves, but who were they? Rather than just harassing me and profiteering, my bank had actually done something I could only agree with. And the police had actually protected me from an actual crime, without my even noticing, with the occult efficiency Norman Lewis said had attached itself to the word “intelligence”.

As far as I know, no money is missing, but I haven’t audited as many as 14 months’ worth of transactions through my current account yet. That’s since this card was issued – they couldn’t give me any other bounds on it. After all, as they said, it was impossible to say how far the list of cards had been sold on by whoever had originally collected them.

Anyway, I didn’t even need to draw any more cash after the first $100. My expenses in Silicon Valley were unusually frugal – the nearest I came to spending significant amounts of money was trying to catch up with two colleagues who’d gone out looking for amusement. (I spent 20 minutes looking for a cab in Palo Alto at 10 o’clock at night and eventually gave up, having noticed that there seemed to be less traffic on the roads at that time of night than I would have expected in a Yorkshire Dales village.) I read the two Operation Ore articles and logged them for future use. As briefed, I called HSBC on my arrival back in Britain and they initiated a new card.

And it was only as I wrote this that I remembered I ought to be scared. After all, it is impossible to say how far the list of cards…

Amusingly for a comment on scalability, I couldn’t post this on D^2’s thread because Blogger was in a state. Anyway, it’s well into the category of “comments that really ought to be posts” so here goes. So various people are wondering how the New York Times managed to spend $50m on setting up their paywall. D^2 reckons that they’re overstating, for basically cynical reasons. I think it’s more fundamental than that.

The complexity of the rules makes it sound like a telco billing system more than anything else – all about rating and charging lots and lots of events in close to real-time based on a hugely complicated rate-card. You’d be amazed how many software companies are sustained by this issue. It’s expensive. The NYT is counting pages served to members (easy) and nonmembers (hard), differentiating between referral sources, and counting different pages differently. Further, it’s got to do it quickly. Latency from the US West Coast (their worst case scenario) to is currently about 80 milliseconds. User-interface research suggests that people perceive a response as instant at 100ms – web surfing is a fairly latency tolerant application, but when you think that the server itself takes some time to fetch the page and the data rate in the last mile will restrict how quickly it can be served, there’s a very limited budget of time for the paywall to do its stuff without annoying the hell out of everyone.

Although the numbers of transactions won’t be as savage, doing real-time rating for the whole NYT website is going to be a significant scalability challenge. Alexa reckons 1.45% of global Web users hit, for example. As comparison, is 0.4% and that’s already a huge engineering challenge (because it’s much more complicated behind the scenes). There are apparently 1.6bn “Internet users” – I don’t know how that’s defined – so that implies that the system must scale to 268 transactions/second (or about 86,400 times the daily reach of my blog!)

A lot of those will be search engines, Internet wildlife, etc, but you still have to tell them to fuck off and therefore it’s part of your scale & scope calculations. That’s about a tenth of HSBC’s online payments processing in 2007, IIRC, or a twentieth of a typical GSM Home Location Register. (The usual rule of thumb for those is 5 kilotransactions/second.) But – and it’s the original big but – you need to provision for the peak. Peak usage, not average usage, determines scale and cost. Even if your traffic distribution was weirdly well-behaved and followed a normal distribution, you’d encounter a over 95th percentile event one day in every 20. And network traffic doesn’t, it’s usually more, ahem, leptokurtotic. So we’ve got to multiply that by their peak/mean ratio.

And it’s a single point of failure, so it has to be robust (or at least fail to a default-open state but not too often). I for one can’t wait for the High Scalability article on it.

So it’s basically similar in scalability, complexity, and availability to a decent sized MVNO’s billing infrastructure, and you’d be delighted to get away with change from £20m for that.

So I had the opportunity to take part in an Augmented Reality standardisation meeting on the fringe of this year’s 3GSM Mobile World Congress. First of all, it was the year the heavens opened (someone on twitter said it was as if the show had turned into Glastonbury) and I got drenched and my shoes went bad, and my cab didn’t take me to the Telefonica R&D building in Via Augusta but instead to the main switching centre, this amazingly domineering building…

2011-02-17 13.07.09 Telcos – they live in places like this, they know where your dog goes to school, but can they tell you if it’s really your bank on the line?

So I got soaked again, and eventually arrived, and spent the first session listening to my shoes rotting. I acted as scribe for the session on AR browser implementations, markup language vs. JSON, native application vs. browser plugin and the like. I hope I contributed something of value. I have a Flickr set of the annotated flip charts here; I’ve been asked to help prepare the final report. Which just goes to show the enduring truth that if you want to influence something, wait until the very end and sum up with a balanced account. Supposedly this used to be the way to pass the Diplomatic Service exams – buy a pipe, puff on it occasionally during the team exercise, then “sum up with a balanced account”. But you’re not allowed to smoke these days.

2011-02-17 19.16.26

Pulling together various resources, I’m beginning to get a picture of what happened with the cut-off and restoration of the Internet in Egypt. First up, at least in some senses, it may be valid to say that the Internet played a role – Arbor Networks observed that traffic to and from Egyptian networks (and between them, in so far as any of them are customers of Arbor’s) had spiked dramatically, almost vertically, in the two hours before the cut-off and that the whole week up to the 28th of January had been one of unusually heavy traffic.

When the cut-off went into effect, at 5.20pm local time on the 27th, it was implemented by forcing all the networks that peer at the Telecom Egypt-controlled Internet exchange to drop their BGP peering sessions with the exception of AS20928, Noor Data Networks. Famously, this is the operator that serves the Central Bank and its payments settlement system. Essentially immediately, 2,576 networks announced by 26 Autonomous Systems became unreachable. The surviving 26 ASNs including, as well as the Central Bank, the Alexandria Library, and the national research & education network, which if it is at all like most NRENs has a lot of its own infrastructure.

On the 31st of January, there was a further wave of cut-offs which removed another 14 ASNs and 134 networks. The list of the last survivors is here – notably, someone had clearly realised that not cutting off the students, of all people, was a missed opportunity, as the NREN isn’t in there. However, one of the mobile operators (UAE incumbent Etisalat’s national opco) stayed online although they had been ordered to cut off the mobile service itself. Perhaps they provide service to the government’s mobile devices?

Interestingly, however, according to posts to NANOG, several of the .eg root DNS servers remained online (not surprisingly, as at least one is outside Egypt). Even more interestingly, even after the BGP sessions with the IX were pulled down, the lower layer equipment stayed active – Egyptian ISPs noticed that there was still link light on the fibre optic lines between their locations, and theoretically it would have been possible to cobble together static routing between their systems.

Similarly, the internal voice network remained operational and so did the international SS7 gateways that link it to other phone systems. As a result, some people found that they could still reach their ISP, whether by dial-up over the voice circuit or even sometimes on DSL. The question, though, was whether there were any routes beyond the ISP’s nearest point of presence. Several foreign ISPs offered free dial-up connectivity over international phone service (notably this French one).

And, it seems, Egyptian ISPs also tried to re-establish internal connectivity after the cut-off, when they noticed that the fibres were still lit up. However, the problem was more subtle than just pointing static routes at each other. Communicating with people outside Egypt wasn’t, after all, the primary need, and anyway, it required passing through the government-controlled exchange.

But the problem with Facebook, Twitter, Gmail or what have you is that unless they have data centres in your country, they’re international traffic. Depending on their internal architecture, even if they do, they might be dependent on international routes. An Egyptian engineer who posted to NANOG during the revolution made the interesting point that, although Egyptian ISPs are relatively well-interconnected among themselves, not that much traffic flows over the interdomain links as so much stuff goes out to the global Internet. It’s analogous to the old problem that the topological centre of the African Internet was 36 Tooley Street, London SE1 (the LINX headquarters), or 111 8th Avenue, New York, depending on whose version of the story you like better, although less pernicious as the infrastructure is there to solve it.

Sometimes this is useful – it’s harder to censor stuff hosted in another jurisdiction. But it’s also a problematic dependency. Back in the Egyptian NOC the New York Times was hosted on, they were struggling to find copies of key software packages to distribute, for example clients for Internet Relay Chat messaging, and also critical data files such as cached DNS zones, lists of domain names and their corresponding addresses. Some ISP engineers are now working on preparing emergency packages of software and data for use in an extreme emergency – for example, regular dumps of the root and local DNS zones, similar snapshots of the local routing table, not to mention PGP signing keys and contacts for as many other engineers as possible.

After all this, what were the government’s aims? The initial cut-off was probably motivated by a combination of wanting to black out sources of independent information and hoping that it would hinder the protestors’ organising. Some of its particular details – for example, leaving 20928 up and not trying to shut down interdomain links within Egypt – may have been an effort to keep some “normal service” going, as well as not preventing VIPs from transferring their money out of the country. It’s also possible that cutting off link light between all Egyptian ISPs without physically grubbing up the fibres was harder than it looked.

So then, why did they bring it back on the Tuesday of camels and thugs? One interpretation is that they were hoping people would go home and update their Facebook statuses, which would have been incredibly patronising. But the Egyptian elite patronised the hell out of the public every time it went on TV, so it can’t be ruled out. Another one is that they hoped to project an impression of returning normality, which didn’t really fit with thugs on horseback swinging knives, but then their response wasn’t characterised by coherence.

Another still is that they hoped it would help to get the government’s propaganda out there. This argument – Gamal Mubarak flipping through his copy of The Net Delusion in a curtained backroom of the palace – has the advantage that when the Internet and the mobile networks were reactivated, there was a rash of reports of loyalist trolls, and one of the first things that happened was that the government forced the mobile operators to send out threatening bulk SMS messages – spam as a weapon. But this was surely incredibly optimistic.

In fact, what did happen was that people started doing precisely what they had only been doing to a limited extent the week before. Twitter feeds from Egypt filled up with what the NANOG crew would term operational content – requests for more medical supplies, reports of a lost child, calls for more protestors to mass at a specific gate into Tahrir Square. This was the real thing – a tactical radio network for the mob – and ironically it was mostly running over SMS and going out to servers elsewhere in the world. And, of course, its major carrier was the much reviled Vodafone Egypt, unwilling deliverer of Central Security’s spam blitz.

Looking back at Tunisia, and forward at Egypt, I think there’s an important point that this post almost hits but not quite.

Specifically, I’m fairly sceptical about “Twitter Revolutions” and such – if your revolution has someone else’s brand name on it, how revolutionary is it? – but I don’t think it’s irrelevant.

I’m feeling a little sorry for Evgeny Morozov at the moment. He’d just hacked out a niche as Mr. Grumpy by royal appointment to the blogosphere, when first Wikileaks and then Tunisia and Egypt came sweeping through, and the Tunisian secret police hacked all the Facebook pages in the country, and the Egyptians turned off the Internet, just pulled all the BGP announcements… Sometimes it’s not your day.

I do think, though, that there is an important way in which a whole lot of Internet tools contributed to the revolutions. I recently posted on the way in which people can at least for a while function as if they were part of an organisation just because they shared certain assumptions. It’s the idea of the imagined community, which can be defined as a group of people who are behaving as if their weak social ties were strong ones. If you want a mental model of this, the revolution happens when enough people change state and start doing this, and it stops again when they revert to pursuing their interests in the normal way. Of course, what happens in between may have changed what those are and how they do it. From a different direction, look at Chris Dillow’s post here – it’s theoretically irrational to take part in politics, until it’s not. The point when it stops being irrational, though, is the point when people stop thinking it’s irrational.

In that sense, a lot of the work of starting a revolution is starting a myth. An ironic salute to this was the Egyptian government’s decision to turn off the Internet, and later the GSM networks as well. If the value of the Internet really had been as a way to pass on the time and place to assemble, this would have been a serious blow to the movement. But once you’re a really angry Egyptian, where else would you protest but Tahrir Square? It wasn’t that they needed it for tactical communication, but rather for strategic propaganda. Also, once they took this step, they had also inadvertently demonstrated to the other world media that This Was It. The mainstream media remains very good at bringing its own connectivity, and the main barrier to them covering the news is usually that they don’t think something is news. Giving Al-Jazeera and friends – who had been heavily criticised on the Web for being soft on the Egyptian government – a monopoly may have been a really bad idea as it forced them to cover the news or look indistinguishable from Nile TV.

I suspect that a lesson here is that the last thing authoritarian governments will do in future is turn the Internet off. For a start, they will increasingly need to keep it up for economic reasons – the ISP that serves the Egyptian stock exchange and central bank was left alone, and with time I would bet that it would become increasingly porous to information. But much more importantly, this is not a policy that has a great track record. Burma managed it, but started with advantages (not many users, only one network, and a strong position to start with). Iran did far better with its throttle-down-and-spy plan. Even though the Tunisians funnelled all the Facebook accounts in the country into one, controlled by the secret police, it didn’t seem to help.

Jamais Casco (via here) asked if you could start a genocide on Twitter – a sensible point, as we know you can do so with the radio, the cinema, television, the newspapers, and (thanks to Serbian turbo-folk) rock’n’roll*. Terrorists tried to start a nuclear war with a spoofed caller-ID. Whether or not you could do that, you can certainly start a mob of quasi-fascist loyalist paramilitaries on QQ. Out of all authoritarian governments, China does best, with strategic trolling and semi-official moderators, which may be more important than direct censorship. Andrew Wilson’s Virtual Politics makes the interesting point that Russia in the early Putin years didn’t so much censor the Internet, as distribute government talking-points and favours to carefully selected bastards.

Then again, was the greatest success of the wumaodang model the 2004 US presidential election? The best way to fight one myth is perhaps with another. And the best ones are distinguished by the fact they are sometimes called principles. The really depressing consequence of this is that Paul Staines probably has a job for life, although the less depressing corollary is that he gets to herd several hundred idiots yelling about ZaNuLiebour for the term of his natural.

A couple of other interesting links: Charles Bwele makes the point that in much of the world, the so-called new media are more like the first ones. Did you know about the Grozny riots of 1958?
*The world’s first genocidal remix is yet to come, but I wouldn’t rule it out by any means. All art aspires to the status of music, and just look what people get up to with books.

I’m not quite as sceptical as some about this. However, it’s not clear to me how this differs from the sort of thing UNOSAT does all the time – here’s their analysis of imagery over Abyei, the key border area between North and South Sudan. Actually it looks like the “Enough Project” is going to be using UNOSAT imagery itself, going by UNOSAT’s own website.

If you follow the link you’ll see that they have more than reasonable capability (50cm resolution) and that they routinely observe the presence of refugees/displaced persons and returnees, construction, and the like. There’s obvious relevance to an effort to monitor potential conflict along the border, especially as oil prospecting is an issue. You can’t easily hide oil exploration from a satellite that can resolve objects 50cm across.

However, the downside is that the UNOSAT report is comparing images over a two-year period. I would suspect that they will need much more frequent passes to be operationally responsive, which is where the costs get interesting.

Also, I’ve just been over to the website and it’s a bit of an unstructured clickaround. What I’ve always liked about MySociety sites is that they all have a function – FixMyStreet reports things in your street that need fixing, WDTK issues Freedom of Information Act requests, TWFY looks up information on MPs, TheStraightChoice logged what candidates promised and said about each other during their campaigns. DemocracyClub, for example, worked because as soon as you logged in it gave you something to do and some feedback about doing it, and then it hassled you to do something more. It had structure.

Notoriously, if you don’t give volunteers something to do as soon as they show up, they’ll wander off. It is nowhere easier to wander off than on the Internet. And so there’s a button to twitbookspace it and a donation link. There isn’t, however, a to-do list or, say, a list of pairs of images that need comparing.

a quick HOWTO

Because someone wanted this: a list of aircraft investigation resources.

First thing: you always need to be able to map registrations to aircraft serial numbers (MSNs) and vice versa, as well as linking registrations to operators and vice versa. So you need to subscribe to one of several commercial databases that provide this information. Otherwise you’d have to follow up each registration individually with the state of registry, which isn’t so bad if it’s the US (you can query the registry at or the UK CAA’s G-INFO. If it’s Equatorial Guinea or Kyrgyzstan, though, it won’t be available online or really at all, and that’s just how the registrants want it.

I use ATDB, but there are others – the biggest one is Airclaims, which is marketed at insurers and debt chasers, but it’s seriously pricey.

You’ll also often want to find out where an aircraft was at a particular date, and if the registration and operator on fBecause someone wanted this: a list of aircraft investigation resources.

First thing: you always need to be able to map registrations to aircraft serial numbers (MSNs) and vice versa, as well as linking registrations to operators and vice versa. So you need to subscribe to one of several commercial databases that provide this information. Otherwise you’d have to follow up each registration individually with the state of registry, which isn’t so bad if it’s the US (you can query the registry at or the UK CAA’s G-INFO. If it’s Equatorial Guinea or Kyrgyzstan, though, it won’t be available online or really at all, and that’s just how the registrants want it.

I use ATDB, but there are others – the biggest one is Airclaims, which is marketed at insurers and debt chasers, but it’s seriously pricey.

You’ll also often want to find out where an aircraft was at a particular date, and if the registration and operator on file matched with the reality. Fortunately the world is full of volunteer spies, plane spotters, who take enormous numbers of photos of anything that might fly and post them on web sites. This is how German Amnesty managed to characterise the CIA rendition planes. is the biggest and has a full-featured search engine, but JetPhotos and a few others are worth trying if you turn up a blank. ATDB will try to pull photos from various servers matching your current query. Try querying by registration as well as MSN, and note that if you know the exact type (all six digits of a Boeing designation rather than just 737) you can often do a type-at-location search.

A lot of airports publish their movements on the web – this is why I did the Viktorfeed, to automate watching Dubai and Sharjah airports. This is useful if you want to know who is going where, or who’s recently been where, and you may be able to find out more by cross-referencing. For example, if Airline X’s flight from Dubai to Baghdad is listed as a passenger flight with a 727 and they only have one 727 with a pax, combi, or quick change configuration, you’ve got the registration and possibly the MSN.

Also, for some parts of the world, you can monitor Air Traffic Control data through sites like FlightTracker or FlightAware and LiveATC. A lot of these are aggregators for people who operate “virtual radar” devices, which receive the SSR transponder data practically all aircraft squawk when they hear a surveillance radar. Here’s an example. The problem is, of course, this depends on owning fancy radio equipment being tolerated in the places you’re interested in. If your BlackBerry is considered subversive, or there’s just no electricity, you’re unlikely to find a virtual radar server – and they won’t work if there’s no radar coverage. (However, if your staff go somewhere weird, why not…)

The Aviation Safety Network website provides a database of accident reports, which can be useful. AirNav has a huge database of airports, although most of them are on Wikipedia. There’s a lookup site for ICAO and IATA codes here. The Great Circle Mapper is a useful calculator for ranges and routes and makes pretty maps. Obviously Google Maps and Earth are useful if you’re planning more complex visualisation.

Things to look out for: networks of companies that repeatedly trade the same aircraft, especially if they’re based in the same location and their corporate registration is somewhere else and bizarre. Inconsistency. Unlikely details (the office that is in Kiev and is a cinema, the manager who is a Soviet ice hockey player and is dead). Things you can’t easily find: details of the cargo or whoever eventually controls the company.

Also, do be sceptical and don’t turn yourself into Evan Kohlmann (via Patrick Lang – check out Adam Silverman’s thoughts in the comments).

I can remember when the Emerson & Kohlmann show sounded fresh and new in late 2001. I can also remember when the Counterterrorism Blog launched and turned out to be really just a lot of drum-banging rightwing propaganda and some really quite horrible guilt-by-association games. Worry about the false positives.

Yahoo! might be going to shut down, the link-sharing and bookmarking website it bought back in 2005 or thereabouts. (They might sell it, too.) This is awful – it’s one of the most useful things on the Web and it’s a key link in the production chain for everything I’ve written in the last six years for this blog, Fistful of Euros, Stable & Principled, Telco 2.0, and God knows what else. As well as providing a bookmarks file you can use anywhere, it also provided a huge quick-reference handbook of stuff other people found useful. During 2004, not only did I start using it, but this blog started to provide a list of RSS items from my account and several other blogs in the sidebar.

Yahoo! never did much with – they managed to retire the original domain name and redirect it to, just as link-shorteners became fashionable, they made the web site more ugly, and they tried to impose some sort of horrible terms of service amendment by asking users to sign in with Yahoo! user IDs. I had nothing against this, but when I saw the lengthy new ToS document, I didn’t bother reading it – it could only be evil, and therefore I refused. Bizarrely, they never even tried putting adverts on the home page, despite being the world leader in display-style Internet advertising, and neither did they ever try to get me or anyone else to subscribe, although they got me and hordes of others to pay for Flickr accounts.

You’ll note that this doesn’t include any new features or anything interesting at all. Also, they never did anything about spam accounts, so a lot of the social functionality became useless as “links for you” were always spam. However, they couldn’t kill a basically useful product. If they sell it, though, it might survive or it might die – look what happened with Technorati.

A common theme about Yahoo! is that although the company drifts strategically, and every now and then gives the Chinese secret police confidential data about dissidents, the engineers are pretty good. True – they released lots of cool and useful stuff. Pipes, YQL, Term Extractor, YUI hackdays. Similarly, the Firefox extension for is very good indeed. It provides a full-text search over your tags, something the web site itself doesn’t, and it can provide offline access to your bookmarks if you need that.

So here’s a tip. The FF extension lets you work with your bookmarks offline and without signing in – so it must store them on your local machine. In fact, it uses quite a lot of Firefox’s bookmarking functionality. And when you sign out, it asks if you want to keep your bookmarks in Firefox. You can add more bookmarks before you sign in again. Therefore, there’s a way to slurp your data out of Yahoo! before it all gets deleted. Obviously they’ll stop maintaining the plugin at some point. But once your data is stored as browser bookmarks, it can’t be too far from being exported to an OPML file, at which point it could be imported anywhere else. Is available?