Archive for the ‘Linux’ Category

Konsidered a waste of time

OK, so I eventually finished listening to the 793 songs in the 2011 SXSW torrent and rating them all. This was a while ago, but it was only yesterday that I reorganised some stuff in the collection and remembered that the couple of gigabytes of mediocrity was sitting there. It was clearly time to implement the TYR Band-Pass filter, my objective methodology for filtering musical slushpiles.

So I frobbed around Amarok until I found the “Automated Playlist Generator” hiding under a rock, and then fiddled with it until I understood the UI-only-a-hacker-could-imagine. Seriously, it would have been easier to just provide a command prompt on the underlying database. (Does a “Match All” Constraint Group match both any rules of its own and also the output of a “Match Any”? Search me, guv, because you can search your hard disk and not find any documentation.)

And it gave me 32 tracks, all with a rating of zero. Now that is a valid output from the filter. Or it would be if there were no tracks rated above the upper limit, 3.5. And I gave out quite a few 5s. So I check in the pile. All the ratings are gone. This isn’t quite as bad as the phase KAddressBook and Akonadi went through a couple of years ago when they regularly, randomly, truncated my contacts file from 269KB to 10.8KB – always exactly the same – and inserted helpful invalid characters. (Fortunately they also left a renamed copy of the original file, so you could just restore from backup.) But it’s pretty shit. Any software that randomly destroys user data has failed and failed horribly. It’s the antithesis of polite software.

But it did produce 32 tracks, so there must be a wrong copy of the data somewhere, which suggests that there might also be a right one.

Meanwhile, I’ve been reading the traffic on kdepim-l about KMail 2 with horror and an increasing sense that KDE is going spongy. Even without anything related to Akonadi actually working, long after the last lot of performance bugs were closed, it still has a nasty habit of keeping the hard disk active for half an hour at a time, doing what? KM2 users report rampant loss of data and of meta-data. And I don’t have a working desktop search utility despite years of promises about Akonadi and Nepomuk and Strigi and “semantic desktop”.

Think about it like this – a new era KDE application that needs to read data from your contacts file, a vCard sitting somewhere in your .kde4 directory, is meant to go to an “akonadi_vcal_resource” that’s mediated by the common Akonadi API and no less than two RDF triplestore databases (Redland and Virtuoso). What happened to the filesystem?

So, I’m going to initiate a new, innocent laptop into the twisted cult this week. And I think I’m quitting the KDE world. I’m not the only one – from 25 killer Linux apps to When you first launch KMail,

it will terminate with a ‘Failed to fetch the resource collection’ error. KMail doesn’t have a default incoming mail directory configured, which causes this error. The workaround involves using Akonadi to specify a maildir location for KMail. To do this, launch the Akonadi Configuration tool and point the Local Folders to /.kde4/share/apps/kmail2/.

Well, I’ve never had that error but my install crashes every time it launches, and only ever works on the second time of asking. Of course, I could spend all my time maintaining this particular e-mail client. Don’t all write at once.

So OpenSUSE11.4 was out this week. As the Jedi said here:

gah! suse is never totally easy

Indeed. I thought I’d do an online upgrade, so I scheduled this to happen when I was in the office and therefore had a fast Internet link available. I applied all the remaining 11.3 updates, configured the three additional repos, did a “zypper ref” and then a “zypper dup”, paged through the Flash player licence, and watched it report 500 odd MB of packages to grab. Much churning later, it started to miss packages, which I installed manually. Eventually, it finished, and I ran “zypper verify” to check it out. This reported that vim-data was missing, so I installed it, and went for a reboot.

Oh dear, the new distro apparently didn’t know what an ext4 filesystem was. And although I could still start 11.3 from the boot menu, KDE wasn’t working. So, back at home, I downloaded the ISO image (2 hours 20 odd minutes at home), burned a disc, and prepared for a clean install, which failed with a message about running out of processes in this runlevel. You guessed it, dodgy install media. Wiped and downloaded again. I check the MD5 hash. It’s a miss. I start the download again and go out. I come back to find the laptop has rebooted and has got to the failure point in 11.4. How? What? I restart in Windows and discover that 678 of 695MB has been fetched before something happened. It dawns on me that Microsoft has force-rebooted the bastard through Windows Update although I set it to do nothing of the sort. I’m getting seriously pissed off now. I download it again, from a different mirror (ox.ac.uk rather than Kent Uni mirrorservice.org). More hours. I check the MD5 hash. What do you know, it’s wrong. And it’s the same hash as last time. As an experiment, I burn it anyway, boot it, and run the media check utility.

Which fails at 63%, block 226192, in exactly the same location as the first time around. Riight, it looks like Novell has pushed a crappy image out to all the damn mirrors. Well, I can still get a Linux shell in 11.3, so I run it up, hook an ethernet cable to the linksys box, run dhclient, and repeat the command-line distro upgrade. Although Zypper still thinks all the dependencies are in place, when I tell it to “zypper dup”, it still manages to find 258 package changes left to do from the original upgrade. It takes an age, but eventually, completes, and it’s shutdown -r now time. And everything now works, right down to hibernated browser tabs.

Except for Python packages, of course. Pythonistas tend to dote on easy_install, but I’m still annoyed that I have to update this stuff out of sync with my linux environment, especially as it lives in my root partition. Would it be so hard to put everything in PyPi into an RPM repository and never worry about it ever again? This is actually an important lesson about the mobile app stores, and the original app store itself, Firefox extensions. Freedom goes with structure.

Lessons from this: once an upgrade shows any signs of weirdness, abort it and start again. And don’t expect online upgrade to work first time – this happened to me with a past OpenSUSE upgrade, come to think of it, but I clearly learned nothing.

Image number four here has a certain additional spice, doesn’t it? What a week. As well as WikiLeaks being the website they couldn’t hang, 4chan became a geopolitical actor, thus fulfilling my prediction that in the future, trolls would be considered a strategic resource like oil. The BBC interviewed a builder from Leeds thinking he was a Liberal MP, but as it turned out, it didn’t matter – the real MP did exactly what the fake one said. And of course there was the case of James Naughtie, demonstrating that the BBC really is the voice of the nation. Speak for England! Prince Charles got a nasty surprise from the students, which startled the mainstream media into actually covering the demonstrators’ main grievance for once.

A lot has been written about how Wikileaks is staying online and I don’t propose to add to it – this piece on CNET and this one from Renesys should tell you all you need. If you’re looking for mirrors the list is currently here and there’s a mirror of the mirror list here.

However, there are a couple of good technical points to be made here that I’ve not seen elsewhere.

First of all, Wikileaks is a website designed to be easily cloned. If you look at it, each page is a self-contained file with a flat URI – there are no signs of a query string, and each cable released has a unique ID within the same directory. This is important because it means that the process of creating a mirror is just to copy all the files into the /public_html/ directory of another web server. On a Unix-like system, it could be a one-liner command (the site doesn’t actually let you do this the quickest way) – the utility wget is capable not only of traversing the directory and downloading all files, but also of changing links within them to point to the same filename in the target directory. The -m or –mirror option activates the options -N -r -l inf –no-remove-listing, which will in order ensure you only download material you haven’t already loaded, that wget will get everything in the target directory or directories, and that any directory listings will be preserved. -p requires that everything needed to make up a page, such as a photo of Julian Assange, will be retrieved. -k turns on link conversion.

It might be enough to do: wget -N -p -k wikileaks.wherever /home/you/public_html/

So it’s easy to create a mirror, and it’s trivial to keep it up to date – you could just run your script as a cron job to grab whatever gets released every day. Anyone thinking of a really controversial Internet project should, IMHO, consider design-for-cloning to be a useful pattern. The clone count is now in the thousands.

Second technical point: what a horrible idea Mastercard SecureCode (and its pal Verified by Visa) is. I already hated it before this – it’s a password, that should be a strong password because it’s financial, but that I don’t use that often and therefore can’t remember, and it trains you to accept the idea of typing confidential information into a random web site you didn’t ask for. Essentially all phishing requires you to type your bank details into something that you didn’t ask for. Forcing the public to type their bank details into some random website they didn’t ask for is howling insane. Right?

Also, the failure case is horrible – you get to reset the password by disclosing a whole lot of confidential information into the same random website you didn’t ask for, so an attacker who managed to inject a frame into the original merchant’s website could fake a failed payment and harvest all the information they would need to empty your bank account. And the service support when they imposed it on me was dire, especially as the SecureCode web site went down part way through the process.

But it’s worse than that. An important part of the way card payments are accepted on the Web is that, as is also true in shops, you interact with the merchant, whose bank interacts with the wider infrastructure. So you should know who you’re dealing with. Further, the bank should at least have some idea who its merchants are – they are customers after all – and restrict access to the system to them. And there’s more than one bank that provides merchant service, so there are no single points of failure.

The SecureCode (and its Visa twin) websites, though, are customer-facing, so they have to accept traffic from the whole of the Internet. And all the Mastercard payments from the Web have to go via the SecureCode website. So you have a critical operational function, that is a single point of failure, and that is exposed to every last dog on the Internet. It’s only surprising that somebody didn’t bring it down earlier, especially when things like Bees with machine guns! are available.

40 years of UNIX

40 years of UNIX!

If you want a monument, J. P. Rangaswami of BT recently said in my presence that “I can’t think of any BT product that doesn’t include Linux in some way.”

KDE4 user? Been having problems with the DCOP and DBUS servers? Do your K-applications no longer launch from the graphical user interface? Do they sometimes not even launch from the command line? Have you got references to “file retriever failed: 149” in your .xsession-errors log?

Well, you’re in luck. I’ve been having this problem recently – it’s especially annoying as things like KNetworkManager and KPowersave can be affected, although non-KDE apps still work (except, of course, when they have to integrate with a K application in some way). It’s associated with KDE4.1 in some way, as KDE3 apps still work, including the KDE3 versions of the same apps. But I’m not any more. Trying to fix the problem, I decided to have a look in the /tmp/ directory, and what did I find? North of 250MB worth of worthless cruft, old prefetched web pages, prefetched ads from old web pages, blank cache files, that sort of thing.

So I removed it all; just to be vindictive I ran the KGPG Shredder function through it as well. And then I moved on to the /yorksranter/tmp/, which was also stuffed with crap. Upshot? Everything works again, faster and better than before. Presumably, the stuffed /tmp/ caused KLauncher to time out before it got something registered somewhere. 250MB is a lot of cruft, but it’s not that much; I think the problem was caused by the number of directory entries rather than their size. Perhaps I should set up a script to flush the e-crapper automatically.

Met up with regular reader and Linux hacker Duane Griffin to discuss the ORGANISE project . As a result, here’s a new and much amended specification that supersedes the first one.

So I actually bought a printer; in fact, a printer/scanner. And I considered buying two pairs of jeans after showing up at the count with interesting new holes. Am I descending into bovine consumerism? And the obvious next step was to qualify it with the Linux Lappeh.

It wasn’t quite the “And then my troubles began…” experience like the BIOS reflash in January, but I was very amused by the fact that XSANE both throws a dialog box containing the following words:

You are trying to run Xsane as root! This is DANGEROUS! Please do not file bug reports for anything that happens when running Xsane as root: YOU ARE ON YOUR OWN!

and also suggests running as root as a generic troubleshooting option in its documentation. Well, I did, and all went OK. As I said to Soizick: the great thing about using Linux is that you get to feel like a mad scientist.

I recommend and endorse hplip.

Ha! Fixed.

Despite the Mozilla Foundation telling you to install the Java Runtime like this:

su root
password: yourrootpasswordhere
cd /usr/java/
chmod x+a jre-6u5-linux-i585-rpm.bin
/home/yourusername/Desktop/jre-6u5-i586-linux.rpm.bin

It won’t work. But if you follow the rest of these instructions it should. Assuming you want Java in a directory called /usr/java/ and you downloaded the thing to desktop by default, that is. Read the EULA, press space to scroll, type yes at the prompt.

rpm -iv ./jre-6u5-i586-linux.rpm
cd /usr/lib/mozilla/plugins/
ln -s /usr/java/jre1.6.0_05/plugin/i386/ns7/libjavaplugin_oji.so

Now, going by the official documentation, you would think this was time to restart the browser and call it a day. But it won’t work. So you need to:

cd /usr/lib/firefox/plugins/
ln -s /usr/java/jre1.6.0_05/plugin/i386/ns7/libjavaplugin_oji.so

Now it will. Does anyone think this could be more user friendly?

My ISP just did something naughty. BT, Virgin Media, and Carphone Warehouse have been caught in a scheme to sell details of all your Web activity to marketers (…and anyone else) without your knowledge or consent. And the full details are genuinely worrying. The Register has a major scoop, in that it’s got hold of the network architecture diagrams; they imply that any and all Web traffic transiting BT Retail’s core network is going to be tapped off and passed to several instances of the ad system. This will intercept your page request, place a cookie on your computer or update it if it’s already there, then (and only then) pass the request to the remote host.

(Some people are concerned that other ISPs using BT Wholesale’s IPStream service might be affected. My reading of the diagram is that the taps are within BT Retail systems, which suggests they’re OK. On the diagram, the ad system is beyond the RADIUS billing/authentication server.)

When the page comes back, the system reads the data in the cookie and decides what ads to squirt into it (or conceivably, to remove from it). Put it another way, they’re going to make their network lie to you. Here’s how to stop them, or at least mitigate the damage. (More data is here. The issue has already sprouted a website, here.)

First: go to Internet Options/Privacy in Microsoft Internet Explorer, Edit/Preferences/Privacy in Mozilla Firefox, and choose “show cookies”. You’re looking for anything from webwise.com, oix.com, or sysip.net. Delete any you see. Now choose Exceptions, enter these names, and choose Block.

Second: if you’re running linux, unix, or MacOS, log in as root and open your /etc/ directory. Look for a file called /etc/hosts; open it. Add the following line: 127.0.0.1 dns.sysip.net (there should be two tabs between them). Save the file. You’ve just told your computer that whatever the DNS server says, dns.sysip.net is located at your own machine, so any attempt to send it information will fail. If you’re running Windows, same procedure, but the file is in a different place.

Third: consider using another DNS server. OpenDNS is handy; open whatever tool your system uses to manage networks, select your home connection, and untick the option to get DNS servers from DHCP. Instead, add 208.67.222.220 and 208.67.222.222 in the boxes provided. The question is, however, whether you trust OpenDNS; if they can’t find a page they return a Google search rather than a 404 or nxdomain, which can break things.

Fourth: go right over here and submit a complaint to the Information Commissioner’s Office.

Fifth: consider getting another ISP. I’m on Virgin Media and I will be.

Dear Lazyweb: I have a mystifying Linux problem.

To begin the tale, I’ve been running Mandriva Linux on my laptop since last October with considerable satisfaction; I use KDE and the Metisse 3D window manager. However, I had to disable the ACPI power management support to get the thing working; eventually I grew tired of this and decided to sort it. Research suggested that the solution might be to update the BIOS; a check showed my machine was running version 4 and the current one was version 11. So I booted in Windows, got the new firmware, reflashed, and rebooted in Linux; and the ACPI CPU tuning, suspend-to-RAM, and suspend-to-disk began working. But later that night I had a crisis – the thing will not boot with ACPI enabled from the Mandriva console, and the KDE desktop wouldn’t start.

Eventually I was able to uninstall the whole KDE and reinstall it, which got us working. But the Metisse desktop doesn’t work; enable it, and the X server crashes on login. Trying to startkde from a command line gives what appears to be a common error, “$DISPLAY not set”. Without Metisse, either no 3D or Compiz works; setting ACPI=ON at the kernel switch allows some but not all functions to work (still no brightness control, but CPU tuning, shutdown, and suspend).

There was an Xorg update on Friday which I hoped might change things, but no. Today saw a big KDE update, but I haven’t experimented on it yet. Any ideas?