Britain's postal-code database online at Wikileaks: produced at public expense, not owned by the public
One odd thing about Britain is that databases produced at public expense -- maps of the country, lists of postal codes, transcripts of Parliamentary debate and so on -- do not belong to the public. In order to use this data, you have to pay gigantic licensing fees to the government, who accordingly threaten to sue people who use them without permission.
It's a pretty bizarre idea. After all, none of these programmes are remotely self-sustaining -- the license fees cover just a tiny fraction of the overall money used to pay for their ongoing upkeep. Imagine if this was how private enterprise worked: an entrepreneur (the government) decides to map all of Britain, so she approaches an investor (the public) for £50,000,000 to cover the expenses. Having spent all 50 mil, she then approaches a second investor (license fee payers) for an extra £5,000,000 for additional operating capital. In the real world, the investors would likely end up split like this:
Initial investor: 60%
Entrepreneur: 35%
Second round investor: 5%
And why not? The initial investor assumed all the risk, while the second round investors merely threw a little money into a proven business.
But in the British scenario, the split looks like this:
Entrepreneur: 51%
Second round investor: 49%
Initial investor: 0%
That is, the entrepreneur (the government) gets total control over the product (maps of Britain, post-code databases, etc). The second round investor (a licensee) gets to commercially exploit the product, subject to oversight from the government.
But the initial investor (the public), gets nothing. If they want, they can become second-round investors and buy licenses from the government. Or they can buy or use products made by the second round investors (the licensees).
This isn't capitalism, nor is it socialism. It's a kind of corporatism in which the risk -- the money spent speculatively mapping Britain, arguing in Parliament, drawing up postal code boundaries -- is entirely assumed by the public, but the reward -- access and profit-taking -- are entirely given to the private sector.
(Many thanks to Paula LeDieu from the British Film Institute for this analysis)
So now we've got the postal code database online and that means that any second, someone from government is going to start threatening lawsuits, telling the people who paid to create it that they don't have the right to own it, build on it and improve it.
the latest
latest episodes
But you make the assumption that the output of these projects exist as a product - not as (respectively) a necessity for government and the management of the country's infrastructure, a safeguard of democratic process and the foundation of a nation-wide postal service.
That seems to be par for the course in Britain; there is still a residual assumption that the public are subjects rather than citizens, and should know their place, which isn't to tinker with and mash up data that their taxes paid for. For example, the third highest-grossing iPhone application in the UK is one for looking up rail timetables. It is the only iPhone application licensed to access timetable data, and costs £4.99, because they can get away with it. (There was another application which could fetch timetable data, but it was locked out after the officially licensed one came out.)
I don't get it. How do Britons send mail? How could postal codes possibly be secret?
The reason for this might be incomprehensible to an American who is a citizen-sovereign. The British Government (and Canadian) is owned as chattel property of Her Majesty the Queen Elizabeth. Same holds true for the military, all natural resources, all land except that titled out to nobles. People in the UK and Canada do not get title to land, merely transferable privileges to real estate, a deed.
There are no citizens in the UK or Canada, but merely subjects of the Queen. All work product of Her Majesty’s Government is wholly owned by the Queen.
Just for the sake of argument-- if an artist gets arts funding from the government (NEA grant in the US, something else elsewhere I imagine), should he or she be able to sell it when she's done? Or should the result belong entirely to the public?
And let's not start considering if the first Harry Potter book should be public domain right now for similar reasons...
The US isn't entirely different.
Much of the IP paid for by the federal government is in the public domain. But not all.
Some, if not most or all, of the states claim a copyright on much of the IP they produce.
My city claims copyright on most if not all of the IP they produce.
@4: A better question: why should the public invest in an entrepreneurial transaction (art produced for sale) if it doesn't get some rights in the product of its investment? E.g., the painter can sell the painting, but the public gets a high-rez scan that it can use for its own projects.
If a town commissions a sculptor to put a statue in the main square, is it a good bargain to leave the copyright with the sculptor, who then gets to charge rent any time someone takes a picture that captures the statue? (This isn't hypothetical: Chicago struck as similar bargain when it commissioned "The Cloud" for its public park).
Artists who are engaged in commerce -- producing material for commercial sale -- have every right to demand all that they can get. The question is, why should the government, negotiating on the public's behalf, give in to demands that leave the public with little or no value in the work that it pays for?
The public paid for the collection and maintenance of the data for public purposes, like delivering mail. If corporations want to benefit from this data, why should they not have to support the system of maintaining and distributing this data? When companies use this data they use it to profit. I think it is reasonable for government agencies, who are supposed to be working in the public interest after all, to have corporations pay a fair price for the value they are receiving from public assets.
@5 Cory- Now, was the government negotiating on the public's behalf, or on the government's behalf?
The postcodes aren't "secret", it's just we have this database copyright which means that any large collection of facts is copyright. This makes it very difficult to build, say, an website which figures out distances based on you entering your postcode without licensing the data or using a service that does.
The national grid system (for identifying co-ordinates on OS Landranger hiking maps) is similarly encumbered.
This kind of thing is why OpenStreetMap is so popular here, and leads to websites like these:
http://www.freethepostcode.org/
http://www.dracos.co.uk/play/locating-postboxes/
http://www.npemap.org.uk/
"This isn't capitalism, nor is it socialism. It's a kind of corporatism...."
I believe Mussolini called it "fascism".
That's a pretty cynical question. If the government is doing its job, then it represents the people.
Um, it's not all that different in the US. My wife works for a GIS company, and if you want access to the USPS postal code database, you have to pay mondo fees.
The reason is this: maintaining a postal code DB is friggin' expensive. The USPS can't keep solvent unless it charges access fees for that data- it's just not in the budget. Yes, it'd be nice if that were accounted for in its budget so that it can give the data away for free, but that's not going to happen.
@Cory Doctorow: That's a very narrow view of governments, limited to a very small subset of possible governments. Having met people, I'm dubious that they should be represented by their government, or at all.
Is this not all about earning additional income streams, from existing data.
i.e. Postcodes are actually created by the tv licensing authority in order to do what they do, they then give them to the post office to do what they do, the nhs and stats offices etc get them for there uses. So the cost of generating and maintaining this data is going to hit government and the people anyway. But because there is also a commercial demand for this data, from all sorts of businesses who want the data for there own reason and are happy to pay for it, so the government then very sensibly are in a position to license this data and recouporate some of the money it cost to generate in the first place, saving the people from at least some of the burdon on the bill.
As far availablitity of the data to the man in the street, if you enter a postcode into google maps or any postcode to long/lat or grid ref conversion system it will give you the same info, so unless your building some kind of commercial product that will need to access thousand of such look ups there isn't a lot of value in having the data, which is actually too large for most common software like excel to handle anyway.
@7- Cory- It is cynical, yes. It's also a good question to ask whether anyone in government for long enough-- living, working there rather than out in and among the general public- isn't eventually going to have his frame of allegiance shift from the public at large to the government itself.
I'm loathe to use class-based terms, but how long does it take to shift from "Public Representative" to "Member of the Governing Class"?
@3 "There are no citizens in the UK or Canada, but merely subjects of the Queen."
Can't speak for Candada, but that hasn't been true since, arguably, the Magna Carta, but certainly since the 1981 Nationality Act: http://www.opsi.gov.uk/RevisedStatutes/Acts/ukpga/1981/cukpga_19810061_en_1
For the record, Charles Arthur from The Guardian pointed out on Twitter yesterday that the file posted on WikiLeaks only contains postcodes and their corresponding geo-coordinates, and does not contain any actual addresses. That is, it is not the actual PAF (Postcode Address File). Nevertheless, it is still a huge chunk of apparently copyrighted data.
Good post above, thanks for helping draw attention to the extortion practised by our government and its failure to put this data into the public domain where it may be used by all.
Roger Lancefield
The postcode information is not secret. The postcode database is (alas) a commercial product; the high cost means a lot of useful stuff isn't done, small business update their records infrequently, and so on. This pseudobusiness model applies to a number of agencies and it's well past its sell-by date.
But the _data_ is nothing special.
There's a project called Free The Postcode! (http://www.freethepostcode.org/) which aims to gather the postcode data from primary sources. The new data will be the same as the existing stuff, only it'll be available under FtP's license instead of the exploitative model the post office are using.
FtP is an OpenStreetMap spinoff. Clicky the link and add yours now - that'll show the Man. :)
As ACB said, this seems symptomatic of the attitude of our governments, that they're our rulers instead of our representatives. Apparently we exist to support the state rather than the other way around.
In mitigation, the Royal Mail do offer a free online Postcode Finder. It's limited to 15 searches a day, ostensibly to "make our system efficient and fair for everyone". Rubbish for businesses and people who want to take on mapping projects, but occasionally useful for everyone else.
The list of data packages you can buy from the Royal Mail is pretty fascinating. Lists of all UK addresses, counts of how many letterboxes are in a given postcode area and something that "Links Postcode data to other geo-demographic UK data". I'd love to spend some time playing with these sets.
They're all offered under a licence rather than sold outright; presumably a clause in that licence is why no online mapping services let you click on a road to find out its postcode.
This is exactly the kind of thing you benefit from when you're a tax payer. An armed force to protect you, a health service to take care of you, a broadcasting service to entertain you.
Applying this Adam Smith market bollocks to everything is surely going to expose a few flaws while ignoring the greater good. Efforts such as the theatre or public transport are not done to make money but to provide a service for everyone.
I personally don't attribute this one to malice, just an inevitably different perspective: The schmo in the government thinks "Well, we need postal mapping...and here's a company that'll do it for just money (easy to spend) rather than us having to do it (hard to think and plan). So that'll cost us some dough...I understand that. And hey, we'll get to make some revenue back in license fees, which makes the cost of the thing not sting quite as much when I'm explaining how much this'll cost us. And think of the jobs we'll create! Everyone loves more jobs!"
At the end of the day, the folks probably thought they'd done a very good deal for the public.
Mapping is a bad example.
You might want to read http://en.wikipedia.org/wiki/Ordnance_Survey#Access_to_data_and_criticisms
The OS is in fact mostly self funding now, through the sale of maps and map data. There are still issues about some aspects of its work, and in fact it has pushed to be allowed to make more of its own data public.
The postcode issue is different, in my view, since the Royal Mail makes a pittance out of it, and would be really useful to have in the public domain.
We can send mail just fine, since we know our postcodes, but you can't get the full list, or the geographical data that accompanies them, which really sucks. Years ago I looked into building a web service that would have used your postcode to find things near you, but I couldn't get the geo-data without paying through the nose, which made it impossible. Of course now I'd tap into google's mapping API which can turn a post-code into a lat/long.
Although it should also be noted that *in theory* the Royal Mail should operate at a profit (or at least not at a loss), but due to continuing terrible management and unions currently has to be bailed out by the tax-payer.
If the Torys get in the RM will almost certainly get privatised to some degree.
Oh, and please shut your cake-holes about all this "subject" vs "citizen", chattels of the queen rubbish. It just makes you look silly.
Even as a strong anti-monarchist, I know that all that stuff is just technical hangovers from old systems. The monarch hasn't had any real power in the UK (influence is another issue), since a certain King Charles got made six inches shorter. Even the "Queen's Speech" at the opening of parliament is written by the Prime Minister. All the powers that fall under the "Royal Prerogative" are really just executive powers that can be used by the PM.
Not that we don't have issues, but this holier-than-thou "look at us with all our unassailable fundamental rights because we have a written constitution" attitude from extraordinary renditioning/NSA wiretapping/indefinitely detaining/secret en-prisoning/police brutalising/drug warring/child executing through cruel and unusual methods (I could go on) Americans is really, really starting to get annoying.
I should say that I've never met a yank I didn't like, but I do seem to encounter a lot online...
Right, time to go and have a calming cup of tea now I've got that off my chest.
The OSM may be self-sustaining now, but if you consider the lifetime capitalization of the programme, it is nowhere near at the point where the majority investor is the private sector.
And since early investments carry higher valuations, and since inflation should also be accounted for, the private sector's contribution to the OSM should be viewed as the pittance it truly is.
That's not very different from US governments selling public roads for companies to charge tolls. Of course there is a positive side to that: companies can afford to maintain the roads, while governments can't.
@Cory
I kind of agree with you, but maps are only worthwhile if someone keeps them up-to-date.
Personally I think that the OS could probable give away some of it's data, perhaps down to a certain level of detail, but sell anything more accurate, and still make enough to stay solvent.
Personally I'd rather have the OS self funding and continue to produce some of the best maps in the world, than have it suck state funding from rather more important things.
Perhaps what they need to do is release it under a CC-noncommercial license, that way they'd still make money from the likes of TomTom, but all us citizens can use it to our hearts content for other purposes. I don't think just giving it all away is the answer. I do think giving away the postcode data is the answer, since charging for it doesn't achieve anything.
Interesting thread but I take issue with a couple of points. Royal Prerogative is plenary power that is absolute in nature. I hate that “just a figurehead” argument that the Queens subjects make. It matters and it matters huge.
In the UK and Canada, you serve under an unelected hereditary Monarch. This Monarch also appoints (as in unelected) the PM, Governor General, Senators and all manner of lesser officials.
To be a true citizen, you must elect your Heads of State and Chief Executive, This means that their names must appear on a ballot for the elected position. No Ballot = No Democracy = Subject status and not Citizenship.
Simple test to see if you re a subject versus a citizen. Can you fire your Head of State in the voting booth? If no, you are a subject. If yes, you are indeed a citizen.
Great analogy.
I think things are slowly changing - Paula Le Dieu's appointment to BFI as Head of Digital after a stint at Creative Commons, for instance, a good sign of this.
I'm just in the contract-signing stage for some UK public R&D investment, and had to have the discussion with the funding body a few weeks back about our desire to open source the final product, as their standard contract focused on protecting closed IPR.
Their answer was that they had invested in us based on commercial potential; thus provided there was still a business that could be built around the open sourced technology (a la Red Hat, Mozilla, Kaltura, etc) than 'fair play' to us.
(It should also be remembered that the UK has a well established history of inventing things only to see more aggressive entrepreneurs in other countries profit enormously from them - tax-payer funded investment also has a responsibility to protect those tax-payer's jobs.)
I suppose the thinking here is that as long as it's effectively kept secret, it can be resold - thus reducing the costs for the tax payer.
A similar justification could be applied to the proposed iPlayer HD broadcast flag and the TV licence fee.
This is about far more than the copyright of the postcode data - it exposes the sham that is the 'database right' (special copyright for collections of facts - ie databases) in the EU.
Liberate the postcode!
www.ilovemypostcode.com
@EPLURIBUSUNUM
You are technically correct but the reality is rather different.
About the only *real* power the Queen has is in choosing who goes to Buck-Palace garden parties, and I bet the government has a say in that as well.
As a republican, I can't wait for Charles to be King, I suspect that unless he learns to shut his cake-hole, he will find that he is the last to be head of state. And I'm really hoping that he doesn't keep quiet, I hope he sticks his ore in everywhere.
I hope he tries to use some of that theoretical power that they technically have.
It would be the surest way to guarantee the UK becoming a republic of some sort in fairly short order. Monarchs have been forced to abdicate by the government in the past (see Edward VIII). Currently the government has more power over the monarchy than they have over themselves because while people in the UK are (sadly) on the whole ok with having a constitutional monarchy, it wouldn't take much to swing the population against them. The government knows this, and the royals know this.
"It's a kind of corporatism in which the risk -- the money spent speculatively mapping Britain, arguing in Parliament, drawing up postal code boundaries -- is entirely assumed by the public, but the reward -- access and profit-taking -- are entirely given to the private sector."
This is incorrect. A public good - like the Ordnance Survey or the Meteorological Service - is undertaken by the Government on behalf of the public it represents. The government then charges user fees to defray the cost to the wider public of the service it provides. The mapping of Britain was't undertaken for speculative reasons, but since the government has created and maintains a valuable asset it has a duty to manage it on the publics behalf.
EPluribusUnum is right about the British Constitution - just stuck 200 years in the past.
IIRC a significant part of the BBC iPlayer DRM furore is that the US networks who provide programmes like Heroes won't do so if we reshow them without the evil bit set.
Sigh, what a lot of silly political posturing.
None of this is about the constitutional nature of the UK. The postcode database (and several other things) are, plain and simple, vestigial political sillyness. It's like how the US government pays for most medical research via grants but then lets the pharma companies charge huge fees for them "to recoup the research expenses" (double-dipping, in other words). At some point in the distant future it will be fixed, but nobody really cares much about the postcode database right now, so it hasn't been done yet.
Funny story: we do reshow them without the evil bit, and the US networks keep providing them anyway. They were bluffing.
I've stopped putting postcodes on letters, instead I put a note on the stating that while the postoffice are going to charge me for access to postcode data (as well as for postage) then I will not include postcodes on letters.
Worth having a read of
http://www.freeourdata.org.uk
for some of the issues around various uk datasets. See blog page for updates
If the Queen wanted to exercise the authority of a real monarch, she would need to assume the responsibilities of a feudalism, as well. You guys should disband the military, and have Liz lead an army of her relatives + Elton John, Mick Jagger, etc. in defense of her subjects.
love how things get turned into .com speak. perhaps it's because the tc50 is going on now.
but rly, how does "she then approaches a second investor (license fee payers) for an extra £5,000,000" make sense?
the license isn't any tradeable commodity that reflects the value or future prospects of the post office. like, is a license to run Oracle an investment in Oracle?
it's more , it's a product they sell as part of their business?
@ HAL in theory thats what taxes are for.
While I don't know about the actual break down for the post office and OS fees, in the 1990s I worked on academic access to the Census. Basically the UK Govt through a university consortium spent UKP 5 Million to purchase the census data for the use of university researchers and students. I asked a census rep how much of the Billion or so the census cost was recovered and apparently it was a few 10s of millions and more than half of it came from other parts of the government. In 2001 the census decided to just give most of the data away (http://www.neighbourhood.statistics.gov.uk/dissemination/).
I know the UK university sector and local authorities also pay a lot of government money to OS to buy map data.
Oh yes, none of the above figures cover the costs of administering, securing access to the data and storing copies of it or the lawyers who draw up the agreements.
"the legitimate object of goverment is to do for a community of people, whatever they need to have done, but can not, or can not, so well so, for themselves - in thier seperate, and individual capacities"
I am not British but, who cares if the OSM is self sustaining? Yeah(!) the government can turn a profit.
The truth we all know is that private industry will provide the services if the business is self sustaining.
In the age of ones and zeros charging for government data is a crime.
It is a proper understanding of goverment's role that clarifies this issue for me. And Goverment is anatham to private industry (until it was co-oped but Free-Marketists in the 90's)
Robot Mistake
@5 Cory Doctrow -
A tiny hometown-pride correction: The name of Chicago's copyright-encumbered public sculpture is "Cloud Gate", colloquially called "The Bean".
Oh goodness, and I fulfilled Internet rule #47: when correcting someone else, you inevitably make a mistake yourself. Sorry for misspelling your name, Mr. Doctorow.
@19: "To be a true citizen, you must elect your Heads of State and Chief Executive, This means that their names must appear on a ballot for the elected position. No Ballot = No Democracy = Subject status and not Citizenship."
I think you mean "to be a true citizen of America...".
@Fuzzy: If there's no name for that rule yet, can we call it Fuzzy's Law?
The OSM may be self-sustaining now, but if you consider the lifetime capitalization of the programme, it is nowhere near at the point where the majority investor is the private sector
Well of course, because the original purpose of OS mapping, which justified the investment, was not a commercial venture. It was intended to assist the defence of Britain in the event of a Napoleonic invasion. Until the last few years, this has been the primary and overriding purpose of the maps - an insurance investment if you will. As such, the original investor can't expect a return on his investment! However, the ongoing costs of keeping the maps up-to-date are met from the sale of mapping, helping to offset the cost to the public purse. I can't see much of a problem here...
EPLURIBUSUNUM @19: it all revolves around the definition of "head of state" in real terms vs legal terms. In many European countries, the legal "head of state" is substantially powerless, and as such you cannot say that political institutions derive their powers from him/her.
For example, in the UK, there is no authority above Parliament; no authority whatsoever, not even the Royal Family or the UN, can force the British Parliament to do anything. This is the basic principle of the British system, established with the last Civil War.
Now, is a country more democratic (i.e.: closer to a Greek-inspired utopia of government "by the people for the people") when electing a dictator -- which is, in substance, what the US did with GWB at least once, as he had already substantially redefined presidential powers before his second term -- or when electing a Parliament? You'll find this is still being debated all over the world :)
If I were to own a part of everything I paid for as US citizen, then I own all the demographic data in the US collected by the government, then I could get all that data and abuse it.
Licensing agreements exist to limit the use of such data. The debate should not be whether governments should or shouldn't open up all data to everyone; the debate should be over which external uses are appropriate, and how can we make license agreements which reflect this?
#2 - why bother paying and downloading an 'app' to look at rail timetables when you can just point your browser at www.nationalrail.co.uk or any of the other rail company websites and search for free
#2 - why bother paying and downloading an 'app' to look at rail timetables when you can just point your browser at www.nationalrail.co.uk or any of the other rail company websites and search for free
The state of Oregon, my home, has tried to pull off the same thing by making all, or most, public records proprietary. You can look at some of them online but, you may not copy them or share them
Bill Harbaugh, an Oregon University Economics professor, has copied the Attorney General's Public Records and Meetings Manual on to his website. This is only published in print form and the AG charges $25 for it. Harbaugh is, in effect, challenging the Attorney General of Oregon to sue him.
Unfortunately, if I understand this correctly, the UK government could just label their records Official Secrets and throw anyone who publishes them into prison as a terrorist.
We're not quite there in the US.
It's going to be interesting to see how the two countries deal with this.
Note to Brits: Will the last to leave please turn of the lights, OR grab your scythes an pitchforks and recapture your government and sanity.
@EPLURIBUSUNUM,
So the residents of Guam, US Virgin Islands, Puerto Rico are subjects of the American kingdom?