03.14.13
My Dream Job
I imagine some epidemiologist somewhere, who has statistics on the something like the measles rate by postal code, who wants to see if there is a geographic trend, like if warmer places have more measles. She has a spreadsheet with the postal codes and the number of cases in that postal code, and wants to turn that into a map where each postal code’s colour represents the number of cases per capita in that postal code.
She should not need to know what a shapefile is, should not need to know that the name of the map type she wants is “choropleth”, and should not have to dig up the population of that postal code. The boundaries of the jurisdictions she cares about (postal codes, in this case) and the population are well-understood and don’t change often; the technology to make such a map out to be invisible to her. She should be able to upload a spreadsheet and get her map.
I find it almost morally wrong that it is so hard to make a map.
Making that possible would be my dream job. It is a small enough job that I could do it all by myself, but it is a large enough job that it would effectively prevent me from doing other paying work for probably about a year, and I can’t see a way to effectively monetize it.
The challenges are not in creating a map that is displayed onscreen — that’s the easy part. To develop this service would require (in order of difficulty):
- code and resources to enable users to store their data and map configurations securely;
- code to pick out jurisdiction names and data columns from spreadsheets, and/or a good UI to walk the user through picking the columns;
- fuzzy matching code which understands that e.g. “PEI” is really “Prince Edward Island”, a province in Canada; that “St John, LA” is actually “Saint John the Baptist Parish”, a county-equivalent in Louisiana; that there are two St. Louis counties in Misouri; that Nunavut didn’t exist before 1999;
- code to allow users to share their data if they so choose;
- UI (and underlying code) to make the shared data discoverable, usable, and combinable;
- code (and perhaps UI) to keep spammers from abusing the system;
- code to generate hardcopy of a user’s map (e.g. PNG or PDF);
- code for a user account mechanism and UI for signing in
This service would give value to many people: sales managers trying to figure out how to allocate sales districts, teachers developing lesson plans about migration of ethnic minorities, public health officials trying to understand risk factors, politicians targeting niche voters, urban planning activists trying to understand land use factors, etc.
Unfortunately, for the people to whom this really matters, if they already have money, they already ponied up the money for an ESRI mapping solution. If they don’t have money, then they won’t pay for this service.
GeoCommons tries to do this. GeoCommons makes maps from users’ data, and you stores and shares users’ data, but their map making is so slow it is basically unusable, and it is not easy to combine data from multiple sources into one map.
It might be that one of the “big data” organizations, e.g. Google or Amazon, might provide this as an enticement for getting people to use their other services. Google, for example, has a limited ability to do this kind of thing with their Fusion Tables (although if you want to do jurisdictions other than countries, then you have to provide a shapefile). Amazon provides a lot of data for use with the Amazon Web Services.
However, it would be almost as difficult for Google or Amazon to monetize this service as it would for me. Google could advertise and Amazon could restrict it to users of its AWS service, but it isn’t clear to me how much money that could bring in.
If anybody does figure out a way to monetize it, or wants to take a gamble on it being possible, please hire me!
Andre West said,
May 2, 2013 at 9:54 am
Three wrongs don’t make a right Speaking from a recent experience trying to use the geocoder.ca data, I can tell you as a matter of fact it is quite inaccurate. The postal code point do not map to the centre of the geographic coverage areas (at least not in the major urban centre I was looking at), many don’t map to the right area period and the polygons are simply all wrong. I suppose I’m one of the few who actually checked the data before proceeding to use it, going by geocoder’ca’s extensive list of ‘clients’. That list includes numerous government agencies and NGOs. I think there’s potential for more harm than good when agencies people rely on every day rely on inaccurate, if not completely incorrect, data. I don’t doubt Canada Post has seen inefficiencies and incurred extra costs as a result of the inaccurate data geocoder.ca provides. However, it helped create that very problem by not making the official postal code data freely available itself, leaving a gap for geocoder.ca and others to fill with data of rather dubious origin / quality. While the federal government tries to play good cop by saying it would prefer to see the postal code data open to the public, it had a hand in creating this mess. It cut funding to the crown corporation, deeming it a for-profit corporation that was expected to be self-sufficient. The federal government would be rather disingenuous to turn around now and criticise Canada Post for seeking to profit from its assets.
Lorenzo W. Arnold said,
June 17, 2013 at 3:58 am
For the new generations including my generation it is hard to know our postal codes , we are not like the older generations which had more access to the Postal services. I understood the importance of knowing the postal codes when I began to order stuff online. This website is really helpful for those who want to know what their postal code is in Egypt : Egypt Postal Codes | الرقم البريدي المصري . It does not include all the areas in Egypt but at least it is a great try.