Tag Archives: London

Yesterday I spent the day at Leicester for a Mashed Museum day organised by Mike Ellis of Eduserv in co-operation with Ross Parry from Leicester University. The day had around 20 people from an array of institutions (for example the National Maritime Museum, Arch & Anth Museum – (my favourite objects were their shrunken heads when I was growing up in Cambridge!), Eduserv, Museum of London and Flickr to mention just a few.) The day was spent taking a variety of Museum data sources to see what you could do with the output within something that you built. The variety of things that people put together was quite interesting and I will mention these first.

Fiona Romeo from the NMM used a data visualisation tool which took free text input from a set of database records and then this analysed the content for correlations in text usage and produced a chained output that allowed you to navigate between records.

Jim from the NMM used Yahoo!’s data extraction API to enrich the data of around 5000 records from the NMM’s collections to see if he could tag multiple records with information and speed up the information production experience.

Dan Zambonini from BoxUK and Mike Ellis built upon the hoard.it application that screen scraped information from several large institutions and produced a random object feed and also a top trumps application.

Frankie Roberto from the NMSI used the freebase application to upload data from their events database and then produce a timeline within the SIMILIE application (MIT) – I’m using this in the rebuild of the Scheme’s database- which was quite interesting.

Carl from the Arch and Anth museum at Cambridge produced a map of the voyage data held within their collection on Poly9‘s version of Google Earth. This seemed to have better imagery than GE, so I think I will have to play around with that soon. He also talked to me about his new collections management system that he’s rebuilding for the Museum. It looks quite swish!

Mia Ridge from the Museum of London, no doubt did something quite interesting, but I went out to use my mobile when she was talking, but I think she was playing with Yahoo! pipes. The Pipes system seemed to go down quite badly with the people who attended.

Steve from Eduserv used the OpenCalais service from Reuters to enrich data pulled from the Hoard.it application. I’m planning to use this and the Yahoo! term extraction service to enrich the Scheme’s records when I finish the database rebuild, which is going to be using Zend Framework as I found Cakephp a bit restrictive.

I tried out several things during the day, some didn’t come off and some were reasonably useful for my actual working environment. First off, I tried to create a Heritage explorer application based around Fire eagle from Yahoo! This failed for two reasons, one I don’t have the correct version of PHP on either my laptop or work server, and secondly I couldn’t get an accurate geocoded set of data from the csv file provided by the 24 Hour Museum. Their dataset was created with postcodes and no lat or long, or indeed grid refs. The Post Office charge for use of their postcode lookup service and I’m not keen on paying for publicly funded data sets. What I wanted to do was to create an application where one can be told the following:

a) Museums within 10km of current location
b) Objects recorded with the Scheme within 10km of current location (no exact details revealed)
c) Any World Heritage sites within 10km derived from the National Monument Record set.

I then wanted to plot these onto a Google map. However that didn’t work and might take me a while to implement anyway.

Next I created a PHP twitter update script using a section of code that I found on the internet and then modified to run from a simplepie iteration. I tried to use the Twitterfeed application, but this couldn’t ping their server regularly….
This used CURL (which doesn’t work on my server!) and had to be run off my rugby club server; however as of today, this now works on our server. This updated the new twitter identity bm_pants (I work at the BM and portable antiquities often gets shortened to portants or pants) and can be run off a server cronjob. This twitterbot runs off a combined RSS feed of the following:

a) New pictures on flickr
b) All finds recorded on the database
c) Notable finds
d) Blog entries

I tried to create this with pipes, but I kept finding that the feed would break and go out of sequence. Therefore I went back to the tried and tested simplepie class which allows you to roll a multiple feed. To do this, I created this script:

[PHP]
< ?php
//Set the header for the script
header(“Content-Type: text/xml;charset=utf-8″);
//Include your simplepie instance
include_once(‘%path to simple pie%/simplepie.inc’);
//Set the root
echo ”;
?>
http://www.finds.org.uk Updates for Twitter bot
en-en
< ?php
$feed = new SimplePie();
// Create a new instance of SimplePie
$feed->set_feed_url(array(
‘http://www.findsdatabase.org.uk/rss.php/project/pas/feed/406′,
‘http://www.findsdatabase.org.uk/rss.php/project/pas/feed/423′,
‘http://api.flickr.com/services/feeds/photos_public.gne?id=10257668@N04〈=en-us&format=rss_200′,
‘http://www.finds.org.uk/wordpress/index.php/feed/’,
‘http://www.finds.org.uk/rss/twfy.php’
));
$feed->set_cache_duration (120); //The cache duration, I’ve set my cronjob to run 10 seconds after the cache expiry time
$feed->enable_xml_dump(isset($_GET['xmldump']) ? true : false);
$success = $feed->init(); // Initialize SimplePie
$feed->handle_content_type(); // Take care of the character encoding
?>
//Next section sets the feed to deliver just 1 item
< ?php if ($success): ?>
< ?php $itemlimit=1; ?>
< ?php foreach($feed->get_items() as $item): ?>
< ?php if ($itemlimit==2) { break; } ?>
//Print out the items
< ?php echo $item->get_permalink(); ?> < ?php echo $item->get_description(); ?>

< ?php echo $item->get_date(‘j M Y, g:i a’); ?>
< ?php $itemlimit++ ?>
< ?php endforeach; ?>
< ?php endif; ?>

[/php]

This results in www.finds.org.uk/multifeed/twitterfeed.php and updates after initialising a cronjob. It will most frequently update with latest find, which might be a bit much, so I might roll it back to just notable finds. The code for this script is:

[PHP]
< ?php
/* This script builds on one I discovered: RSS to Twitter v0.1 by paul stamatiou
of http://paulstamatiou.com. This uses Simplepie instead of lastrss.
*/
require($_SERVER['DOCUMENT_ROOT'] . ‘/php/simplepie.inc’); //Include the simplepie script
$twitter_username = ”; //For example “bananaman” for twitter.com/bananaman, or your email address that you signed up with.
$twitter_password = ”; //Your Twitter password.
$twitter_url = ‘http://twitter.com/statuses/update.xml’; //The update page from Twitter’s API docs.
$feed = new SimplePie(); // Create a new instance of SimplePie
$feed->set_feed_url(”); //Set your feed location
$feed->set_cache_location($_SERVER['DOCUMENT_ROOT'] . ‘/cache’); //set the location of your cache
$success = $feed->init(); // Initialize SimplePie
$feed->handle_content_type();
?>
< ?php if ($success): ?>
< ?php $itemlimit=1; ?>
< ?php foreach($feed->get_items() as $item): ?>
< ?php if ($itemlimit==2) { break; } ?>
< ?php
$title = $item->get_title();
$URL =$item->get_permalink();
?>
< ? $itemlimit++ ?>
< ?php endforeach; ?>
< ?php endif; ?>
< ?php
//$tiny_url = file_get_contents(“http://tinyurl.com/api-create.php?url=” . $URL);
//$status = $title . ” ” . $tiny_url;
$status = $title . ” ” . $URL;
echo $status;//just for status if you are directly viewing the script
$curl_handle = curl_init();
curl_setopt($curl_handle,CURLOPT_URL,”$twitter_url”);
curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,2);
curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
curl_setopt($curl_handle,CURLOPT_POST,1);
curl_setopt($curl_handle,CURLOPT_POSTFIELDS,”status=$status”);
curl_setopt($curl_handle,CURLOPT_USERPWD,”$twitter_username:$twitter_password”);
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);
if (empty($buffer)){echo ‘
message’;}else{echo ‘
Your update to Twitter was successful’;}
?>
[/php]
So if you want to chat to this twitterbot, feel free. I’ll add the code for the twitterbot tomorrow once clean.

After creating this, I then went on to creating a KML layer from the database of all finds that contain a grid reference and also an image. The ouput for this can be seen below:

View Larger Map

This is a degraded set of data, and the images won’t pull through to live feed just yet as I’m playing with these at the moment. Once the database has been rebuilt, this will become a more standard offering to users, with the grid refs degraded to protect privacy of finders and land owners.

This needed a JOIN query from several tables and caused a small server outage at one point for my work colleagues back in Londinium (columns weren’t indexed!). The basic query is below, but the actual SQL is modified to return degraded spatial data. Not revealing that on here!

[code]SELECT finds.old_findID, finds.secuid, finds.broadperiod, slides.imageID, findspots.declong, findspots.declat
FROM finds
LEFT JOIN findspots ON finds.secuid = findspots.findID
LEFT JOIN findximage ON finds.secuid = findximage.findID
LEFT JOIN slides ON slides.secuid = findximage.imageID
WHERE findspots.declong IS NOT NULL
AND findspots.declat IS NOT NULL
ORDER BY finds.created_on ASC[/code]

The output could be transformed into either geoRSS or KML and allows one to plot these finds quite accurately onto any mapping programme that accepts these feeds.

One of the things that I’ve been trying to do for a couple of years now, is to get local museums to display objects discovered and reported to us by their local consituents displayed on their websites. This can be done easily now by consuming RSS or geoRSS and displaying Google maps. So for example, Exeter museum could have discoveries recorded by their FLO, or the Museum of London could have a map of objects discovered by the Mudlarks. Easy to do now, anyone game?

I therefore achieved a few things I wanted to do, and probably should have shown them off at the end, but I didn’t feel that they warranted displaying. It was nice to meet some other people who work on Museum techie stuff, I find myself a bit isolated within the Scheme’s structure as I build all our webstuff myself (and it isn’t really the major part of my job!) So if any of you read this, nice to meet you and if I missed your show and tell application off, it is because I’m forgetful rather than I didn’t like it. And if you come to another one of these, always worth bringing a bottle opener for post mash beers. It might also be good to get a bit more structure and push people to actually achieve something. I think I’d like to learn more about Rails, so perhaps would have been good to do something with someone like that.

On the way back on the speedy train, I came across some very strange unsecured home wireless networks, “swampworker-athome” you really should secure your network! If you want help, send me an email. I still don’t really like Leicester, even if it is 10 years since I last went there and lost to the Tigers 76 -3; however Welford Road is a brilliant stadium to play at!!!!!