Previously all the behaviour was specified by a configuration array with a dispatcher class. I found that was limiting the flexibility I needed and the “simple” configuration array was becoming decidedly complex. The Dispatcher class has been replaced by a new UriSpace class which is responsible for identifying the resources identified by a group of URIs. Applications can create classes derived from UriSpace to encapsulate the behaviour of their resources. Resources are split into three categories: documents that can be served straight up, abstract resources and descriptions of abstract resources. The last two are where the interesting bits of Paget lie. An application will typically override the get_description method to return a custom description derived from ResourceDescription. This class does all the hard work of finding triples about the requested abstract resource.

A class derived from ResourceDescription can override several methods to customise the RDF returned:

get_resources

This method returns an array of resource URIs that the description will consider when generating its RDF. The default behaviour is simply to chop the file extension off of the description’s URI. So, the description at http://iandavis.com/id/me.rdf will have a resource of http://iandavis.com/id/me.

get_generators

This returns a list of generators that seed the triples in the descrpition. The ResourceDescription class calls each generator’s add_triples method once for each resource returned by the get_resources method. Paget has some pre-defined generators that can read triples from a local file or from a platform store. The default behaviour is to do nothing.

get_augmentors

This returns a list of augmentors that add triples to the description. Paget comes with a few built-in augmentors to augment with RDF from a platform store, annotate properties with human readable labels and even do some limited inferencing. By default the simple property labeller is returned as an augmentor.

get_label

This just calculates a sensible label for the description that could be used in the title of a web page or a link. The default behaviour is to look for an rdfs:label, dc:title or foaf:name for the primary resource in the description (which is the first one returned by get_resources). Applications could override this to use whatever heuristics make sense for their data.

get

This is the dispatch point for HTTP GET requests. At a later date I hope to handle other methods too, but for now Paget is a read only system

get_html

This is called by the get method to generate an HTML representation of the description. By default it uses Paget’s SimpleHtmlRepresentation class but this is the point at which most customisations will take place for rendering linked data.

The HTML output from Paget has been revised too. The basic layout of the page is handled by the SimpleHtmlRepresentation class but some type-specific logic has been broken out into a number of “widgets”. There’s one for OWL ontologies, RDF classes and properties and a general one that can render any RDF data. The html representation chooses an appropriate widget based in the type of the primary resource being rendered. I’m thinking about adding widgets for people and various other common classes. This is all very early and experimental. Ideally I would like the page to adapt itself completely dynamically based on the underlying data. Switching on the class of a resource is rather simplistic, but it will do as a starter.

Here’s an example of how I’m using Paget in my personal data space http://iandavis.com/id/me. All the data is held in a Talis Platform store. I handle requests to http://iandavis.com/id/ with some .htaccess rules that ensure every request is handled by a file called index.php which contains the code hooking the space up to Paget. In index.php I create a subclass of UriSpace called StoreBackedUriSpace that maps the URIs beneath http://iandavis.com/id/ to resources and their descriptions. That class creates instances of StoreBackedResourceDescription that use a StoreDescribeGenerators to fetch the descriptions from the platform store. The entire code for index.php (less PHP includes etc) is shown here:

class StoreBackedUriSpace extends PAGET_UriSpace {
  function get_description($uri) {
    return new StoreBackedResourceDescription($uri);
  }
}

class StoreBackedResourceDescription extends PAGET_ResourceDescription {
  function get_generators() {
    return array( new PAGET_StoreDescribeGenerator("http://api.talis.com/stores/iand") );
  }
}

$space = new StoreBackedUriSpace();
$space->dispatch();

That’s basically the pattern for publishing data using Paget: derive a class from UriSpace and override the get_description method to return a custom ResourceDescription. I do that to publish some vocabularies on vocab.org such as Bio and Whisky. The UriSpace for those locations returns a resource description class that uses the FileGenerator class to read the schemas from local RDF documents and the simple property labeller and the simple inferencer to augment the results. My other deployment, at placetime.com, uses a custom resource description for each type of resource with custom generators that create the raw triples based on the requested URI.

So far it seems that Paget is flexible enough to deal with these varied scenarios of data publishing. The next step is to start looking at editing of the data and providing more application functionality.

I’ve been looking carefully at the core patterns that my PHP applications have been following to see if there’s anything else I could pull out. This is generally how I prefer to build new libraries: extracting them from several different projects. Assuming you know how your library is going to work before you’ve written any applications is almost always wrong. I like using libraries that have distilled the essence of repeated attempts at solving the same problem. That’s why I never think about modularization of a codebase until I need to.

I’ve been gravitating towards Konstrukt because it appears to be the least intrusive of the PHP web application frameworks out there and it keeps fairly true to REST principles. I used it to build Kniblet as part of a platform tutorial. However, there are some quirks that it has that I don’t like. For example, to return anything other than HTML requires you to throw an exception. That mechanism works quite well for most applications but doesn’t really suit data-rich applications that have multiple output formats.

It’s with this in mind that I’ve started a new PHP web application framework called Paget. Calling it a framework is somewhat of an overstatement. It’s a few classes that make it easy to publish RDF as linked data. It’s very primitive at the moment, but it’s quite versatile.

It uses a simple configuration array that is passed to a dispatcher that handles the request. The application’s default behaviour is specified using this configuration. One part sets up a series of regular expressions that match URI paths handled by the application and map them to the resources it provides. The data about each resource is obtained by using one or more “generators”. These are simply classes that generate RDF for the given resource. Paget runs each generator to gather the RDF data describing the resource and then handles the serving up of that data according to linked data principles. Right now that’s just enough behaviour to function as a generic linked data publishing framework.

I have three different deployments of Paget that are publishing three RDF data sets using different generators. Each of these was quite trivial to set up, being a few lines of confiiguration. For my own site’s data space I wrote a generator that fetched RDF directly from one of my platform stores (this one) and served it up as HTML and various flavours of RDF. See, for example, http://iandavis.com/id/me which is URI that identifies me.

My second deployment was for PlaceTime, a URI space that I have operated since 2003. It provides RDF data for timelike entities like instants and intervals and spacelike points. However, it hasn’t been fully linked data compliant (mainly because it pre-dated the decision on httpRange-14). I wrote a generator for each type of entity that creates trivial RDF for each valid URI in the space. Some examples:

• http://placetime.com/instant/gregorian/2003-05-19T08:00:00Z - an instant in time
• http://placetime.com/interval/gregorian/1970-06-15T19:31:00Z/P1D - an interval in time
• http://placetime.com/geopoint/wgs84/X-126.817Y46.183 - a point in space

Finally, I created a generator that reads a local RDF file. I then used it to serve up the whisky vocabulary that Tom, I and several others created at the recent VoCamp Oxford

Admittedly, all these datasets and spaces look pretty similar but this is still early days for Paget. I have some ideas for future development that will flesh out Paget into a fully-fledged RDF driven application framework. For example: as well as generators I plan to add filters, augmenters and transformers that alter the generated data in arbitrary fashions. These could be used to trim the data down, or to convert it to a more usable structure. I can imagine that it would be very useful to be able to pull in more RDF from arbitrary locations on the Web to supplement data in the initial set, e.g. with schema information or additional details. In my opinion that’s one of the significant differences between the web of data and the web of documents: the web of data is going to enable more information to be brought automatically together for the user rather than forcing them to seek it out.

Paget’s HTML rendering of RDF is very primitive at the moment, making only basic attempts to make it human readable. It’s still extremely tabular which is hardly a great use of structured information. One area that I’ve been interested in exploring is that of dynamic user interfaces that adapt to the underlying data automatically. RDF is particularly amenable to building these kinds of interfaces because of its uniform data model. A lot of work on this was done by the Fresnel project and it would be interesting to apply some of the learnings from that project to building dynamic web applications. My goal here is to code as little specific behaviour into the application as possible, instead making the application detect patterns in the data and provide suitable user interface behaviours at runtime. This is really the only way we’re going to be able to build true open world applications, i.e. those that are tolerant of missing data and can adapt to new and unanticipated data.

What I’m still experimenting with is whether these user interface additions should be server-side or passed on to the client. Some of the augmentations could make more sense when actioned by the client based on user activity.

There’s lots to research here and hopefully some of these ideas will make it into Paget very soon.

Update: see my follow up post that describes the major revisions I made to Paget after this was written

“Who knew it was going to get posted on the Web? It’s shocking,” said one House Democratic chief of staff, who requested anonymity to discuss her personal finances. “Now that anybody can look it up on the Web, I don’t know if I like it anymore.”

Her forms for 2006, which were filed last spring, included her home address and 32 pages of detailed statements about bank accounts under the name of her husband and daughter. That prompted her to raise concerns about identity theft at a chiefs of staff meeting in March.

These people are upset but their anger is misdirected. The problem isn’t that this is private information, after all these are government employees being held to account. Nor is the Web to blame for making this information trivial to access. The blame squarely lies with the ease with which having access to this information can be used to commit fraud. The reaction though is a startling illustration of how the banking industry has subtly shifted responsibility for financial security from themselves to their customers. They use the phrase “identity theft” like it’s our fault! They should be focusing on fraud prevention rather than throwing up smokescreens.

The Web opens and connects enormous quantities of data from all over the world and, as the Semantic Web gains momentum, we’ll see more connections and exposure of many more types and sources of data. Hiding this data from other people isn’t an option. Taking control of your own data and having the tools and services that let you find out where and why it’s being used is. But we should also expect every organization to accept responsibility for fraud prevention and guarding their customers against abuse. Pretending it’s someone else’s fault is just abdication of that responsibility.

Try thinking of it this way instead. You are going to serve some representation on the web, for this thing. Are those going to be (a) ABOUT the thing, or (b) the CONTENT of this thing denoted by the URI? If the former, you must use # or 303. If the later, you can serve the representations with 200 from that URI. You see, 200 means (basically) “Here comes the content of the document you asked for” and 303 means “Here is the URI of document ABOUT the thing you asked for.

That seems a pretty good characterisation of the 303 decision. Information resources can serve up representations of themselves, other resources cannot so you have to make do with descriptions. (Regular readers will already know that I don’t fully agree with this model, but it’s the accepted one in the SemWeb community)

There appears to be an asymmetry about this though and I think it’s a limitation of the model.

Suppose I have a resource “R” with URI http://example.org/R. If it is an “Information Resource” then I can arrange things so that a GET request for its text/html representation responds with a 200 and the HTML in the body of the response. I could also arrange for a request for its application/rdf+xml representation to respond with a 303 status and the URI of another information resource “RDESC” (e.g. http://example.org/RDESC). In this example the 303 response meand that “R” cannot be represented as RDF, but there’s an alternative RDF document that is a description of R. The user can then re-issue the request on http://example.org/RDESC to obtain that description.

Now, I can arrange for http://example.org/RDESC to return an RDF representation in a 200 response. But, here’s the asymmetry. How can I allow the user to obtain a description of RDESC? The representation I send back is the content of RDESC, not its description. I can’t use the media type to distinguish the type of request any more.

In case you think this is an artificial distinction, it’s not. We’re dealing with it right now at Talis (and have been for a number of months). We give access to a number of RDF graphs via HTTP where naturally the user wants to obtain the content of the graph as the response to a GET. We also serve RDF descriptions of those graphs containing some configuration information. There’s no standard way to link the two things together so that users can select either the description or the content using HTTP.

Now, I happen to think that there is an interesting solution to this asymmetry. Suppose we created a new HTTP header called “resource-description” whose value was the URI of a description of the given resource. Note that it’s a description of the resource, not of any representation that is being sent as part of the response. The asynmmetry goes away because this gives you a method of pointing to the description regardless of the status code and/or content negotiation going on in the request.

Things get even more interesting if you allow multiple resource-description headers: what a great way to cross link to other people’s descriptions of your resource.

I seem to recall something similar to this being proposed a few years back but my Googling doesn’t turn anything up. Going back further takes us to Patrick Stickler’s attempts to solve the description problem using URIQA, which took the approach of introducing another verb, MGET, to obtain the description. This was almost universally disliked, but the underlying problem has remained unsolved in the meantime.

And given where my head has been for the past few weeks, I have to ask what decision would have been taken on the httpRange-14 issue if this header had already existed. Would instead we be returning 406 responses when we cannot supply a suitable representation for resources, or even a 204. Both of those could work with the header pointing to an appropriate description of the resource.

Update: it seems that timbl and I were touched by the same muse tonight: Alternative to 303 response: Description-ID: header

RDF uses an RDF URI Reference, which may include a fragment identifier, as a context free identifier for a resource. RFC 2396 [URI] states that the meaning of a fragment identifier depends on the MIME content-type of a document, i.e. is context dependent.

These apparently conflicting views are reconciled by considering that a URI reference in an RDF graph is treated with respect to the MIME type application/rdf+xml [RDF-MIME-TYPE]. Given an RDF URI reference consisting of an absolute URI and a fragment identifier, the fragment identifer identifies the same thing that it does in an application/rdf+xml representation of the resource identified by the absolute URI component. Thus:

• we assume that the URI part (i.e. excluding fragment identifier) identifies a resource, which is presumed to have an RDF representation. So when eg:someurl#frag is used in an RDF document, eg:someurl is taken to designate some RDF document (even when no such document can be retrieved).
• eg:someurl#frag means the thing that is indicated, according to the rules of the application/rdf+xml MIME content-type as a “fragment” or “view” of the RDF document at eg:someurl. If the document does not exist, or cannot be retrieved, or is available only in formats other than application/rdf+xml, then exactly what that view may be is somewhat undetermined, but that does not prevent use of RDF to say things about it.
• the RDF treatment of a fragment identifier allows it to indicate a thing that is entirely external to the document, or even to the “shared information space” known as the Web. That is, it can be a more general idea, like some particular car or a mythical Unicorn.
• in this way, an application/rdf+xml document acts as an intermediary between some Web retrievable documents (itself, at least, also any other Web retrievable URIs that it may use, possibly including schema URIs and references to other RDF documents), and some set of possibly abstract or non-Web entities that the RDF may describe.

This provides a handling of URI references and their denotation that is consistent with the RDF model theory and usage, and also with conventional Web behavior. Note that nothing here requires that an RDF application be able to retrieve any representation of resources identified by the URIs in an RDF graph.

I’ve been thinking about this for a couple of days and I’m still not entirely sure what to make of it. What it appears to be saying is that RDF ignores the Web Architecture principle that fragment identifiers are given meaning by the representation that is retrieved.

So this ensures that RDF is self-consistent. I can refer to anything I like using a fragment identifier in my URI and I’m guaranteed not to have my intended meaning upset by anything messy like a network operation. This alleviates one of my major concerns at using these kinds of URIs in RDF, but at what cost? If anything this increases my concerns over the shadow web since by circumventing the web architecture it sets RDF further away from today’s web of documents. For example, when I use “http://www.w3.org/TR/webarch/#media-type-fragid” as a URI in my RDF, it probably doesn’t refer to the thing you think it does. You, as a human (if you are), get to see a representation of that section of the document when you click on the link, but an RDF-aware agent must treat that URI as though rdf/xml had been retrieved. Unfortunately there isn’t any RDF there and the Web Architecture actually forbids you from serving up both HTML and RDF documents at the same URI.

What does that mean? How are we supposed to interpret that? One interpretation is that it really doesn’t matter what you do outside of RDF. You can throw up all kinds of other representation formats and it won’t affect yours or anyone else’s RDF. They might use the same identifiers, and occasionally, coincidentally they may identify the same things, but in general RDF is partitioned into its own little world. RDF can only link to RDF.

How can RDF co-exist with other formats on the Web if it ignores their semantics? If you just want the Semantic Web to be built using RDF then you probably don’t care. But if, like me, you want to see an inclusive Semantic Web built from a mix of RDF, microformats, topic maps, RDDL and all the other ways to express semantics, then it’s a very very big problem. I don’t want two webs competing for attention, I want one strong one.

Hence the title of this post. It is OK to use URIs with fragments in RDF, but only if you don’t particularly care about relating to the existing web. If you do care then avoid fragments at all costs. Use standard URIs and stick 303 redirects on them if you need to. It’ll work and the whole web will be better for it.

Nad’s written a good summary over on his blog and managed to capture the questions I was asked at the end too, which is nice to see after the event. The whole day was brilliant with lots of chances to natter and catch up with everyone. I met lots of new people too and everyone seemed to be doing something interesting. I even managed to harangue Stuart Williams off of the W3C TAG on my recent Web architecture posts. Andy said about 50 people were attending despite there being no marketing of the event at all which is a good indicator of the rising popularity of the Semantic Web. Talis are planning to host a SWIG meeting like this in the middle of next year - hopefully we can get more people from the Midlands interested.

There were plenty of other cool presentations on the day too. Nad and Rob were blogging but couldn’t post them live for some reason so check out their sites and Nodalities too in the coming days. I particularly enjoyed Leigh’s talk on Facet, another templating framework for RDF, this time in bog-standard Java allowing the use of JSP and/or Velocity; Richard’s talk on Sindice (which I didn’t get to see at ISWC and for us ignorant Brits is pronounced “sin-dee-chee”); and Graham’s talk on image publication. All great stuff!

You want to link from your webpage to Tim Berners-Lee’s URI <http://www.w3.org/People/Berners-Lee/card#i>, except you can’t because that link points to something that can never contain the #i fragment in its HTML. It can only ever link to RDF because Tim is relying on RDF’s semantics for the meaning of the #i fragment. Tough luck if you can’t read RDF or don’t want to have to learn.

1. A hashless URI should be allowed to denote any resource whatsoever. Documents, books, people, galaxies and unicorns. There is no ambiguity here, the URI denotes a single thing. More than one URI can denote the same thing, so I can have a URI that denotes the city of London, and Danny can have a different URI that also denotes London.
2. A representation of a resource can be obtained by issuing an HTTP GET on a URI. The representation is a sequence of bits that somehow stands in for the resource the URI denotes. Content negotiation can be used to select an appropriate format for the representation, withouth changing the actual resource being denoted. Perhaps my URI denoting London can respond with an HTML document containing essential facts and figures about the city, a JPEG aerbyial photograph, an SVG streetmap or a sound recording of the sounds encountered while in the city itself. None of these things are London, but they all can stand in for it in some limited fashion. I could retrieve them all to obtain a better sense of London itself, but I cannot actually obtain London using HTTP.
3. URIs containing hashes are constrained in what they may denote and have an inherent ambiguity due to their reliance on the particular representation obtained. Their denotations vary depending on the URI plus a set of HTTP headers used during the request.
4. There is no such thing as an “Information Resource”. All resources are made equal. However for many resources, the only representation available happens to be identical to the resource itself. Still, you cannot obtain the actual resource using HTTP, but you can get a copy in the form of a representation. The majority of HTML documents on the web behave in this manner, a single representation that is a copy of the resource itself.

These aren’t huge changes and they’re backwards compatible with the existing web. On the other hand they greatly reduce the reliance on fragment identifiers and they encourage people to use real unambiguous URIs to refer to things other than documents, weaving the Semantic Web right into today’s Web.

For background, you might like to read my earlier posts on this subject:

• Fragmentation
• Is the Semantic Web Destined to be a Shadow?
• What are Information Resources Good For?