Using interface encapsulation to listen to linked data predicates

Introduction

The article was a submission to ISWC 2014 Developer Workshop

Linked data and its RDF model is known for its flexible schema less structure and integrated meta-data. Although this is great for expressiveness of the data, its is a nightmare for interface developers since all the modern interface paradigms are tree based.

The standard way of interacting with data in these user interfaces is to use application logic to interact with interface elements which needs to query the data and decide which piece of information to put in which element. Trying to push free format RDF in a tree is not a easy tasks and often results in very static interface for a specific graph structure.
Wouldn’t it be more intuitive to use the meta data in the RDF to decide which user interface element treats which property? And even more intuitive let the UI be able to decide to get more data from other linked resources without needing specific application logic ?

Traditional Model

Lets try to go through the traditional model with static data:

Data:
 Bart, van leeuwen, @semanticfire, http://netage.nl/dir.ttl
Code:
 Select firstname, lastname,handle,companyprofile from data
 interface.firstname.setValue(resultRow.interface)
Interface:
 Field firstname
 Field lastName
 Field handle
 Field companyProfile

If any changes are made on the interface, it is up to the application logic to extract it from the interface and modify the underlying data model.
So if we get a new data structure as input we need to change both logic and interface to react to that. When we want to react to different types of input structures both the interface and logic are getting more and more complex. This is mainly due to the fact that the data is pretty ‘dumb’ we need the logic to identify various pieces of data and match them to user interface elements, the other way around is even more complex. Getting the modified data an sending it back to the underlying data model. There we need a extra step to make sure both UI and application logic are in sync with the constraints of the data store.
The traditional solution while dealing with RDF data is to use SPARQL queries to get somewhat structured data which resembles the result of the SQL query above. Although we can use traditional methods to show the data, we loose a significant part of the semantics around it, the data is ‘dumb’ again. Updating data is even more complex since SPARQL results might come from multiple endpoints and that information is not present in the SPARQL result set.

A different approach

Now how can we use the context in RDF to eliminate application logic and put all the logic in the interface. If we are able to do this it would allow us to simply be ready when the piece of information we are able to show comes by and complies to the constraints we have set on it.
During the development of some of the front ends of our system we started out with the traditional SPARQL way. One of the important reasons to use linked data in our solution is the agile nature, adding extra properties or replacing them is easy and doesn’t require large schema updates. However on the front end we were still struggling with multiple SPARQL queries for the various incarnations of our data. This should be done in a smarter way.

Introducing EnyoJS

In our search for toolkits to create mobile applications we came across EnyoJS [1] a Javascript framework which originated on the Palm Pre smart phones as a part of WebOS and evolved in a Framework for platform independent applications. The current Enyo 2.4 version is the basis for LG’s latest SmartTV Platform. The concept of EnyoJS is encapsulation, components should have no knowledge about their place in the hierarchy, they communicate with bubbling up and water-falling down events. They have no control who receives these notifications and events. There is also no limitation on which components encapsulate each other. The flow of events and notifications is left undisturbed by encapsulation, adding a extra component around another doesn’t break the chain, unless the inserted component deliberately listens and ends the event chain. Components can directly talk to elements they define but should not talk to the inner components of those elements. Neither should they directly talk to their parent components.

The EnyoJS way

The encapsulation gave us the inspiration to see if we could use this method to add a linked data property to components. By encapsulating a standard UI component with a linked data aware component we would be able to define at interface level which element should contain values of predicates.
So if we would have a UI Container C which contains a Textbox T traditionally it would look like

C1 → T

The container would use the traditional SPARQL way to get data and update the value v of T . Our approach would be that in between C and T we place a property component P which is configured to react to predicate “example:foo” and propagate the value of “example:foo” to its components.

C2 → P → T

and instead of the traditional SPARQL way we simply let C announce to its children it has a graph available, P will pick up any triple which has the “?s example:foo ?o” pattern and extracts the value ?o and propagates this to its underlying components. P will stop the waterfall. This method would still allow to insert a extra component in between, e.g. a Box B

C2_2 → P → B → T

Since B is does not have a “setvalue” called by P we will simply call T directly, remember we can call methods of defined components directly.

If the value of ?o is a URI, P will try to load that resource and announce to its components that it has a Graph. So if I would like to know “foaf:name” from someone I “foaf:knows” it would look a bit like this.

C3 → P¹ → P² → T

Where P¹ listens to “foaf:knows” which will load the graph and announces it to its components where P² which then will set the value of T to “foaf:name”
Presenting multiple properties is no problem either if I am also interested in the “foaf:mbox” value of my “foaf:knows” relations ships this would work as well.

C4 → P¹ → [ P² → T¹, P³ → T² ]

Here P³ represents “foaf:mbox”
The above example snippets can be found in the github repo [2] from the samples you can clearly see that the ‘meta-data’ stays close to the interface level and we only fetch and announce the source graph once. The snippets just give a rough overview of what is possible with this approach.

Additional possibilities

The EnyoJS platform knows the concept of repeaters, which allows components to contain a virtually unlimited amount of instance of another component. This would solve a cardinality issue.

C → P¹ → R → P² → T

Here Repeater R is used to create multiple T components from all the “foaf:knows” relations I have.

Its also possible to combine multiple components inside a repeaters, e.g. P³ listens to foaf:mbox

C → P¹ → R → [ P² → T¹ , P³ → T² ]

this would result in a list of names and emails addresses of all the “foaf:knows” relationships in my profile.
The way back.
By maintaining the ?s on the P components propagating updates back is relatively easy

C ← P ← T

if we update T, and configure P to listen to the updates ?u to value ?o while maintaining ?s related to P we can simply construct a triple out of that

 ?s → P:predicate → ?u

which replaces

?s → P:predicate → ?o

We now would be able to either construct a SPARQL Update, provided we know which endpoint to use, or use LDP spec to to do a patch on ?s
Of course you might want to constraint the actual execution of water-falling based on property values, e.g. only show properties which are no older then one week. And only show a certain component if there actually is data, or have property component just be responsible for containing other properties without generating any UI. It even allows to listen to the same property with different constraints.
e.g.

C → P(ex:foo >= 50 ) → Tgreen
C → P(ex:foo <50 ) → TRed

If T would be a editable field passing the threshold of 50 would in the end propagate the update up to C which would then announce a updated graph, which in turns would show the T with the right color.
For formating back and forth EnyoJS provides all sorts of transition hooks which would allow to format a xsd:DateTime to something human readable, and back to xsd:DateTime after modification

Conclusion

By using the encapsulation mechanism in EnyoJS we are able to to combine both UI description and the related RDF properties in a single place without being bothered by external code to sync the model and the interface. Although surely in its infancy the approach described here is not only a nice idea, it is practically being used in our applications.

The code is not in the public domain yet, but this is certainly something we will be doing in the near future.

Samples

The sample snippets are made available on Github [2]. Handling RDF data is done with rdflib.js [3]
This is none working code right now, I’ll assemble working samples before the conference
Current state and usage
By the time of writing the technique described in this paper is used deployed actively on several monitors at fire stations in the Netherlands

Future work

One of the more obvious additions would be language support., set a preferred language on literals so you can react to multi language RDF.
Multiple property update predicate relation, as in the example above with Tgreen and Tred, maybe a P which updates and listens to the change of ?o but also modifies another property on T, the color in this case.
Using Json-LD to define properties, this would allow the omission of full URI’s and use a context to solve them
User property definitions and constraints based on the work off the future Shape expressions Working group [4] or the Resource Shapes 2.0 submission [5]

References

[1] http://enyojs.com
[2] https://github.com/semanticfire/iscw2014_dev_workshop
[3] https://github.com/linkeddata/rdflib.js/
[4] http://www.w3.org/2001/sw/wiki/ShEx
[5] http://www.w3.org/Submission/2014/SUBM-shapes-20140211/

The confusing shared visual representation

You just finished a novel from your favorite author, while reading you have build a visual image of the scenery described by the author. A bit later the filming by a well known Hollywood director makes it to a cinema nearby. A big fan as you are, you’ll be surely going out to see this movie! But after the opening scene you will find yourself confused and disappointed, the depiction created by the director doesn’t match yours at all.

You wake up in the middle of the night during a holiday in a hotel, you have no idea where you are and it takes a few seconds to realize the environment you are in is the hotel you checked in the day before.

This phenomenon is perfectly described in a recent article (http://www.sciencedaily.com/releases/2011/09/110928131800.htm) , and there is even a good explanation in it !

In the Fire Captain training in the Netherlands you are taught not to compose an image based on the verbal information you receive from the dispatch unit over the radio. Even the on site explanation by an informant should be listened to carefully. The reason for doing this is to prevent the same confusion as described in the first two paragraphs. Composing an image based on information you haven’t yet confirmed yourself creates a great risk. Being a Fire Fighter you want to march into action, the decisions you make are not the best, but the most acceptable for that moment in time. If you construct a plan based on an image you didn’t confirm you might find yourself confused.

Recent developments in the information management field for first responders are more and more facilitating the composition of shared visual representation. The idea behind this concept is that we create a common user interface with a visual representation where all the information that is available is displayed. We have some very large concerns with that approach:

  1. The assumption is made that every single player understands the way the information is presented. GIS based applications are generally used for these visualizations, that is fine for people who work with building plans in their daily life, but volunteer fire fighters may only be confronted with this type of visualization while working for the fire department. It adds to the confusion
  2. Not all the first responders have the time to interpret and analyze all the data shown on the interface, select what they need for their job, and realize that the actual situation might be different from what they concluded from the shared visual representation ( see paragraph 1)
  3. The detail that can be shown on this interfaces is really high, but that creates a expectation. Experience in Amsterdam has shown that the satellite navigation system sometimes shows the wrong location for a certain address, it happened more then once that the unit rushing to the scene focused completely on the navigation system and passed the actual address.
  4. With the detail level of the interfaces comes another issue, the data you display on these interface has to be of equally high quality. The recently released dutch building register for example contains a building that is roughly 320km² big !

So with all these great techniques and available datasets we actually create more confusion! The techniques assume that everybody understands the visual representation, while the datasets are expected to be correct. In my previous blog post I already expressed my concerns about opendata.

Pushing all the data and information into one representation is done to generate a context for it. Most bits and pieces of data and information have no context themselves, deep knowledge of its structure is needed to be able to understand it. So what could we do if the bits and pieces of information supplied actually contain the meta-data to describe them? Then we could use intelligent software agents to collect those bits an pieces of information which are needed for our current situation and present them in a way that makes sense to us, and doesn’t assist in creating confusion!

The technique to include the meta-data is called linked data, linked data contains the vocabulary to describe it, and is machine readable. Another important feature is the linking to external datasets, which allows us to infer relationships and automatically enrich the data with these external sources. Since linked data is  machine readable it’s very suitable to be processed by intelligent software agents, who can collect information based on the needs of the operator and the context they have to operate in. Another important feature of linked data is the distributed nature, we do not have to generate a large data management solution that contains all the information we might need, we simply reference external sources which are being kept up to date by their respective owners.

Fire department Amsterdam-Amstelland is using linked data already to assist in creating a small assistant for navigation, this is already described in previous post.

Afraid for Open Data

After visiting two Open Data related events in Amsterdam, Apps4nl and Open Innovation Festival, I found myself being afraid of what I had to conclude after these events. This might raise some eyebrows, coming from someone who claims to be a Linked Data enthusiast, and then blogging about being afraid for Open Data.

The primary reason that got me started with Linked Data were the problems I saw with the lack of operational access to information. When I needed to explain this problem to the board of directors of the Fire Department Amsterdam I had to explain this in a non technical way.

My fear

To express the problems I saw in the current situation of information management at operational level, I decided to formulate it as a fear:

During an operational incident an accident will happen where me or one of my colleagues will get injured or killed. Then after an investigation we will come to the conclusion that we had all the information available to prevent the accident from happening, but we just did not have the means to access it.

I considered this to be a very bold statement. But to my surprise it was acknowledged by some of the board members and I got green light for my experiments. Although we primarily focused on the data available in the organization, it is not that hard to draw the exact same conclusion for data available at the partners of the Fire Department.

The fear basically consists of two parts:

  1. Although the data might be available, the interfaces and devices at operational level are not capable of accessing it
  2. The information is available but not accessible digitally by machines to be used at operational level

Point 2 is exactly where the whole open data movement comes in to play. Wouldn’t it be great if we could access partner data while we drive to an incident? Let me illustrate this with an example.

If buildings are under control of Fire Department Inspectors because of their usage it should be possible to get the status on those permits in the organization. However there are numerous situations where the Fire Department does not require a permit for the building. That of course does not mean there are no problems with the building. If they would be able to access this information straight at district level they could solve this problem.

I am aware of a project in the Netherlands called DBK “Digital Object Map” which promises exactly that, but does so in a traditional closed model legacy system. That might have worked about 6 years ago, but with the current high interest for Open Data movements, it is smarter to ride that wave instead of forcing all the different districts, we need to deal with, to use a single purpose proprietary system.

So now that data becomes available, what is it that I am so afraid for? The current approach a lot of local governments take is the ‘Over the hedge’ method, we extract a dataset from our systems, put it on-line free to use, problem solved. That is not going to help us as this means we always have to rely on two important factors, the data is updated frequently, and we implement the changes in the right places as well. That is not a sustainable way to make me feel save again.

Linked Data

If a Fire Department wants to literally put the lives of their Fire Fighters in the hands of data, they cannot accept that the data they use is based on an occasionally exported dataset of which they hope the latest version is made available. There needs to be a possibility to query the source of the information directly, which will expose a dataset that is self explanatory, or at least gives the possibility to infer the relationships in that dataset . That dataset should of course reflect the real time situation of the source data!

Steps to take

It is really great that so many governmental organizations are opening up their data of course, but now that this topic is so high on the agenda we should dare to take the final step as well! Link the data! Make it sustainable and not just a simple dump. Only then organizations like the Fire Department Amsterdam-Amstelland can really take full benefit of all the efforts that have been put in this Open Data movement.

Linked Data Now!

 

How it all started, answers to questions that weren’t asked.

About 3 years ago a group of fire fighters at a fire station in Amsterdam was confronted with a new navigation assistant in the fire truck. From a conceptual point of view the idea of having navigation assistance in a fire truck might be nice, but there are practical implications when using a system designed for normal traffic in an emergency vehicle , remember fire trucks are allowed to defy all the rules in traffic! Without elaborating on why it is not working as it should, the more abstract point is that it gives answers to questions we never asked.

The answer you get from a navigation assistant is based on the question ‘What is my next step in getting to my destination?’ where fire fighters are more interested in ‘What is my final destination?’. Sitting down and discussing the subject the conclusion was that they were interested in an overview on a map with both the location of the fire station as well as the location of the incident marked.

The route itself is not that interesting as the knowledge of the fire fighters about the local area is at such a high level that they easily beat a navigation assistant.

To solve this problem, we were confronted with a few technical requirements:

  1. Real time, the system has to respond to an alarm at the fire station in a few seconds.
  2. It should not require any manual interaction.
  3. Low maintenance at station side.
  4. We should have access to the raw data

Items 1, 2 and 3 were pretty easy to solve, item 4 however turned out  to be a more challenging one.
Although the full message was sent ‘over the air’ via a paging protocol, it wasn’t really a structured data format. It was just a compression of various seemingly random chosen fields into a maximum width of 140 characters.
After some trial and error we were able to extract the various fields with about a 98% accuracy. Now that we had the data and the various pieces of information we needed to distribute this data. In our team we had some discussions about what format to use.

During that time we got introduced to RDF, where the flexibility of the data model was what was intriguing us most. This would perfectly fit with requirement 3. We could still enrich the data dynamically without breaking already installed systems at the fire stations.

How does it work now

After tying all loose ends together the structure of the solution is very elegant and simple.

  1. We scan the via air broad casted message for station ID’s we are interested in. Whenever a station ID we want is detected we generate RDF data.
  2. The vocabularies used are partly standard, partly domain specific.
    The standard ones include:

    • Meta data, all the information about the publishing of the data,who/when/where/what: OWMS / DCMI
    • Events, the alarm is an event, and we record everything relative to the event: SEM
    • Location, the crew is dispatched to a location: GEO
    • Address, the incident takes place at a certain address: vCard

    The domain specific vocabulary is a subclass of event, describing:

    • Units dispatched to the incident
    • Fire station involved
    • Deployment plans used.
  3. This information is stored as RDF data on a public URI. This URI is then dispatched to the relevant fire stations.
  4. The software at the stations uses Semantic Web techniques to retrieve the data from the URI, and show the relevant information on the map.

Advantages

Due to the flexible structure of the data model we have been able to enhance and update the data model without touching the client software. If the clients needed additional functionality it needed a software update of course, but the software update was not necessary to accommodate a data model update.

From a Semantic Web point of view we now start very early in the incident describing the data, we actually do know what we are talking about. Because we gain knowledge about the actual incident at a very early stage we are able to assist and connect multiple sources of information in a later stage.

From a non technical point of view we have some results we didn’t expect. Because the truck driver now has a better idea of where he needs to go, he is more relaxed and better concentrated behind the wheel, resulting in more secure and save driving.
As there is a map at the station, the fire fighters look at the map more often, and get a better idea of their work area.

Outside in development

Due to the close connection with the people actually using the system, we were able to implement modifications and wishes really fast. We didn’t go through highly detailed project plans and endless meetings, but decided to start where we thought it was easiest and just took it from there, never loosing sight of the technical strategical direction we wanted to go and the targets to achieve.

Future plans

Our current focus now is to extend the screens with information from the deployment plans. More on that in the near future….

Welcome

Our company, netage.nl, has been active in the fire fighting business for over 10 years now. With a team of people, all close to or active in the job, we have learned a lot about how fire departments treat Information Technology. Over the years we have seen huge changes in the landscape of fire fighting and Information Technology. netage.nl however is not very pleased with what we see right now.

In the current hype of open data and open government initiatives, fire departments are confronted with huge amounts of data. The enormous amount of attention that goes out to open data applications on various platforms clearly underlines the problem we are facing: we are just trading the open data for mirrors and beads, they are shiny and grasp our attention, but are not tackling the real problems.

To really leverage the gained knowledge in all the open data, we need to start to act now and understand the language we speak within the fire department organizations and align that with the new possibilities in the IT landscape.This modern structure of organizing our newly found knowledge is in need of a fresh vision at the methods we use to interact with all this data.

Over the past 4 years netage.nl has been closely involved with our technology partner netlabs.org to develop a new framework for accessing information. You can find them under ‘technology partners’.

This blog is intended to share our ideas on how things should change in the fire fighting business, thereby improving the service to the public, reducing risks for the fire fighters and saving lives.