Using interface encapsulation to listen to linked data predicates

Introduction

The article was a submission to ISWC 2014 Developer Workshop

Linked data and its RDF model is known for its flexible schema less structure and integrated meta-data. Although this is great for expressiveness of the data, its is a nightmare for interface developers since all the modern interface paradigms are tree based.

The standard way of interacting with data in these user interfaces is to use application logic to interact with interface elements which needs to query the data and decide which piece of information to put in which element. Trying to push free format RDF in a tree is not a easy tasks and often results in very static interface for a specific graph structure.
Wouldn’t it be more intuitive to use the meta data in the RDF to decide which user interface element treats which property? And even more intuitive let the UI be able to decide to get more data from other linked resources without needing specific application logic ?

Traditional Model

Lets try to go through the traditional model with static data:

Data:
 Bart, van leeuwen, @semanticfire, http://netage.nl/dir.ttl
Code:
 Select firstname, lastname,handle,companyprofile from data
 interface.firstname.setValue(resultRow.interface)
Interface:
 Field firstname
 Field lastName
 Field handle
 Field companyProfile

If any changes are made on the interface, it is up to the application logic to extract it from the interface and modify the underlying data model.
So if we get a new data structure as input we need to change both logic and interface to react to that. When we want to react to different types of input structures both the interface and logic are getting more and more complex. This is mainly due to the fact that the data is pretty ‘dumb’ we need the logic to identify various pieces of data and match them to user interface elements, the other way around is even more complex. Getting the modified data an sending it back to the underlying data model. There we need a extra step to make sure both UI and application logic are in sync with the constraints of the data store.
The traditional solution while dealing with RDF data is to use SPARQL queries to get somewhat structured data which resembles the result of the SQL query above. Although we can use traditional methods to show the data, we loose a significant part of the semantics around it, the data is ‘dumb’ again. Updating data is even more complex since SPARQL results might come from multiple endpoints and that information is not present in the SPARQL result set.

A different approach

Now how can we use the context in RDF to eliminate application logic and put all the logic in the interface. If we are able to do this it would allow us to simply be ready when the piece of information we are able to show comes by and complies to the constraints we have set on it.
During the development of some of the front ends of our system we started out with the traditional SPARQL way. One of the important reasons to use linked data in our solution is the agile nature, adding extra properties or replacing them is easy and doesn’t require large schema updates. However on the front end we were still struggling with multiple SPARQL queries for the various incarnations of our data. This should be done in a smarter way.

Introducing EnyoJS

In our search for toolkits to create mobile applications we came across EnyoJS [1] a Javascript framework which originated on the Palm Pre smart phones as a part of WebOS and evolved in a Framework for platform independent applications. The current Enyo 2.4 version is the basis for LG’s latest SmartTV Platform. The concept of EnyoJS is encapsulation, components should have no knowledge about their place in the hierarchy, they communicate with bubbling up and water-falling down events. They have no control who receives these notifications and events. There is also no limitation on which components encapsulate each other. The flow of events and notifications is left undisturbed by encapsulation, adding a extra component around another doesn’t break the chain, unless the inserted component deliberately listens and ends the event chain. Components can directly talk to elements they define but should not talk to the inner components of those elements. Neither should they directly talk to their parent components.

The EnyoJS way

The encapsulation gave us the inspiration to see if we could use this method to add a linked data property to components. By encapsulating a standard UI component with a linked data aware component we would be able to define at interface level which element should contain values of predicates.
So if we would have a UI Container C which contains a Textbox T traditionally it would look like

C1 → T

The container would use the traditional SPARQL way to get data and update the value v of T . Our approach would be that in between C and T we place a property component P which is configured to react to predicate “example:foo” and propagate the value of “example:foo” to its components.

C2 → P → T

and instead of the traditional SPARQL way we simply let C announce to its children it has a graph available, P will pick up any triple which has the “?s example:foo ?o” pattern and extracts the value ?o and propagates this to its underlying components. P will stop the waterfall. This method would still allow to insert a extra component in between, e.g. a Box B

C2_2 → P → B → T

Since B is does not have a “setvalue” called by P we will simply call T directly, remember we can call methods of defined components directly.

If the value of ?o is a URI, P will try to load that resource and announce to its components that it has a Graph. So if I would like to know “foaf:name” from someone I “foaf:knows” it would look a bit like this.

C3 → P¹ → P² → T

Where P¹ listens to “foaf:knows” which will load the graph and announces it to its components where P² which then will set the value of T to “foaf:name”
Presenting multiple properties is no problem either if I am also interested in the “foaf:mbox” value of my “foaf:knows” relations ships this would work as well.

C4 → P¹ → [ P² → T¹, P³ → T² ]

Here P³ represents “foaf:mbox”
The above example snippets can be found in the github repo [2] from the samples you can clearly see that the ‘meta-data’ stays close to the interface level and we only fetch and announce the source graph once. The snippets just give a rough overview of what is possible with this approach.

Additional possibilities

The EnyoJS platform knows the concept of repeaters, which allows components to contain a virtually unlimited amount of instance of another component. This would solve a cardinality issue.

C → P¹ → R → P² → T

Here Repeater R is used to create multiple T components from all the “foaf:knows” relations I have.

Its also possible to combine multiple components inside a repeaters, e.g. P³ listens to foaf:mbox

C → P¹ → R → [ P² → T¹ , P³ → T² ]

this would result in a list of names and emails addresses of all the “foaf:knows” relationships in my profile.
The way back.
By maintaining the ?s on the P components propagating updates back is relatively easy

C ← P ← T

if we update T, and configure P to listen to the updates ?u to value ?o while maintaining ?s related to P we can simply construct a triple out of that

 ?s → P:predicate → ?u

which replaces

?s → P:predicate → ?o

We now would be able to either construct a SPARQL Update, provided we know which endpoint to use, or use LDP spec to to do a patch on ?s
Of course you might want to constraint the actual execution of water-falling based on property values, e.g. only show properties which are no older then one week. And only show a certain component if there actually is data, or have property component just be responsible for containing other properties without generating any UI. It even allows to listen to the same property with different constraints.
e.g.

C → P(ex:foo >= 50 ) → Tgreen
C → P(ex:foo <50 ) → TRed

If T would be a editable field passing the threshold of 50 would in the end propagate the update up to C which would then announce a updated graph, which in turns would show the T with the right color.
For formating back and forth EnyoJS provides all sorts of transition hooks which would allow to format a xsd:DateTime to something human readable, and back to xsd:DateTime after modification

Conclusion

By using the encapsulation mechanism in EnyoJS we are able to to combine both UI description and the related RDF properties in a single place without being bothered by external code to sync the model and the interface. Although surely in its infancy the approach described here is not only a nice idea, it is practically being used in our applications.

The code is not in the public domain yet, but this is certainly something we will be doing in the near future.

Samples

The sample snippets are made available on Github [2]. Handling RDF data is done with rdflib.js [3]
This is none working code right now, I’ll assemble working samples before the conference
Current state and usage
By the time of writing the technique described in this paper is used deployed actively on several monitors at fire stations in the Netherlands

Future work

One of the more obvious additions would be language support., set a preferred language on literals so you can react to multi language RDF.
Multiple property update predicate relation, as in the example above with Tgreen and Tred, maybe a P which updates and listens to the change of ?o but also modifies another property on T, the color in this case.
Using Json-LD to define properties, this would allow the omission of full URI’s and use a context to solve them
User property definitions and constraints based on the work off the future Shape expressions Working group [4] or the Resource Shapes 2.0 submission [5]

References

[1] http://enyojs.com
[2] https://github.com/semanticfire/iscw2014_dev_workshop
[3] https://github.com/linkeddata/rdflib.js/
[4] http://www.w3.org/2001/sw/wiki/ShEx
[5] http://www.w3.org/Submission/2014/SUBM-shapes-20140211/

Using IBM DB2 NoSQL Graph Store in Websphere Application Server Community Edition

This is the first of a series of blog posts about our experience with the IBM DB2 Express-C NoSQL Graph Store (hereafter DB2 RDF) in combination with IBM WebSphere Application Server Community Edition (hereafter WASCE).

The DB2 RDF product allows the storage and manipulation of RDF data. The data can be stored in graphs, all according to W3C Recommendations. The DB2 RDF product uses the Apache Jena programming model to interact with the underlying store. In the very detailed documentation there is an outline of products and tools needed to get the basic DB2 RDF programming environment going.

This series of articles is specifically about using the tools inside the WASCE environment. While developing our RESC.Info product we gathered a lot of experience which we like to share with the community using this article. We will also be presenting our experience during this years IBM Information On Demand 2013 in Las Vegas.

The series will cover the following topics:

  • Configuring WASCE data sources
  • Assembling the correct Jena distribution
  • Dealing with transactions

This first article is about configuring WASCE data sources for use with the DB2 RDF and Jena programming model. This is NOT meant to be an extensive installation guide for these products, you should refer to the respective product documentation for more information on installation.
It is very important to select the correct versions of the various products:

  • IBM DB2 Express-C 10.1.2
  • IBM WebSphere Application Server Community Edition 3.0.4

Creating a DB2 database and graph store

To  be able to use the DB2 RDF features we need to create a standard database with some specific parameters. The DB2 documentation contains extensive information for this task. To create a database ‘STORE’ that supports DB2 RDF we issue the following commands:

db2 CREATE DATABASE STORE PAGESIZE 32 K
db2 UPDATE DATABASE CONFIGURATION FOR STORE USING LOGFILSIZ 20000
db2 CREATE TABLESPACE SYSTOOLSPACE IN IBMCATGROUP MANAGED BY AUTOMATIC STORAGE EXTENTSIZE 4

For the correct administration we also need to execute:

db2set DB2_ATS_ENABLE=YES
db2 UPDATE DB CONFIG USING AUTO_MAINT ON AUTO_TBL_MAINT ON AUTO_RUNSTATS ON
db2 alter bufferpool IBMDEFAULTBP IMMEDIATE SIZE 15000 AUTOMATIC

Now that the database is created we still need to create a graph store inside the database. This is done with the following command:

createrdfstore rdfStore  -db STORE -user db2admin -password XXX -schema public 

This can take a while. After completion you will have a graph store ‘rdfStore’ inside your database ‘STORE’. To check the presence of this store issue the following command when connected to ‘STORE’:

db2 SELECT * FROM SYSTOOLS.RDFSTORES

The resulting table should contain reference to our store ‘rdfStore’ in schema ‘public’

WebSphere Application Server Community Edition installation

Install WASCE with the installer, but do not start it yet. WASCE is distributed with some older DB2 JDBC drivers which interfere with the DB2 JDBC4 drivers that are needed for the DB2 RDF interface. In the repository directory of WASCE look for the path

<wasce_install>/repository/com/ibm

and delete the db2 sub-directory. Run WASCE with the -clean parameter, which causes WASCE to cleanup all references to the included DB2 JDBC drivers.

geronimo.[sh/bat] run -clean

Installing db2jcc4.jar

Now it is time to install the JDBC4 driver into WASCE repository. In the advanced mode of the console you will find the Resources/Repository tab where you can add new jars to the repository. Select the db2jcc4.jar from your <DB2_INST>/java directory and fill out the fields as shown in the image and click ‘Install’.
wascerepos

Creating a Database Pool

Once the correct jar is installed the creation of the connection to the database is the same as any other regular database connection. Select DB2 XA as ‘Database type’ and fill out the connection information. You should only see one JDBC driver here, the one we just installed. Fill out the details of your regular DB2 database ‘STORE’ and click ‘Deploy’.

After a database source object is created we can use it in the simple SQL entry field, select the newly created data source and issue the following command:

SELECT * FROM SYSTOOLS.RDFSTORES

The result should be the same as the result we had after issuing this query from the command line.

Conclusion

Now we have setup a DB2 RDF connection inside WASCE with the correct version of both products and connecting drivers. The next step will be to create a simple Jena based application to interact with the store.