Copying one RDF datasource into another

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Copying one RDF datasource into another

nathanpbell
Hi all, I keep finding conflicting documentation on how to use RDF
datasources in XUL (I'm using XULRunner with local chrome and remote
RDF) so I thought I'd ask the group!

I have a remote database that is significantly large and changes
frequently.  I have a XUL client and I want to retrieve the
RDF-formatted database on startup and store it in memory.  I then want
it to be able to poll the server for updates, download the updates in
RDF and 'commit' them to the in-memory store.

The server side works fine.  I can give the server side a timestamp and
it returns an RDF of all records that have changed since that timestamp.
  Theoritically my XUL client can just make requests to the server and
include the timestamp of it's last update in the request.

However, I can't for the life of me figure out how to take the RDF
update (which could contain both new Subjects (database records) or new
Targets of old Subjects (database column values)) and add to/overwrite
the in-memory store.

I will be making many of these update requests over the runtime of the
software (hundreds or thousands), so I also want to be careful about
memory leaks.  It seems to me that continually adding these update RDF's
the the tree's composite datasource would result in very large and
unnecessary memory usage. Ideally I'd like to download the update RDF,
overlay it's assertions with the in-memory store, and then discard the
update RDF from memory.  Which docs should I be reading, or what methods
should I be looking at?

Thanks all!
-Nathan


code snippets:
------XUL-------------------

<tree datasources="rdf:product-datasource"
       ref="&remote.baserdf;/Products"
       flags="dont-build-content"
       id="productTree">
<...Template code here...>


-----Remote update RDF ------------------

<RDF:Description
  RDF:about="http://www.example.com/rdf/Products/1654684684"
  Products:product_id="1654684684" Products:qty="15" />

<RDF:Seq RDF:about="http://www.example.com/rdf/Products">
<RDF:li RDF:resource="http://www.example.com/rdf/Products/1654684684"/>
</RDF:Seq>

----- JavaScript ---------------------
var datasource =
rdfService.GetDataSource("http://www.example.com/dbupdate/" +
lastUpdateTimestampe);
// Combine the data sources some how???
_______________________________________________
dev-tech-rdf mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-rdf
Reply | Threaded
Open this post in threaded view
|

Re: Copying one RDF datasource into another

Neil-4
Nathan Bell wrote:

> I have a remote database that is significantly large and changes
> frequently.  I have a XUL client and I want to retrieve the
> RDF-formatted database on startup and store it in memory.  I then want
> it to be able to poll the server for updates, download the updates in
> RDF and 'commit' them to the in-memory store.

That heavily depends on what you mean by updates. Remember that RDF has
no concept of your data structure, so you need to figure out which of
the following cases you intend to identify and how to handle them:
1. Adding a new item to a list
2. Removing an item from a list
3. Setting a property on an item
4. Changing a property on an item
5. Removing a property from an item

--
Warning: May contain traces of nuts.
_______________________________________________
dev-tech-rdf mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-rdf
Reply | Threaded
Open this post in threaded view
|

Re: Copying one RDF datasource into another

nathanpbell
Neil wrote:

> Nathan Bell wrote:
>
>> I have a remote database that is significantly large and changes
>> frequently.  I have a XUL client and I want to retrieve the
>> RDF-formatted database on startup and store it in memory.  I then want
>> it to be able to poll the server for updates, download the updates in
>> RDF and 'commit' them to the in-memory store.
>
> That heavily depends on what you mean by updates. Remember that RDF has
> no concept of your data structure, so you need to figure out which of
> the following cases you intend to identify and how to handle them:
> 1. Adding a new item to a list
> 2. Removing an item from a list
> 3. Setting a property on an item
> 4. Changing a property on an item
> 5. Removing a property from an item
>

Good catch, thank you.  I should have explained that better.

Really, I am only tracking one table in the database.  There are no
deletions.  There are only 1) Adding new rows and 3) Overwriting a
property of an already existing row.

Intuitively it would make sense that there's a way to just graft one
datasource onto another (since I'm not doing any deletes of items or
properties, just new values and overwriting old values), but I'm
beginning to understand that is not the case, is it?
_______________________________________________
dev-tech-rdf mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-rdf
Reply | Threaded
Open this post in threaded view
|

Re: Copying one RDF datasource into another

Axel Hecht
Nathan Bell wrote:

> Neil wrote:
>> Nathan Bell wrote:
>>
>>> I have a remote database that is significantly large and changes
>>> frequently.  I have a XUL client and I want to retrieve the
>>> RDF-formatted database on startup and store it in memory.  I then
>>> want it to be able to poll the server for updates, download the
>>> updates in RDF and 'commit' them to the in-memory store.
>>
>> That heavily depends on what you mean by updates. Remember that RDF
>> has no concept of your data structure, so you need to figure out which
>> of the following cases you intend to identify and how to handle them:
>> 1. Adding a new item to a list
>> 2. Removing an item from a list
>> 3. Setting a property on an item
>> 4. Changing a property on an item
>> 5. Removing a property from an item
>>
>
> Good catch, thank you.  I should have explained that better.
>
> Really, I am only tracking one table in the database.  There are no
> deletions.  There are only 1) Adding new rows and 3) Overwriting a
> property of an already existing row.
>
> Intuitively it would make sense that there's a way to just graft one
> datasource onto another (since I'm not doing any deletes of items or
> properties, just new values and overwriting old values), but I'm
> beginning to understand that is not the case, is it?

There is one tricky detail, that is, in RDF, one source can have
multiple arcs with the same predicate to multiple targets, that is,
loading in a datasource to *move* an arc from one to the other doesn't work.

You'll have to hardcode that, which makes it less painful that you have
do code the whole thing yourself ;-).

Axel
_______________________________________________
dev-tech-rdf mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-rdf
Reply | Threaded
Open this post in threaded view
|

Re: Copying one RDF datasource into another

Neil-4
In reply to this post by nathanpbell
Nathan Bell wrote:

> Intuitively it would make sense that there's a way to just graft one
> datasource onto another (since I'm not doing any deletes of items or
> properties, just new values and overwriting old values), but I'm
> beginning to understand that is not the case, is it?

New values are easy, but RDF doesn't have a concept of overwriting old
values; it basically allows multiple values everywhere but provides
functions for convenient access to the first value.

Put it another way, if you were to say "the sky is blue" and later say
"the sky is grey", then when asked "what colour is the sky" you are
equally likely to get the answer "blue" or "grey", and if you ask "what
colours are the sky" it will say "blue and grey" or "grey and blue".

RDF does have a function "the sky is no longer blue, it is grey" but you
do need to know that the sky was blue before.

As for lists, the items in the list are typically numbered for
convenience, so you might prefer to use rdf container objects to manage
listing elements from your new values and adding them to your memory data.

--
Warning: May contain traces of nuts.
_______________________________________________
dev-tech-rdf mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-rdf
Reply | Threaded
Open this post in threaded view
|

Re: Copying one RDF datasource into another

Axel Hecht
Neil wrote:

> Nathan Bell wrote:
>
>> Intuitively it would make sense that there's a way to just graft one
>> datasource onto another (since I'm not doing any deletes of items or
>> properties, just new values and overwriting old values), but I'm
>> beginning to understand that is not the case, is it?
>
> New values are easy, but RDF doesn't have a concept of overwriting old
> values; it basically allows multiple values everywhere but provides
> functions for convenient access to the first value.
>
> Put it another way, if you were to say "the sky is blue" and later say
> "the sky is grey", then when asked "what colour is the sky" you are
> equally likely to get the answer "blue" or "grey", and if you ask "what
> colours are the sky" it will say "blue and grey" or "grey and blue".
>
> RDF does have a function "the sky is no longer blue, it is grey" but you
> do need to know that the sky was blue before.
>
> As for lists, the items in the list are typically numbered for
> convenience, so you might prefer to use rdf container objects to manage
> listing elements from your new values and adding them to your memory data.
>

I actually think that it's pretty easy to implement, one just needs to do a
if ((old = GetTarget..))
  was-that-move-or-change(..)
else
  assert

No reason to introduce arbitrarily numbers and fiddle the vocabulary.

Axel
_______________________________________________
dev-tech-rdf mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-rdf
Reply | Threaded
Open this post in threaded view
|

Re: Copying one RDF datasource into another

Neil-4
Axel Hecht wrote:

> No reason to introduce arbitrarily numbers and fiddle the vocabulary.

I thought RDF containers introduced them for you.

--
Warning: May contain traces of nuts.
_______________________________________________
dev-tech-rdf mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-rdf
Reply | Threaded
Open this post in threaded view
|

Re: Copying one RDF datasource into another

Axel Hecht-2
Neil wrote:
> Axel Hecht wrote:
>
>> No reason to introduce arbitrarily numbers and fiddle the vocabulary.
>
> I thought RDF containers introduced them for you.
>

Well, if you use RDF containers in aggregation, you would have to do
some special handling to actually enumerate the container while reading
the one datasource and then use container apis when doing the other.

Anyway, I understood the original post to be somewhat orthogonal. More
like table column than table row.

Axel
_______________________________________________
dev-tech-rdf mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-rdf
Reply | Threaded
Open this post in threaded view
|

Re: Copying one RDF datasource into another

Neil-4
Axel Hecht wrote:

> I understood the original post to be somewhat orthogonal. More like
> table column than table row.

Whereas I understood him to want to support new table rows as well as
changed table column values for existing table rows.

--
Warning: May contain traces of nuts.
_______________________________________________
dev-tech-rdf mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-rdf