Reference Guide | AllegroGraph 3.0

This is the reference guide for AllegroGraph 3.0. The overview tutorial can be found here. An introduction to AllegroGraph covering all of its many features at a high-level is in the AllegroGraph Introduction.

Conceptual Triple-Store Structure

AllegroGraph is a graph database that makes an excellent RDF triple-store. This is a conceptual diagram of the data AllegroGraph manages:

AllegroGraph's logical structure

The bulk of an AllegroGraph triple-store is composed of assertions called triples. For historical reasons, each triple has five fields:
- subject (s)
- predicate (p)
- object (o)
- graph (g)
- triple-id (i)

Each assertion can be viewed as an edge p between graph nodes s and o with additional data g or as the statement "subject predicate object (in the context of graph)".

All of s, p, o, and g are strings of arbitrary size. String data is associated with Unique Part Identifiers (UPIs) and managed by the string dictionary.
To speed queries, AllegroGraph creates indices which contain the assertions plus additional information (see below for more details).
AllegroGraph can also perform freetext searching in the assertions using its freetext indices (see the section on freetext indexing for details).
and Finally, AllegroGraph keeps track of deleted triples

Triple-data generally comes into AllegroGraph as strings either from pure RDF/XML or as the more verbose but simpler N-Triple format. Once the triples are in AllegroGraph, it is possible to manipulate them, perform logical inference, and execute extremely fast queries.

The string dictionary manages efficient storage and fast lookup of the strings in the triple-store. We call the process of hashing a string into its UPI and storing it in the string dictionary interning.

Basic Triple-Store Operations

Adding Triples

Triples stored in files using the N-Triples ¹ , RDF/XML ² , and other formats can be loaded into the triple-store with the following functions.

Add triples from source to the triple store.

source - can be a stream, a pathname to an N-Triples file or a string that can be coerced into a pathname to an N-Triples file. Source can also be a list of any of these things. In this case, each item in the list will be imported in turn. This can be more efficient than iterating over the items yourself since AllegroGraph will only synchronize at the end of the loading.

The following keyword parameters can be used to control the loading process:

:db - specifies the triple-store into which triples will be loaded. This defaults to the value of *db*.
:graph - the graph to which the triples from source will be placed. It defaults to nil which is interpreted as db's default graph. If supplied, it can be:
- a string representing a URIref, or a UPI or future-part encoding a URIref, which adds the triples in source to a graph named by that URI
- the keyword :source (in which case the source argument will be interned as a URI and the loaded triples added to a graph named by that URI). This has the effect of associating the file or URL of source with the new triples.
:verbose - specifies whether or not progress information is printed as triples are loaded. It defaults to the value of (ag-property :verbose).
:always-save-string-literals - determine whether or not to save the strings of a triple's object field when the object can be encoded directly into the triple. If true (the default) then the strings will be saved. If false, then only the encoded values will be preserved (this may prevent exact round-trips if data is coerced during the encoding process).

load-ntriples returns the number of triples added, and the graph used, as multiple values.

Add all of the triples from the string to the triple store.

The following keyword parameters can be used to control the loading process:

:db - specifies the triple-store into which triples will be loaded. This defaults to the value of *db*.
:graph - the graph to which the triples from source will be placed. It defaults to nil which is interpreted as db's default graph. If supplied, it can be:
- a string representing a URIref, or a UPI or future-part encoding a URIref, which adds the triples in source to a graph named by that URI
- the keyword :source (in which case the source argument will be interned as a URI and the loaded triples added to a graph named by that URI). This has the effect of associating the file or URL of source with the new triples.
:verbose - specifies whether or not progress information is printed as triples are loaded. It defaults to the value of (ag-property :verbose).
:always-save-string-literals - determine whether or not to save the strings of a triple's object field when the object can be encoded directly into the triple. If true (the default) then the strings will be saved. If false, then only the encoded values will be preserved (this may prevent exact round-trips if data is coerced during the encoding process).

load-ntriples-from-string returns the number of triples added, and the graph used, as multiple values.

Load the file named by manifest, loading the triples from the graphs it references.

Add triples from the named RDF/XML file to the triple-store. The additional arguments are:

:db - specifies the triple-store into which triples will be loaded; defaults to the value of *db*.
:base-uri - this defaults to the name of the file from which the triples are loaded. It is used to resolve relative URI references during parsing. To use no base-uri, use the empty string "".
:graph - the graph to which the triples from source will be placed. It defaults to nil which is interpreted as db's default graph. If supplied, it can be:
- a string representing a URIref, or a UPI or future-part encoding a URIref, which adds the triples in source to a graph named by that URI
- the keyword :source (in which case the source argument will be interned as a URI and the loaded triples added to a graph named by that URI). This has the effect of associating the file or URL of source with the new triples.
use-rapper-p - If use-rapper-p is true, then the RDF/XML file will be piped through the open source tool rapper using run-shell-command. Obviously, rapper must be both installed and in your path for this to work. If rapper is not in your path, you can supply it explicitly as the value of use-rapper-p.

Treat string as an RDF/XML data source and add it to the triple-store. For example:

   (load-rdf/xml-from-string  
    "<?xml version="1.0"?>  
<rdf:RDF xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\"  
         xmlns:ex=\"http://example.org/stuff/1.0/\">  
  <rdf:Description rdf:about=\"http://example.org/item01\">  
    <ex:prop rdf:parseType=\"Literal\"  
             xmlns:a=\"http://example.org/a#\"><a:Box required=\"true\">  
         <a:widget size=\"10\" />  
         <a:grommit id=\"23\" /></a:Box>  
    </ex:prop>  
  </rdf:Description>  
</rdf:RDF>  
")

See load-rdf/xml for details on the parser and the other arguments to this function.

RDF in the form of named graphs can also be loaded using load-trix, which understands the TriX format.

Load a TriX document into the triple store named by db.

source - a string or pathname identifying a file, or a stream.
:db - specifies the triple-store into which triples will be loaded; defaults to the value of *db*.
:verbose - if true, information about the load progress will be printed to *standard-output*.
:default-graph - a future-part or UPI that identifies a graph, or nil. If it is non-null, any graphs in the TriX document that are equal to the given URI will be treated as the default graph in db.
default-graph can also be the symbol :source, in which case namestring is called on the source and treated as a URI.

We have implemented a few extensions to TriX to allow it to represent richer data:

Graphs can be named by <id>, not just <uri>. (A future revision might also permit literals.)
The default graph can be denoted in two ways: by providing <default/> as the name for the graph; or by providing a graph URI as an argument to load-trix.
In the interest of generality, the predicate position of a triple is not privileged: it can be a literal or blank node (just like the subject and object), not just a URI.

Load a string containing data in TriX format into db

string - a string containing TriX data
:db - specifies the triple-store into which triples will be loaded; defaults to the value of *db*.
:verbose - if true, information about the load progress will be printed to *standard-output*.
:default-graph - a future-part or UPI that identifies a graph, or nil. If it is non-null, any graphs in the TriX document that are equal to the given URI will be treated as the default graph in db.

See load-trix for more details.

If you need to load a format that AllegroGraph does not yet support (for example, Turtle) we suggest you use the excellent free tool rapper from http://librdf.org/raptor/rapper.html to create either an N-Triples or RDF/XML file that AllegroGraph can load.

Note that both load-rdf/xml* and load-ntriples* have been removed from AllegroGraph 3.0 and beyond because their ability to load multiple triples sources simultaneously has been incorporated directly into load-rdf/xml and load-ntriples

Adding Programmatically

You can also add triples to a triple-store programatically with the function add-triple. The three required arguments, representing the subject, predicate, and object of the triple to be added can be expressed either as:

strings in the N-Triples syntax for URI references and literals;
UPIs such as are returned by the functions intern-resource, intern-literal, intern-typed-literal, and new-blank-node;
Encoded-UPIs created with functions like value->upi; or
future-parts created using the !-reader..

Add a new triple to the triple-store with the given subject, predicate and object and graph, specified either as UPIs or strings in N-Triples format. The :db keyword argument specifies the triple store to which the triple will be added and defaults to the value of *db*. Returns the numeric id of the new triple.

Note that duplicate triples can be added to a triple-store but indices will only refer to the same triple once.

Triple Manipulation

Triple parts: Resources, Literals, UPIs and more

Each triple has five parts (!), a subject, a predicate, an object, a graph and a (unique, AllegroGraph assigned) ID. In RDF, the subject must be a "resource", i.e., a URI or a blank node. The predicate must be a URI. The object may be a URI, a blank node or a "literal". Literals are represented as strings with an optional type indicated by URI or with a (human) language tag such as en or jp. ³ ; Blank nodes are anonymous parts whose identity is only meaningful within a given triple-store.

Resources, literals and blank nodes are represented as strings in RDF/XML or N-Triple syntax. AllegroGraph stores these strings in a string dictionary and hashes them to compute a Unique Part Identifier (UPI) for each string. A UPI is a length 12 octet array. One byte of the array is used to identify its type (e.g., is it a resource, a literal, or a blank node). The other 11-bytes are used to either store a hash of the string or to store an encoding of the UPIs contents (see type mapping below for more information about encoded UPIs).

From Strings to Parts

Resources and literals can be denoted with plain Lisp strings in the syntax used in N-Triples files. However this isn't entirely convenient since the N-Triples syntax for literals requires quotation marks which then need to be escaped when writing a Lisp string. For instance the literal whose value is "foo" must be written in N-Triples syntax as "\"foo\"". Similarly -- though not quite as irksome -- URIs must be written enclosed in angle brackets. The string "http://www.franz.com/simple#lastName", passed as an argument to add-triple will be interpreted as a literal, not as the resource indicated by the URI. To refer to the resource in N-Triples syntax you must write "<http://www.franz.com/simple#lastName> ". Literals with datatypes or language codes are even more cumbersome to write as strings, requiring both escaped quotation marks and other syntax.

To make it easier to produce correctly formatted N-Triple strings we provide two functions resource and literal. (The ! reader macro, discussed below, can also be used to produce future-parts and UPIs suitable to use as arguments for most of AllegroGraph's API.):

Create a new future-part with the provided values.

:language - If provided, language should be a valid RDF language tag.
:datatype - If provided, the datatype must be a resource. I.e., it can be a string representation of a URI (e.g., "http://foo.com/") or a future-part specifying a resource (e.g., !<http://foo.com>). If it does not specify a resource, a condition of type invalid-datatype-for-literal-error is signaled. An overview of RDF datatypes can be found in the W3C's RDF concepts guide.

Only one of datatype and language can be used at any one time. If both are supplied, a condition of type datatype-and-language-specified-error will be signaled.

Return the provided string as a future-part naming a resource. If namespace is provided, then string will be treated as a fragment and the future-part returned will be the URIref whose prefix is the string to which namespace maps and whose fragment is string. I.e., if the namespace prefix rdf maps to <http://www.w3.org/1999/02/22-rdf-syntax-ns#>, then the parts created by

(resource "Car" "rdf")

and

(resource "http://www.w3.org/1999/02/22-rdf-syntax-ns#Car")

will be the same.

Some examples (we will describe and explain the ! notation below):

> (resource "http://www.franz.com/")  
!<http://www.franz.com/>  
 
> (literal "Peter")  
!"Peter"  
 
> (literal "10" :datatype  
  "http://www.example.com/datatypes#Integer")  
!"10"^^<http://www.example.com/datatypes#Integer>  
 
> (literal "Lisp" :language "EN")  
!"Lisp"@en

Another issue with using Lisp strings to denote literals and resources is that the strings must, at some point, be translated to the UPIs used internally by the triple-store. This means that if you are going to add a large number of triples containing the same resource or literal and you pass the resource or literal value as a string, add-triple will have to repeatedly convert the string into its UPI.

To prevent this repeated computation, you can use functions like intern-resource, intern-literal, or intern-typed-literal to compute the UPI of a string outside of the add-triple loop. The function new-blank-node (or the macro with-blank-nodes) can be used to produce the UPI of a new anonymous node for use as the subject or object of a triple.

Returns true if upi is a blank node and nil otherwise. For example:

> (blank-node-p (new-blank-node))  
t  
> (blank-node-p (literal "hello"))  
nil

Compute the UPI of the string URI, make sure that it is stored in the string dictionary, and return the UPI.

:db - specifies the triple-store into which the uri will be interned. This defaults to the value of *db*.
:upi - if supplied, then this upi will be used to store the uri's UPI; otherwise, a new UPI will be created using make-upi.

Compute the UPI of value treating it as an untyped literal, possibly with a language tag. Ensure that the literal is in the store's string dictionary and return the UPI.

:language - if supplied, then this language will be associated with the literal value. See rfc-3066 for details on language tags.
:db - specifies the triple-store into which the uri will be interned. This defaults to the value of *db*.
:upi - if supplied, then this upi will be used to store the uri's UPI; otherwise, a new UPI will be created using make-upi.

Compute the UPI of value treating it as an typed literal with datatype datatype. Ensure that the literal is in the store's string dictionary and return the UPI.

value - must be a string
datatype - a URI, a future-part specifying a URI or a UPI specifying a URI.
:db - specifies the triple-store into which the uri will be interned. This defaults to the value of *db*.
:upi - if supplied, then this upi will be used to store the uri's UPI; otherwise, a new UPI will be created using make-upi.

Create a new blank-node in the triple-store db and return the UPI. If a upi is not passed in with the :upi parameter, then a new UPI structure will be created.

This convenience macro binds one or more variables to new blank nodes within the body of the form. For example:

(with-blank-nodes (b1 b2)  
  (add-triple b1 !rdf:type !ex:Person)  
  (add-triple b1 !ex:firstName "Gary")  
  (add-triple b2 !rdf:type !ex:Dog)  
  (add-triple b2 !ex:firstName "Abbey")  
  (add-triple b2 !ex:petOf b1))

The following example demonstrates the use of these functions. We use intern-resource to avoid repeatedly translating the URIs used as predicates into UPIs and then use new-blank-node to create a blank node representing each employee and intern-literal and intern-typed-literal to translate the strings in the list employee-data into UPIs. (We could also use literal to convert the strings but using intern-literal is more efficient.)

(defun add-employees (company employee-data)  
  (let ((first-name (intern-resource "http://www.franz.com/simple#firstName"))  
    (last-name (intern-resource "http://www.franz.com/simple#lastName"))  
    (salary (intern-resource "http://www.franz.com/simple#salary"))  
    (employs (intern-resource "http://www.franz.com/simple#employs"))  
    (employed-by (intern-resource "http://www.franz.com/simple#employed-by")))  
    (loop for (first last sal) in employee-data do  
     (let ((employee (new-blank-node)))  
       (add-triple company employs employee)  
       (add-triple employee employed-by company)  
       (add-triple employee first-name (intern-literal first))  
       (add-triple employee last-name (intern-literal last))  
       (add-triple  
        employee salary  
        (intern-typed-literal sal "http://www.franz.com/types#dollars"))))))

Note that the only difference between resource and intern-resource is that the former only computes the UPI of a string whereas the latter both does this computation and ensures that the string (and its UPI) are present in the triple-store's string dictionary.

Working with Triples

The functions subject, predicate, object, graph, and triple-id provide access to the part UPIs of triples returned by the cursor functions cursor-row and cursor-next-row or collected by get-triples-list.

Sets and gets the graph UPI of a triple.

In addition to being returned, the optional upi argument will be filled in with the graph's UPI. If not passed in, a new UPI will be created.

Sets and gets the object UPI of a triple.

In addition to being returned, the optional upi argument will be filled in with the object's UPI. If not passed in, a new UPI will be created.

Sets and gets the predicate UPI of a triple.

In addition to being returned, the optional upi argument will be filled in with the predicate's UPI. If not passed in, a new UPI will be created.

Sets and gets the subject UPI of a triple.

In addition to being returned, the optional upi argument will be filled in with the subject's UPI. If not passed in, a new UPI will be created.

Returns the ID of triple.

Working with UPIs

You can also create UPIs and triples programmatically; determine the type-code of a UPI, and compare them. You may want to make your own copy of a triple or UPI when using a cursor since AllegroGraph does not save triples automatically (see, e.g., iterate-cursor for more details).

If using encoded-triples or UPIs, then the functions upi->value and value->upi will be essential. You can also check the type of a UPI with either upi-type-code or upi-typep (see type-code->type-name and its inverse type-name->type-code for additional information.)

Copy a triple.

triple - the triple to copy
new - (optional) If supplied, then this must be a triple and the copy of triple will be placed into it. Otherwise, a new triple will be created.

This function is useful if you want to keep a reference to a triple obtained from a cursor returned by query functions such as get-triples since the cursor reuses the triple data structure for efficiency reasons.

Create a copy of a UPI.

upi - the UPI to copy.

Creates and returns a new triple data structure. The subject, predicate, object and graph slots will have UPIs whose upi-type-code is +rdf-unencoded-upi+ and the triple-id will be zero.

Creates and returns a new UPI data structure. The initial contents of the UPI returned are unspecified.

Compares triple1 and triple2 and returns true if they have the same contents and false otherwise. Only the subject, predicate, object and graph slots of each triple will be compared. See triple= if you need to compare triples including their IDs.

Compares triple1 and triple2 and returns true if they have the same contents and triple-id and false otherwise. See triple-contents= if you need to compare triples regardless of their IDs.

Decodes UPI and returns the value, the type-code and any extra information as multiple values.

upi - the UPI to decode
:db - specifies the triple-store from which the UPI originates.

The value, type-code, and extra information are interpreted as follows:

value - a string representing the contents of the UPI
type-code - an integer corresponding to one of the defined UPI types (see supported-types for more information). You can use type-code->type-name to see the English name of the type.
extra - Some encoded UPIs contain additional information which will be placed in extra if it is there. Examples include the language code or datatype of a literal. If no extra information is present, then nil will be returned as the extra information.

Examples:

> (upi->value (value->upi 22 :byte))  
22  
18  
nil  
 
> (upi->value (upi (literal "hello" :language "en")))  
"hello"  
3  
"en"

See value->upi for additional information.

Decodes UPI and returns the multiple values encoded by the upi. This is intended for UPIs that intrinsically carry more than a single value, such as longitude and latitude for a geospatial UPI. Operates the same as upi->value for hashed UPIs.

Returns the type-code of a UPI. This is a one-byte tag that describes how the rest of the bytes in the UPI should be interpreted. Some UPIs are hashed and their representation is stored in a string-table. Other UPIs encode their representation directly (see upi->value and value->upi for additional details).

Returns true if upi has type-code type-code.

upi - a UPI
type-code - either a numeric UPI type-code or a keyword representing a type-code. See type-code->type-name and type-name->type-code for more information on type-codes and their translations.

Compare upi-1 and upi-2 bytewise and return true if upi-1 is less than upi-2.

Test if two UPIs are equal. UPIs are equal if they are eq (i.e., the same object) or if they are equal as octet arrays.

Returns true if thing appears to be a UPI. Recall that every UPI is a octet array 12 bytes in length but not every length 12 octet array is a UPI. It is possible, therefore, that upip will return true even if thing is not a UPI.

Encodes value into UPI as type of encode-as. The encode-as argument can be a type code or a type name (see supported-types and type-name->type-code for details. If a upi keyword argument is not supplied, then a new UPI will be created. See upi->value for information on retrieving the original value back from an encoded UPI.

Comparing parts of triples

Return true if and only if the graph field of triple is the same as part using part= for comparison. The part may be a UPI, a string (in N-Triples syntax) that can be converted to a UPI, or a future-part. See graph-upi= if you know that you will be comparing UPIs.

Return true if and only if the object field of triple is the same as part using part= for comparison. The part may be a UPI, a string (in N-Triples syntax) that can be converted to a UPI, or a future-part. See object-upi= if you know that you will be comparing UPIs.

Return true if and only if the predicate field of triple is the same as part using part= for comparison. The part may be a UPI, a string (in N-Triples syntax) that can be converted to a UPI, or a future-part. See predicate-upi= if you know that you will be comparing UPIs.

Return true if and only if the subject field of triple is the same as part using part= for comparison. The part may be a UPI, a string (in N-Triples syntax) that can be converted to a UPI, or a future-part. See subject-upi= if you know that you will be comparing UPIs.

Return true if and only if the graph field of triple is the same as upi using upi= for comparison. The upi must be a UPI. See graph-part= if you need to compare future-parts or convert strings into UPIs.

Return true if and only if the object field of triple is the same as upi using upi= for comparison. The upi must be a UPI. See object-part= if you need to compare future-parts or convert strings into UPIs.

Return true if and only if the predicate field of triple is the same as upi using upi= for comparison. The upi must be a UPI. See predicate-part= if you need to compare future-parts or convert strings into UPIs.

Return true if and only if the subject field of triple is the same as upi using upi= for comparison. The upi must be a UPI. See subject-part= if you need to compare future-parts or convert strings into UPIs.

Future-Parts and UPIs

Future-parts (which are discussed in detail in their own section) can take the place of UPIs in many of the functions above. For efficiencies sake, functions like upi= assume that they are called with actual UPIs. AllegroGraph provides more general functions for the UPI only variants when it makes sense to do so. For example, future-part= works only with future-parts whereas part= works equally well with any combination of UPIs, future-parts or even strings.

A future-part defers namespace resolution and string interning until it is required. Along with the !-reader, they make writing and using UPIs significantly more readable and easier.

Return whatever is extra in the string of the future-part. The meaning of extra depends on the type of the part:

resource - the prefix (if applicable)
typed-literal - the datatype
language-literal - the language
other - the value will always be nil

See future-part-type and future-part-value if you need to access the part's other parts.

Returns the type-code of the future-part. The code can be converted to a keyword using type-code->type-name.

Return the value of the future-part. The meaning of value depends on the type of the part:

resource - the namespace (if any)
typed-literal - the literal without the datatype
language-literal - the literal without the language
other - the value will always be nil

See future-part-type and future-part-extra if you need to access the others parts (no pun intended) of the part.

Test if two future-parts are equal. Future-parts are equal if they both resolve to the same UPI or, if they cannot yet be resolved, if their strings are string=.

Create a 'future' part that will intern itself in new triple stores as necessary.

Compare any two things that can be coerced into UPIs as UPIs.

Returns the UPI associated with the future-part future-part.

If the future-part uses namespaces, then calling upi will resolve the namespace mapping. An error will be signaled if upi is called and there is no namespace mapping defined. You can use the errorp keyword parameter to disable the error and return nil instead.

Querying Triples

You can get triples out of a triple-store as a list or a cursor. The list structure is convenient but unwieldy if your query returns millions of triples (since every triple must be returned before you will see any of them). A cursor is like a database cursor from the RDBMS-world. It lets you traverse through the results of your query one step at a time.

Cursors

Cursors supply the functions cursor-next, cursor-row, and cursor-next-p for basic forward iteration. For convenience we include cursor-next-row which advances the cursor and returns the next row immediately. Cursors reuse the triple data-structure as they move through the result set so if you want to accumulate triples, make sure to use the copy-triple function.

Return true if and only if cursor has at least one more triple in it. Note that cursor-row, cursor-next and cursor-next-p are lower-level cursor manipulation routines. You may be better served by using collect-cursor, count-cursor, map-cursor and iterate-cursor.

Moves cursor forward to the next triple in the collection.

Returns t if there was another row on which to move and nil if the cursor is exhausted. I.e., if cursor-next returns t, then cursor-row will return a triple.

Note that cursor-row, cursor-next and cursor-next-p are lower-level cursor manipulation routines. You may be better served by using collect-cursor, count-cursor, map-cursor and iterate-cursor.

Returns the next triple from the cursor. The actual triple object returned is always the the same (eql) object. If you want to hold onto a triple for use after you advance the cursor, use the function copy-triple to make a copy of the value returned by cursor-next-row

Note that cursor-row, cursor-next, cursor-next-row and cursor-next-p are lower-level cursor manipulation routines. You may be better served by using collect-cursor, count-cursor, iterate-cursor and map-cursor.

Returns the triple that cursor is currently pointing at.

If the cursor is exhausted, then cursor-row returns nil.

Note that cursor-row, cursor-next and cursor-next-p are lower-level cursor manipulation routines. You may be better served by using collect-cursor, count-cursor, map-cursor and iterate-cursor.

There are several natural cursor idioms, the following functions handle many of them. We suggest building your own functions using these as building blocks since there may be internal optimizations made possible only through these. ⁴

Iterate over the cursor and collect a list of its triples. The :transform keyword can be used to modify the triples as they are collected. It defaults to the copy-triple function but you can use any function that takes one triple as an argument. (collect-cursor reuses the row argument so make sure that you use copy-triple in your own code if necessary).

Returns the number of triples remaining in a cursor and also exhausts cursor in the process.

Iterate over the triples in cursor binding var to each triple. Use the :count keyword with a fixnum value to limit the maximum number of triples that iterate-cursor visits. The binding to var is the same EQ triple in each iteration. Make sure to use copy-triple if you are retaining any of the triples that you visit.

The macro creates a block nil around its expansion. If iteration finishes normally by satisfying the count or by exhausting the cursor, iterate-cursor returns the number of triples visited.

Iterate over the triples in cursor while applying the function fn to each one. Use the count keyword to limit the maximum number of triples that map-cursor visits. In each iteration fn is applied to the current triple and to the list of arguments in args. map-cursor reuses a single triple as it iterates so make sure to use copy-triple if you retain any of the triples you visit.

The function returns the number of triples visited.

There are many other functions that either query a triple-store directly or return information about a triple-store. For example, count-query determines the number of matches to a query very quickly (because it doesn't bother to iterate over them) and pprint-subject lets you easily explore more about particular resources.

Return the count of the number of triples in db using only the information stored in the indices. Unindexed triples will not be included. An error will be signaled if the triple-store is missing the required index for the query.

Return an estimated count of the number of triples in db using only the information stored in the indices. Unindexed triples will not be included. The estimate can be off by as much as twice the triple store's metaindex-skip-size for each index chunk involved.

Returns the first triple found matching the search pattern specified by s, p, o and g. You may also specify that only encoded or unencoded triples by searched; whether or not to search deleted triples; and whether or not a filter should be used.. A new triple will be created unless you pass in one to use using the triple keyword parameter. nil is returned if no triples match.

Locate the triple whose triple-id is id and return it. The keyword argument db can be used to specify the triple-store in which to search. It defaults to the current triple-store, *db*. Get-triple-by-id allocates a new triple (using make-triple. You can prevent this by passing in your own triple using the keyword argument :triple. The data in the triple you pass in will by overwritten.

Query a triple store for triples matching the given subject, predicate, object, and graph. These can be specified either as UPIs, future-parts, strings in N-Triple format, or the wildcard nil. Returns a cursor object that can be used with cursor-next-p and cursor-next-row.

The following example finds every triple that starts with !ub:Kevin.

> (add-triple !ub:Kevin !ub:isa !"programmer")  
8523645  
> !ub:Kevin  
!<http://www.w3.org/1999/02/22-rdf-syntax-ns#Kevin>  
> (get-triples :s !ub:Kevin)  
#<row-cursor #<triple-record-file @ #x13c1a87a> 2019 [1 - 2018] @  
  #x14942bba>  
> (print-triples *)  
<http://www.w3.org/1999/02/22-rdf-syntax-ns#Kevin>  
  <http://www.w3.org/1999/02/22-rdf-syntax-ns#isa>  
  "programmer" .

The function get-triples takes the following arguments:

s, p, o, g - specify the query pattern. Use nil as a wildcard. Each of these can be a UPI, a future-part, or a string that can be converted into a part. These can also take on the value :minimum if a corresponding ?-end parameter is specified. If :minimum is used, then the range query will run from the smallest value up to the ending value.
s-end, p-end, o-end, g-end - Allows for range queries over encoded triples (triples whose parts are encoded UPIs). Each ?-end parameter may only be used in conjunction with its corresponding starting value parameter. Each of these can be a UPI, a future-part, or a string that can be converted into a part. These can also take on the value :maximum if a corresponding starting parameter is specified. If :maximum is used, then the range query will run from the starting value up to the maximum value in the triple-store.
The :db keyword argument specifies the triple store to query, defaulting to the value of *db*.
filter - if supplied, the filter should be a predicate of one parameter, a triple. If the filter function returns nil, then the triple will not be included in the result set.
include-deleted - if set to true, then triples that are flagged as deleted will not be filtered out by the cursor.
return-encoded-triples - If true, then get-triples returns triples with encoded parts; i.e., triples that use directly encoded UPIs rather than strings stored in the dictionary. The default value is true.
return-non-encoded-triples - if true, then get-triples will return triples all of whose UPIs are stored as strings. This is set to true unless overridden.

The return value is a cursor object. The functions cursor-next-row and cursor-next-p can be used to step through the cursor.

If it can be determined that one of the search parameters is not interned in the triple-store, then get-triples will return a null-cursor (i.e., a cursor that has no rows) and a second value of :part-not-found.

Query a triple store for triples matching the given subject, predicate, object, and graph, specified either as part IDs (UPIs), future-parts, strings in N-Triples format, or the wildcard nil. Returns a list of matching triples. The get-triples-list function supports a multitude of options:

db - This keyword argument specifies the triple store to query, defaulting to the value of *db*.
s, p, o, g - controls the actual query pattern. Use nil as a wildcard.
s-end, p-end, o-end, g-end - Allows for range queries over encoded triples (triples whose parts are encoded UPIs). Each ?-end parameter may only be used in conjunction with its corresponding starting value parameter.
cursor - if cursor is supplied then AllegroGraph will use it to return more triples rather than building a new cursor.
if-fewer - controls the behavior of get-triples-list when fewer than limit triples match the query. Possible values are (defaults to nil):
- nil - return list of actual results
- :error - signal error if fewer results than limit
- other -- return other instead of short list
if-more - controls what happens when more results than limit are available. Possible values are (defaults to :cursor):
- nil - return list of limit results
- :error - signal error if more results than limit
- :cursor - return the list of limit results and a second value of a cursor that will yield the remaining results
- other - return other instead of truncated list
limit - This keyword argument can be used to place a cap on the maximum number of triples returned. It defaults to the value of the special variable get-triples-list-limit. If set, then get-triples-list will return no more than :limit triples. If it is nil, then get-triples-list will return all of the triples found by the query. Warning: setting limit to nil can cause get-triples-list to return every triple in the triple-store; this can be a very bad thing over a serial connection.
return-encoded-triples - If true, then get-triples-list returns triples with encoded parts; i.e., triples that use directly encoded UPIs rather than strings stored in the dictionary. The is set to true unless overridden.
return-non-encoded-triples - if true, then get-triples-list will return triples all of whose UPIs are stored as strings. This is set to true unless overridden.
use-reasoner - Deprecated. If true, then the RDFS++ reasoner will be used to return inferred triples. If left unspecified, this will be true if the db is a reasoning-triple-store and nil otherwise.
Note that most of the arguments to get-triples-list do not make sense when reasoning is turned on. AllegroGraph will signal an error if you try to combine reasoning with other parameters that it cannot use.

If it can be determined that one of the search parameters is not interned in the triple-store, then get-triples-list will return immediately return nil.

Print information about part down to a maximum depth of maximum-depth using the format format. Triples for which part is an object and their children will be printed. See part->string and (ag-property :default-print-triple-format) for information about part printing. See pprint-subject to display information based on objects rather than subjects.

Print information about part down to a maximum depth of maximum-depth using the format format. Triples for which part is a subject and their children will be printed. See part->string and (ag-property :default-print-triple-format) for information about part printing. See pprint-object to display information based on subjects rather than objects.

Returns true if a triple matching s, p and o (and optionally g) can be found in the designated triple-store. If left unspecified, the triple-store designated by *db* will be searched. This is handy when you care only about the presence of a triple and not its contents. If you want to use the triple, then use get-triple instead.

The default number of triples to return from get-triples-list. If nil, then all triples will be returned.

Range Queries

Both get-triples and get-triples-list support fast range queries using AllegroGraph's encoded data types. Though RDF requires that subjects be resources or blank nodes and predicates by resources, AllegroGraph allows you to store and query for encoded data types in any of a triple's fields. ⁵ In each case, to execute a range query, you must supply a starting and ending encoded UPI. You can only make a range query on a single field in each query but you can include other fields which act as additional filters. For example:

> (get-triples :o (value->upi "12:01" :time))  
               :o-end (value->upi "12:59" :time)  
               :p !ex:startTime)

will return a cursor that iterates only all of the triples whose predicate is !ex:startTime and whose object falls between 12:01 and 12:59. ⁶

SPARQL Overview

AllegroGraph includes twinql, an implementation of the powerful SPARQL query language. SPARQL is covered both in this reference guide and in the twinql reference and tutorial. AllegroGraph also includes a SPARQL client and Server.

Prolog Select Overview

With pure Lisp as the retrieval language, you use a combination of functional and procedural approaches to query the database. With Prolog, you can specify queries in a much more declarative manner. Allegro CL Prolog and AllegroGraph work very well together. The reference guide includes more details and the tutorial provides an introduction to using Prolog and AllegroGraph with many complete examples.

RDFS++ Reasoning Overview

See the Reasoner tutorial for more details on using AllegroGraph's RDFS++ reasoner. It works with get-triples, get-triples-list, SPARQL and the Prolog q functor.

Deleting and Undeleting triples

AllegroGraph lets us delete and undelete triples. The function delete-triples deletes triples from the triple store, using the same query syntax as get-triples.

Marks the triple whose id is id as deleted.

See also undelete-triple and undeleted-triple-p.

The :db keyword argument specifies the triple store in which the triple should be marked as deleted.

Delete triples matching the given subject, predicate, object, and graph, specified either as part IDs, strings in N-Triples format, or the wildcard nil. Returns the number of triples deleted.

The :db keyword argument specifies the triple store to query, defaulting to the value of *db*.

Returns a boolean indicated whether the triple whose ID is id is deleted.

The :db keyword argument specifies the triple store in which to check.

Unsets the deleted flag from the triple whose id is id.

The :db keyword argument specifies the triple store in which the triple should be marked as undeleted.

Undelete triples matching the given subject, predicate, object, and graph, specified either as part IDs, strings in N-Triples format, or the wildcard nil. Returns the number of triples undeleted.

The :db keyword argument specifies the triple store to query, defaulting to the value of *db*.

Serializing Triples

There are several methods by which you can create a textual representation of your triple-store:

using print-triples,
using one of the serialization functions like serialize-rdf/xml or serialize-rdf-n3
using the serialization methods in twinql (AllegroGraph's SPARQL sub-system).

The print-triples function provides a simple mechanism to output triples to *standard-output* or a stream. It's easy to build an export function on top of it:

(defun export-triples (triples file)  
  (with-open-file (output file  
                          :direction :output  
                          :if-does-not-exist :create  
                          :if-exists :error)  
    (print-triples triples  
                   :limit nil :stream output :format :ntriple)))

The other techniques provide more control over the output format.

Print a triple returned by cursor-next-row or get-triples-list. The keyword argument :format, which defaults to the value of (ag-property :default-print-triple-format), specifies how the triple should be printed. The value :ntriples specifies that it should be printed in N-Triples syntax. The value :long indicates that the string value of the part should be used. And the value :concise causes it to use a more concise, but possibly ambiguous, human-readable format.

Display the triples in triple-container which can be either a triple store object, a list of triples such as is returned by get-triples-list, or a cursor such as is returned by get-triples. If the keyword argument :limit is supplied, then at most that many triples will be displayed. The :format keyword argument controls how the triples will be displayed, in either :ntriples, :long, or :concise format. It defaults to (ag-property :default-print-triple-format). The stream argument can be used to send output to the stream of your choice. If left unspecified, then output will go to standard-output.

Write from, which should be a triple store, a list, or a cursor, to to, which should be a stream, file path, t (print to *standard-output*), or nil (return a string).

If from is a triple-store or a list of triples, and prepare-namespaces-p is t, it is first traversed to build a hash of namespaces to prefixes for all properties in the store. The value of db.agraph::*namespaces* is used as a seed.

If you can ensure that every property used in the triple-store has a defined prefix, you can pass nil for prepare-namespaces-p to gain a speed improvement from omitting this phase.

If error-on-invalid-p is t, the serializer will throw an error if it encounters a type or predicate that it cannot abbreviate for RDF/XML.

If a namespace prefix table is built, it will be returned as the second value.

If from is a cursor, it cannot be traversed multiple times, so prepare-namespaces-p is ignored. If a property is encountered that cannot be abbreviated with the current available prefixes, an error will be signaled, unless error-on-invalid-p is nil. You should be aware of this before using this serializer: serializing the output of a cursor can fail if you do not first prepare namespace mappings, or specify that invalid output is acceptable.

if-exists and if-does-not-exist are arguments to the Common Lisp open function which apply when to is a filename.

If memoize-abbrev-lookups-p is t, a upi-hash-table is built to store the mappings between resources in the store and string abbreviations. This hash-table will contain as many entries as there are types and properties in the data to be serialized. For some datasets disabling this caching will yield a significant reduction in space usage in exchange for a possible loss of speed.

If indent is non-nil, then it specifies the initial indentation of elements.

If nestp is t, then (subject to the order of triples in the triple store) some nesting of RDF/XML elements will be applied. nestp of nil will cause a flat tree to be produced, where each resource is its own top-level element.

If output-types-p is t, then additional queries will be performed for each resource to decorate the RDF/XML with types. If nil, then rdf:type elements alone will be used.

Please note that the RDF/XML serializer is not guaranteed to work correctly with value-encoded literals.

Serialize source according to exchange-of-named-rdf-graphs. Serialization will probably open at least as many files as there are graphs in the source.

Returns the manifest path and the number of graphs saved.

If single-stream-p only a single file is open at any one time. This is slower, but guaranteed not to fail with large numbers of graphs.

Write from, which should be a triple store, a list, or a cursor, to to, which should be a stream, file path, t (print to standard-output), or nil (return a string). if-exists and if-does-not-exist are arguments to open which apply when to is a filename. If indent is non-nil, then it specifies the initial indentation of elements.

Write source, which should be a triple store, cursor, or list of triples, to output. output must be t (equivalent to *standard-output*), nil (which returns the serialization as a string), a string or pathname, which will be opened for output respecting the if-exists and if-does-not-exist arguments, or a stream.

The resulting output is produced according to the value of constraints. This must be one of the following symbols:

:canonical: each graph appears once in the output. All graphs and triples are sorted lexicographically. All graphs are named, even when a name must be generated.
:collated: each graph appears once in the output.
:lenient (or nil, the default): graphs can appear multiple times in the output. No sorting occurs. load-trix can consume this, but not all other TriX parsers necessarily can.

The value of the default-graph argument applies when the default graph has to be serialized. If it is nil, the default, the AllegroGraph-specific <default/> element is used. If it is non-null, it should be a URI string or a valid part that will be substituted into the output.

Managing a triple-store

These functions are used to create, delete, and examine a triple-store. All of a triple-store's data is kept in a single directory whose name it shares. For convenience, many operations act by default on the current triple-store which is kept in a special variable named *db*. The with-triple-store macro makes it easy to call other code with a particular store marked current. There are also several reports that describe the triple-store in detail.

Each triple-store has two parameters that help you to manage its indices automatically: unindexed-triple-count-threshold and unmerged-chunk-count-threshold. The first controls the maximum number of unindexed triples in the store; the second controls the maximum number of index chunks or fragments. When it ships, each triple-store has both of these parameters set to zero (0) which means that no automatic indexing and merging will take place -- everything is left to your control. Here are some of the things that you will want to consider when setting these values.

Ideally, you would like to have all of your triples indexed into a single index-fragment.
The lower you set unindexed-triple-count-threshold, the fewer unindexed triples you will have but you will also have more index chunks.
Each query must examine each index chunk so having too many will slow down query times.

We suggest that you experiment with variations on the parameters. Franz is very open to hearing your feedback on how you would like to manage your triple-store.

One final point is that the automatic indexing and merging is subject to checks by the AllegroGraph manager process (see manager-period elsewhere in this documentation).

The default triple store instance. API functions that take a db keyword argument default to the value of this variable.

While completely unrelated to AllegroGraph, there is another symbol with the same name exported from the dbi.oracle package. If you try to have a package use both the dbi.oracle and the db.agraph packages, these two symbols will signal a package conflict error. Assuming you want to have the db.agraph symbol to be available without a package qualifier, you should define your package something like this

    (defpackage :my-package  
      (:use :cl :dbi.oracle :triple-store ...)  
      (:shadowing-import-from :triple-store :*db*)  
      ...)

or else execute this form

    (shadowing-import 'db.agraph:*db*)

before using either package.

Close all open triple-stores. The :wait keyword parameter is passed along to close-triple-store. It defaults to nil

Close the triple store, after saving all persistent data to disk as if by sync-triple-store. Close-triple-store has the following keyword arguments:

:db - defaults to the value of *db* and specifies which triple-store to close. *db* is set to nil after the triple store is closed. A triple-store only needs to be closed once regardless of how many times open-triple-store has been called on it.
:if-closed - controls the behavior when the db is either not open or nil. If if-closed is :error then an error will be signaled in this situation. If it is :ignore, then close-triple-store will just return without signaling an error. The argument defaults to :ignore.
:verbose is true, then a message will be printed to *debug-io* before the triple store is closed.
:wait is true then close-triple-store will not return until all of its concurrent activity is complete. This includes synchronizing its data, building indices and running queries. The value of :wait will be true unless overridden. If it is false, then close-triple-store will return immediately but the triple-store will no longer be available to use. Its actual in-memory structures will not be available to the garbage-collector until all of its background activity is complete and it is finally closed.

Create a new triple store with the given name. This is both the name of the triple-store and its location on disk. It can be the full path to a directory or just a directory name. If it is a simple name, then the directory argument is used to convert it to a complete path.

For example, to create a triple-store named 'animals' in the directory 'c:/datafiles/biology', you could use either

(create-triple-store  
 "c:/datafiles/biology/animals")

(create-triple-store "animals"  
 :directory "c:/datafiles/biology/")

The directory defaults to the directory component of *default-pathname-defaults*. You cannot use a name containing a directory path and the directory argument simultaneously.

Create-triple-store takes numerous keyword arguments:

:if-exists - controls what to do if the data directory already exists. The default value, :supersede, will cause create-triple-store to delete the old triple store and create a new one; :error causes create-triple-store to signal an error.
:expected-unique-resources - sets the initial size of the triple-store. It determines the number of unique names (e.g., resources and literals) that the triple store is expected to hold and defaults to the value of default-expected-unique-resources. The triple-store will grow if this number is exceeded but setting the value initially will provide better performance because no resizing will be required.
:with-indices - adds indices to the newly created triple-store. It is equivalent to calling add-index once for each index in the list of indices. It defaults to the value of the AllegroGraph property standard-indices.
:include-standard-parts - if true, then AllegroGraph will add the following strings to the triple-store at creation time: rdf:type, owl:sameAs, owl:inverseOf, rdfs:subPropertyOf, rdfs:subClassOf, rdfs:range, rdfs:domain, owl:transitiveProperty. The value defaults to the ag-property(:include-standard-parts).

Create-triple-store returns a triple-store object and sets the value of the variable *db* to that object.

Navigation

AllegroGraph 3.0 Reference Guide

Table of Contents