Commit 181c92c5 authored by Laurent Wouters's avatar Laurent Wouters
Browse files

[doc] Configuration for readthedocs.org

parent 8be54e42a561
......@@ -2,5 +2,7 @@
"files.watcherExclude": {
".hg/**": true,
"**/node_modules/**": true
}
},
"python.pythonPath": "/usr/bin/python",
"restructuredtext.confPath": "${workspaceFolder}"
}
\ No newline at end of file
.. _label-top:
============================
CubicWeb Linked Data Browser
============================
The toolkit for the production of a browser of the web of data.
.. toctree::
:maxdepth: 2
docs/Repository
docs/ServerLDCompliance
docs/WritingNewViews
from recommonmark.parser import CommonMarkParser
source_parsers = {
'.md': CommonMarkParser,
}
source_suffix = ['.rst', '.md']
master_doc = 'docs/README'
\ No newline at end of file
source_suffix = ['.rst']
master_doc = 'INDEX'
project = 'CubicWeb Linked Data Browser'
author = 'Laurent Wouters <lwouters@cenotelie.fr>'
copyright = 'Copyright 2003-2018 LOGILAB S.A. (Paris, FRANCE), all rights reserved'
\ No newline at end of file
==========
Repository
==========
--------------------
Repository structure
--------------------
This repository contains multiple packages. It is structured as follow:
- `libview` contains the sources for the NPM package `@logilab/libview`.
This package defines the common interface that has to be implemented
by views for the web of data browser.
- `extension` contains the sources for the Chrome and Firefox web
extension (`@logilab/ld-browser`) that really implements the browser
for the web of data. To use the browser, this extension has to be
deployed in a regular web browser.
- `views-*` contain the sources for specialized views that can be
leveraged by the web of data browser.
They mostly take the form of NPM projects.
- `views-logilab` contains the sources for the Person and Book views.
----------------
General concepts
----------------
The general idea for this browser of the web of data is to provide uniform
views for data regardless of their providers.
Views could be specialized according to the types of data by should be
uniform across providers.
In this project, the browser for the web of data is implemented as an
extension to a web browser.
The views themselves are designed to be defined externally and can be
externally defined and dynamically loaded by the browser so that it is
possible to contribute new views without having to modify the browser.
A view itself is defined by two artifacts: a descriptor (JSON object)
and an implementation (Javascript module).
For the browser to detect and use the view, a corresponding source has
to be added in the browser's configuration, pointing to the location of
the descriptor.
The descriptor then points to the location of the Javascript file containing
the implementation that can be dynamically loaded by the browser.
------------
How to build
------------
This project uses a build environment defined as a Docker image in the
`.releng` folder.
You will need Docker to build.
If not already present, the Docker image for the build environment will
be automatically built, by executing `./.releng/build-env.sh`.
You can manually run this command if you wish to refresh the build
environment.
To build the project, run:
.. code:: bash
./build.sh
Alternatively, to locally build the project without using the provided
environment, run:
.. code:: bash
./build-src.sh
Doing so, the minimal working versions of node.js and npm are:
* `node --version`: `v8.10.0`
* `npm --version`: `3.5.2`
---------------------------
Access to build environment
---------------------------
The build environment can be invoked using the `cmd` script at the
repository's root by passing the wanted command as a parameter to it.
The sources repository is mapped to the `/src` folder in the build
environment.
For example, to execute the `build-src.sh` script in the context of
the specified build environment (instead of the local host):
.. code:: bash
./cmd /src/build-src.sh
The `cmd` script tries to lookup the current folder relative to the
repository's root in order to execute the requested command within
the same folder.
It is then possible to execute:
.. code:: bash
cd extension
../cmd npm install
-----------
Run linters
-----------
Linters for the components are called during the normal process.
It is also possible to manually invoke the configured linter as follow:
.. code:: bash
cd extension
../cmd npm run linter
----------
How to use
----------
This project contains a set of contributed views that must be served
(by a web server) in order to be used.
In order to use this project, the definition of a runtime environment
is provided in the form of a `docker-compose.yml` file.
Once the project has been built, simply execute:
.. code:: bash
docker-compose up -d
Run `docker-compose down` to terminate the environment.
This environment launches a web server (Apache), accessible at
`http://localhost/ <http://localhost/>`_
, that serves the contributed views.
The descriptors for the views can then be accessed at
`http://localhost/index.vd.json <http://localhost/index.vd.json>`_.
This is the URI that can be defined as a source of views for the browser.
Then, the web extension implemented in the `extension` repository has
to be deployed into a compatible web browser such as Firefox or Chrome.
The web extension is made into the `build` directory and can be deployed
as an unpacked extension in Chrome or Firefox for debuggin pruposes.
In Chrome, go to `chrome://extensions/ <chrome://extensions/>`_
and activate the developer mode (toggle un upper-right corner).
Then click on the appearing `Load Unpacked` button and select the
`build/` directory in this repository (after build).
.. image:: https://developer.chrome.com/static/images/get_started/load_extension.png
In Firefox, go to `about:debugging <about:debugging>`_, click
"Load Temporary Add-on" and open the extension's directory and select
the `build/manifest.json` file. Details at
`Mozilla MDN <https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Temporary_Installation_in_Firefox>`_.
Once this is done, go to a web page with data attached to it,
such as `dbpedia Claude Debussy <http://dbpedia.org/page/Claude_Debussy>`_.
The extension icon shall lit up whenever data has been detected for the page.
To register custom views, click on the extension's icon, then on the gear
in the appearing popup.
There should be two fields available to add add a new source of views.
Simply enters the name (can be anything) for the source and the targer URI,
for example `http://localhost:8080/index.vd.json <http://localhost:8080/index.vd.json>`_.
Then validate to register the source.
The corresponding views should then be automatically loaded and used
when appropriate.
---------------------
How can I contribute?
---------------------
The simplest way to contribute is to:
- Fork this repository.
- Fix `some issue <https://www.logilab.org/project/cweb>`_ or implement a new feature.
- Create a pull request and subimit it.
-------
License
-------
This software is licenced under the Lesser General Public License (LGPL) v3.
Refers to the `LICENSE.md` file at the root of the repository for the
full text,
or to `the online version <http://www.gnu.org/licenses/lgpl-3.0.html>`_.
----------------
Acknowledgements
----------------
This work is financed by `Logilab <https://www.logilab.fr/>`_, France.
Copyright 2003-2018 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
=============================
Linked Data Server Compliance
=============================
The :ref:`label-top` is a browser to navigate the web of data.
The point of this document is not to completely explain the web of data;
but it is important to important to understand one of its underlying promise:
Namely that data or resources can be referred to by URIs and that by connecting
to those URIs, interpreted as locations, the data associated with the URIs can
be retrieved, potentially through content negotiation.
For example, a RDF resource (file) refers to entities identified by URIs.
The promise is then that the data(sets) about these entities can be fetched at
the URIs.
Therefore, in the same way that a browser for the web of documents can navigate
from pages to pages through links using URLs,
a browser for the web of (linked) data could navigate within and between
datasets through the URIs of the entities referred to and find new datasets
about them.
For this ideal scenario, the web servers that in the end serve the content,
in this case the data, should *play nice*, i.e. be compliant with the web
of data. In this context, this mainly mean that when asked about an URI in
its scope, a web server should answer with data about the requested URI,
at least if explicitely asked to using HTTP headers related to content
negotiation. For example, a compliant server of linked data handling resources
for `example.com` should readily (or through content negotiation) answers
with the data related to
`http://example.com/some_resource`.
Example of compliance issues:
- The server always answers HTML, or redirect to an HTML page,
despite content negotiation requesting RDF data.
- Possible fix: Rely on content negotiation to answer with RDF content.
HTML can still be returned by default.
- Instead of replying with data, the server redirects to `http://example.com/some_resource.rdf`.
- Possible dix: Do not redirect to different URIs depending on RDF syntaxes
(.n3, .nt, .rdf, etc.) but directly return the content in the appropriate
syntax, as requested through content negotiation.
- The datasets returned by the server make use of multiple writings for the
same logical entity, for example using unicode escape sequences `\uxxxx`
in URIs.
- Possible fix: Try to canonicalize data to a single URI writing.
- Multiple datasets have been located for different writing of the same
URI and they have different content.
- The provided datasets use language specific URIs with different data
for different languages.
- Possible fix: Unify the datasets in a language-neutral naming scheme
for the entities and use language-tag on language-specific RDF literals.
The :ref:`label-top` still tries to cooperate with servers that are
not already compliant. It does the following to try to detect datasets:
- Simple content negotiation by requesting RDF data at the URI for the
resource (case for compliant servers).
- Follow redirections to URIs for datasets serialized in specific syntaxes.
- Try to detect datasets linked to from HTTP `Link` header.
- When HTML was obtained, try to detect datasets linked to from HTML
`Link` headers.
- When HTML was obtained, try to detect links to corresponding datasets,
i.e. links starting with the URI of the requested resource and ending
with a file extension for a known RDF syntax (.n3, .nt, .rdf, etc.).
==================================================
Writing new views for CubicWeb Linked Data Browser
==================================================
The :ref:`label-top` relies on views to render RDF datasets.
By default, it only provides a single view that simply lists the
RDF triples from the loaded dataset.
All other views are user-defined and dynamically discovered and
loaded by the browser.
This document explains the basics of writing new views.
At its core, a view is simply a piece of Javascipt code (or any language
that compiles to it) that implements a simple interface.
There are a few other requirements, as listed below:
- The view implementation must conform to the
`ViewImplementation` interface.
- The Javascript resource containing the implementation must contain
an entrypoint for the browser.
- The view implementation must be referred to by a endpoint serving
a descriptor of the implementation.
This document go through all these requirements and explain
how to fulfill them. In the end, the implementation of a view does
not require any special behavior from the server it.
They are simply static files.
---------------------------------------
Step 1 - Implement `ViewImplementation`
---------------------------------------
A library has been designed to help at the implementation of views.
It is available on npm as `@logilab/libview`.
This library is also used by the browser and therefore provides
all the necessary APIs and interfaces.
First, add this library as a dependency in your project:
.. code:: bash
npm install @logilab/libview
The next step is to start the implementation of the main
interface to provide: `ViewImplementation`.
This interface is located in the `implementation` namespace within
`@logilab/libview`. A good starting point is (in Typescript):
.. code:: typescript
import { application, definition, implementation } from "@logilab/libview";
import * as $rdf from "rdflib";
class MyView implements implementation.ViewImplementation {
descriptor: definition.ViewDescriptor;
constructor() {
this.descriptor = ...;
}
priorityFor(store: $rdf.Formula, target: application.Resource): number {
return 0;
}
render(
renderer: application.ViewRenderer,
context: application.RenderingContext,
target: application.Resource
): implementation.ViewRendering {
return ...;
}
}
This interface is basically three members:
- `descriptor` is a public attribute that must contain
the descriptor (metadata for the view).
- `priorityFor` is a method that, given a RDF dataset and a target resource
(URI) to be rendered, must return the self-evaluated priority of the view.
This value is used by the browser to select an appropriate view for the
dataset. A higher (positive) number indicates a higher priority.
A negative number can be return to indicate that the view *must not* be
selected by the browser. As of this writing, the browser primarily relies
on these self-evaluation by the views, unless the user explicitely select
a particular view.
- `render` is the single entrypoint for the view implementation used by the
browser. Given a rendered, a contact and a target resource (URI) to be
endered, it must return a rendering that may or may not include an HTML
DOM sub-tree.
The basic idea for a view implementation is for the browser to specify the RDF
dataset and the main resource to render and for the view to provide an HTML DOM tree.
How the DOM tree is produced is left to the discretion of the view implementation.
Any framework may be used, such as React, Angular, Vue, etc.; or no framework at all.
Descriptor
==========
The view descriptor is an object that conform the `ViewDescriptor` interface.
It *must* have the following fields:
- `identifier`, a unique identifier for the view.
- `name`, a short human-readable name for the view.
- `description`, a more lengthier description for the view. It may expands of
the kind of RDF data that the view is able to handle.
- `entrypoint`, the name of an exported variable that contains an instance of
the view implemention. See Step 2.
- `resourceCss`, an array of potentially remote CSS resources that should be
loaded by the browser for this view.
- `resourceJs`, an array of potentially remote additional Javascript resources
that should be loaded by the browser for this view.
- `resourceMain`, points to the potentially remote Javascript resource that
contains the view implementation.
Example:
.. code:: typescript
const MY_VIEW_DESCRIPTOR: definition.ViewDescriptor = {
identifier: "::SeriousBusiness::MyView",
name: "A short human-readable name",
description:
"A lengthier description of this view. What kind of RDF data does it handle?",
entrypoint: "VIEW_ENTRYPOINT", // name of the entrypoint object
resourceCss: [
{
location: "remote",
uri:
"https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css"
}
],
resourceJs: [],
resourceMain: {
location: "remote",
uri: "http://views.example.com/myview.js"
}
};
Priority evaluation
===================
When computing the priority of a view, the raw and complete RDF dataset is
passed to the `priorityFor` method of the view implementation, along with
the target of rendering, i.e. the primary entity to be rendered by this view.
It is the responsability of the view implementation to determine its priority
regarding the dataset and the target entity.
For example, a view specific to books *should* try to detect the type of the
target resource and *should* deactivate itself if it does not look like a book.
In the case the view desires to deactivate itself, the
`VIEW_PRIORITY_INAPPROPRIATE` constant can be returned.
The strategy to be used to determine whether the view is appropriate and what
is its priority is left to the view itself.
A common strategy is to look for the triples with the `rdf:type` predicate on
triples with the target entity as subject.
For example:
.. code:: typescript
priorityFor(store: $rdf.Formula, target: application.Resource): number {
// use an entity repository for easier manipulation of the RDF dataset
let repo = new rdfEntities.RdfEntityStore(store);
// get the entity identified as the target
let entity = repo.entityForUri(target.uri);
// determine whether the target is really is target, or a proxy of another entity with a focus predicate
// this step may not be applicable to all use cases
let focus = entity.getEntityForS("http://xmlns.com/foaf/0.1/focus");
if (focus !== null) entity = focus;
// list all the types of the entity (using rdf:type)
let entityTypes = entity.getEntitiesForS("http://www.w3.org/1999/02/22-rdf-syntax-ns#type");
if (entityTypes.length > 0) {
// look for a specific entity type
let x = entityTypes.find(
(entity: rdfEntities.RdfEntity) =>
entity.uris.indexOf("http://example.com/Book") >= 0
);
if (x) {
// The entity is a Book
return 10;
}
}
// could not find the types, the view may not be appropriate
return implementation.VIEW_PRIORITY_INAPPROPRIATE;
}
Rendering
=========
Once a view has been selected, it can be rendered by the browser,
by calling the `render` method on the view implementation.
This method takes 3 parameters:
- `renderer`, an object that can be used to interact
with the rendering process.
- `context`, an object that holds all the contextual
information for this call.
- `target`, the target RDF resource (URI) to be used
as the primary entity to be rendered.
The renderer, of type `ViewRenderer` makes available a few methods
for the view implementation to interact with the rendering process.
.. code:: typescript
/**
* A renderer for an application
*/
interface ViewRenderer {
/**
* Selects the languages to use for a target resource
* @param context The current rendering context
* @param target The target resource for this view
*/
getLanguagesFor(context: RenderingContext, target: Resource): Language[];
/**
* Renders a resource belonging to an RDF store
* @param context The context for the rendering
* @param target The target resource for this view
*/
render(context: RenderingContext, target: Resource): RenderingResult;
}
- The `getLanguagesFor` method can be used to compute the list of languages,
from the highest priority down, that should be used for a specific RDF
resource.
The methods takes into account the user preferences in the form of the
rendering context.
- Moreover, the `render` method can be used by a view implementation to
recursively perform the rendering of (probably another) RDF resource,
so that it can be included into its own rendering.
This mechanism can be used by a view to include snippets about other
resources as rendered by other view implementations.
The rendering context, of type `RenderingContext`, holds various data
related to the rendering.
.. code:: typescript
/**
* Represents the context of a rendering
*/
interface RenderingContext {
/**
* The RDF store holding the data to be rendered
*/
store: $rdf.Formula;
/**
* The root resource
*/
root: Resource;
/**
* The current rendering options
*/
options: RenderingOptions;
/**
* The handler of user events
*/
event: UserEvents;
/**
* The language used by the browser
*/
browserLanguage: Language;
}
- Most notable, the `store` field gives access to the current RDF dataset.
- The `root` field is set to the root RDF resource that is being rendered.
- The `options` field is a dictionary that holds the preferences
(view, language, etc.) selected by the user for various RDF resources.
- The `browserLanguage` field simply holds the language that has been
detected for the browser.
- The `events` field contains an object that holds methods that can be
invoked for certain events related to the user interection with the
rendered view.
The `onRequestNavigateTo` method can be used to trigger the navigation
of the browser to a RDF resource when a user click on a link to another
resource in the rendered view.
The `onSelectAsPrimaryTopic` method can be used to update the primary
topic (root resource) of the current dataset.
This method is notably used by the built-in RDF triples view to enable
the user to select the primary topic.
In the end, the view implementation *must* produce a `RenderingResult`
object to be returned.
.. code:: typescript
/**
* Represents the rendering result of a resource
*/
interface RenderingResult {
/**
* The DOM elements representing the rendering
*/
dom: HTMLElement | null;
/**
* The identifier of the selected view
*/
viewId: string;
/**
* The URI of resources that could be of interest to the view that produced this rendering
*/
suggestedResources: Resource[];
/**
* The evaluations of the possible views
*/
evaluations: RenderingEvaluation[];
/**
* The used languages
*/
languages: Language[];
}
This object contains the rendering itself as a DOM tree with the field
`dom`, as well as some metadata.
Note that returning a rendering object without a DOM tree can be used
to indicate a failure to render.
The second field of interest is `suggestedResources`.
This is a list of RDF resources that have been identifier of importance
by the view implementation (probably because they are linked to by
the resource being rendered).
They can be used to indicate of resource of interest to the browser
so that it can be loaded as well.
An example of use case is that the browser asks to render the resource A.
Resource A links to resource B, but the data about B is not in the same dataset.
The view implementation can suggest B so that data about it can be loaded.
When the browser finally loads data about B, a second rendering pass is invoked.
This time the view implementation can use the data about B, for example to render a user-friendly label instead of just an URI.
The `RenderingResult` object possesses other fields, such as the identifier of the view implementation that produced the rendering.
Those are automatically re-written by the renderer (so that they cannot be faked) ans therefore can be safely let empty.
Facilities to manipulate RDF data
=================================
Although view implements can rely on the raw dataset for rendering, `libview`
also provides a few abstraction layers that can ease the manipulation of the