Skip to content

GitLab

Commit 5daa2689 authored by Simon Chabot's avatar Simon Chabot
Browse files

Remove everything but libview source code

parent c1a8e1f96446
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 12757 deletions
.releng/Dockerfile deleted 100644 → 0
View file @ c1a8e1f9
FROM buildpack-deps:18.04-curl
LABEL maintainer="Laurent Wouters <lwouters@cenotelie.fr>" vendor="Logilab SA" description="Build environment for the CubicWeb Linked Data Browser"
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash -
RUN apt-get update && apt-get dist-upgrade -y && apt-get install -y --no-install-recommends \
build-essential zip mercurial nodejs
RUN mkdir /npm
RUN chmod -R a+rwX /npm
\ No newline at end of file
.releng/README.md deleted 100644 → 0
View file @ c1a8e1f9
## Release Engineering ##
This folder contains the definition of the build environment for this project as a Docker image.
* `build-env.sh`: Script to build the Docker image that contains the build environment
* `build-src.sh`: Script to build the application sources. It is meant to be called from within the build environment.
.releng/build-env.sh deleted 100755 → 0
View file @ c1a8e1f9
#!/bin/sh
# Build the build environment as a Docker image
SCRIPT="$(readlink -f "$0")"
RELENG="$(dirname "$SCRIPT")"
ROOT="$(dirname "$RELENG")"
VERSION=$(hg -R "$ROOT" id -n)
HASH=$(hg -R "$ROOT" --debug id -i)
# Build the new image
docker build --tag "logilab/ld-build-env:latest" --rm --label changeset="$HASH" "$RELENG"
.vscode/settings.json deleted 100644 → 0
View file @ c1a8e1f9
{
"files.watcherExclude": {
".hg/**": true,
"**/node_modules/**": true
},
"python.pythonPath": "/usr/bin/python",
"restructuredtext.confPath": "${workspaceFolder}"
}
\ No newline at end of file
README.md deleted 100644 → 0
View file @ c1a8e1f9
# CubicWeb Linked Data Browser
The toolkit for the production of a browser of the web of data.
## Repository structure
This repository contains multiple packages. It is structured as follow:
- `libview` contains the sources for the NPM package `@logilab/libview`.
This package defines the common interface that has to be implemented by views for the web of data browser.
- `extension` contains the sources for the Chrome and Firefox web extension (`@logilab/ld-browser`) that really implements the browser for the web of data.
To use the browser, this extension has to be deployed in a regular web browser.
- `views-*` contain the sources for specialized views that can be leveraged by the web of data browser.
They mostly take the form of NPM projects.
- `views-logilab` contains the sources for the Person and Book views.
## General concepts
The general idea for this browser of the web of data is to provide uniform views for data regardless of their providers.
Views could be specialized according to the types of data by should be uniform across providers.
In this project, the browser for the web of data is implemented as an extension to a web browser.
The views themselves are designed to be defined externally and can be externally defined and dynamically loaded by the browser so that it is possible to contribute new views without having to modify the browser.
A view itself is defined by two artifacts: a descriptor (JSON object) and an implementation (Javascript module).
For the browser to detect and use the view, a corresponding source has to be added in the browser's configuration, pointing to the location of the descriptor.
The descriptor then points to the location of the Javascript file containing the implementation that can be dynamically loaded by the browser.
## How to build
This project uses a build environment defined as a Docker image in the `.releng` folder.
You will need Docker to build.
If not already present, the Docker image for the build environment will be automatically built, by executing `./.releng/build-env.sh`.
You can manually run this command if you wish to refresh the build environment.
To build the project, run:
```sh
./build.sh
```
Alternatively, to locally build the project without using the provided environment, run:
```sh
./build-src.sh
```
Doing so, the minimal working versions of node.js and npm are:
* `node --version`: `v8.10.0`
* `npm --version`: `3.5.2`
## Access to build environment
The build environment can be invoked using the `cmd` script at the repository's root by passing the wanted command as a parameter to it.
The sources repository is mapped to the `/src` folder in the build environment.
For example, to execute the `build-src.sh` script in the context of the specified build environment (instead of the local host):
```sh
./cmd /src/build-src.sh
```
The `cmd` script tries to lookup the current folder relative to the repository's root in order to execute the requested command within the same folder.
It is then possible to execute:
```sh
cd extension
../cmd npm install
```
## Run linters
Linters for the components are called during the normal process.
It is also possible to manually invoke the configured linter as follow:
```sh
cd extension
../cmd npm run linter
```
## How to use
This project contains a set of contributed views that must be served (by a web server) in order to be used.
In order to use this project, the definition of a runtime environment is provided in the form of a `docker-compose.yml` file.
Once the project has been built, simply execute:
```sh
docker-compose up -d
```
Run `docker-compose down` to terminate the environment.
This environment launches a web server (Apache), accessible at [http://localhost/](http://localhost/), that serves the contributed views.
The descriptors for the views can then be accessed at [http://localhost/index.vd.json](http://localhost/index.vd.json).
This is the URI that can be defined as a source of views for the browser.
Then, the web extension implemented in the `extension` repository has to be deployed into a compatible web browser such as Firefox or Chrome.
The web extension is made into the `build` directory and can be deployed as an unpacked extension in Chrome or Firefox for debuggin pruposes.
In Chrome, go to [chrome://extensions/](chrome://extensions/) and activate the developer mode (toggle un upper-right corner).
Then click on the appearing `Load Unpacked` button and select the `build/` directory in this repository (after build).
![Chrome developer mode](https://developer.chrome.com/static/images/get_started/load_extension.png)
In Firefox, go to [about:debugging](about:debugging), click "Load Temporary Add-on" and open the extension's directory and select the `build/manifest.json` file. Details at [Mozilla MDN](https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Temporary_Installation_in_Firefox).
Once this is done, go to a web page with data attached to it, such as [dbpedia Claude Debussy](http://dbpedia.org/page/Claude_Debussy).
The extension icon shall lit up whenever data has been detected for the page.
To register custom views, click on the extension's icon, then on the gear in the appearing popup.
There should be two fields available to add add a new source of views.
Simply enters the name (can be anything) for the source and the targer URI, for example [http://localhost:8080/index.vd.json](http://localhost:8080/index.vd.json).
Then validate to register the source.
The corresponding views should then be automatically loaded and used when appropriate.
## How can I contribute?
The simplest way to contribute is to:
- Fork this repository.
- Fix [some issue](https://www.logilab.org/project/cweb) or implement a new feature.
- Create a pull request and subimit it.
## License
This software is licenced under the Lesser General Public License (LGPL) v3.
Refers to the `LICENSE.md` file at the root of the repository for the full text, or to [the online version](http://www.gnu.org/licenses/lgpl-3.0.html).
## Acknowledgements
This work is financed by [Logilab](https://www.logilab.fr/), France.
Copyright 2003-2018 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
bitbucket-pipelines.yml deleted 100644 → 0
View file @ c1a8e1f9
image: nexus.cenotelie.fr/logilab/ld-build-env:latest
pipelines:
default:
- step:
name: Build and test
caches:
- npmextension
- npmlibview
- npmviews
script:
- ./build-src.sh
artifacts:
- extension/build/ld-browser-extension.zip
- libview/build/**
- views-logilab/build/**
definitions:
caches:
npmextension: extension/node_modules
npmlibview: libview/node_modules
npmviews: views-logilab/node_modules
build-src.sh deleted 100755 → 0
View file @ c1a8e1f9
#!/bin/sh
# Build the sources
SCRIPT="$(readlink -f "$0")"
ROOT="$(dirname "$SCRIPT")"
VERSION=$(hg -R "$ROOT" id -n)
HASH=$(hg -R "$ROOT" --debug id -i)
# Cleanup
hg --config extensions.purge= -R "$ROOT" purge --all
echo ""
echo "=========================================="
echo "= Building @logilab/libview"
echo "=========================================="
(cd "$ROOT/libview"; rm -rf build)
(cd "$ROOT/libview"; npm install)
(cd "$ROOT/libview"; npm run build)
(cd "$ROOT/libview"; npm run test)
echo ""
echo "=========================================="
echo "= Building @logilab/views-logilab"
echo "=========================================="
(cd "$ROOT/views-logilab"; rm -rf build)
(cd "$ROOT/views-logilab"; rm -rf node_modules/@logilab/libview)
(cd "$ROOT/views-logilab"; npm install)
(cd "$ROOT/views-logilab"; npm run build)
# (cd "$ROOT/views-logilab"; npm run test)
echo ""
echo "=========================================="
echo "= Building @logilab/ld-browser"
echo "=========================================="
(cd "$ROOT/extension"; rm -rf build)
(cd "$ROOT/extension"; rm -rf node_modules/@logilab/libview)
(cd "$ROOT/extension"; npm install)
(cd "$ROOT/extension"; npm run build)
(cd "$ROOT/extension"; npm run test)
echo ""
echo "=========================================="
echo "= Packaging @logilab/ld-browser"
echo "=========================================="
(cd "$ROOT/extension/build"; zip -r -FS ld-browser-extension.zip *)
\ No newline at end of file
build.sh deleted 100755 → 0
View file @ c1a8e1f9
#!/bin/sh
SCRIPT="$(readlink -f "$0")"
ROOT="$(dirname "$SCRIPT")"
# Execute the build
"$ROOT/cmd" /src/build-src.sh
# Build the new image
docker build --tag "logilab/ld-demo-views:latest" --rm --label changeset="$HASH" "$ROOT/views-logilab"
cmd deleted 100755 → 0
View file @ c1a8e1f9
#!/bin/sh
SCRIPT="$(readlink -f "$0")"
ROOT="$(dirname "$SCRIPT")"
BUILDER=$(docker images | grep -o -E '^logilab/ld-build-env(\s)+latest' | wc -l)
if [ "$BUILDER" -lt "1" ]; then
echo "=> Build environment must be created"
"$ROOT/.releng/build-env.sh"
fi
CD=$(pwd)
WD="/src/"
case "$CD" in
"$ROOT"* )
if [ "$CD" != "$ROOT" ]; then
S=$((${#ROOT} + 1))
SUB=$(echo "$CD" | cut -c "$S-${#CD}")
WD="/src$SUB"
fi
;;
esac
docker run -it --rm --user=$(id -u) -v "$ROOT:/src" -e "HOME=/npm" -w "$WD" -p 8080:8080 "logilab/ld-build-env:latest" $@
conf.py deleted 100755 → 0
View file @ c1a8e1f9
#!/usr/bin/python
# -*- coding: UTF-8 -*-
from subprocess import call
exclude_patterns = ['**/node_modules/**']
source_suffix = ['.rst']
html_static_path = ['libview/build/docs']
master_doc = 'index'
project = 'CubicWeb Linked Data Browser'
author = 'Laurent Wouters <lwouters@cenotelie.fr>'
copyright = 'Copyright 2003-2018 LOGILAB S.A. (Paris, FRANCE), all rights reserved'
call(["npm", "install", "--prefix", "libview"])
call(["npm", "run", "doc", "--prefix", "libview"])
docker-compose.yml deleted 100644 → 0
View file @ c1a8e1f9
version: '3'
services:
views-logilab:
build:
context: ./views-logilab
image: logilab/ld-demo-views:latest
hostname: views
restart: unless-stopped
ports:
- "80:80"
environment:
- HOSTPORT=localhost
\ No newline at end of file
docs/README.md deleted 100644 → 0
View file @ c1a8e1f9
# CubicWeb Linked Data Browser
This is the developer-oriented documentation for the CubicWeb Linked Data Browser.
* [Writing new views](WritingNewViews.md)
* [Linked data server compliance](ServerLDCompliance.md)
docs/Repository.rst deleted 100644 → 0
View file @ c1a8e1f9
==========
Repository
==========
--------------------
Repository structure
--------------------
This repository contains multiple packages. It is structured as follow:
- `libview` contains the sources for the NPM package `@logilab/libview`.
This package defines the common interface that has to be implemented
by views for the web of data browser.
- `extension` contains the sources for the Chrome and Firefox web
extension (`@logilab/ld-browser`) that really implements the browser
for the web of data. To use the browser, this extension has to be
deployed in a regular web browser.
- `views-*` contain the sources for specialized views that can be
leveraged by the web of data browser.
They mostly take the form of NPM projects.
- `views-logilab` contains the sources for the Person and Book views.
----------------
General concepts
----------------
The general idea for this browser of the web of data is to provide uniform
views for data regardless of their providers.
Views could be specialized according to the types of data by should be
uniform across providers.
In this project, the browser for the web of data is implemented as an
extension to a web browser.
The views themselves are designed to be defined externally and can be
externally defined and dynamically loaded by the browser so that it is
possible to contribute new views without having to modify the browser.
A view itself is defined by two artifacts: a descriptor (JSON object)
and an implementation (Javascript module).
For the browser to detect and use the view, a corresponding source has
to be added in the browser's configuration, pointing to the location of
the descriptor.
The descriptor then points to the location of the Javascript file containing
the implementation that can be dynamically loaded by the browser.
------------
How to build
------------
This project uses a build environment defined as a Docker image in the
`.releng` folder.
You will need Docker to build.
If not already present, the Docker image for the build environment will
be automatically built, by executing `./.releng/build-env.sh`.
You can manually run this command if you wish to refresh the build
environment.
To build the project, run:
.. code:: bash
./build.sh
Alternatively, to locally build the project without using the provided
environment, run:
.. code:: bash
./build-src.sh
Doing so, the minimal working versions of node.js and npm are:
* `node --version`: `v8.10.0`
* `npm --version`: `3.5.2`
---------------------------
Access to build environment
---------------------------
The build environment can be invoked using the `cmd` script at the
repository's root by passing the wanted command as a parameter to it.
The sources repository is mapped to the `/src` folder in the build
environment.
For example, to execute the `build-src.sh` script in the context of
the specified build environment (instead of the local host):
.. code:: bash
./cmd /src/build-src.sh
The `cmd` script tries to lookup the current folder relative to the
repository's root in order to execute the requested command within
the same folder.
It is then possible to execute:
.. code:: bash
cd extension
../cmd npm install
-----------
Run linters
-----------
Linters for the components are called during the normal process.
It is also possible to manually invoke the configured linter as follow:
.. code:: bash
cd extension
../cmd npm run linter
----------
How to use
----------
This project contains a set of contributed views that must be served
(by a web server) in order to be used.
In order to use this project, the definition of a runtime environment
is provided in the form of a `docker-compose.yml` file.
Once the project has been built, simply execute:
.. code:: bash
docker-compose up -d
Run `docker-compose down` to terminate the environment.
This environment launches a web server (Apache), accessible at
`http://localhost/ <http://localhost/>`_
, that serves the contributed views.
The descriptors for the views can then be accessed at
`http://localhost/index.vd.json <http://localhost/index.vd.json>`_.
This is the URI that can be defined as a source of views for the browser.
Then, the web extension implemented in the `extension` repository has
to be deployed into a compatible web browser such as Firefox or Chrome.
The web extension is made into the `build` directory and can be deployed
as an unpacked extension in Chrome or Firefox for debuggin pruposes.
In Chrome, go to `chrome://extensions/ <chrome://extensions/>`_
and activate the developer mode (toggle un upper-right corner).
Then click on the appearing `Load Unpacked` button and select the
`build/` directory in this repository (after build).
.. image:: https://developer.chrome.com/static/images/get_started/load_extension.png
In Firefox, go to `about:debugging <about:debugging>`_, click
"Load Temporary Add-on" and open the extension's directory and select
the `build/manifest.json` file. Details at
`Mozilla MDN <https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Temporary_Installation_in_Firefox>`_.
Once this is done, go to a web page with data attached to it,
such as `dbpedia Claude Debussy <http://dbpedia.org/page/Claude_Debussy>`_.
The extension icon shall lit up whenever data has been detected for the page.
To register custom views, click on the extension's icon, then on the gear
in the appearing popup.
There should be two fields available to add add a new source of views.
Simply enters the name (can be anything) for the source and the targer URI,
for example `http://localhost:8080/index.vd.json <http://localhost:8080/index.vd.json>`_.
Then validate to register the source.
The corresponding views should then be automatically loaded and used
when appropriate.
---------------------
How can I contribute?
---------------------
The simplest way to contribute is to:
- Fork this repository.
- Fix `some issue <https://www.logilab.org/project/cweb>`_ or implement a new feature.
- Create a pull request and subimit it.
-------
License
-------
This software is licenced under the Lesser General Public License (LGPL) v3.
Refers to the `LICENSE.md` file at the root of the repository for the
full text,
or to `the online version <http://www.gnu.org/licenses/lgpl-3.0.html>`_.
----------------
Acknowledgements
----------------
This work is financed by `Logilab <https://www.logilab.fr/>`_, France.
Copyright 2003-2018 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
docs/ServerLDCompliance.md deleted 100644 → 0
View file @ c1a8e1f9
# Linked Data Server Compliance
The [CubicWeb Linked Data Browser]() is a browser to navigate the web of data.
The point of this document is not to completely explain the web of data; but it is important to important to understand one of its underlying promise:
Namely that data or resources can be referred to by URIs and that by connecting to those URIs, interpreted as locations, the data associated with the URIs can be retrieved, potentially through content negotiation.
For example, a RDF resource (file) refers to entities identified by URIs.
The promise is then that the data(sets) about these entities can be fetched at the URIs.
Therefore, in the same way that a browser for the web of documents can navigate from pages to pages through links using URLs,
a browser for the web of (linked) data could navigate within and between datasets through the URIs of the entities referred to and find new datasets about them.
For this ideal scenario, the web servers that in the end serve the content, in this case the data, should _play nice_, i.e. be compliant with the web of data.
In this context, this mainly mean that when asked about an URI in its scope, a web server should answer with data about the requested URI, at least if explicitely asked to using HTTP headers related to content negotiation.
For example, a compliant server of linked data handling resources for `example.com` should readily (or through content negotiation) answers with the data related to `http://example.com/some_resource`.
Example of compliance issues:
- The server always answers HTML, or redirect to an HTML page, despite content negotiation requesting RDF data.
- Possible fix: Rely on content negotiation to answer with RDF content. HTML can still be returned by default.
- Instead of replying with data, the server redirects to `http://example.com/some_resource.rdf`.
- Possible dix: Do not redirect to different URIs depending on RDF syntaxes (.n3, .nt, .rdf, etc.) but directly return the content in the appropriate syntax, as requested through content negotiation.
- The datasets returned by the server make use of multiple writings for the same logical entity, for example using unicode escape sequences `\uxxxx` in URIs.
- Possible fix: Try to canonicalize data to a single URI writing.
- Multiple datasets have been located for different writing of the same URI and they have different content.
- The provided datasets use language specific URIs with different data for different languages.
- Possible fix: Unify the datasets in a language-neutral naming scheme for the entities and use language-tag on language-specific RDF literals.
The [CubicWeb Linked Data Browser]() still tries to cooperate with servers that are not already compliant.
It does the following to try to detect datasets:
- Simple content negotiation by requesting RDF data at the URI for the resource (case for compliant servers).
- Follow redirections to URIs for datasets serialized in specific syntaxes.
- Try to detect datasets linked to from HTTP `Link` header.
- When HTML was obtained, try to detect datasets linked to from HTML `Link` headers.
- When HTML was obtained, try to detect links to corresponding datasets, i.e. links starting with the URI of the requested resource and ending with a file extension for a known RDF syntax (.n3, .nt, .rdf, etc.).
docs/ServerLDCompliance.rst deleted 100644 → 0
View file @ c1a8e1f9
=============================
Linked Data Server Compliance
=============================
The :ref:`label-top` is a browser to navigate the web of data.
The point of this document is not to completely explain the web of data;
but it is important to important to understand one of its underlying promise:
Namely that data or resources can be referred to by URIs and that by connecting
to those URIs, interpreted as locations, the data associated with the URIs can
be retrieved, potentially through content negotiation.
For example, a RDF resource (file) refers to entities identified by URIs.
The promise is then that the data(sets) about these entities can be fetched at
the URIs.
Therefore, in the same way that a browser for the web of documents can navigate
from pages to pages through links using URLs,
a browser for the web of (linked) data could navigate within and between
datasets through the URIs of the entities referred to and find new datasets
about them.
For this ideal scenario, the web servers that in the end serve the content,
in this case the data, should *play nice*, i.e. be compliant with the web
of data. In this context, this mainly mean that when asked about an URI in
its scope, a web server should answer with data about the requested URI,
at least if explicitely asked to using HTTP headers related to content
negotiation. For example, a compliant server of linked data handling resources
for `example.com` should readily (or through content negotiation) answers
with the data related to
`http://example.com/some_resource`.
Example of compliance issues:
- The server always answers HTML, or redirect to an HTML page,
despite content negotiation requesting RDF data.
- Possible fix: Rely on content negotiation to answer with RDF content.
HTML can still be returned by default.
- Instead of replying with data, the server redirects to `http://example.com/some_resource.rdf`.
- Possible dix: Do not redirect to different URIs depending on RDF syntaxes
(.n3, .nt, .rdf, etc.) but directly return the content in the appropriate
syntax, as requested through content negotiation.
- The datasets returned by the server make use of multiple writings for the
same logical entity, for example using unicode escape sequences `\uxxxx`
in URIs.
- Possible fix: Try to canonicalize data to a single URI writing.
- Multiple datasets have been located for different writing of the same
URI and they have different content.
- The provided datasets use language specific URIs with different data
for different languages.
- Possible fix: Unify the datasets in a language-neutral naming scheme
for the entities and use language-tag on language-specific RDF literals.
The :ref:`label-top` still tries to cooperate with servers that are
not already compliant. It does the following to try to detect datasets:
- Simple content negotiation by requesting RDF data at the URI for the
resource (case for compliant servers).
- Follow redirections to URIs for datasets serialized in specific syntaxes.
- Try to detect datasets linked to from HTTP `Link` header.
- When HTML was obtained, try to detect datasets linked to from HTML
`Link` headers.
- When HTML was obtained, try to detect links to corresponding datasets,
i.e. links starting with the URI of the requested resource and ending
with a file extension for a known RDF syntax (.n3, .nt, .rdf, etc.).
docs/WritingNewViews.md deleted 100644 → 0
View file @ c1a8e1f9
# Writing new views for CubicWeb Linked Data Browser
The [CubicWeb Linked Data Browser]() relies on views to render RDF datasets.
By default, it only provides a single view that simply lists the RDF triples from the loaded dataset.
All other views are user-defined and dynamically discovered and loaded by the browser.
This document explains the basics of writing new views.
At its core, a view is simply a piece of Javascipt code (or any language that compiles to it) that implements a simple interface.
There are a few other requirements, as listed below: