Commit b00c5b9d authored by Sylvain Thénault's avatar Sylvain Thénault
Browse files

[doc] enhanced setup section: fix sections'level, more postgres tips, merge...

[doc] enhanced setup section: fix sections'level, more postgres tips, merge environment doc with cwconfig.py module docstring using custom autodocstring directive

--HG--
branch : stable
parent 910e021131d1
# -*- coding: utf-8 -*-
"""common configuration utilities for cubicweb
#:organization: Logilab
#:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2.
#:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr
#:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses
:organization: Logilab
:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2.
:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr
# docstring included in doc/book/en/admin/setup.rst
"""
.. _ResourceMode:
Resource mode
-------------
If cubicweb is a mercurial checkout (eg `CWDEV` is true), located in
`CW_SOFTWARE_ROOT`:
A resource *mode* is a predifined set of settings for various resources
directories, such as cubes, instances, etc. to ease development with the
framework. There are two running modes with |cubicweb|:
* main cubes directory is `<CW_SOFTWARE_ROOT>/../cubes`. You can specify
another one with `CW_INSTANCES_DIR` environment variable or simply add some
other directories by using `CW_CUBES_PATH`.
* 'user', resources are searched / created in the user home directory:
* cubicweb migration files are by default searched in
`<CW_SOFTWARE_ROOT>/misc/migration` instead of
`/usr/share/cubicweb/migration/`(unless another emplacement is specified
using `CW_MIGRATION_DIR`.
- instances are stored in :file:`~/etc/cubicweb.d`
- temporary files (such as pid file) in :file:`/tmp`
* Cubicweb will start in 'user' mode (see below)
* 'system', resources are searched / created in the system directories (eg
usually requiring root access):
- instances are stored in :file:`<INSTALL_PREFIX>/etc/cubicweb.d`
- temporary files (such as pid file) in :file:`/var/run/cubicweb`
On startup, Cubicweb is using a specific *mode*. A mode corresponds to some
default setting for various resource directories. There are currently 2 main
modes : 'system', for system wide installation, and 'user', fur user local
installation (e.g. no root privileges).
where `<INSTALL_PREFIX>` is the detected installation prefix ('/usr/local' for
instance).
'user' mode is activated automatically when cubicweb is a mercurial checkout
(e.g. has a .hg directory). You can also force mode by using the `CW_MODE`
environment variable, to:
* use system wide installation but user specific instances and all, without root
privileges on the system (`export CW_MODE=user`)
* use local checkout of cubicweb on system wide instances (requires root
privileges on the system (`export CW_MODE=system`)
Here is the default resource directories settings according to mode:
Notice that each resource path may be explicitly set using an environment
variable if the default doesn't suit your needs. Here are the default resource
directories that are affected according to mode:
* 'system': ::
CW_INSTANCES_DIR = /etc/cubicweb.d/
CW_INSTANCES_DIR = <INSTALL_PREFIX>/etc/cubicweb.d/
CW_INSTANCES_DATA_DIR = /var/lib/cubicweb/instances/
CW_RUNTIME_DIR = /var/run/cubicweb/
......@@ -50,27 +46,82 @@ environment variable, to:
CW_INSTANCES_DATA_DIR = ~/etc/cubicweb.d/
CW_RUNTIME_DIR = /tmp
Cubes search path is also affected, see the :ref:Cube section.
By default, the mode automatically set to 'user' if a :file:`.hg` directory is found
in the cubicweb package, else it's set to 'system'. You can force this by setting
the :envvar:`CW_MODE` environment variable to either 'user' or 'system' so you can
easily:
* use system wide installation but user specific instances and all, without root
privileges on the system (`export CW_MODE=user`)
* use local checkout of cubicweb on system wide instances (requires root
privileges on the system (`export CW_MODE=system`)
If you've a doubt about the mode you're currently running, check the first line
outputed by the :command:`cubicweb-ctl list` command.
Also, if cubicweb is a mercurial checkout located in `<CW_SOFTWARE_ROOT>`:
* main cubes directory is `<CW_SOFTWARE_ROOT>/../cubes`. You can specify
another one with :envvar:`CW_INSTANCES_DIR` environment variable or simply
add some other directories by using :envvar:`CW_CUBES_PATH`
* cubicweb migration files are searched in `<CW_SOFTWARE_ROOT>/misc/migration`
instead of `<INSTALL_PREFIX>/share/cubicweb/migration/`.
.. _ConfigurationEnv:
Environment configuration
-------------------------
Python
``````
If you installed |cubicweb| by cloning the Mercurial forest or from source
distribution, then you will need to update the environment variable PYTHONPATH by
adding the path to the forest `cubicweb`:
Add the following lines to either :file:`.bashrc` or :file:`.bash_profile` to
configure your development environment ::
export PYTHONPATH=/full/path/to/cubicweb-forest
If you installed |cubicweb| with packages, no configuration is required and your
new cubes will be placed in `/usr/share/cubicweb/cubes` and your instances will
be placed in `/etc/cubicweb.d`.
CubicWeb
````````
Here are all environment variables that may be used to configure |cubicweb|:
.. envvar:: CW_MODE
Resource mode: user or system
Resource mode: user or system, as explained in :ref:ResourceMode.
.. envvar:: CW_CUBES_PATH
Augments the default search path for cubes
Augments the default search path for cubes. You may specify several
directories using ':' as separator (';' under windows environment).
.. envvar:: CW_INSTANCES_DIR
Directory where cubicweb instances will be found
Directory where cubicweb instances will be found.
.. envvar:: CW_INSTANCES_DATA_DIR
Directory where cubicweb instances data will be written
Directory where cubicweb instances data will be written (backup file...)
.. envvar:: CW_RUNTIME_DIR
Directory where pid files will be written
.. envvar:: CW_MIGRATION_DIR
Directory where cubicweb migration files will be found
Directory where pid files will be written
:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses
.. |cubicweb| replace:: *CubicWeb*
"""
__docformat__ = "restructuredtext en"
_ = unicode
......
......@@ -8,16 +8,17 @@ Installation and set-up of a *CubicWeb* environment
Installation of `Cubicweb` and its dependencies
-----------------------------------------------
*CubicWeb* is packaged for Debian and Ubuntu, but can be installed from source
|cubicweb| is packaged for Debian and Ubuntu, but can be installed from source
using a tarball or the Mercurial version control system.
.. _DebianInstallation:
Debian and Ubuntu packages
```````````````````````````
Depending on the distribution you are using, add the appropriate line to your list
of sources (for example by editing ``/etc/apt/sources.list``).
Depending on the distribution you are using, add the appropriate line to your
list of sources (for example by editing ``/etc/apt/sources.list``).
For Debian Lenny::
......@@ -37,21 +38,26 @@ You can now install the required packages with the following command::
apt-get update
apt-get install cubicweb cubicweb-dev
`cubicweb` installs the framework itself, allowing you to create
new instances.
`cubicweb-dev` installs the development environment allowing you to
develop new cubes.
`cubicweb` installs the framework itself, allowing you to create new instances.
`cubicweb-dev` installs the development environment allowing you to develop new
cubes.
There is also a wide variety of cubes listed on http://www.cubicweb.org/Project available as debian packages and tarball.
There is also a wide variety of cubes listed on the `CubicWeb.org Forge`_
available as debian packages and tarball.
The repositories are signed with `Logilab's gnupg key`_. To avoid warning on
"apt-get update":
The repositories are signed with `Logilab's gnupg key`_. To avoid warning on "apt-get update":
1. become root using sudo
2. download http://ftp.logilab.org/dists/logilab-dists-key.asc using e.g. wget
3. run "apt-key add logilab-dists-key.asc"
4. re-run apt-get update (manually or through the package manager, whichever you prefer)
.. _`Logilab's gnupg key`: http://ftp.logilab.org/dists/logilab-dists-key.asc
.. _`CubicWeb.org Forge`: http://www.cubicweb.org/project/
.. _SourceInstallation:
......@@ -66,6 +72,11 @@ You can download the archive containing the sources from our `ftp site`_ at::
Make sure you have installed the dependencies (see appendixes for the list).
|cubicweb| should soon be pip_ installable, stay tuned (expected in 3.8).
.. _pip: http://pypi.python.org/pypi/pip
Install from version control system
```````````````````````````````````
......@@ -85,32 +96,31 @@ Do not forget to update the forest itself (using `cd path/to/forest ; hg up`).
Make sure you have installed the dependencies (see appendixes for the list).
.. _WindowsInstallation:
Windows installation
````````````````````
Base elements
_____________
~~~~~~~~~~~~~
Setting up a windows development environment is not too complicated
but requires a series of small steps. What is proposed there is only
an example of what can be done. We assume everything goes into C:\ in
this document. Adjusting the installation drive should be
straightforward.
Setting up a windows development environment is not too complicated but requires
a series of small steps. What is proposed there is only an example of what can be
done. We assume everything goes into `C:\\` in this document. Adjusting the
installation drive should be straightforward.
You should start by downloading and installing the Python(x,y)
distribution. It contains python 2.5 plus numerous useful third-party
modules and applications::
You should start by downloading and installing the Python(x,y) distribution. It
contains python 2.5 plus numerous useful third-party modules and applications::
http://www.pythonxy.com/download_fr.php
At the time of this writting, one gets version 2.1.15. Among the many
things provided, one finds Eclipse + pydev (an arguably good IDE for
python under windows).
At the time of this writting, one gets version 2.1.15. Among the many things
provided, one finds Eclipse + pydev (an arguably good IDE for python under
windows).
Then you must grab Twisted. There is a windows installer directly
available from this page::
Then you must grab Twisted. There is a windows installer directly available from
this page::
http://twistedmatrix.com/trac/
......@@ -129,11 +139,9 @@ The python drivers for posgtresql are to be found there::
http://www.stickpeople.com/projects/python/win-psycopg/#Version2
Please be careful to select the right python (2.5) and postgres (8.4)
versions.
Please be careful to select the right python (2.5) and postgres (8.4) versions.
Pyro enables remote access to cubicweb repository instances. Get it
there::
Pyro enables remote access to cubicweb repository instances. Get it there::
http://sourceforge.net/projects/pyro/files/
......@@ -144,26 +152,26 @@ package. Windows binaries are available from::
Check out the latest release.
Having graphviz will allow schema drawings, which is quite recommended
(albeit not mandatory). You should get an msi installer there::
Having graphviz will allow schema drawings, which is quite recommended (albeit
not mandatory). You should get an msi installer there::
http://www.graphviz.org/Download_windows.php
Simplejson will be provided within the forest, but a win32 compiled
version will run much faster::
Simplejson will be provided within the forest, but a win32 compiled version will
run much faster::
http://www.osuch.org/python-simplejson%3Awin32
Tools
_____
~~~~~
Get mercurial + its standard windows GUI (TortoiseHG) there (the
latest is the greatest)::
Get mercurial + its standard windows GUI (TortoiseHG) there (the latest is the
greatest)::
http://bitbucket.org/tortoisehg/stable/wiki/download
If you need to peruse mercurial over ssh, it can be helpful to get an
ssh client like Putty::
If you need to peruse mercurial over ssh, it can be helpful to get an ssh client
like Putty::
http://www.putty.org/
......@@ -173,10 +181,9 @@ it. Instructions are set there, in the `Download & Install` section::
http://www.vectrace.com/mercurialeclipse/
Setting up the sources
______________________
~~~~~~~~~~~~~~~~~~~~~~
You need to enable the mercurial forest extension. To do this, edit
the file::
You need to enable the mercurial forest extension. To do this, edit the file::
C:\Program Files\TortoiseHg\Mercurial.ini
......@@ -185,8 +192,8 @@ In the [extensions] section, add the following line::
forest=C:\Program Files\TortoiseHg\ext\forest\forest.py
Now, you need to clone the cubicweb repository. We assume that you use
Eclipse. From the IDE, choose File -> Import. In the box, select
`Mercurial/Clone repository using MercurialEclipse`.
Eclipse. From the IDE, choose File -> Import. In the box, select `Mercurial/Clone
repository using MercurialEclipse`.
In the import main panel you just have to:
......@@ -194,28 +201,26 @@ In the import main panel you just have to:
* check the 'Repository is a forest' box.
Then, click on 'Finish'. It might take some time to get it all. Note
that the `cubicwin32` forest contains additional python packages such
as yapps, vobject, simplejson and twisted-web2 which are not provided
with Python(x,y). This is provided for convenience, as we do not
ensure the up-to-dateness of these packages, especially with respect
to security fixes.
Then, click on 'Finish'. It might take some time to get it all. Note that the
`cubicwin32` forest contains additional python packages such as yapps, vobject,
simplejson and twisted-web2 which are not provided with Python(x,y). This is
provided for convenience, as we do not ensure the up-to-dateness of these
packages, especially with respect to security fixes.
Environment variables
_____________________
~~~~~~~~~~~~~~~~~~~~~
You will need some convenience environment variables once all is set
up. These variables are settable through the GUI by getting at the
'System properties' window (by righ-clicking on 'My Computer' ->
properties).
You will need some convenience environment variables once all is set up. These
variables are settable through the GUI by getting at the 'System properties'
window (by righ-clicking on 'My Computer' -> properties).
In the 'advanced' tab, there is an 'Environment variables'
button. Click on it. That opens a small window allowing edition of
user-related and system-wide variables.
In the 'advanced' tab, there is an 'Environment variables' button. Click on
it. That opens a small window allowing edition of user-related and system-wide
variables.
We will consider only user variables. First, the PATH variable. You
should ensure it contains, separated by semi-colons, and assuming you
are logged in as user Jane::
We will consider only user variables. First, the PATH variable. You should ensure
it contains, separated by semi-colons, and assuming you are logged in as user
Jane::
C:\Documents and Settings\Jane\My Documents\Python\cubicweb\cubicweb\bin
C:\Program Files\Graphviz2.24\bin
......@@ -231,13 +236,13 @@ From now, on a fresh `cmd` shell, you should be able to type::
... and get a meaningful output.
Running an instance as a service
--------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This currently assumes that the instances configurations is located
at C:\\etc\\cubicweb.d.
This currently assumes that the instances configurations is located at
C:\\etc\\cubicweb.d.
For a cube 'my_cube', you will then find C:\\etc\\cubicweb.d\\my_cube\\win32svc.py
that has to be used thusly::
For a cube 'my_cube', you will then find
C:\\etc\\cubicweb.d\\my_cube\\win32svc.py that has to be used thusly::
win32svc install
......@@ -248,17 +253,6 @@ This should just register your instance as a windows service. A simple::
should start the service.
PostgreSQL installation
```````````````````````
Please refer to the `PostgreSQL project online documentation`_.
.. _`PostgreSQL project online documentation`: http://www.postgresql.org/
You need to install the three following packages: `postgresql-8.3`,
`postgresql-contrib-8.3` and `postgresql-plpython-8.3`.
Other dependencies
``````````````````
......@@ -271,103 +265,115 @@ You can also install:
* `python-ldap` if you plan to use a LDAP source on the server
.. _ConfigurationEnv:
Environment configuration
-------------------------
.. _DatabaseInstallation:
If you installed *CubicWeb* by cloning the Mercurial forest, then you
will need to update the environment variable PYTHONPATH by adding
the path to the forest ``cubicweb``:
Databases configuration
-----------------------
Add the following lines to either `.bashrc` or `.bash_profile` to configure
your development environment ::
Whatever the backend used, database connection information are stored in the
instance's :file:`sources` file. Currently cubicweb has been tested using
Postgresql (recommanded), MySQL, SQLServer and SQLite.
export PYTHONPATH=/full/path/to/cubicweb-forest
.. _PostgresqlConfiguration:
If you installed *CubicWeb* with packages, no configuration is required and your
new cubes will be placed in `/usr/share/cubicweb/cubes` and your instances
will be placed in `/etc/cubicweb.d`.
PostgreSQL configuration
````````````````````````
You may run a system-wide install of *CubicWeb* in "user mode" and use it for
development by setting the following environment variable::
For installation, please refer to the `PostgreSQL project online documentation`_.
export CW_MODE=user
export CW_CUBES_PATH=~/lib/cubes
export CW_INSTANCES_DIR=~/etc/cubicweb.d/
export CW_INSTANCES_DATA_DIR=$CW_INSTANCES_DIR
export CW_RUNTIME_DIR=/tmp
.. _`PostgreSQL project online documentation`: http://www.postgresql.org/
.. note::
The values given above are our suggestions but of course
can be different.
You need to install the three following packages: `postgresql-8.X`,
`postgresql-client-8.X`, and `postgresql-plpython-8.X`. If you run postgres
version prior to 8.3, you'll also need the `postgresql-contrib-8.X` package for
full-text search extension.
If you run postgres on another host than the |cubicweb| repository, you should
install the `postgresql-client` package on the |cubicweb| host, and others on the
database host.
Databases configuration
-----------------------
.. Note::
.. _ConfigurationPostgresql:
If you already have an existing cluster and PostgreSQL server running, you do
not need to execute the initilization step of your PostgreSQL database unless
you want a specific cluster for |cubicweb| databases or if your existing
cluster doesn't use the UTF8 encoding (see note below).
PostgreSQL configuration
````````````````````````
* First, initialize a PostgreSQL cluster with the command ``initdb``.
::
.. note::
If you already have an existing cluster and PostgreSQL server
running, you do not need to execute the initilization step
of your PostgreSQL database.
$ initdb -E UTF8 -D /path/to/pgsql
* First, initialize the database PostgreSQL with the command ``initdb``.
::
Notice the encoding specification. This is necessary since |cubicweb| usually
want UTF8 encoded database. If you use a cluster with the wrong encoding, you'll
get error like::
new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII)
HINT: Use the same encoding as in the template database, or use template0 as template.
$ initdb -D /path/to/pgsql
Once initialized, start the database server PostgreSQL
with the command::
Once initialized, start the database server PostgreSQL with the command::
$ postgres -D /path/to/psql
If you cannot execute this command due to permission issues, please
make sure that your username has write access on the database.
::
If you cannot execute this command due to permission issues, please make sure
that your username has write access on the database. ::
$ chown username /path/to/pgsql
* The database authentication can be either set to `ident sameuser`
or `md5`.
If set to `md5`, make sure to use an existing user
of your database.
If set to `ident sameuser`, make sure that your
client's operating system user name has a matching user in
the database. If not, please do as follow to create a user::
* The database authentication can be either set to `ident sameuser` or `md5`. If
set to `md5`, make sure to use an existing user of your database. If set to
`ident sameuser`, make sure that your client's operating system user name has a
matching user in the database. If not, please do as follow to create a user::
$ su
$ su - postgres
$ createuser -s -P username
The option `-P` (for password prompt), will encrypt the password with
the method set in the configuration file ``pg_hba.conf``.
If you do not use this option `-P`, then the default value will be null
and you will need to set it with::
The option `-P` (for password prompt), will encrypt the password with the
method set in the configuration file :file:`pg_hba.conf`. If you do not use this
option `-P`, then the default value will be null and you will need to set it
with::
$ su postgres -c "echo ALTER USER username WITH PASSWORD 'userpasswd' | psql"
This login/password will be requested when you will create an
instance with `cubicweb-ctl create` to initialize the database of
your instance.
.. Note::
The authentication method can be configured in file:`pg_hba.conf`.
The above login/password will be requested when you will create an instance with
`cubicweb-ctl create` to initialize the database of your instance.
.. note::
The authentication method can be configured in ``pg_hba.conf``.
Notice that the `cubicweb-ctl db-create` does database initialization that
may requires a postgres superuser. That's why a login/password is explicitly asked
at this step, so you can use there a superuser without using this user when running
the instance. Things that require special privileges at this step:
* database creation, require the 'create database' permission
* install the plpython extension language (require superuser)
* install the tsearch extension for postgres version prior to 8.3 (require superuser)
.. FIXME Are these steps really necessary? It seemed to work without.
To avoid using a super user each time you create an install, a nice trick is to
install plpython (and tsearch when needed) on the special `template1` database,
so they will be installed automatically when cubicweb databases are created
without even with needs for special access rights. To do so, run ::
* Installation of plain-text index extension ::
# Installation of plpythonu language by default ::
$ createlang -U pgadmin plpythonu template1
$ psql -U pgadmin template1
template1=# update pg_language set lanpltrusted=TRUE where lanname='plpythonu';
cat /usr/share/postgresql/8.3/contrib/tsearch2.sql | psql -U username template1
Where `pgadmin` is a postgres superuser. The last command is necessary since by
default plpython is an 'untrusted' language and as such can't be used by non
superuser. This update fix that problem by making it trusted.
* Installation of plpythonu language by default ::
To install the tsearch plain-text index extension on postgres prior to 8.3, run::
createlang -U pgadmin plpythonu template1
cat /usr/share/postgresql/8.X/contrib/tsearch2.sql | psql -U username template1
.. _MySqlConfiguration:
MySql configuration
```````````````````
......@@ -378,19 +384,22 @@ Yout must add the following lines in ``/etc/mysql/my.cnf`` file::
default-character-set=utf8
max_allowed_packet = 128M
.. note::
.. Note::
It is unclear whether mysql supports indexed string of arbitrary lenght or
not.
.. _SQLServerConfiguration:
SQLServer configuration
-----------------------
```````````````````````
As of this writing, sqlserver support is in progress. You should be
able to connect, create a database and go quite far, but some of the
generated SQL is still currently not accepted by the backend.
As of this writing, sqlserver support is in progress. You should be able to
connect, create a database and go quite far, but some of the generated SQL is
still currently not accepted by the backend.
The `source` configuration file may look like this (specific parts
only are shown)::
The `source` configuration file may look like this (specific parts only are
shown)::
[system]
db-driver=sqlserver2005
......@@ -402,17 +411,43 @@ only are shown)::
db-encoding=utf8
.. _SQLiteConfiguration:
SQLite configuration
````````````````````
SQLite has the great advantage of requiring almost no configuration. Simply
use 'sqlite' as db-driver, and set path to the dabase as db-name. Don't specify
anything for db-user and db-password, they will be ignore anyway.
.. Note::
SQLite is great for testing and to play with cubicweb but is not suited for
production environments.
.. _PyroConfiguration:
Pyro configuration
------------------
If you use Pyro, it is required to have a name server Pyro running on your
network (by default it is detected by a broadcast request).
If you want to use Pyro to access your instance remotly, or to have multi-source
or distributed configuration, it is required to have a name server Pyro running