GitLab

Since, for some reason, exception log messages are not visible when
running tests, it's very hard to debug when sendmails() method fails
during tests. So, as in some other places, let's raise the exception
when in test mode.

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

The script should be installed normally as it is a setuptools' entry
point.

--HG--
branch : 3.26

We are not actually running tests during package build. (This was
disabled for python2, now it is also for python3.) And having these
build-depends makes it hard to rebuild the package.

--HG--
branch : 3.26

Extid can be bytes or text.

This avoid AttributeError: 'str' object has no attribute 'decode'

--HG--
branch : 3.26

What was happening
==================

The bug appears when ldapfeed tries to insert a user while another user exists
exists with a different source.

Simple use case to reproduce:

- create a local user in cubicweb (source=system)
- sync with ldap
- ldapfeed will stop complaining user already exists.

Without next patch the test MUST fail with message:
cubicweb/server/sources/native.py:714: UniqueTogetherError

The ldapfeed is thus stopped ignoring any further ldap entries.


The proposal
============

Prior to this patch, the insertion was trying to create all CWUser with the
computed login from eeimporter.extid2eid (method process of DataFeedLDAPAdapter).

When a CWUser existed with a different cw_source ("system" for user created
with cubicweb for instance), it created a conflict.

To avoid collisions, in the extentities_generator method a bypass
was added at the insertion layer.

Prior to insertion the absence of the computed login is checked on a list of
all existing login from a different source.

If collision is detected, we skip the ldap record.

By short circuiting at the CWUser entity insertion level we also avoid to treat
CWGroup and EmailAddress related to this user.
Hence ensuring a behaviour that will not break existing instances.
(conservative approach: faced with ambiguity better do nothing than guess).

An error message is added stating explicitly the conflict.

--HG--
branch : 3.26

Until someone works on fixing these, this should make our CI green
again.

I tried to use setupModule() to check for python version, but
pre_setup_database() is apparently called even when a SkipTest exception
is raised there. So handle this in that method.
(grafted from 4d68d20427de)

--HG--
branch : 3.26

--HG--
branch : 3.26

* Remove usage of unicode() and decode the base64-encoded string in
  encrypt();
* Encode the string received in decrypt() as (I supposed) it should come
  from the result of encrypt().

Add tests for this module along the way.

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

From psycopg2 >= 2.8, specific exceptions are raised corresponding to
postgresql errors. E.g. a CheckViolation exception is raised instead of
a generic IntegrityError previously when a constraint violation occurs.
The way we intercept database errors, especially for constraint
violation, is not compliant with that because we do not catch subclasses
of IntegrityError in native source's doexec() method.

We fix this by checking for the presence of IntegrityError error in
exception class's mro. This is still overcomplicated and clumsy, because
we still use string comparison, but this is the best we can do as far as
I know. (A better fix would be 'isinstance(ex, IntegrityError)' but we
have no engine-independent error classes, so this is not possible.
Something like sqlalchemy's DBAPI Errors [1] might help:
https://docs.sqlalchemy.org/en/latest/errors.html#dbapi-errors)

--HG--
branch : 3.26

commit() might raise a AttributeError too.

Use getattr(req, 'cnx', req) instead, which is a form already used to get the real cnx
in some code:

cubicweb/rset.py:577:        cnx = getattr(self.req, 'cnx', self.req)
cubicweb/schema.py:353:                with getattr(_cw, 'cnx', _cw).security_enabled(read=False):

We could use if hasattr(req, 'commit') here too but it lead to 3 additionals lines.
Maybe we should have commit() and rollback() on
cubicweb.web.request.ConnectionCubicWebRequestBase too ?

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

Based on django source code in case SystemRandom is not available.

According to python documentation:

https://docs.python.org/2/library/random.html
> Warning
> The pseudo-random generators of this module should not be used for security
> purposes. Use os.urandom() or SystemRandom if you require a cryptographically
> secure pseudo-random number generator.

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

Try to make it more accessible, when we first encountered these errors
it it didn't made any sense to us.

--HG--
branch : 3.26

Avoids the following warning:

  The psycopg2 wheel package will be renamed from release 2.8; in order
  to keep installing from binary please use "pip install psycopg2-binary"
  instead. For details see:
  <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi

--HG--
branch : 3.26

This includes changeset aa4a3a3d0c18 fixing import of
os.errno.

--HG--
branch : 3.26

--HG--
branch : 3.26

With Python3.7, this test fails as the error message returned by
rest_publish() is:

  <p>an error occurred while interpreting this rql directive: ObjectNotFound('toto')</p>

while we previously assumed a comma after 'toto'.

Fixing this by only shortening the prefix used in out.startswith().

--HG--
branch : 3.26

--HG--
branch : 3.26

This gets rid of the deprecation warning from logilab.common.registry:

  DeprecationWarning: instantiate MyAFS with __module__=__name__

and solves a test error on Python 3.7.

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

According to a discussion on XMPP we should recommend pip instead.

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26