GitLab

PyCryptodome comes in two flavors: “an almost drop-in replacement for the old
PyCrypto library” and “a library independent of the old PyCrypto”. The former
uses the Crypto namespace, and is shipped as `pycryptodome` while the latter
uses Cryptodome instead and lies in the `pycryptodomex` package.

Given the reason to switch to PyCryptodome is that PyCrypto in unmaintained,
its probably better to avoid any mistake and mandate the specific usage of
the Cryptodome namespace by requiring `pycryptodomex` instead of
`pycryptodome`.

A more present reason is that Debian buster will only provide a package
with the separate namespace flavor. The current Recommends is not working with
the current code. Although it's important to note that the package name will
probably have to be changed to `python3-pycryptodomex` once
https://bugs.debian.org/886291 is solved.

The default branch only provides support for Python 3.

As suggested by Denis Laxalde, we can merge python3-cubicweb-pyramid into
python3-cubicweb as all other web backend in the default branch.

As suggested by Denis Laxalde, we can let pybuild generate the list
of Depends from the egg requirements.

The list we get is slightly less specific in terms of needed versions,
but given the age of the previous ones, it's unlikely to cause issues.
Also we lose python3-six from the list… which is correct as it was
removed from the dependencies in 26744ad37953.

As pointed by Lintian, the “Apps” section is only used in menu,
not in doc-base. We can just remove it.

[pkg] Use “dependency package” instead of “virtual package” to describe python3-cubicweb-postgresql-support

The term “virtual” was too imprecise and is not accepted for empty
packages by Lintian anymore.

Just use the now preferred location.

cubicweb-ctl shipped an empty /etc/bash_completion.d as a leftover
of shipping a bash completion script. As pointed by Lintian, this
directory is not obsolete, so let's get rid of it for good.

The documentation package was missing a Built-Using field to record
the provenance of the templates.

Empty packages or meta packages do not require Depends on a specific
Python version, so we can remove the relevant substvars.

Quoting Lintian, “Since Debian Policy version 4.0.1, the priority extra
has been deprecated.” So let's use “optional” instead.

As suggested by Lintian, remove build dependency on obsolete package
dh-systemd. Instead, specify a more precise version of debhelper 9 that
added support for systemd.

Add support for autopkgtest. The test “unittest” will run the test
suite using pytest. As we need PostgreSQL in the test environment
and open network connection, the test is marked with
`isolation-container` and requires at least LXC to be run, e.g:

    autopkgtest -s -- lxc -s autopkgtest-sid

This forces us to base the Debian package on the source of the Python
package (as produced by running `python3 setup.py sdist`). While
it might seem cumbersome, this greatly reduces likelyhood of mismatch
between an installation via `pip` and one using the Debian package.

`dpkg-source` will also document for us what is currently in the
source repository but not in the Python package. Currently the missing
file are:

PKG-INFO
cubicweb/sobjects/test/data/cubicweb_comment/schema.py
cubicweb/sobjects/test/data/cubicweb_comment/__init__.py
cubicweb/sobjects/test/data/cubicweb_comment/__pkginfo__.py
cubicweb/sobjects/test/data/cubicweb_card/schema.py
cubicweb/sobjects/test/data/cubicweb_card/entities.py
cubicweb/sobjects/test/data/cubicweb_card/__init__.py
cubicweb/sobjects/test/data/cubicweb_card/__pkginfo__.py
cubicweb/server/test/data-migractions/cubicweb_comment/schema.py
cubicweb/server/test/data-migractions/cubicweb_comment/__init__.py
cubicweb/server/test/data-migractions/cubicweb_comment/__pkginfo__.py
cubicweb/server/test/data-migractions/cubicweb_localperms/schema.py
cubicweb/server/test/data-migractions/cubicweb_localperms/__init__.py
cubicweb/server/test/data-migractions/cubicweb_localperms/__pkginfo__.py
cubicweb/server/test/data-migractions/cubicweb_basket/schema.py
cubicweb/server/test/data-migractions/cubicweb_basket/__init__.py
cubicweb/server/test/data-migractions/cubicweb_basket/__pkginfo__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_comment/schema.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_comment/__init__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_comment/__pkginfo__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_localperms/schema.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_localperms/__init__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_localperms/__pkginfo__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_basket/schema.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_basket/__init__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_basket/__pkginfo__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_tag/schema.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_tag/__init__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_tag/__pkginfo__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_card/schema.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_card/__init__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_card/__pkginfo__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_file/schema.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_file/__init__.py
cubicweb/server/test/data-migractions/migratedapp/cubicweb_file/__pkginfo__.py
cubicweb/server/test/data-migractions/cubicweb_tag/schema.py
cubicweb/server/test/data-migractions/cubicweb_tag/__init__.py
cubicweb/server/test/data-migractions/cubicweb_tag/__pkginfo__.py
cubicweb/server/test/data-migractions/cubicweb_card/schema.py
cubicweb/server/test/data-migractions/cubicweb_card/__init__.py
cubicweb/server/test/data-migractions/cubicweb_card/__pkginfo__.py
cubicweb/server/test/data-migractions/cubicweb_file/schema.py
cubicweb/server/test/data-migractions/cubicweb_file/__init__.py
cubicweb/server/test/data-migractions/cubicweb_file/__pkginfo__.py
cubicweb/test/data-rewrite/cubicweb_localperms/schema.py
cubicweb/test/data-rewrite/cubicweb_localperms/__init__.py
cubicweb/test/data-rewrite/cubicweb_localperms/__pkginfo__.py
cubicweb/test/data-rewrite/cubicweb_card/schema.py
cubicweb/test/data-rewrite/cubicweb_card/entities.py
cubicweb/test/data-rewrite/cubicweb_card/__init__.py
cubicweb/test/data-rewrite/cubicweb_card/__pkginfo__.py

The current manifest will duplicate several files that are stored as
symlinks in the Mercurial repository, hence the long list of ignored
files in `extend-diff-ignore`.

With changeset 234ca3cbbb46, clicking in the schema of an entity with
cubicweb in py3 on "vue en boite" will probably result in an infinite
spinner (which implies cw > 3.26)

What happened ?

This "vue en boite" used to work at least until...

hg diff -c a8c1ea390400 cubicweb/schema.py


    @@ -993,10 +992,6 @@ class CubicWebRelationSchema(PermissionM
                         return False
             return True

    -    @deprecated('use .rdef(subjtype, objtype).role_cardinality(role)')
    -    def cardinality(self, subjtype, objtype, target):
    -        return self.rdef(subjtype, objtype).role_cardinality(target)
    -

     class CubicWebSchema(Schema):
         """set of entities and relations schema defining the possible data sets


But, wait ...
If I open a shell on an instance of cw 3.24 something seems off

>>> list(schema['CWUniqueTogetherConstraint'].relation_definitions())[0][0].cardinality
# <bound method CubicWebRelationSchema.wrapped of <constraint_of [CWUniqueTogetherConstraint,CWEType]>>

We have been sorting on a method the whole time ? Is it possible what
were the effects ?

1) We cannot sort function can't we ?

>>> def adder(i): return lambda x: x+i
>>> sorted(map(adder,range(10)))
[<function __main__.<lambda>>,
 <function __main__.<lambda>>,
...

Yes we can.

2) what does it means.

>>> { adder(1) : 1 }
Out[19]: {<function __main__.<lambda>>: 1}

In fact the function object as a __hash__ method (which is practical for making
memoizers (cache)), and return truly random results (pseudo random).

My take on this patch is relations have NEVER been sorted by cardinality.
No one never ever noticed. Hence, I propose to not fix a bug that never was
reported.

py2 support being dropped, passing the code in full py3 str/bytes syntax

--HG--
branch : 3.26

--HG--
branch : 3.26

--HG--
branch : 3.26

Instead of git which require git to be installed. Also downloading archive is faster than cloning the repository.

--HG--
branch : 3.26

The script should be installed normally as it is a setuptools' entry
point.

--HG--
branch : 3.26

We are not actually running tests during package build. (This was
disabled for python2, now it is also for python3.) And having these
build-depends makes it hard to rebuild the package.

--HG--
branch : 3.26

Extid can be bytes or text.

This avoid AttributeError: 'str' object has no attribute 'decode'

--HG--
branch : 3.26

What was happening
==================

The bug appears when ldapfeed tries to insert a user while another user exists
exists with a different source.

Simple use case to reproduce:

- create a local user in cubicweb (source=system)
- sync with ldap
- ldapfeed will stop complaining user already exists.

Without next patch the test MUST fail with message:
cubicweb/server/sources/native.py:714: UniqueTogetherError

The ldapfeed is thus stopped ignoring any further ldap entries.


The proposal
============

Prior to this patch, the insertion was trying to create all CWUser with the
computed login from eeimporter.extid2eid (method process of DataFeedLDAPAdapter).

When a CWUser existed with a different cw_source ("system" for user created
with cubicweb for instance), it created a conflict.

To avoid collisions, in the extentities_generator method a bypass
was added at the insertion layer.

Prior to insertion the absence of the computed login is checked on a list of
all existing login from a different source.

If collision is detected, we skip the ldap record.

By short circuiting at the CWUser entity insertion level we also avoid to treat
CWGroup and EmailAddress related to this user.
Hence ensuring a behaviour that will not break existing instances.
(conservative approach: faced with ambiguity better do nothing than guess).

An error message is added stating explicitly the conflict.

--HG--
branch : 3.26

To avoid these pytest error when collecting the whole test suite:

import file mismatch:
imported module 'unittest_utils' has this __file__ attribute:
  cubicweb/cubicweb/server/test/unittest_utils.py
which is not the same as the test file we want to collect:
  cubicweb/cubicweb/test/unittest_utils.py


Move cubicweb/server/test/unittest_security.py to cubicweb/server/test/unittest_security.py
and cubicweb/test/unittest_utils.py to cubicweb/test/unittest_server_utils.py

The hook's regid is check_no_subsidiary_cycle.

See the changelog https://cubicweb.readthedocs.io/en/3.26/changes/changelog/#id14

pycrypto isn't maintained anymore, the last release 2.6.1 is from 2014.

Use the drop-in replacement fork pycryptodome instead.
pycryptodome is packaged in debian starting from stretch-backports.

pycrypto accept both bytes or string but pycryptodome, which will be introduced
in next changeset, doesn't accept this.

For backward compatibility, encode unicode strings in _cypherer()

Move from:

    (Pdb++) raise some_exception
    *** ViolatedConstraint:

To:

    (Pdb++) raise some_exception
    *** ViolatedConstraint: constraint 'cstr56c2ab4b3154f21d08b067742ce5bd9d' is being violated by the query 'ALTER TABLE cw_Bibliography ADD CONSTRAINT cstr56c2ab4b3154f21d08b067742ce5bd9d CHECK(cw_item_type IN ('journalArticle', 'note', 'book', 'thesis', 'film', 'web page', 'manuscrit', 'tapuscrit'))'. You can run the inverted constraint on the database to list the problematic rows.

And save hours of debugging to actually understand what is going on.

We either ignore errors in case of name re-exports and drop re-exports
from old backwards compatibility. We fix the only occurrence of
NoSelectableObject being imported from cubicweb in tests and update the
import to use lgc instead.