README.rst 5.67 KB
Newer Older
Laurent Peuch's avatar
Laurent Peuch committed
1
2
3
4
5
6
7
8
9
10
11
12
13
Cube Doctor
===========

Cube doctor is a tool used to send generated refactoring and QA improvement MR
to logilab's projects.

Usage
-----

CLI pattern looks like:

::

14
    GITLAB_TOKEN=... python3 doctor_hg.py [-h] [-a] [-i] [-m] [-n NUMBER] [-q QUERY] {fix-README,add-tox,add-pytest-deprecated-warnings-cmd,replace-set-attributes-by-set-cw,update-licence-dates}
Laurent Peuch's avatar
Laurent Peuch committed
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


Example:

::

    GITLAB_TOKEN=... python3 doctor_hg.py fix-README

Explanation
-----------

The current workflow for cube doctor is:

- select a refactoring command (for example fix-README will rename a README into README.rst and do other needed stuff)
- download our generated heptapod.trig that contains informations about all the public repositories of logilab's forge https://forge.extranet.logilab.fr/
- do a sparql query on that heptapod.trig to select repositories to target (each command has a default query to select only repositories on which it will be able to work)
- alternatively, if no query is provided (but right now all commands provides a query) all cubes of the cubicweb/cube group will be used
32
- for each of those cubes, apply the refactoring command, show what would have been done, but do nothing
Laurent Peuch's avatar
Laurent Peuch committed
33
- except if the "-a"/"--apply" flag is provided then the modification is pushed and a new merge request is created (and it is logged in create_mr.log)
34
- or if the "-i"/"--interactif" flag is provided then for each cube, the user will be prompted to tell if he wants to create a MR
Laurent Peuch's avatar
Laurent Peuch committed
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

Commands
--------

Current existing refactoring commands

fix-README
~~~~~~~~~~

This command will:

- rename README to README.rst
- replace "README" by "README.rst" in setup.py if present
- replace "README" by "README.rst" in MANIFEST.in if present or add "include README.rst" at the end of it

add-pytest-deprecated-warnings-cmd
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This command will:

- add :ref:`git+https://github.com/psycojoker/pytest-capture-deprecatedwarnings` to the deps section of the [testenv] of the tox.ini
- or if a dev-requirements.txt exist, add it into it instead
- modify .gitlag-ci.yml "py3" section to add "py3-deprecated-warnings.json" has an artifact

add-tox
~~~~~~~

This command is a bit WIP. It will:

- add a base tox.ini in a repository if it doesn't exist
- it will also launch this tox and fail if the tox fails

67
68
69
70
71
72
73
74
replace-set-attributes-by-set-cw
--------------------------------

This command will:

- retrieve all the Python files inside the cube
- for each file, replace all occurences of set_attributes calls by set_cw calls

75
76
77
78
79
80
81
82
update-licence-dates
--------------------

This command will:
- retrieve all the Python files inside the cube
- for each file, if it has a licence part in comments, extend the licence end date to
  the current year.

Laurent Peuch's avatar
Laurent Peuch committed
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
Command CLI arguments
---------------------

All commands share the same CLI arguments. They are:

::

    -a, --apply
        If used a merge request will be created with the modifcations.

        Default value: False

    -i, --interactif
        If used the user will be prompt before created a merge request (a "hg
        diff" and "hg status" will be run before to show the modifications)

        -a/--apply will overwrite this argument

        Default value: False

    -m, --merge-when-pipeline-succeeds
        If used the created merged request will be set to be merged if the
        tests succeeds. Won't have any effect if no merge request is created.

        Default value: False

    -n, --number
        Limit the number of cubes of which to run the command

        Default value: None (equal to "all cubes")

    -q, --query
        Additional conditions to pass to the sparql query SELECT section.

117
118
119
120
121
122
    -p, --project
        If specified, select one specific project instead of using the query.
        Arguments supported are: project id (an it), project url and project name

        Default value: None (unused)

123
124
125
126
127
128
129
    -t, --target
        Can has the value "cubes", "core" or "clients". Will select the family
        of projects specified. Can also be used with "auto-mr" and will skip
        refactoring commands not meant for the specified family

        Default value: cubes

Laurent Peuch's avatar
Laurent Peuch committed
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
Sparql query reference
----------------------

The sparql base query looks like this:

::

    prefix lgg: <http://data.logilab.fr/logigraphe/>
    prefix lon: <http://data.logilab.fr/ontology/network/>
    prefix dep: <http://ontologi.es/doap-deps#>
    prefix doap: <http://usefulinc.com/ns/doap#>

    SELECT ?projectId

    WHERE {
      ?project a doap:Project .
      ?project doap:name ?projectName .
      ?project lgg:heptapod_id ?projectId .
      ?project lgg:is_cube true
    } ORDER BY ?projectName

And all the field of a project looks like this (but this evolve quickly so
check the heptapod.trig to be sure):

::

    lgg:has_black false ;
    lgg:has_capture_deprecated_warnings true ;
    lgg:has_check_manifest true ;
    lgg:has_doc false ;
    lgg:has_flake8 true ;
    lgg:has_from_forge false ;
    lgg:has_gitlab_ci true ;
    lgg:has_mypy false ;
    lgg:has_py2_tests true ;
    lgg:has_py3_tests true ;
    lgg:has_pytest true ;
    lgg:has_readme_rst false ;
    lgg:has_readthedocs false ;
    lgg:has_tox true ;
    lgg:heptapod_id 143 ;
    lgg:is_cube true ;
    lgg:is_new_format true ;
    lgg:last_activity_at "2020-09-17T07:16:55.785000"^^xsd:dateTime ;
    lgg:visibility "public" ;
    dc:description "cubicweb / cubes / activitystream" ;
    dc:title "cubicweb-activitystream" ;
    doap:name "cubicweb-activitystream" ;
    doap:release <https://forge.extranet.logilab.fr/cubicweb/cubes/activitystream/-/releases/0.2.0> .

So, an example query to only select the activitystream could be done like this:

::

    GITLAB_TOKEN=... python3 doctor_hg.py fix-README -q '?project doap:name "cubicweb-activitystream"'