README.rst 4.71 KB
Newer Older
Laurent Peuch's avatar
Laurent Peuch committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
Cube Doctor
===========

Cube doctor is a tool used to send generated refactoring and QA improvement MR
to logilab's projects.

Usage
-----

CLI pattern looks like:

::

    GITLAB_TOKEN=... python3 doctor_hg.py [-h] [-a] [-i] [-m] [-n NUMBER] [-q QUERY] {fix-README,add-tox,add-pytest-deprecated-warnings-cmd}


Example:

::

    GITLAB_TOKEN=... python3 doctor_hg.py fix-README

Explanation
-----------

The current workflow for cube doctor is:

- select a refactoring command (for example fix-README will rename a README into README.rst and do other needed stuff)
- download our generated heptapod.trig that contains informations about all the public repositories of logilab's forge https://forge.extranet.logilab.fr/
- do a sparql query on that heptapod.trig to select repositories to target (each command has a default query to select only repositories on which it will be able to work)
- alternatively, if no query is provided (but right now all commands provides a query) all cubes of the cubicweb/cube group will be used
- for each of those cube, apply the refactoring command and show what it would have done but do nothing
- except if the "-a"/"--apply" flag is provided then the modification is pushed and a new merge request is created (and it is logged in create_mr.log)
- or if the "-i"/"--interactif" flag is provided then for each cube the user will be prompt to tell if it wants to create a MR

Commands
--------

Current existing refactoring commands

fix-README
~~~~~~~~~~

This command will:

- rename README to README.rst
- replace "README" by "README.rst" in setup.py if present
- replace "README" by "README.rst" in MANIFEST.in if present or add "include README.rst" at the end of it

add-pytest-deprecated-warnings-cmd
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This command will:

- add :ref:`git+https://github.com/psycojoker/pytest-capture-deprecatedwarnings` to the deps section of the [testenv] of the tox.ini
- or if a dev-requirements.txt exist, add it into it instead
- modify .gitlag-ci.yml "py3" section to add "py3-deprecated-warnings.json" has an artifact

add-tox
~~~~~~~

This command is a bit WIP. It will:

- add a base tox.ini in a repository if it doesn't exist
- it will also launch this tox and fail if the tox fails

Command CLI arguments
---------------------

All commands share the same CLI arguments. They are:

::

    -a, --apply
        If used a merge request will be created with the modifcations.

        Default value: False

    -i, --interactif
        If used the user will be prompt before created a merge request (a "hg
        diff" and "hg status" will be run before to show the modifications)

        -a/--apply will overwrite this argument

        Default value: False

    -m, --merge-when-pipeline-succeeds
        If used the created merged request will be set to be merged if the
        tests succeeds. Won't have any effect if no merge request is created.

        Default value: False

    -n, --number
        Limit the number of cubes of which to run the command

        Default value: None (equal to "all cubes")

    -q, --query
        Additional conditions to pass to the sparql query SELECT section.

Sparql query reference
----------------------

The sparql base query looks like this:

::

    prefix lgg: <http://data.logilab.fr/logigraphe/>
    prefix lon: <http://data.logilab.fr/ontology/network/>
    prefix dep: <http://ontologi.es/doap-deps#>
    prefix doap: <http://usefulinc.com/ns/doap#>

    SELECT ?projectId

    WHERE {
      ?project a doap:Project .
      ?project doap:name ?projectName .
      ?project lgg:heptapod_id ?projectId .
      ?project lgg:is_cube true
    } ORDER BY ?projectName

And all the field of a project looks like this (but this evolve quickly so
check the heptapod.trig to be sure):

::

    lgg:has_black false ;
    lgg:has_capture_deprecated_warnings true ;
    lgg:has_check_manifest true ;
    lgg:has_doc false ;
    lgg:has_flake8 true ;
    lgg:has_from_forge false ;
    lgg:has_gitlab_ci true ;
    lgg:has_mypy false ;
    lgg:has_py2_tests true ;
    lgg:has_py3_tests true ;
    lgg:has_pytest true ;
    lgg:has_readme_rst false ;
    lgg:has_readthedocs false ;
    lgg:has_tox true ;
    lgg:heptapod_id 143 ;
    lgg:is_cube true ;
    lgg:is_new_format true ;
    lgg:last_activity_at "2020-09-17T07:16:55.785000"^^xsd:dateTime ;
    lgg:visibility "public" ;
    dc:description "cubicweb / cubes / activitystream" ;
    dc:title "cubicweb-activitystream" ;
    doap:name "cubicweb-activitystream" ;
    doap:release <https://forge.extranet.logilab.fr/cubicweb/cubes/activitystream/-/releases/0.2.0> .

So, an example query to only select the activitystream could be done like this:

::

    GITLAB_TOKEN=... python3 doctor_hg.py fix-README -q '?project doap:name "cubicweb-activitystream"'