5. Development¶

This section only needs to be read by developers of the Yagot project, including people who want to make a fix or want to test the project.

5.1. Repository¶

The repository for the Yagot project is on GitHub:

https://github.com/andy-maier/python-yagot

5.2. Setting up the development environment¶

If you have write access to the Git repo of this project, clone it using its SSH link, and switch to its working directory:
```
$ git clone git@github.com:andy-maier/python-yagot.git
$ cd python-yagot
```
If you do not have write access, create a fork on GitHub and clone the fork in the way shown above.
It is recommended that you set up a virtual Python environment. Have the virtual Python environment active for all remaining steps.
Install the project for development. This will install Python packages into the active Python environment:
```
$ make develop
```
This project uses Make to do things in the currently active Python environment. The command:
```
$ make
```
displays a list of valid Make targets and a short description of what each target does.

5.3. Building the documentation¶

The ReadTheDocs (RTD) site is used to publish the documentation for the project package at https://yagot.readthedocs.io/

This page is automatically updated whenever the Git repo for this package changes the branch from which this documentation is built.

In order to build the documentation locally from the Git work directory, execute:

$ make builddoc

The top-level document to open with a web browser will be build_doc/html/docs/index.html.

5.4. Testing¶

All of the following make commands run the tests in the currently active Python environment. Depending on how the yagot package is installed in that Python environment, either the directories in the main repository directory are used, or the installed package. The test case files and any utility functions they use are always used from the tests directory in the main repository directory.

The tests directory has the following subdirectory structure:

tests
 +-- plugintest          Plugin tests
 +-- unittest            Unit tests

The unit tests and plugin tests are run by executing:

$ make test

Test execution can be modified by a number of environment variables, as documented in the make help (execute make help).

To run the unit and plugin tests in all supported Python environments, the Tox tool can be used. It creates the necessary virtual Python environments and executes make test (i.e. the unit and function tests) in each of them.

For running Tox, it does not matter which Python environment is currently active, as long as the Python tox package is installed in it:

$ tox                              # Run tests on all supported Python versions
$ tox -e py27                      # Run tests on Python 2.7

5.5. Contributing¶

Third party contributions to this project are welcome!

In order to contribute, create a Git pull request, considering this:

Test is required.
Each commit should only contain one “logical” change.
A “logical” change should be put into one commit, and not split over multiple commits.
Large new features should be split into stages.
The commit message should not only summarize what you have done, but explain why the change is useful.

What comprises a “logical” change is subject to sound judgement. Sometimes, it makes sense to produce a set of commits for a feature (even if not large). For example, a first commit may introduce a (presumably) compatible API change without exploitation of that feature. With only this commit applied, it should be demonstrable that everything is still working as before. The next commit may be the exploitation of the feature in other components.

For further discussion of good and bad practices regarding commits, see:

Further rules:

The following long-lived branches exist and should be used as targets for pull requests:
- master - for next functional version
- stable_$MN - for fix stream of released version M.N.
We use topic branches for everything!
- Based upon the intended long-lived branch, if no dependencies
- Based upon an earlier topic branch, in case of dependencies
- It is valid to rebase topic branches and force-push them.
We use pull requests to review the branches.
- Use the correct long-lived branch (e.g. master or stable_0.2) as a merge target.
- Review happens as comments on the pull requests.
- At least one approval is required for merging.
GitHub meanwhile offers different ways to merge pull requests. We merge pull requests by rebasing the commit from the pull request.

5.6. Releasing a version to PyPI¶

This section describes how to release a version of Yagot to PyPI.

It covers all variants of versions:

Releasing the master branch as a new (major or minor) version
Releasing a fix stream branch of an already released version as a new fix version

The description assumes that the project repo is cloned locally. Their upstream repos are assumed to have the remote name origin.

Switch to your work directory of the project repo (this is where the Makefile is), and perform the following steps in that directory.

Set shell variables for the version and branch to be released.

When releasing the master branch:

$ MNP="0.2.0"          # Full version number M.N.P of version to be released
$ MN="0.2"             # Major and minor version number M.N of version to be released
$ BRANCH="master"      # Branch to be released

When releasing a fix stream branch:

$ MNP="0.1.1"          # Full version number M.N.P of version to be released
$ MN="0.1"             # Major and minor version number M.N of version to be released
$ BRANCH="stable_$MN"  # Branch to be released

Check out the branch to be released, make sure it is up to date with upstream, and create a topic branch for the version to be released:
```
$ git checkout $BRANCH
$ git pull
$ git checkout -b release_$MNP
```
Edit the version file:
```
$ vi yagot/_version.py
```
and set the version to be released:
```
__version__ = 'M.N.P'
```
where M.N.P is the version to be released, e.g. 0.2.0.

You can verify that this version is picked up by setup.py as follows:
```
$ ./setup.py --version
0.2.0
```
Edit the change log:
```
$ vi docs/changes.rst
```
To make the following changes for the version to be released:
- Finalize the version to the version to be released.
- Remove the statement that the version is in development.
- Update the statement which fixes of the previous stable version are contained in this version. If there is no fix release of the previous stable version, the line can be removed.
- Change the release date to today´s date.
- Make sure that all changes are described. This can be done by comparing the changes listed with the commit log of the master branch.
- Make sure the items in the change log are relevant for and understandable by users of the project.
- In the “Known issues” list item, remove the link to the issue tracker and add text for any known issues you want users to know about.
  
  Note: Just linking to the issue tracker quickly becomes incorrect for a released version and is therefore only good during development of a version. In the “Starting a new version” section, the link will be added again for the new version.
Perform a complete build (in your favorite Python virtual environment):
```
$ make clobber
$ make all
```
If this fails, fix and iterate over this step until it succeeds.

Commit the changes and push to upstream:

$ git status    # to double check which files have been changed
$ git commit -asm "Release $MNP"
$ git push --set-upstream origin release_$MNP

On GitHub, create a Pull Request for branch release_$MNP. This will trigger the CI runs in Travis and Appveyor.

Important: When creating Pull Requests, GitHub by default targets the master branch. If you are releasing a fix version, you need to change the target branch of the Pull Request to stable_$MN.
Perform a complete test using Tox:
```
$ tox
```
This will create virtual Python environments for all supported Python versions that are installed on your system and will invoke make test in each of them.
If any of the tests mentioned above fails, fix the problem and iterate back to step 6. until they all succeed.
On GitHub, once the CI runs for the Pull Request succeed:
- Merge the Pull Request (no review is needed)
- Delete the branch of the Pull Request (release_$MNP)
Checkout the branch you are releasing, update it from upstream, and delete the local topic branch you created:
```
$ git checkout $BRANCH
$ git pull
$ git branch -d release_$MNP
```

Tag the version:

This step tags the local repo and pushes it upstream:

$ git status    # double check that the branch to be released (`$BRANCH`) is checked out
$ git tag $MNP
$ git push --tags

If you released the master branch it will be fixed separately, so it needs a new fix stream.
- Create a branch for its fix stream and push it upstream:
```
$ git status    # double check that the branch to be released (`$BRANCH`) is checked out
$ git checkout -b stable_$MN
$ git push --set-upstream origin stable_$MN
```
- Log on to RTD, go to the project, and activate the new branch stable_$MN as a version to be built.
On GitHub, edit the new tag, and create a release description on it. This will cause it to appear in the Release tab.
On GitHub, close milestone M.N.P.

Note: Issues with that milestone will be moved forward in the section “Starting a new version”.
Upload the package to PyPI:
```
$ make upload
```
Attention!! This only works once. You cannot re-release the same version to PyPI.

Verify that it arrived on PyPI: https://pypi.python.org/pypi/yagot/

5.7. Starting a new version¶

This section shows the steps for starting development of a new version of the Yagot project in its Git repo.

It covers all variants of new versions:

A new (major or minor) version for new development based upon the master branch.
A new fix version based on a stable_$MN fix stream branch.

Switch to the work directory of your repo clone and perform the following steps in that directory.

Set shell variables for the version to be started and for the branch it is based upon.

When starting a new major or minor version based on the master branch:

$ MNP="0.2.0"          # Full version number M.N.P of version to be started
$ MN="0.2"             # Major and minor version number M.N of version to be started
$ BRANCH="master"      # Branch the new version is based on

When releasing a fix version based on a fix stream branch:

$ MNP="0.1.1"          # Full version number M.N.P of version to be started
$ MN="0.1"             # Major and minor version number M.N of version to be started
$ BRANCH="stable_$MN"  # Branch the new version is based on

Check out the branch the new version is based upon, make sure it is up to date with upstream, and create a topic branch for the new version:
```
$ git checkout $BRANCH
$ git pull
$ git checkout -b start_$MNP
```
Edit the version file:
```
$ vi yagot/_version.py
```
and set the version to the new development version:
```
__version__ = 'M.N.P.dev1'
```
where M.N.P is the new version to be started, e.g. 0.2.0.

Edit the change log:

$ vi docs/changes.rst

To insert the following section before the top-most section:

yagot 0.2.0.dev1
------------------------------------------

This version contains all fixes up to yagot 0.1.x.

Released: not yet

**Incompatible changes:**

**Deprecations:**

**Bug fixes:**

**Enhancements:**

**Cleanup:**

**Known issues:**

* See `list of open issues`_.

.. _`list of open issues`: https://github.com/andy-maier/python-yagot/issues

Commit the changes and push to upstream:

$ git status    # to double check which files have been changed
$ git commit -asm "Start $MNP"
$ git push --set-upstream origin start_$MNP

On Github, create a Pull Request for branch start_$MNP.

Important: When creating Pull Requests, GitHub by default targets the master branch. If you are starting a fix version, you need to change the target branch of the Pull Request to stable_$MN.
On GitHub, once all of these tests succeed:
- Merge the Pull Request (no review is needed)
- Delete the branch of the Pull Request (release_$MNP)
Checkout the branch the new version is based upon, update it from upstream, and delete the local topic branch you created:
```
$ git checkout $BRANCH
$ git pull
$ git branch -d start_$MNP
```
On GitHub, create a new milestone M.N.P for the version that is started.
On GitHub, list all open issues that still have a milestone of less than M.N.P set, and update them as needed to target milestone M.N.P.

5. Development¶

5.1. Repository¶

5.2. Setting up the development environment¶

5.3. Building the documentation¶

5.4. Testing¶

5.5. Contributing¶

5.6. Releasing a version to PyPI¶

5.7. Starting a new version¶

Table of Contents

Previous topic

Next topic

This Page