Developer’s Manual

This section provides some resources for developers interested in contribute to MCS as a solution for their own MIMA instrument control software.

Note

Large parts of this document came from the Pandas Contributing Guide.

MCS is built with modularity and flexibility in mind and makes use of the GitLab infrastructure for version control and to support collaborative development. Therefore it shall be easy for its user & developer community to directly contribute and expand the software capabilities by adding their own models, hardware controllers and processing feautures.

Bug reports and enhancement requests

Bug reports are an important part of making MCS more stable. Having a complete bug report will allow others to reproduce the bug and provide insight into fixing.

See this stackoverflow article and this blogspot for tips on writing a good bug report.

Trying the bug-producing code out on the master branch is often a worthwhile exercise to confirm the bug still exists. It is also worth searching existing bug reports and merge requests to see if the issue has already been reported and/or fixed.

Bug reports must:

  1. Include a short, self-contained python snipper reproducing the problem. You can format the code nicely by using GitLab Flavored Markdown:

    ```python
    >>> from mcs.io import load
    >>> cfg = load('config.yml')
    ...
    ```
    

#. Include the full version string of mcs and its dependencies. You can use the built in function:

>>> import mcs
>>> mcs.__version__
  1. Explain why the current behavior is wrong/not desired and what you expect instead.

The issue will be show up to the MCS community and be open to comments/ideas from others.

Working with the code

Now that you have an issue you want to fix, enhancement to add, or documentation to improve, you need to learn how to work with GitLab and the MCS code base.

Version control, Git, and GitLab

To the new user, working with Git is one of the more daunting aspects of contributing to MCS. It can very quickly become overwhelming, but sticking to the guidelines below will help keep the process straightforward and mostly trouble free. As always, if you are having difficulties please feel free to ask for help.

The code is hosted on GitLab. To contribute you will need to sign up for a free GitLab account. We use Git for version control to allow many people to work together on the project.

Some great resources for learning Git:

Getting started with Git

GitLab has instructions for installing git, setting up your SSH key, and configuring git. All these steps need to be completed before you can work seamlessly between your local repository and GitLab.

Get the source code by forking

You will need your own fork to work on the code. Go to the MCS project page and hit the Fork button (top right). You have to do this operation only once.

After that you will want to clone your fork to your machine. The following command creates the directory MCS.

$ git clone https://gitlab.com/YOUR-USER-NAME/mcs.git
$ cd mcs

Then the following command connects your repository to upstream (main project) MCS repository.

$ git remote add upstream https://gitlab.com/ppmime/mima.git

Now you can push/pull your _fork_ with git push and git pull.

Creating a development environment

To test out code changes, you’ll need to build MCS from source, which requires a Python environment. If you’re making documentation changes, you can skip to Contributing to the documentation but you won’t be able to build the documentation locally before pushing your changes.

Creating a Python Environment (conda)

Before starting any development, you’ll need to create an isolated MCS development environment:

We’ll now kick off a two-step process:

  1. Install the build dependencies

  2. Build and install MCS

# Create the new build environment (once)
$ conda env create -f environment.yml

# Activate the build environment
$ conda activate mcs-dev

# or with older versions of Anaconda:
$ source activate mcs-dev

# Build and install MCS in the new environment
(mcs-dev) $ pip install -e .

At this point you should be able to import MCS from your locally built version:

# Start an interpreter
$ python
>>> import mcs
>>> mcs.__version__
'2.0.3.dev9+g766507a.d20200703173647'

This will create the new environment, and not touch any of your existing environments, nor any existing Python installation.

To view your environments:

$ conda info -e

To return to your root environment:

$ conda deactivate

See the full conda docs here.

Creating a Python Environment (pip)

If you aren’t using conda for your development environment, follow these instructions:

  • You’ll need to have at least python3.6 installed on your system.

  • Make sure that you have cloned the repository

  • cd to the MCS source directory

# Create a virtual environment
# Use an ENV_DIR of your choice. We'll use ~/virtualenvs/mcs-dev
# Any parent directories should already exist
$ python3 -m venv ~/virtualenvs/mcs-dev

# Activate the virtualenv
$ . ~/virtualenvs/mcs-dev/bin/activate

# Install the build dependencies
$ python -m pip install -r requirements-dev.txt

# Build and install pandas
$ python -m pip install -e .

At this point you should be able to import MCS from your locally built version:

# Start an interpreter
$ python
>>> import mcs
>>> mcs.__version__
'2.0.3.dev9+g766507a.d20200703173647'

Creating a branch

You want your master branch to reflect only production-ready code, so create a feature branch for making your changes. For example:

$ git branch shiny-new-feature
$ git checkout shiny-new-feature

The above can be simplified to:

$ git checkout -b shiny-new-feature

This changes your working directory to the shiny-new-feature branch. Keep any changes in this branch specific to one bug or feature so it is clear what the branch brings to MCS. You can have many “shiny-new-features” and switch in between them using the git checkout command.

To update this branch, you need to retrieve the changes from the master branch:

$ git fetch upstream
$ git rebase upstream/master

This will replay your commits on top of the latest MCS git master. If this leads to merge conflicts, you must resolve these before submitting your merge request. If you have uncommitted changes, you will need to git stash them prior to updating. This will effectively store your changes and they can be reapplied after updating.

Contributing to the documentation

If you’re not the developer type, contributing to the documentation is still of huge value. You don’t even have to be an expert on MCS to do so! In fact, there are sections of the docs that are worse off after being written by experts. If something in the docs doesn’t make sense to you, updating the relevant section after you figure it out is a great way to ensure it will help the next person.

About the MCS documentation

The documentation is written in reStructuredText, which is almost like writing in plain English, and built using Sphinx. The Sphinx Documentation has an excellent introduction to reST. Review the Sphinx docs to perform more complex changes to the documentation as well.

Some other important things to know about the docs:

  • The MCS documentation consists of two parts: the docstrings in the code itself and the docs in this folder mcs/docs/.

    The docstrings are meant to provide a clear explanation of the usage of the individual functions, while the documentation in this folder consists of tutorial-like overviews per topic together with some other information (what’s new, installation, etc).

  • The docstrings follow the Numpy Docstring Standard, which is used widely in the Scientific Python community. This standard specifies the format of the different sections of the docstring. See this document for a detailed explanation, or look at some of the existing functions to extend it in a similar manner.

  • The tutorials make heavy use of the ipython directive sphinx extension. This directive lets you put code in the documentation which will be run during the doc build. For example:

    .. ipython:: python
    
        x = 2
        x**3
    

    will be rendered as:

    In [1]: x = 2
    
    In [2]: x**3
    Out[2]: 8
    

    Almost all code examples in the docs are run (and the output saved) during the doc build. This approach means that code examples will always be up to date, but it does make the doc building a bit more complex.

  • Our API documentation for models in docs/models.rst houses the auto-generated documentation from the docstrings. For classes, there are a few subtleties around controlling which methods and attributes have pages auto-generated.