.. _dev_manual: ################## Developer's Manual ################## This section provides some resources for developers interested in contribute to MCS as a solution for their own MIMA instrument control software. .. contents:: Table of contents: :local: .. note:: Large parts of this document came from the `Pandas Contributing Guide `_. MCS is built with modularity and flexibility in mind and makes use of the GitLab infrastructure for version control and to support collaborative development. Therefore it shall be easy for its user & developer community to directly contribute and expand the software capabilities by adding their own models, hardware controllers and processing feautures. Where to start? =============== TBW Bug reports and enhancement requests ==================================== Bug reports are an important part of making *MCS* more stable. Having a complete bug report will allow others to reproduce the bug and provide insight into fixing. See `this stackoverflow article `_ and `this blogspot `_ for tips on writing a good bug report. Trying the bug-producing code out on the *master* branch is often a worthwhile exercise to confirm the bug still exists. It is also worth searching existing bug reports and merge requests to see if the issue has already been reported and/or fixed. Bug reports must: #. Include a short, self-contained python snipper reproducing the problem. You can format the code nicely by using `GitLab Flavored Markdown `_:: ```python >>> from mcs.io import load >>> cfg = load('config.yml') ... ``` #. Include the full version string of *mcs* and its dependencies. You can use the built in function: .. code-block:: fish >>> import mcs >>> mcs.__version__ #. Explain why the current behavior is wrong/not desired and what you expect instead. The issue will be show up to the *MCS* community and be open to comments/ideas from others. .. _contributing.gitlab: Working with the code ===================== Now that you have an issue you want to fix, enhancement to add, or documentation to improve, you need to learn how to work with GitLab and the *MCS* code base. .. _contributing.version_control: Version control, Git, and GitLab -------------------------------- To the new user, working with Git is one of the more daunting aspects of contributing to *MCS*. It can very quickly become overwhelming, but sticking to the guidelines below will help keep the process straightforward and mostly trouble free. As always, if you are having difficulties please feel free to ask for help. The code is hosted on `GitLab `_. To contribute you will need to sign up for a `free GitLab account `_. We use `Git `_ for version control to allow many people to work together on the project. Some great resources for learning Git: * the `GitLab help pages `_. * the `NumPy's documentation `_. * Matthew Brett's `Pydagogue `_. Getting started with Git ------------------------ `GitLab has instructions `_ for installing git, setting up your SSH key, and configuring git. All these steps need to be completed before you can work seamlessly between your local repository and GitLab. .. _contributing.forking: Get the source code by forking ------------------------------ You will need your own fork to work on the code. Go to the `MCS project page `_ and hit the ``Fork`` button (top right). You have to do this operation only once. After that you will want to clone your fork to your machine. The following command creates the directory `MCS`. .. code-block:: fish $ git clone https://gitlab.com/YOUR-USER-NAME/mcs.git $ cd mcs Then the following command connects your repository to upstream (main project) *MCS* repository. .. code-block:: fish $ git remote add upstream https://gitlab.com/ppmime/mima.git Now you can push/pull your _fork_ with `git push` and `git pull`. .. _contributing.dev_env: Creating a development environment ---------------------------------- To test out code changes, you'll need to build *MCS* from source, which requires a Python environment. If you're making documentation changes, you can skip to :ref:`contributing.documentation` but you won't be able to build the documentation locally before pushing your changes. .. _contributing.dev_python: Creating a Python Environment (conda) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Before starting any development, you'll need to create an isolated MCS development environment: - Install either `Anaconda3 `_ or `miniconda3 `_ - Make sure your conda is up to date (launch command ``conda update conda``) - Make sure that you have :ref:`cloned the repository ` - ``cd`` to the *MCS* source directory We'll now kick off a two-step process: 1. Install the build dependencies 2. Build and install MCS .. code-block:: fish # Create the new build environment (once) $ conda env create -f environment.yml # Activate the build environment $ conda activate mcs-dev # or with older versions of Anaconda: $ source activate mcs-dev # Build and install MCS in the new environment (mcs-dev) $ pip install -e . At this point you should be able to import *MCS* from your locally built version: .. code-block:: fish # Start an interpreter $ python >>> import mcs >>> mcs.__version__ '2.0.3.dev9+g766507a.d20200703173647' This will create the new environment, and not touch any of your existing environments, nor any existing Python installation. To view your environments: .. code-block:: fish $ conda info -e To return to your root environment: .. code-block:: fish $ conda deactivate See the full conda docs `here `__. Creating a Python Environment (pip) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you aren't using conda for your development environment, follow these instructions: - You'll need to have at least python3.6 installed on your system. - Make sure that you have :ref:`cloned the repository ` - ``cd`` to the *MCS* source directory .. code-block:: fish # Create a virtual environment # Use an ENV_DIR of your choice. We'll use ~/virtualenvs/mcs-dev # Any parent directories should already exist $ python3 -m venv ~/virtualenvs/mcs-dev # Activate the virtualenv $ . ~/virtualenvs/mcs-dev/bin/activate # Install the build dependencies $ python -m pip install -r requirements-dev.txt # Build and install pandas $ python -m pip install -e . At this point you should be able to import *MCS* from your locally built version: .. code-block:: fish # Start an interpreter $ python >>> import mcs >>> mcs.__version__ '2.0.3.dev9+g766507a.d20200703173647' Creating a branch ----------------- You want your master branch to reflect only production-ready code, so create a feature branch for making your changes. For example: .. code-block:: fish $ git branch shiny-new-feature $ git checkout shiny-new-feature The above can be simplified to: .. code-block:: fish $ git checkout -b shiny-new-feature This changes your working directory to the shiny-new-feature branch. Keep any changes in this branch specific to one bug or feature so it is clear what the branch brings to *MCS*. You can have many "shiny-new-features" and switch in between them using the ``git checkout`` command. To update this branch, you need to retrieve the changes from the master branch: .. code-block:: fish $ git fetch upstream $ git rebase upstream/master This will replay your commits on top of the latest *MCS* git master. If this leads to merge conflicts, you must resolve these before submitting your merge request. If you have uncommitted changes, you will need to ``git stash`` them prior to updating. This will effectively store your changes and they can be reapplied after updating. .. _contributing.documentation: Contributing to the documentation ================================= If you're not the developer type, contributing to the documentation is still of huge value. You don't even have to be an expert on *MCS* to do so! In fact, there are sections of the docs that are worse off after being written by experts. If something in the docs doesn't make sense to you, updating the relevant section after you figure it out is a great way to ensure it will help the next person. .. contents:: Documentation: :local: About the *MCS* documentation ------------------------------- The documentation is written in **reStructuredText**, which is almost like writing in plain English, and built using `Sphinx `__. The Sphinx Documentation has an excellent `introduction to reST `__. Review the Sphinx docs to perform more complex changes to the documentation as well. Some other important things to know about the docs: - The *MCS* documentation consists of two parts: the docstrings in the code itself and the docs in this folder ``mcs/docs/``. The docstrings are meant to provide a clear explanation of the usage of the individual functions, while the documentation in this folder consists of tutorial-like overviews per topic together with some other information (what's new, installation, etc). - The docstrings follow the **Numpy Docstring Standard**, which is used widely in the Scientific Python community. This standard specifies the format of the different sections of the docstring. See `this document `_ for a detailed explanation, or look at some of the existing functions to extend it in a similar manner. - The tutorials make heavy use of the `ipython directive `_ sphinx extension. This directive lets you put code in the documentation which will be run during the doc build. For example:: .. ipython:: python x = 2 x**3 will be rendered as:: In [1]: x = 2 In [2]: x**3 Out[2]: 8 Almost all code examples in the docs are run (and the output saved) during the doc build. This approach means that code examples will always be up to date, but it does make the doc building a bit more complex. - Our API documentation for models in ``docs/models.rst`` houses the auto-generated documentation from the docstrings. For classes, there are a few subtleties around controlling which methods and attributes have pages auto-generated.