sqlite-utils/docs/contributing.rst
.. _contributing:
==============
Contributing
==============
Development of ``sqlite-utils`` takes place in the `sqlite-utils GitHub repository <https://github.com/simonw/sqlite-utils>`__.
All improvements to the software should start with an issue. Read `How I build a feature <https://simonwillison.net/2022/Jan/12/how-i-build-a-feature/>`__ for a detailed description of the recommended process for building bug fixes or enhancements.
.. _contributing_checkout:
Obtaining the code
==================
To work on this library locally, first checkout the code. Then create a new virtual environment::
git clone git@github.com:simonw/sqlite-utils
cd sqlite-utils
python3 -mvenv venv
source venv/bin/activate
Or if you are using ``pipenv``::
pipenv shell
Within the virtual environment running ``sqlite-utils`` should run your locally editable version of the tool. You can use ``which sqlite-utils`` to confirm that you are running the version that lives in your virtual environment.
.. _contributing_tests:
Running the tests
=================
To install the dependencies and test dependencies::
pip install -e '.[test]'
To run the tests::
pytest
.. _contributing_docs:
Building the documentation
==========================
To build the documentation, first install the documentation dependencies::
pip install -e '.[docs]'
Then run ``make livehtml`` from the ``docs/`` directory to start a server on port 8000 that will serve the documentation and live-reload any time you make an edit to a ``.rst`` file::
cd docs
make livehtml
The `cog <https://github.com/nedbat/cog>`__ tool is used to maintain portions of the documentation. You can run it like so::
cog -r docs/*.rst
.. _contributing_linting:
Linting and formatting
======================
``sqlite-utils`` uses `Black <https://black.readthedocs.io/>`__ for code formatting, and `flake8 <https://flake8.pycqa.org/>`__ and `mypy <https://mypy.readthedocs.io/>`__ for linting and type checking.
Black is installed as part of ``pip install -e '.[test]'`` - you can then format your code by running it in the root of the project::
black .
To install ``mypy`` and ``flake8`` run the following::
pip install -e '.[flake8,mypy]'
Both commands can then be run in the root of the project like this::
flake8
mypy sqlite_utils
All three of these tools are run by our CI mechanism against every commit and pull request.
.. _contributing_just:
Using Just and pipenv
=====================
If you install `Just <https://github.com/casey/just>`__ and `pipenv <https://pipenv.pypa.io/>`__ you can use them to manage your local development environment.
To create a virtual environment and install all development dependencies, run::
cd sqlite-utils
just init
To run all of the tests and linters::
just
To run tests, or run a specific test module or test by name::
just test # All tests
just test tests/test_cli_memory.py # Just this module
just test -k test_memory_no_detect_types # Just this test
To run just the linters::
just lint
To apply Black to your code::
just black
To update documentation using Cog::
just cog
To run the live documentation server (this will run Cog first)::
just docs
And to list all available commands::
just -l
.. _release_process:
Release process
===============
Releases are performed using tags. When a new release is published on GitHub, a `GitHub Actions workflow <https://github.com/simonw/sqlite-utils/blob/main/.github/workflows/publish.yml>`__ will perform the following:
* Run the unit tests against all supported Python versions. If the tests pass...
* Build a wheel bundle of the underlying Python source code
* Push that new wheel up to PyPI: https://pypi.org/project/sqlite-utils/
To deploy new releases you will need to have push access to the GitHub repository.
``sqlite-utils`` follows `Semantic Versioning <https://semver.org/>`__::
major.minor.patch
We increment ``major`` for backwards-incompatible releases.
We increment ``minor`` for new features.
We increment ``patch`` for bugfix releass.
To release a new version, first create a commit that updates the version number in ``setup.py`` and the :ref:`the changelog <changelog>` with highlights of the new version. An example `commit can be seen here <https://github.com/simonw/sqlite-utils/commit/b491f22d817836829965516983a3f4c3c72c05fc>`__::
# Update changelog
git commit -m " Release 3.29
Refs #423, #458, #467, #469, #470, #471, #472, #475" -a
git push
Referencing the issues that are part of the release in the commit message ensures the name of the release shows up on those issue pages, e.g. `here <https://github.com/simonw/sqlite-utils/issues/458#ref-commit-b491f22>`__.
You can generate the list of issue references for a specific release by copying and pasting text from the release notes or GitHub changes-since-last-release view into this `Extract issue numbers from pasted text <https://observablehq.com/@simonw/extract-issue-numbers-from-pasted-text>`__ tool.
To create the tag for the release, create `a new release <https://github.com/simonw/sqlite-utils/releases/new>`__ on GitHub matching the new version number. You can convert the release notes to Markdown by copying and pasting the rendered HTML into this `Paste to Markdown tool <https://euangoddard.github.io/clipboard2markdown/>`__.