til/python/rye.md
# A few notes on Rye
[Rye](https://github.com/mitsuhiko/rye) is Armin Ronacher's new experimental Python packaging tool. I decided to take it for a test-run.
## Installing Rye
Rye is built in Rust, and currently needs to be installed using Cargo. I had Cargo from previous dalliances with Rust, so I could install Rye using:
cargo install --git https://github.com/mitsuhiko/rye rye
This resulted in a 6.4MB binary file in:
~/.cargo/bin/rye
I added the following line to my `~/.zshrc` file to make it available on my path:
export PATH=$PATH:$HOME/.cargo/bin
## Installing Python with Rye
The Rye docs suggested running this:
rye pin cpython@3.11
As far as I can tell all this did was write `3.11.1` to a `~/.python-version` file. I'm not sure if there are any tools other than Rye which pay attention to this file. (UPDATE: [pyenv uses this file too](https://github.com/pyenv/pyenv#understanding-python-version-selection))
Actually fetching versions of Python can be done using `rye toolchain fetch VERSION`, for example:
```
% rye toolchain fetch 3.9
Downloading cpython@3.9.16
success: Downloaded cpython@3.9.16
```
This placed a whole bunch of files in `~/.rye/py/cpython@3.9.16/` - `find . | wc -l` reported 4,085 in that directory.
The one that matters most is:
~/.rye/py/cpython@3.9.16/install/bin/python3
Which gives me a 3.9.16 Python interpreter.
This is the feature of Rye I'm most excited about: the Python bundles it installs come from Gregory Szorc's [indygreg/python-build-standalone](https://github.com/indygreg/python-build-standalone) project.
This should mean that they completely ignore the many other weird ways that Python can end up installed on a system. Admittedly, this is a _new_ weird way to install Python - but at least it shouldn't clash with anything else.
Normally though you wouldn't use `rye toolchain fetch` at all. The Rye suggested workflow is to run `rye init` in a new directory to create a `pyproject.toml` file:
```
% cd /tmp
/tmp % mkdir my-project
/tmp % cd my-project
my-project % rye init
success: Initialized project in /private/tmp/my-project/.
my-project % ls
README.md pyproject.toml
my-project % cat pyproject.toml
[project]
name = "my-project"
version = "0.1.0"
description = "Add a short description here"
authors = [
{ name = "Simon Willison", email = "...@gmail.com" }
]
dependencies = []
readme = "README.md"
requires-python = ">= 3.8"
license = { text = "MIT" }
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.rye]
managed = true
my-project % cat README.md
# my-project
Describe your project here.
* License: MIT
```
To add dependencies to that project, use `rye add`:
rye add httpx
This added the following line to `pyproject.toml`:
dependencies = ["httpx~=0.24.0"]
Then to actually install the dependencies, run `rye sync`:
```
my-project % rye sync
Initializing new virtualenv in /private/tmp/my-project/.venv
Python version: cpython@3.11.1
Generating production lockfile: /private/tmp/my-project/requirements.lock
Generating dev lockfile: /private/tmp/my-project/requirements-dev.lock
Installing dependencies
...
Successfully built my-project
Installing collected packages: sniffio, idna, h11, certifi, anyio, httpcore, httpx, my-project
Successfully installed anyio-3.6.2 certifi-2022.12.7 h11-0.14.0 httpcore-0.17.0 httpx-0.24.0 idna-3.4 my-project-0.1.0 sniffio-1.3.0
Done!
```
This adds a `requirements.lock` file that looks like this:
```
# generated by rye
# use `rye lock` or `rye sync` to update this lockfile
-e file:.
anyio==3.6.2
certifi==2022.12.7
h11==0.14.0
httpcore==0.17.0
httpx==0.24.0
idna==3.4
sniffio==1.3.0
```
And a `requirements-dev.lock` file with identical contents. Presumably this starts to differ as you install dev requirements (another Rye feature).
Your folder will now have a `.venv` hidden folder in it. Inside that is a Python virtual environment containing your installed dependencies and a file called `rye-venv.json` which just contains:
```json
{
"python": "cpython@3.11.1"
}
```
## Installing global applications
The `rye add X` command adds a dependency to your current virtual environment / Rye project.
`rye install` does something completely different: it installs new global packages, in a similar fashion to `pipx` in that they get their own isolated environments so their dependencies don't clash with other installed applications.
```
% rye install cowsay
Collecting cowsay
Downloading cowsay-5.0.tar.gz (25 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: cowsay
Building wheel for cowsay (pyproject.toml) ... done
Created wheel for cowsay: filename=cowsay-5.0-py2.py3-none-any.whl size=25707 sha256=de872e9ef328d25cd9ac58df693c7bfd913033f696282c60562854d0db38737e
Stored in directory: /Users/simon/Library/Caches/pip/wheels/d4/2d/c7/c018bd8e6f825d6b47ae38f28baabd4588b3857e0e7dbc8cd3
Successfully built cowsay
Installing collected packages: cowsay
Successfully installed cowsay-5.0
installed script cowsay
```
You can now run `cowsay` with `~/.rye/shims/cowsay`:
```
% ~/.rye/shims/cowsay hello
_____
| hello |
=====
\
\
^__^
(oo)\_______
(__)\ )\/\
||----w |
|| ||
```
You can add `~/.rye/shims` to your `$PATH` to make these commands available everywhere.
## Failing and then succeeding to install Datasette
I tried to install Datasette using `rye install datasette` and got this error:
```
% ~/.rye/shims/datasette
Traceback (most recent call last):
File "/Users/simon/.rye/shims/datasette", line 5, in <module>
from datasette.cli import cli
File "/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/cli.py", line 17, in <module>
from .app import (
File "/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/app.py", line 14, in <module>
import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'
```
Rye has strong opinions, including omitting `pip` and `setuptools` entirely from the environments that it creates.
[ **UPDATE**: I released [Datasette 0.64.3](https://docs.datasette.io/en/stable/changelog.html#v0-64-3) with a fix for this and now it installs correctly under Rye ]
It turns out Datasette includes code that imports `pkg_resources`, assuming that `setuptools` will be present because it's usually there as a Python environment default!
I added `setuptools` to Datasette's `setup.py` dependencies in an attempt to fix that, in [issue #2065](https://github.com/simonw/datasette/issues/2065).
When I'm using `pip` I often install development copies of my projects by feeding them the URL to a `.zip` generated by GitHub, for example:
```
pip install https://github.com/simonw/datasette/archive/main.zip
```
I tried that with `rye install` and got an error:
```
% rye install https://github.com/simonw/datasette/archive/main.zip
Error: Expected one of `@`, `(`, `<`, `=`, `>`, `~`, `!`, `;`, found `:`
https://github.com/simonw/datasette/archive/main.zip
^
```
Evidently Rye doesn't (yet?) support installing from URLs.
> **UPDATE**: Armin [showed me a way](https://twitter.com/mitsuhiko/status/1651470124015054848) to do this:
>
> rye install 'datasette @ https://github.com/simonw/datasette/archive/main.zip'
>
> This worked - running `~/.rye/shims/datasette --version` confirmed the installation.
Instead, I downloaded the `main.zip` file and ran Rye install against that directly:
```
% rye install main.zip
Processing ./main.zip
Installing build dependencies ... done
Getting requirements to build wheel ... done
...
Successfully built datasette
...
```
That seemed to work. Oddly it didn't add `datasette` to the `.rye/shims` directory - but I did find a working installation here instead:
~/.rye/tools/main-zip/bin/datasette