Well, it’s been a while since I’ve had my development environment running. It is crazy how things can devolved. To remind everyone, in my normal world, I use asdf, direnv and pipenv (and now poetry and conda) to manage things. The basic idea is that when I switch to a new directory, I get a new environment that is set up with a dotfile at the root of a github repo:
- direnv and .envrc. This sets environment variables. A good example would be a specific AWS token that I might need for a particular project. The main point here is *not to check that in!* The alternative I’ll probably move to is to put all such security tokens into 1Password as they’ve improved their dev support alot. My legacy way is to use my own handbuilt veracrypt secure disk which has the secrets and then symlink into .ssh all my secrets. It works well, but is brittle.
- poetry. This is used by my make files to run the right environment. You can do a poetry install to put specific pip packages into the environment and you can poetry run to run a command line or poetry shell to drop into it interactively.
- pipenv. I used to use pipenv for this purpose, but it was so slow in creating environments. On the other hand, it was super convenient because just changing into a directory would change the python environment.
- asdf. With .tool-versions, you get much the same as pipenv, that is a virtual environment for python that changes by directory. It is more general than pipenv in that you can switch basically anything (like java for instance) and it can control direnv too.
- make and Makefiles. This is a lot to keep up with so I have a system where a single Makefile let’s me bootstrap all the files that I need. It is sort of like a github template in that way (and I may switch to that mode), but a “cp Makefile.base Makefile && make install-repo” does the trick.
- Conda. conda is more powerful than poetry in that it also installs binaries and in this case, with all this machine-learning stuff, you need those. So I normally have a single conda environment with everything that I need in it and then “conda activate” it as needed
Pipenv you can forget it is there
My problem was that one particular project kept failing both neovim and pre-commit installation of jupytext. I went crazy for a while, but the ways to debug this are:
- Run neovim and then “:SystemHealth” this will tell you what python it is looking for. I got a strange result. Normally with poetry, you can look at the version and see the directory “poetry” in it. But this time it wasn’t it. Turns out that when I moved from pipenv to poetry, I didn’t do a “pipenv –rm” in that directory. Once done, this let Neovim find the asdf poetry
- The difficult part is recognizing which virtual environment tool uses what hidden directories in your home directory. Pipenv things look like ~/pyenvs/_dir_/ so that’s a giveaway. The command ‘pipenv —venv’ will tell you if there’s a Pipenv under there
