-
Notifications
You must be signed in to change notification settings - Fork 1
Checkouts directory
A technique for developing libraries
Development, Libraries, Leiningen
A checkouts directory is a great tool for simultaneous development of
multiple projects.
The Clojure ecosystem strongly encourages us to write single-purpose libraries, rather than monolithic frameworks. This means that if you are working on a project of any real size, you should probably split sections off into their own libraries.
Your main project will then depend on these new libraries. More precisely, it will depend on a particular version of each library.
If you are simultaneously coding your main project and one or more libraries, this can quickly get difficult. It is hard to keep the right versions loaded together, especially if you are working in a REPL and making continuous changes.
To make this work better, you can do a few things. Let's review snapshot versions and lein install, and then look at how to use the checkouts directory feature in Clojure and ClojureScript projects.
Clojure libraries usually have version numbers consisting of three integer fields,
separated by periods (e.g., "3.0.47"). Often, this numbering follows
semantic versioning guidelines.
Library versions can also be suffixed with the "-SNAPSHOT",
e.g. "3.0.48-SNAPSHOT". This has a special meaning to Leiningen and the other
build tools: It says that this version is still changing. If your project depends on a
snapshot version of a library, Leiningen will check for new versions daily.
For loosely coupled or slowly changing libraries, this is often a sufficient solution.
Important note: Snapshot dependencies are intrinsically unstable. A library snapshot may point to a completely different version tomorrow, or next year. Any version of your project that needs to remain unchanged (whether for deployment or repeatable testing) must not depend on snapshot versions of any libraries.
Normally, Leiningen finds dependencies online, downloading them from Clojars or
Maven. But, if you are developing a library, you need to test it locally before you
deploy. You can install a project directly into your computer's local cache with the
command lein install.
If you are actively running your main project, you will then need to reload the dependency. this needs to be explained, for the various tool chains.
There is one solution that lets you work smoothly and simultaneously on your main
and library project. This involves creating a project directory called checkouts.
This technique is discussed in detail in the checkout-dependencies section of the Leiningen documentation, and is pretty simple:
- In your main project, create a directory
checkouts - In
checkouts, create a symbolic link to the root of your library project. - The library project must also be installed with the current version number. (Again, this is explained in more detail in the Lein docs. The basic idea is that the checkouts directory overrides where Lein looks for the library files, but it does not subvert the full version search mechanism).
Under Linux or Macintosh, this is just:
cd ~/my-project
mkdir checkouts
cd checkouts
ln -s ~/my-lib my-lib
cd ~/my-lib
lein install
I don't know if this is possible under Windows. I just discovered that Windows
now has symlinks with
the mklink command, but I have never tried this.
Later: I just discovered lein-checkouts. I've not yet tried it, but this looks like it should make it much easier to work with checkouts.
Finally, the real point of this essay!
The checkouts directory solution works nicely for pure Clojure projects. But, the picture gets more complicated for modern ClojureScript projects which usually include complex tooling including Figwheel and Cljsbuild.
The quick answer is that you can make it work with small changes to your main
project's project.clj. Search for :source-paths and add in a reference
to the library link in checkouts. So, for example, you should get something
like:
:source-paths ["src" "checkouts/my-lib/src"]
Now, you can make changes in both projects, and Figwheel will automatically pick up all the changes in real time.
This technique is discussed in more detail here.
A few subtleties:
- This is definitely needed for the
:source-pathsinside{:cljsbuild {:builds [{:id "dev" :source-paths ...}]}}. Maybe in your other builds too, if you need to actively test them locally. - In my projects, I've also done this for the top-level
:source-paths. I don't remember why I did this. It might not be needed. - I've only started using this recently, and have not yet automated deployment of my projects using this. I can see this being a problem, since the checkouts directory will not generally exist. On my manual deployments, I got a warning, though it seemed to be harmless.
Some open issues:
- There has been active work and discussion this year (2017) to make this be more automated: See this PR.
- This blog post from a while back, discusses some related issues.
You can help expand this article if you can write about: