I've kicked off a somewhat ambitious hobby project in Clojure (as discussed here). I keep itching for types, which got me thinking about what problems I actually want to solve. Obviously (or mabye not, I hadn't given it that much thought before) types aren't an end to themselves, so what are the concrete problems I actually want to solve:
Out of date comments are one of my bugbears. I read a comment, take it at face value, and then go off and make something that doesn't work. Types offer mechanically checked comments; if the type of a function is (-> int int int), and the program still compiles, at least you know something about that function.
Clojure doesn't have annotations, but argument destructuring seems like an often adequate replacement. It's worth noting first, yes, my current project is young, but I've been satisfied with how, e.g. a function looking like (fn [{:foo/keys [bar baz]}] (...)) tells me up front that it takes a map as its first argument and the map must have the keys :foo/bar and :foo/baz.
So, the Clojure approach is flexible, maybe not as flexible as the Type based approach, but it seems sufficient if things can be kept simple[0] which seems, if not easy, at least possible.
I worked on a Python 2 project for several years. The codebase was in the hundreds of thousands of lines, and changing a function always felt risky. When Python's optional type system and MyPy became widely available I started adding types to everything and it felt great; you could finally move things around with some confidence.
A few aspects of refactoring MyPy provided assurances for. First is when a function argument type changes, which is really subset of distinguishing similar types so I won't discuss that much here. Second, function arity changes; third functions that are moved, removed, or renamed.
It turns out Clojure does check and at least emit warnings if a function is missing or does not exist at the expected arity. It's also the case that Clojure is simple to parse and manipulate at the AST level, so there are refactoring tools which supposedly handle this for you[1]. As such We're just left wanting a way to [distinguishing similar types](#Distinguish similar types).
It is useful to distinguish type in general, but if you expect an integer and a passed a kazoo then the error should be caught pretty quickly. More pernicious are types which are pretty close. I'm again thinking of my experience with Python 2, in particular unicode and byte strings. File handling and email seemed particularly prone to these kinds of problems, despite our automated testing efforts – everything would work fine in development until the program went out and it would turn out some buried file read was only handling ASCII file paths.
Python's solution is two fold. Famously Python 3 makes some breaking changes so byte strings are not the default literal type and are less convenient to use in string processing contexts. This forces some discipline around encoding and decoding. Python also began offering a gradual typing system with PEP 484. I liked the static type system and spent a great deal of time annotating our code and adding .pyi stub files for our dependencies.
The trouble with gradual typing is it is basically useless until you have sufficient coverage. If you're testing a function by passing a kazoo in tests and it's passed a string in production then it's still going to blow up at runtime and your static checks offered only false confidence. The problem with gradual typing as mechanically checked documentation is similar – until type checking is pervasive your annotations are approximate at best.
Adding these annotations is time consuming, and there are running-code edge cases which are nearly impossible to type check. We used a framework called Twisted which, with Python 2 at least, wrapped Generator functions to allow for synchronous looking async code (i.e. your function would yield on async calls and an event loop would re-enter the generator once it had a result). The trouble is that async is contagious so many of our functions used this pattern and Generator supports only one yield type. So what do you do for a function that calls multiple async functions internally (which is quite common)? Well, we just couldn't type the yielded values any more precisely than Any. I spent weeks working on and off on an alternate implementation which would allow functions to be invoiced something like my_func(d, some, args, here)(yield d). The idea was that the caller would be of type Generator[Deferred, YieldedType], my_func would return a function which returned a casting function from (-> YieldedType DesiredType). At runtime my_func would add itself to a stack held by d, yielding on d would evaluate the function in the typical twisted style (i.e. re-enter the generator once the event fired) and the casting function simply existed to make type checks happen – it was an alias for identity. All this makes weird looking function bodies and the implementation was complicated, so it was never shipped and we just lived with swaths of unchecked code..
Anyway; Python 3 adds async and await which are recognized by MyPy et alia, so that particular problem is solved-ish, but I think the point can be generalized a bit. Type systems help avoid some problem classes, and often adding them is low overhead[2], but you occasionally run into an edge case that eats a couple weeks of your life (want to make a generic function in Go? Use interface {} and good luck to you if you want compile time checks).
So, now we round the bend back to Clojure. There is a gradual type system for Clojure, which I may try at some point, but there's a different Official-ish solution: runtime schemes (i.e. spec or, less officially, malli and others). Rich Hicky seems adamant that these runtime schemes are a better solution than types; so – well, I haven't used them enough to say really, but they do have different trade offs. The schema based solution is twofold; first the schema can be used to generate random values, and second the schema can be used to validate input/output at run time. Using the random values in tests and ensuing the same scheme is followed in production (or at least during manual testing) covers the same ground as a type system, but it's probabilistic rather than deterministic. Probabilistic seems worse, but in the low annotation case it does just work pretty well. A schema can drive a functional test usefully without annotating all the internal functions. So, weaker guarantees for cheaper. On the third, the runtime schemes can provide arbitrary predicates to verify relatively complex behaviors (i.e. you can check that your function [:vector any?] => [:vector any] is reversed or what have you), so even if the schema is non deterministic it is in some other sense a more powerful way to express the expected behavior (at least compared to the Python/Rust/Go/Haskell systems I've tried. I think it's safe to generalize though; if you're allowing arbitrary predicates they can't be generally checked at compile time (assuming you always want your compiler to terminate)). You also have the schema available at runtime for meta-programming, which is neither here nor there, but can be useful. Is that a worthwhile trade off? I don't know yet, but I'm giving it a shot.
[0]: Or, if not uniformly simple at least keeping complexity localized. [1]: Python also has such tools, I just never had confidence they updated everything in a complex program (becacuse usually they didn't). [2]: Although, I certainly spent tens of hours (maybe a hundered?) adding trivial type annotations to the 200k line Python project, so maybe the overhead isn't low but rather hidden by amortization in the typical cases.