A Year (and Change) of Rust

#

I've been writing Rust for a few years now. My first (unfinished) Rust project was committed to GitHub in 2018. However, it is only since April of last year (2021) that I have been writing Rust full time. It is now September of 2022, which means I am five months late in writing this post, which I originally intended to get done for my one-year anniversary at Spec. Better late then never, I suppose!

My intention in this post is to take a fair look at Rust given what is now a significant amount of experience and expertise. I have built large, complex projects in Rust, worked on a team with other Rust engineers, and shipped huge amounts of Rust to production. What initial assumptions did I have that turned out to be false? What were the surprises? Am I disillusioned at all with the language? Given that it was an explicit goal in my last job search to work with Rust if possible, would I make that decision again today? Do I intend to keep writing Rust in future jobs? Read on to find out!

The Good

#

Tooling

#

Rust's tooling continues to be among the best I've ever worked with. Little things like offline docs still wow me, especially given that I take at least one or two trips by train per year, with little to no WiFi for several days running. I'll split this section up into particular tools that I think are great.

rust-analyzer

#

The long de-facto and now official standard language server for Rust, this is simply one of the best pieces of engineering in modern development tooling, period. It is incredibly fast, even on very large projects, it can be easily configured to run cargo clippy instead of cargo check, so you get nearly instantaneous detailed feedback on the code you're writing. It provides inlay hints, which show you as you're writing code what the compiler will infer any given type to be. I find the inlay hints helpful everywhere, but especially helpful in long chains of Result- or Option-handling calls like .and_then(), .map(), etc.

rust-analyzer also provides a ton of really helpful code actions. Some of my most used are:

• rename: renames every usage of an identifier across the entire codebase, incredibly reliable
• add_impl_missing_members: adds missing trait method signatures to an impl for that trait
• add_missing_match_arms: automatically fills match statements for enums and other types
• generate_new: intelligently creates a new() function for a struct, adding an impl block if needed
• generate_documentation_template: actually does a pretty solid job at auto-documenting what your functions do, based on their name and other info
• flip_comma: a deceptively useful little action to flip two arguments around a comma, very nice when refactoring function parameters
• make_raw_string, and its inverse make_usual_string: flips a regular string to a raw string or vice versa. Super useful when you're in the middle of writing a string and want to add quotation marks
• remove_dbg: easily rip out all the extra dbg!() calls from when I'm trying to figure something out
• auto_import: adds imports to the file automatically as I'm writing code

There are a bunch more. You can see the full list here.

Anyway, at this point, rust-analyzer is a significant part of why I love working with Rust. I've never seen anything better in another language, and I hope it serves as the gold-standard in language tooling moving forward.

Cargo

#

Cargo is great, and its extension API means that third-parties can fill whatever gaps it has. Cargo prevents us from needing to worry about maintaining complicated Makefiles to build our code (we do use Makefiles for other things, but it's nice to have the Rust stuff taken care of). Adding packages is easy, cross-compiling for other targets is easy, updating dependencies is easy, and (with third-party libraries) checking for outdated packages and security vulnerabilities is easy.

Language Design

#

The Rust language is a pragmatic blend of functional and imperative programming, and I think the result is a language that feels productive and high-level, while promoting good habits and helping developers fall into the Pit of Success. The language itself makes it hard to make certain classes of mistakes. The Monad-ish Result and Option types are a great example of this: one cannot simply fail to handle an error or a null case, because the type system won't let you. Similarly, the various Guard types, such as the one returned by Mutex, make brilliant use of Rust's borrow checking and Drop semantics to ensure that you can't forget to unlock a lock or do whatever other cleanup you should be doing.

Traits are a great way to enforce interfaces without opting into all the complexity of object-orientation. I find traits a lot easier to reason about than inheritance trees, especially for anything complex. The interaction of traits with generics via trait bounds I think is also mostly very pleasant. The ability to make a RealDataStore and a FakeDataStore, which both implement the DataStore trait, and then pass either of these into a function that takes a store: impl DataStore parameter is wonderful, especially because this does not require opting into dynamic dispatch or any kind of heap allocation.

It's relatively easy to follow the Parse, Don't Validate philosophy in Rust, designing types whose structure and instantiation enforce invariants in the system. This leads to robust code that's easy to understand, and easy to refactor. Rust definitely provides an "if it compiles, it works" feeling, which is something that I've gotten very used to and that it's hard to do without.

I also appreciate that the language design (and the community) is pretty unopinionated, which is a nice breath of fresh air coming from Python. I see plenty of code using functional-style chaining, and plenty of code using regular old loops. In our codebase, we tend to use whatever solution feels better for the current problem (async code often winds up in loops because it's hard to do async in closures).

The Compiler / the Borrow Checker

#

The borrow checker gets a lot of heat outside of the Rust community, especially it seems from veteran C/C++ devs who are upset that it doesn't let them write code exactly the same way they are used to writing it. It also gets heat from beginners who feel as though it makes the language too hard to learn. Both things are true to some degree. For the former, the Rust compiler team would eventually like to make as much correct code verifiable as possible, meaning the bounds of what the compiler considers safe should continue to increase as Rust evolves (as indeed it has been so far, see e.g. non-lexical lifetimes from the 2018 edition). For the latter, the team that works on Rust errors views the compiler as a teaching tool, and is constantly working on making the error messages more informative, useful, and accessible.

From the perspective of someone who has been writing almost exclusively Rust, though, I love the compiler and the borrow checker. As I've worked with Rust, the experience of getting things to work and of reading the compiler messages when they don't have helped me to better understand the underlying reasoning behind what the borrow checker considers valid and what it doesn't. Most of the time now, I write code the borrow checks correctly on the first try, without thinking too much about it, because I have internalized many of the rules around ownership. Being able to lean on the compiler and to just know that my code is safe to run and likely to be correct as long as my it satisfies the type system and the borrow checker is a glorious feeling, especially when I'm in a hurry and trying to just get things done.

The other thing the borrow checker enables, and one of its major raisons d'être, is "fearless concurrency." The main purpose of most of Rust's ownership rules is to ensure that shared state is safe, whether that state is shared within a single thread or across threads. I can say that in my experience at least this promise is realized. Data races simply aren't a thing.

There is also the sort of subtle inverse benefit that you are able to trust the Rust compiler when removing safety. A great example of this in our codebase was removing an Arc around a piece of shared data and replacing it with immutable references. We didn't have to go through and introspect everywhere the value was being used to be sure the swap was safe: we could just that if we removed it, and the compiler didn't complain, we were good to go. This allows us to default to the minimum possible overhead, letting the compiler tell us when we need more. This keeps everything lean and avoids the issue where we preemptively wrap shared data in an Arc<T> or an Arc<Mutex<T>> just in case.

The compiler also makes many refactors a breeze and significantly increases the confidence in the result of refactored code. With my editor (emacs), I can run cargo check, pop over to the results window, press enter on an error, and it takes me right to the code. This makes stepping through all of the places where I need to fix things during a refactor trivially easy. When there are many places that takes a similar change, I can even apply those changes with a keyboard macro for almost zero-effort refactors.

Finally, the compiler has also helped me to become a better programmer (at least in my estimation). I think a lot more about the data flow within my programs now, which I think has helped me to keep the architecture simpler and more focused. By focusing on the data (in the same style as in much of functional programming), it becomes more natural to create simple, composable systems. Rust has also helped me to become a more pragmatic engineer: because of the trust I am able to place in the compiler and my confidence in refactoring, I find myself less worried about ensuring that everything is done the "right" way, and more concerned about ensuring that things are done in a way that works well enough right now and is easy to change later.

Libraries

#

The Rust ecosystem is still young and relatively small, depending on your niche, but I have generally found that the quality of third-party libraries is very high. Certain crates make their way into almost every Rust crate I write (regex, anyhow, thiserror, itertools, serde, chrono, uuid, etc.), and these have been rock-solid for the entire duration of my tenure with Rust.

There are also libraries that do things I've never seen done in other languages, often through the power of procedural macros. The ability to simply inline another language or specification format (à la the json!() macro in serde_json) is really useful. The sqlx library, which we use for communication with our database, has a query!() macro that is able to validate queries against a local database as you write them, including inferring column names and types on the returned data! This surfaces as compilation errors both simple sql mistakes like trailing commas and more complex errors like passing in an incorrect type for a bind parameter. This is another thing that I don't want to live without, now that I've got it.

Robustness

#

The Rust code I write just turns out to have fewer bugs than the code I write in most other languages. I don't think that I've improved so much as an engineer in the last year and a half as to make a significant difference in the number of bugs I write, so I have to assign this to the language itself. Potentially, this would also be true of other statically typed languages, but I'm not convinced. I think the expressiveness of Rust's type system (sum types, Result/Options, traits), combined with the borrow checker's prevention of certain classes of data-access-related bugs, really does make for more robust software at the end of the day. A lot of the time, it feels like you get the stability of a functional programming language with the ease-of-use of an imperative language and the speed of a low-level language, which is a really wild ride.

Speed

#

I almost didn't mention this, but Rust is incredibly fast, even without spending a lot of time optimizing. Some of this is probably because we mostly avoid dynamic dispatch, but I have never not been impressed by the speed whenever we've measured it.

WebAssembly

#

Our frontend is obviously in JavaScript, and it's a real killer feature to be able to take the domain types that we use to define invariants and business logic on the Rust side and expose them straight to the frontend. This all being a part of the same codebase means that we are able to roll changes across the stack just by making changes to the Rust code. Other languages of course can compile to WebAssembly, but the support for it is quite good in Rust.

The Bad

#

Language Design

#

There's not a lot I don't like about the Rust language, but there are definitely some pain points. Most of these are being worked on by the language teams, so I'll link to proposals and thoughts on them as I go through.

Async/Await

#

I actually think that for the most part, async/await is pretty intuitive and easy to use in Rust. However, there are some very rough edges! One of the major ones is incomplete support for async closures, making it difficult to do .map(), .and_then(), .or_else(), and so on when dealing with futures. The futures crate has some extension traits that help with this, but they can definitely be confusing. I would definitely like to see better support for this.

The combination of async with traits can lead to some very nasty errors. Currently async traits are provided via a third-party library, and their lack of native support occasionally causes issues. For example, I spent several days trying to figure this one out before eventually asking on the Rust forums and getting help. That issue inspired us to set up a git hook to automatically add certain clippy lints to every main.rs and lib.rs file in the codebase, so we would never be bitten by that particular issue again.

All that said, I don't find async code that hard to work with in Rust, and the future of async as detailed by the language team looks very promising, so I'm looking forward to seeing this continue to evolve!

The Compiler

#

My only real complaint about the compiler is that it (probably necessarily) operates in multiple passes: the first infers types, the second checks trait coherence, and the third does type checking. This is all well and good, but let's say you change the signature of a struct somewhere deep in your code, or a trait that is used in a lot of places, to include a lifetime. You dutifully go through and add the lifetime everywhere you need to add it, following the compiler's error trail, and then, once you finish, you're greeted with new errors that show that your update to your struct/trait won't work anyway, because the new reference causes ownership issues in one of the many places it's being used. There is no way around it: this is painful. I think I understand why it is the way it is, and why making it otherwise is an extremely hard problem, but the fact remains that this is one of the few pain points I have with Rust development (although like with most other issues, less often now than initially, since I can generally see things coming better).

Unfounded Fears

#

In this section, I'll talk about some worries that I had based on my earlier experience writing Rust, all of which turned out to be either entirely or largely untrue.

Writing Rust Will Be Hard

#

Rust is not the easiest language to learn, and when I was still writing at a hobby level, I inevitably ran into situations where something about my architecture would jive poorly with the strictness of the compiler. Often, because there was nothing forcing me to complete these projects, I'd give up and do something else. I worried that this would be a problem when writing Rust "for real," and while it has occasionally been a problem, it becomes less of a problem by the day.

First, when it has come up that some design runs afoul of the compiler, being forced to figure it out (by virtue of needing to do my job) has helped me to understand the borrow checker much better. Understanding the borrow checker has helped me to either a) design my data flow better from the get-go, or b) to understand what workarounds are available and when to apply them.

I'll talk more about this later, but I think that a lot of the times when this has come up, it's because Rust is too good at pretending to be a high-level language. I would imagine some design with lots of objects of different types, all unified by some complex trait, not thinking it through and realizing that that means that in order to work with them I'd generally need to use dynamic dispatch, and would often need to Box them (put them on the heap) rather than working with them in the stack. Of course, these solutions are fine for most use-cases, but we have pretty intense performance requirements at Spec, so more than once I've had to go back to the drawing board to figure out how to design things in such a way that we can get away with generics and more stack allocation. The longer I work with Rust, the more natural thinking through these tradeoffs becomes, and the less I get stuck.

So I guess this one is partially true. It was hard at times, but it's gotten much easier.

Regarding the line-by-line experience of writing Rust, I never found that to be particularly difficult. However, I will say that rust-analyzer helps a lot, especially with inlay hints enabled. It is truly a masterwork of modern development tooling, and I miss it sorely any time I'm writing in any other language.

Writing Rust Will Be Slow

#

My early forays into Rust were definitely not speedy, especially compared to getting things done in Python, which was the language I was working in at the time. I worried that the strictness of the compiler and fighting with the borrow checker would lead to a significant productivity drop. In fact, it did, but not for very long, and I think probably not longer than I would have experienced in any less familiar language.

However, I do think that at this point my writing code velocity has picked back up to what I'm used to, and I think in many cases is quite a bit faster because of the quality of the diagnostics and the trust I'm able to place in the compiler. There are certainly entire classes of tests I don't need to write now, when I'm able to represent invariants in the type system. I also spend less time on code review, because I spend less time looking for all the various language "gotchas" that I am used to looking for in other languages. Finally, producing code that is generally robust means that we've spent very little time chasing bugs and fixing old code, which means we've been able instead to focus on new stuff (i.e. the velocity that's really important for the business).

Overall Impressions

#

I really love writing Rust. It's been one of the most fun languages to learn, and it remains fun to write a year and a half in. There are many big and small things that I appreciate about it on a day-to-day basis, and the warts, while present, are small enough that they don't really detract from my overall enjoyment of the language. I've also been pleased to see how the language has evolved so far, and I'm looking forward to seeing what the language and other teams bring in for future editions. I'd make the decision to learn it again in a heartbeat, and I hope to still be working with Rust for the foreseeable future!