Rust for Devops

Published

Normally, the programming languages of us Devops folk consists of Bash, Python, and occasionally Ruby or Perl. Branching out to newer languages like Rust and Go opens up some new problem solving avenues. Though Rust is a bit less friendly to use, it has a number of advantages that make it worth exploring.

If it Compiles, it (Probably) Works

One reason Rust is good for Devops tools is because its compile-time safety means that if your program compiles, it probably works. Compared to Python, whose developement loop looks like "write some code, run it and see if it works, then tweak," Rusts development loop looks more like "write some code, tweak it until it compiles at all, and then run it for some final testing." With Rust, you can't accidentally misspell a function or access the wrong element in a structure or call a method that doesn't exist ('NoneType' object has no attribute 'strip', anyone?).

Serde brings in this helpful rigidity when serializing and deserializing JSON:

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug)]
struct VaultResponseData {
    lease: String,
    lease_max: String
}

#[derive(Serialize, Deserialize, Debug)]
struct VaultResponse {
    data: VaultResponseData
}

fn main() {
    let response: VaultResponse = reqwest::blocking::get("http://some-vault-server")
      .expect("Failed to make fault request")
      .json()
      .expect("Failed to deserialize the JSON into an expected structure");
}

Here, you can guarantee that the response object has the exact schema you would expect, leading to predictable kinds of failures. With Python, you would need additional libraries and unit testing to get this kind of safety.

Self Contained Binaries

When you're distributing scripts, you have to take special care that your environment is correct. Minor Python versions can cause problem, for example when you try and use format strings but the Linux distro you're on is only on Python 3.4. Bash, Busybox's Shell and Zsh all have minor differences that can add confusion to debugging. By now, every Devops engineer has invented their solution to this problem, but mistakes happen.

Compiling a binary sidesteps this issue entirely. Rust, Go, and other compiled languages don't need to be run in any particular kind of runtime environment, as long as your shared libraries are correct. When you're shipping crucial programs to hundreds of servers, this is one less thing to have to worry about.

Side note: I have had issues specifically getting Rust binaries compiled with Ubuntu working in Alpine linux, because Ubuntu/Debian use glibc while Alpine uses Musl, so it couldn't find some shared libraries like OpenSSL. This was over a year ago and I'm not sure if its a problem or not but it's worth mentioning.

Raw Performance

Devops applications aren't typically performance critical, but occasionally they either do need to run as quickly as possible or they need to run in very resource constrained environments (t3.micros and nanos for example).

Actually Doing Stuff with Helpful libraries

Rust's ecosystem is quickly evolving, especially thanks to the buy-in from some major players like Amazon. Here's a list of crates that I use pretty often for Devops work:

  • Rusoto: the AWS library. This is all-async but you can make Tokio runtimes with block_on if you need sync compatibility, for example:
  let client = rusoto_s3::S3Client::new(rusoto_core::region::Region::UsWest2);
  let mut runtime = Runtime::new().unwrap();

  let request = rusoto_s3::ListObjectsV2Request{
      bucket: "MyBucket".to_owned(),
      ..Default::default()
  };

  let response = runtime.block_on(client.list_objects_v2(request)).unwrap();

Oh yeah, and use std::default::Default a lot.

  • Clap: argument parsing. This is my go-to for all CLI's.
  • Regex: A performant and stable regular expression library.
  • Reqwest: This seems to be the most stable HTTP library for Rust that I can find. It works with Serde for JSON so it's easy to use with REST APIs.
  • Glob: Makes it super easy to find files matching a certain pattern
  • Tar and Zstd: I used this in a recent project to make backups as quickly as possible and it worked very well. For example, here's some Buffer Inception (bufferception?) to create a Tar archive object from a zstd-compressed file:
  let file = std::fs::File::open("mycooltar.tar.zst").unwrap();
  let zst_buffer = zstd::stream::Decoder::new(file).unwrap();
  let archive = tar::Archive::new(zst_buffer);
  • Rusqlite: Sqlite's Full Text Search has a special place in my heart since it bailed me out on a tricky problem I had over a decade ago as a student programmer. I had to make a few hundred of these accreditation documents searchable and the database we were using at the time was unusably slow for this task, so I mirrored the relevant tables in Sqlite with the FTS extension enabled and synced it every half hour. It worked perfectly.