It's a few days until the new year, but I wanted to make a list of some of the software I'm thankful to have in 2019, in no particular order.
Hashicorp's Vault is one of the most useful software I've used all year. Its ability to generate temporary credentials has drastically improved the security of the company I'm currently at. In the past, we've had to settle for people generating their own AWS credentials and then reminding them when it expires, which is at least 60 days, all the while those keys are sitting on their laptop. Now, every day people just generate 10 hour credentials for only the environments their using. We have also started to use the database secrets engine for generating short-lived Postgres and Elasticsearch credentials as well. No more storing long lived keys!
I use their PKI backend too. It's super easy to have Vault generate a CA, install on whatever machines, and then generate short lived certificates as needed. This allows us to use transport TLS in Elasticsearch and mutual TLS in Etcd.
To top it all off, Vault is a tiny Go binary. I think the MacOS build is less than 20 megabytes for all this.
Docker has been a game changer in the way we deploy our applications. In the past, deploying one of my Python applications meant either butchering my system level dependencies or trying to wrangle virtual environments and some kind of process runner like Supervisor or Systemd. By encapsulating the whole environment in a Docker container, I can have Python 3.8 even if my version of Ubuntu doesn't yet support it, without fear of polluting my system. What's more is that I can change languages entirely and as long as the docker container runs, nothing about the host system cares.
It might not be the fastest or the most feature rich database but by golly does Sqlite meet all of my needs and more. I've clocked it at being able to reach tens of thousands of basic queries per second on a single core, and hundreds of requests per second when using its immensely useful Full Text Search support. While of course tools like Elasticsearch and Solr will be much more powerful than Sqlite's FTS engine, you can absolutely power a basic (and fast) search engine with it.
Grafana and InfluxDB
Speaking of getting the most performance bang for your buck, InfluxDB has shown me that it can handle a lot of time-series data points on tiny hardware, which is great for me as someone who runs everything on Raspberry Pi's or Digitalocean $5-$15 dollar servers. It has a simple rest API for sending data to it and a simple authentication scheme, and has a way to specify retention periods.
For visualizing this data, Grafana is the most powerful visualization tool I've ever used. It can do all the graphing you'd expect from a tool like this, but also has powerful features like being able to repeat panels, query for variables before the dashboard is loaded, and supports a whole bunch of data sources including InfluxDB. While it technically has support for alerting, it kind of sucks when dealing with a lot of hosts.
While I've used the tools above heavily this year, there's a lot of other tools I could go on and on about. They certainly deserve an honorable mention:
- Rust: of course Rust deserves a shout out for being a very well thought out language with a promising future. I enjoy all of the language features like pattern matching, destructuring, iterators, and closures. I have not yet used it in any production tools - the main thing holding me back is that writing any tool that needs to use HTTPS (which is 90% of what I do) will pull in a myriad of dependencies, a few of which are land mines waiting to go off. If any of the dependencies break or are yanked (especially Ring for TLS), your app can no longer compile until the whole chain of dependencies is brought up to date.
- Terraform: Terraform is also a very well thought out tool that makes my life as a Infrastructure Engineer a lot easier. Statefiles and planning are the way to go. I even managed to make a couple custom providers in Go without too much trouble
- Ansible: Ansible is also great for Infra stuff, and it's my go to for configuring hosts. I've always thought that the perfect tool would be Ansible plus statefiles so you can plan out what it's going to do.
- Python 3.8: The walrus operator reduces a lot of line bloat. Python in general is just great.
- Blender: Blender is a miraculous project. I've been using it off and on since 2005 because it was free and I had aspirations of making videogames as a teenager (still do!), and watching Blender go from an average 3d modeller to something that is used in movies and TV shows is amazing and inspiring.
- Godot: I've only ever played around with Unity but never got very good at it, and then came across Godot this year and started to really invest time in learning it. I hope it goes the way of Blender one day; it seems pretty close as it is.
- Fd and Ripgrep:
grep -Rand use these modern tools.
- Visual Studio Code: VS Code is great, please don't add any more bloat to it as it feels like it's getting slower and slower.