Simple over easy for operations

Mar 11, 2026

Building a workflow engine for infrastructure operations is not trivial. Most people start with a simple mental model: a desired state and a sequence of functions that produce side effects. In Clojure, this looks like a simple thread-first macro:

(-> {}
    fn1
    fn2
    ...)

Your state {} is threaded through fn1 and fn2. However, real-world operations are rarely linear. They require complex branching, error handling, and conditional jumps (e.g., “if success, continue; otherwise, jump to cleanup”).

Wiring the Engine

To handle non-linear flows, we associate functions with qualified keywords (steps). Together with the next step, they form the “wiring”. You can override sequential execution by providing a next-fn to handle custom branching.

The core execution loop looks like this:

(loop [step first-step
       opts opts]
  (let [[f next-step] (wire-fn step step-fns)
        new-opts (f opts)
        [next-step next-opts] (next-fn step next-step new-opts)]
    (if next-step
      (recur next-step next-opts)
      next-opts)))

Workflow Example

Here is how we use this engine to create a client-side lock for Terraform using Git tags. The opts map represents our “World State”, shared across all functions.

We invoke it like this: (lock [] {}). The first argument is a list of middleware-style step functions, and the second is the starting state.

(->workflow {:first-step ::generate-lock-id
             :wire-fn (fn [step _]
                        (case step
                          ::generate-lock-id [generate-lock-id ::delete-tag]
                          ::delete-tag [delete-tag ::create-tag]
                          ::create-tag [create-tag ::push-tag]
                          ::push-tag [push-tag ::get-remote-tag]
                          ::get-remote-tag [(comp get-remote-tag delete-tag) ::read-tag]
                          ::read-tag [read-tag ::check-tag]
                          ::check-tag [check-tag ::end]
                          ::end [identity]))
             :next-fn (fn [step next-step opts]
                        (case step
                          ::end [nil opts]
                          ::push-tag (choice {:on-success ::end
                                              :on-failure next-step
                                              :opts opts})
                          ::delete-tag [next-step opts]
                          (choice {:on-success next-step
                                   :on-failure ::end
                                   :opts opts})))})

Debugging Made Simple

In many CI/CD systems, debugging is a nightmare of “print” statements and re-running 10-minute pipelines. Because Clojure data structures are immutable and persistent, we can use a debug macro provided by BigConfig and a “spy” function to inspect the state at every step.

(comment
  (debug tap-values
    (create [(fn [f step opts]
               (tap> [step opts]) ;; "Spy" on every state change
               (f step opts))]
            {::bc/env :repl
             ::tools/tofu-opts (workflow/parse-args "render")
             ::tools/ansible-opts (workflow/parse-args "render")})))

Using tap>, you get the result “frozen in time”. You can render templates and inspect them without ever executing a side effect.

Solving the Composability Problem: Nested Options

Operations often require calling the same sub-workflow multiple times. If every workflow uses the same top-level keys, they clash. We solve this with Nested Options.

By using the workflow’s namespace as a key, we isolate state. However, sometimes a child needs data from a sibling (e.g., Ansible needs an IP address generated by Terraform). We use an opts-fn to map these values explicitly at runtime.

The specialized ->workflow* constructor uses this next-fn to manage this state isolation:

(fn [step next-step {:keys [::bc/exit] :as opts}]
  (if (steps-set step)
    (do
      (swap! opts* merge (select-keys opts [::bc/exit ::bc/err]))
      (swap! opts* assoc step opts))
    (reset! opts* opts))
  (cond
    (= step ::end) [nil @opts*]
    (> exit 0)     [::end @opts*] ;; Error handling jump
    :else
    [next-step (let [[new-opts opts-fn]
                     (get step->opts-and-opts-fn next-step [@opts* identity])]
                 (opts-fn new-opts))]))

This logic ensures that if a step is a sub-workflow, its internal state is captured within the parent’s state under its own key. The opts-fn allows us to bridge the gap—for instance, pulling a Terraform-generated IP address into the Ansible configuration dynamically.

The Working Directory and the Maven Diamond Problem

In operations, you must render configuration files before invoking tools. If you compose multiple workflows, you run into the “Maven Diamond Problem”: two different parent workflows sharing the same sub-workflow. To prevent them from overwriting each other’s files, we use dynamic, hashed prefixes for working directories:

.dist/default-f704ed4d/io/github/amiorin/alice/tools/ansible

The hash f704ed4d is dynamic. If a workflow is moved or re-composed, the hash changes, ensuring total isolation during template rendering.

Conclusion: Simple over Easy

Tools like AWS Step Functions , Temporal , or Restate are powerful workflow engines, but for many operational tasks, they are not a good fit. BigConfig has an edge because it is local and synchronous where it counts. It turns infrastructure into a local control loop orchestrating multiple tools.

In the industry, “Easy” (using the same language as the backend, like Go) often wins over “Simple”. But Go lacks a REPL, immutable data structures, and the ability to implement a debug macro that allows for instantaneous feedback.

Infrastructure eventually becomes a mess of “duct tape and prayers” when the underlying tools aren’t built for complexity. If you choose Simple over Easy, Clojure is the best language for operations—even if you’re learning Clojure for the first time.