Folio’s Lightning Automations, our best-in-class supplier connections, are responsible for catalog mirroring and placing orders.

And with this update, they are now self-maintaining.

The result? Buyers benefit from fewer errors, overall. And the Folio engineering team can focus on getting proposed solutions live, faster, rather than investigating problems.

The low-probability but high-stakes problem

Automations at ordering are very low-frequency. But they are still high-stakes. And, thus, urgent to resolve. (A property is trying to source supplies necessary for its operation!)

For example:

1. In Food & Beverage purchasing, supplier inventory can deplete same-day. A later order may mean a chef doesn’t get the goods they want for a specific dish.

2. Buyers are expecting delivery of supplies necessary to run their operation. If the order doesn’t go through, fixes to the property can lag, or guest experiences can suffer.

3. For Folio, any issue or delay erodes trust. When a buyer places an order, it’s GOT to work. Trust is earned in drips and lost in buckets. If we mess it up, and an order doesn’t go through as expected, even once, a buyer may not come back.

The reality is that the error rate will never be zero. There’s a lot of funky stuff that can happen in the time lag between cart creation, actual order approval, and actual order placement. And, inevitably, websites can and will change. (Think: notifications or pop-up boxes.)

So how do we scale our ability to resolve these errors, and do so quickly?

Solution: See bug, squash bug

Now, when there’s a Lightning Automation error, an embedded agent trained on our automations starts investigating the issue. (LLMs are good at navigating our logs and external websites, and only getting better!)

We then automatically drop an LLM-generated diagnosis of the underlying issue across our build systems. For example, the agents add real-time insights to our Slack threads about the error, for internal visibility. We can guide them in the process. (A great demonstration of how robots and humans collaborate within Folio.) They investigate common issues, considering the likelihood of it being a change on the supplier site or a regression in our code.

They then open an LLM-generated PR to fix the problem. The PR includes all of the info necessary for a human reviewer to understand the issue, test the fix and approve the PR.

At Folio, this is consistent with our build strategy for 100% of code to be written by AI, but 100% reviewed by humans.

The error gets resolved, and the fix gets deployed, faster. Time from first error to code change is reduced dramatically.

Buyers may not notice, but Folio sure does

Buyers, holistically, can expect even fewer errors and faster resolution times when issues inevitably happen. They may see orders that are slow in getting submitted (because they errored behind the scenes) get placed faster.

We are actually pretty sure that buyers will not notice. The incidence rate was so low already, with very low order impact given our review processes. Resolution times were already happening quite quickly.

The bigger impact is on Folio engineering capacity. The team can deploy more solutions, faster. We can now focus on reviewing a queue of fixes, rather than digging through logs and investigating issues themselves.

Why this is a fun update

There are a lot of reasons this release is exciting.

• The multiplayer impact! Our buyers are encountering fewer surprises, owners are benefiting from more successfully placed orders, and our eng team can resolve issues faster and focus on other work.

• It demonstrates the value of owning the automations in-house vs. the unfortunate third-party supplier dependencies often introduced through the traditional punchout approach.

• Our Automations are getting smarter and smarter, so every future build and future investigation can incorporate the learnings from every error.

• It’s an example of the compounding benefits of AI when fully leveraged across an organization and integrated into our build processes.

• Always good to see a hackathon project brought fully into production; even more exciting when it’s WITHIN the 2-day Jam.

• There are whole AI companies that help engineers draft PRs for bug resolution. We’ve built it in-house!

• Folio culture encourages self-care. Now we extend that to our robots, too!

Thank you to Hannah Brock for the original idea, and to Henry Clabby for bringing it fully into production during our recent 2-day Folio hackathon!