Bah. I just lost the entire list of test cases that I spent most of yesterday collecting and planning.
Over the past few months, I’ve been using a new text editor that I wrote for myself using AI. I haven’t really reviewed even a single line of code for the text editor, and it’s worked fairly well over the months. Yes, it is vibe-coded in the full meaning of the word, and now the bill has come due.
Today, some weird bug caused the entire text file to get deleted.
This is a bitter lesson for trusting vibe-coded stuff for things that need reliability. AI is ultimately a tool, and it’ll write what I tell it to write, and won’t write what I don’t tell it to write. There’s no point in blaming your tools.
It’s tempting to feel like “I wouldn’t have written such an inconsistent system”, and it’s probably true. But it was my responsibility to get AI to write test cases related to data reliability (and then for me to review those test case implementations thoroughly), and I didn’t do that.
Later edit: Fortunately I remembered the final list, and was able to re-write my lost data from memory! :)
But I’ll really have to audit the data storage logic of my editor now, and add a lot of tests to ensure reliability. It’s pretty scary, I have no idea what the AI wrote.
But there are two ways to react to this. I’m going to continue using my custom editor (it solves an important problem for me), and use this experience and fear of data loss as a forcing function to audit the code, and design the internal data-handling system myself.
And more broadly, the point of this post is a reminder to myself. To ensure that I make sure that the important aspects of a system are provable (via tests) when using AI to build stuff. For e.g. data reliability in this case, probably something else in another project.