A parser combinator library in Ada
A few months ago, I started writing a parser combinator library in Ada.
Fun and odd stuff to do in a language so strict and non-functional that just using the library seemed wrong. The language is constantly fighting you to not do that kind of stuff. I was able to implement the most common combinators and a few character parsers to make string parsing “easier”. The idea was to use this library to write an URL format parser.
I thought it was quite a bit overblown to do that and parsing URL is not hard. So I lost interest in completing the library.
Use AI to write software in Ada/SPARK?
Fast forward to today and I just tried Cursor AI. I needed some code to use as guinea pig and this unfinished project looked like a good candidate. So I loaded Cursor, which is its own Visual Studio Code clone with the AI embedded and asked it to complete the missing parts, add unit tests, SPARK annotations, documentation, etc. I specifically enabled YOLO mode, which allows the AI to run commands, iterate on command output (such as build errors, unit test failures, etc) and fix its own mess.
At first it seemed impressive that it was be able to write missing function implementations,
add more combinators, add unit tests, implement a calculator using combinators, even adding SPARK annotations, you name it. But then, when asked to build the project, things started to fall apart.
The AI, in all its wisdom, took control of the code to fix the issues. Good boy! Except it started making private tagged types, and types with unknown discriminants into public and non discriminated types, even removing the tagged specification, totally wrecking the existing code and breaking encapsulation. It found issues with access types and so it created its own finalization controlled holder type, without actually fixing the underlying issue.
So I had to put the leash on it, constraining its behaviour by telling it to not touch the existing type definitions. That did actually work a bit better, but shows that an AI doesn’t really understand about code or language constructs. It can output code that statistically should be there, but there is no reasoning or understanding of any of it.
It looked promising, but in a deceiving way, making you think that you dispatch it to perform a certain task and it will complete the task successfully without making a mess. That is simply not true. At the current state, at best, you can request a function body to be implemented given a spec and, with enough documentation written, it might be able to do it well. But there
is a huge difference in being able to write code and being able to write correct code that compiles and passes proofs.
The future of AI writing code
I believe that the future of AI might be better off in implementing the body of functions that were correctly specified with contracts, good documentation, good unit tests, so the AI has all the information needed to derive a correct implementation, and with iteration in the build->prove->test cycle it might eventually output the right code.
If this is the future, software engineers will design the abstractions, interfaces, proofs, tests and documentation of projects, while the AI is the implementer.
In my initial tests with this library, I lacked documentation, proofs and tests. I think the next phase would be to provide that and have the AI take care of the missing bits.
Update
I don’t know if someone mentioned this already, but the biggest threat to AI is IP. There are many businesses who will simply not want their (or their partners) IP being sent to someone else’s server. This is a huge issue and has potentially devastating consequences for businesses that fail to restrict the AI use in parts of the code that constitute IP. If you think about this carefully, and you have a business, you certainly don’t want your code to be used for training or stored out there unsafely, or
event stored in the cloud. To me, it doesn’t make sense to store your IP in the cloud, but many companies now default to have their repos in GitHub.