Third Update:
I originally posted this article on icefireair.com which I have stopped working on, and some content has been moved to this new homepage. I have also deleted the IFA Youtube channel, so the SayIntention videos were not available for a while. I’ve posted a new video which walks through missions and Tour Guides and things like that, to show off the capabilities as well as limits of the software – link’s at the end of the article. My “verdict” remains the same even after a couple of updates SAI has published since February.
Second Update:
As stated below in the article I tried the experimental feature “Strict Readback Reinforcement”, and I believe it actually helps up the precision – which is cool. Didn’t have any “experimental” issues either yet.
Also some more fun with TwoToneMurphy and his TourGuide and SAR Crews. “Good times, eh, Matt!”
First Update:
I had a short written exchange with Brian, the CEO of SayIntentions, who isn’t really fond of my original writing here.
After publishing it, I asked Brian for feedback, because I wanted to get the facts right. I think that’s a fair move, considering it’s kind of a ground breaking product in some ways. And I really do want these guys to succeed, because I love the concept of this add-on.
But let’s be clear about the intention of my article: This is not a sales pitch, I am a regular customer who pays the full premium price, and this review reflects my personal experiences with the product, which are not at all flawless.
With that said, and with his permission, I’ll add Brian’s comments and mine. No doubt he is very proud of his creation and looking to sell it, so he fights for it.
Everything written in black is the original unaltered article, additions in green are a result of the discussion I had with Brian.
I quite like interacting with ATC in flight simulators. While I am not very fluent in aviation speak/phraseology, I can judge to some degree the correctness of the structure and content of calls and answers. Microsoft Flight Simulator’s default ATC sucks like a Hoover, it’s neither competent nor immersive. Nobody really likes it.
VATSIM or IVAO or any other of these online networks, whether free or paid, are a realistic alternative with human beings controlling the air spaces; but some sim pilots suffer from mic fright, others from impostor syndrome (they feel like not being good enough) or simply don’t want to join an online network. That’s fine, we’ve got other options.
ATC with AI agents
There are software based alternatives, like Pilot2ATC or BeyondATC which simulate complex ATC communication patterns, and pilots are actually able to talk to these “synthetic” controllers. I tried both, and I think they are quite good at what they intend to do. But their abilities are limited in that these systems do not support freestyle VFR without flight plans, only IFR. They have to rely on clear and concise, deterministic routing to guide you, and only offer a fixed set of speech patterns for input as well as output, without any room for creative verbalisation. You have to stick to their speech patterns to a certain degree, otherwise ATC won’t work properly. BeyondATC seems to be developing a solution which enables VFR by leveraging the capabilities of AI based language models (LLM), but this is still in the making and not available publicly, as far as I’ve seen. And they also struggle quite a bit with voice recognition, at least from my experience, which makes it difficult to establish a reliable, immersive flow of communication.
SayIntentions already has a solution which can do “chaotic” VFR without prior submission of any flight plans. You’re talking to quite naturally sounding AI controllers who have a variety of voices both male and female. Some even simulate foreign accents, depending on where you fly. There are copilots, a cabin crew, tour guides and even a mission system which dynamically generates tasks which pilots can accomplish. You can talk to ATC and your crew in a natural manner and you’ll get some good answers. The voice recognition also works pretty well most of the time.
From Brian: “you failed to mention that there’s a wide variety of AI personalities to choose from…. so if you don’t like the one you’re using, there are tons more to pick from.”
Well, in fact I did mention the “variety of voices”, see paragraph above. There is also a YT video below where I demo the tour guide Janet who can’t stop telling stories about her clumsy friend Tony (she won’t stop annoying me with these anecdotes despite me asking her, several times), and I’ve shown that there are other teams as well, like TwoToneMurphy and his SkyOps SAR gang. Murphy’s personality is fun at the beginning, but wears out quickly because of – I presume – topical limitations to make sure the creativity of the system doesn’t step out of line. What’s pretty neat though is that the system has some kind of memory which can recall previous SkyOps missions, which the AI guys pick up in their sermons.
Guesses as to how SayIntentions actually works
SayIntentions is only vaguely describing what components their system is made of, including a hint that they’re connected to OpenAI. Professionally, I’ve been working quite intensely with Generative AI-based systems for almost 3 years now, so I dare to make some educated guesses (which of course might be wrong) about which components their software stack might consist of.
- Some kind of database system (or API connection to a data provider) which delivers information about airports, including ICAO codes, names, taxiways, runways, radio frequencies etc. – all the information you need in the context of ATC
- Some kind of API connection to data providers (or frequently updated caching system) which offer current weather information relevant to ATC, from all over the planet
- A TTS (text to speech) system which generates speech from written text (including a variety of different voices)
- A STT (speech to text) system which records the voice input from users and transscribes it to text (maybe Whisper from OpenAI for both TTS and STT?)
- An agentic system which interacts with OpenAI, aggregating information from user input and additional data from providers mentioned above
- An extensive set of instructions and guard rails (system prompts) for the LLM component so that it, for example, …
- … knows how to behave (be friendly, don’t make stuff up, rely on facts etc)
- … what to do and say (and what not)
- … which roles it has to play (ATC, copilot, tour guide etc)
- … what kind of information it has to request or give
- … and probably a couple of additional broker scripts/programs to handle data exchange between the systems and put it all together for the end user.
Brian tells me my assumptions as to how the system works are incorrect, and it would be much smarter than I thought. Of course he can’t go into details, considering the innards of SI a trade secret.
I’ll leave it at that.
Update in April 2025: SAI has put up some icons in the software which give hints to which platforms they’re using, among them X.AI, Microsoft Azure, OpenAI, flightradar24 and others. So some of my guesses from above seem not to be too far fetched.
How well does it work?
I have to say, the results are impressive, at least compared to other LLM based multi-agents I’ve seen so far.
Voice interaction feels quite natural, and the info or instructions given by ATC are not bad at all in many cases. How good all of this works depends on a couple of factors, e.g. where you’re flying (airfield database coverage and weather data availability), how accurate your own input is, what your intentions are and how you express them, and what kind of aircraft you’re using.
Still, there’s a catch to using LLM-centric systems. You’re dealing with a probabilistic/non-deterministic approach. This basically means that the output of the system is not 100% reliable, there’s always a chance the LLM makes stuff up, doesn’t find the correct answers or reacts unexpectedly.
Occasionally, I had problems during my tests (see videos below):
- delayed responses or none at all (problably due to overload in some components, most likely OpenAI)
- readback errors were not recognized or STT didn’t transscribe correctly (there’s an experimental fix for this, see green remarks below)
- implausible choice of active runway (wind conditions)
- Repeating wrong answers, ignoring my correction requests
- “Funny crews” making jokes which quickly become annoying and repetitive, anecdotal variety is limited
- Mission control (“SkyOps”) does not reliably detect the kind of aircraft you’re flying, making suggestions for the wrong task types
- During SAR missions, scenery items belonging to the missions are sometimes misplaced at locations which make no sense (ship on a moutain, for example)
Brian mentions that for all the “tiny bugs” I have experienced, I should have provided an in-flight ID for them to debug the issues and correct stuff.
Now, while I can understand his reasoning from a dev/sales perspective, I cannot agree with the notion that I should take the time to file bugs every single time and help them fix their product. I’m a paying customer, and the payment is not even little. I simply don’t have the time nor inclination to be a beta tester. I know it’s become sort of an industry standard to have paying customers take over at least some of the testing, but I don’t agree with that at all.
For the bug where my intentional readback errors were not chalked down by ATC (mentioned above), the team seems to work on a solution. It’s still experimental and “not recommended”, but it’s available and called Strict Readback Enforcement. I haven’t tried that yet, but I will and let you know how well it works.
Is it worth 20 bucks/month?
Now, if this was a beta with reduced monthly fees I’d say: great effort, I’ll support them with a permanent subscription. This is probably getting better all the time. I’d be willing to give up to 10 dollars per month, I guess, for an unfinished but promising platform like this. Unfortunately, SAI is charging almost 20$/mo, and it was even more a while ago.
Brian would like to add that if you pay a year in advance, the monthly cost would be reduced further.
It’s true that 20 bucks per month are only due when you don’t pay a full year in advance. Thing is: I simply won’t pay an entire year in advance if I am not yet fully convinced as to whether the software is worth that much to me. And their 1-day trial isn’t nearly enough for me to check it out properly with all options.
Another of Brian’s reproaches: “Your article also doesn’t really mention why we charge so much, nor does it talk about the things we can do that nobody else can do.”
Actually I do, very clearly, on both accounts. See above comparison to other products like BeyondATC or pilot2ATC, and the following paragraph on my assumptions on why it’s probably expensive for them to run their system.
Now. I think I do understand why they have to charge this much. They’re not trying to squeeze every penny out of their customers. For example, using the OpenAI API is not cheap – it’s pay per use. Depending on the LLM variant used, costs per token can quickly amount to significant sums. The fact that SI is basically offering a flatrate does mean they’re taking a risk. If a pilot is extraordinarily chatty and uses up a lot of tokens torturing the synthetic controllers, SI might even end up with a net loss for high chat volume accounts, at least theoretically.
For the sake of fairness I will also quote their FAQ: “Because we’re using modern, cutting-edge AI services, neural networks, insanely authentic voices, and machine learning, costs are incurred every time you communicate with ATC.” Whatever systems they use beside OpenAI, it’s clear they do invest a lot to make it all work.
Whether this is, from an architectural point of view, actually a good approach, integrating all these expensive external platforms, remains to be seen. There are people who question the quality of SAI’s work (see Flightsimulator.com Forums). We’ll see how an approach with a local LLM might do the trick (or not): BeyondATC seem to be working on a solution which runs locally on the user’s computer, that’s a vector which has its own pitfalls.
So, my recommendation is this: Invest one month, try if this system works for you. You can cancel at any time and resubscribe as well later on. SI will, due to the nature of Generative AI, never be an infallible ATC simulation. But then again, even human ATCs make mistakes. I still have a subscription going because for the long sim tours I’m doing, SAI adds to the immersion quite nicely. But I do hope there’ll be other less expensive products in the future.
To Brian: While we might not agree on some of the issues, I do appreciate the time you took to give some feedback, and my original recommendation for everyone still stands: Try it, pay a month (24h trial is way too short), and see if it works for you. I think it’s all heading in the right direction.