# AI and Security Source: https://research.auditless.com/p/ai-and-security Kiwi News: https://news.kiwistand.com/stories/AI-and-Security?index=0x6ae0cf9f38c2303be6a3bd374757f410b5f1447c9706ce140f021167e6ab2639c63115ff Submitted by tim on 2026-10-27 UTC, 1 upvote, 0 comments ## Summary The author's central point is that the real challenge posed by AI in security is not matching better scanners but building security into design and development, using AI earlier and more proactively. The article says AI scanners can now find vulnerabilities with little supervision and are especially good at exploring integrations and different layers of the stack, though humans still do better at finding critical business-logic issues. It suggests approaches such as narrow formal verification with LLMs automating configuration, as in Verity's work, along with safer languages like Rust, and packaged "security-in-a-box" frameworks. ## Article While the recent PHP vulnerability may be overstated it's highlighted to teams how unprepared they are to deal with and process vulnerabilities. While AI is indeed getting good at finding and exploiting vulnerabilities (we will discuss what that means), the larger challenge is staying ahead of them. Security is in our DNA, in fact Auditless initially started as an automated smart contract security tool. Since then, we've incorporated security in our development processes and a lot of things have changed over the years. Involving human auditors before major releases has been the only constant. We complemented this with internal auditing using our own processes and as many tools that we could throw at the problem (static analysis, visualizers, mutation testing, etc.). Auditor competitions that draw independent security researchers are a more recent phenomenon but one that has been highly effective at discovering vulnerabilities. None of these changes come close to having the impact that AI will have on security. We've been using AI in internal auditing for various workflows beyond just finding issues: threat model generation (my favorite), running static analysis tools, helping triage & validate issues, generating proposed fixes. But the AI scanners have gotten really good recently. The latest models can find vulnerabilities on their own with very little supervision, proprietary tools do even better (a lot better actually). From my experience, AI is especially good at spending unreasonable amounts of time trying approaches that humans may not prioritize such as focusing on integrations, different parts of the stack (hardware vs. software). I think humans are still better at finding “critical” issues in business logic but there is a class of vulnerabilities that will take humans a very long time to find that will be trivial to find in agents and that is rapidly growing. The problem is what do you do with this information? Installing an AI scanner yourself isn't necessarily enough. First of all: If your adversary has a better scanner, they may find a vulnerability you won't; If you can only find vulnerabilities in production or rush to release, an adversary may find the vulnerability before you. Bug bounties mitigate the costs of this but not entirely. This is particularly challenging if all your code is open source (like in crypto) and you don't have much information asymmetry with respect to the hackers. Security has been a somewhat isolated process that is downstream of design and development (it’s even an entirely siloed function in many organizations). But AI scanning leads us to consider broader questions: If you have to supply AI scanners with substantial information about design intent, threat model, deployment configuration, where else can you apply that context? Are scans point-in-time based on the code or ongoing based on production logs, user actions and other information? If issues may affect your designs down the line, should you be using AI much earlier in the process? What happens after a vulnerability is discovered? Do you use Claude/ChatGPT to triage or do you need a proprietary tool for that as well? The problem isn't that AI scanners are getting good and we need to match them. Throwing software at the wall and using the “security process” to make sure its correct is the problem, no matter how robust the process. The solution has to be using AI more proactively, not just for vulnerability scanning. Formally correct One of the technologies we explored early at Auditless was formal verification. At the time formal verification suffered from two problems: using it exhaustively was too computationally intensive but using it narrowly required too much human configuration. Formal verification companies therefore fell in two camps: ones that were really expensive and discovered very little actionable vulnerabilities, and others that were effectively tool-enabled service/R&D companies (read: auditors). But now you can get the best of both worlds. You can use formal verification methods that focus on narrow-verification and use LLMs to automate configuration. Companies like Verity do just that and this seems like a natural stepping stone. Software engineers will seek to establish that their code is correct and safe by design not having to rely on throwing more compute at scanning than their attackers. A lighter way to approach this is to use languages like Rust or Darklang that are inherently safer: Make no mistakes Vibe coding should also seek to become more stable than human-led coding by spending disproportionate amounts of tokens on testing, code refactoring, simplification and other asks that don't produce new features but add layers to security. Packaged security There may be a new generation of application frameworks or website builders with the key feature being “security-in-a-box”. These will build packaged shells for secure and reliable web apps with clear boundaries where users can vibe code their business logic. The next Webflow or Retool won't help you create software quickly, they will provide agents and environments to help you secure it, test it and get it to production. In short, it’s complicated, I'm spending more time on security than before and probably so should you. At least until the dust settles.