By Krystle Kaul
Sandeep Shilawat has spent his career at the intersection of technology and national mission — working with federal agencies including the Department of Homeland Security and the Chief Digital and AI Office, and holding senior roles at ManTech, IBM and the Cloud Council.
Earlier this year, he published “Trustworthy AI: Red Teaming, Risk and Architecture of Secure Intelligence,” arguing that the industry’s long-standing faith in “alignment” — the belief that AI systems can simply be trained to share human values — is a failed approach, and that only continuous, adversarial testing paired with real enforcement can make autonomous AI systems safe to deploy.
In an exclusive for The American Bazaar, Krystle Kaul spoke with Shilawat about what prompted the book, the AI failures organizations are underestimating, and where he sees the biggest opportunities for U.S.-India collaboration on AI in the years ahead.
Krystle Kaul: What prompted you to write Trustworthy AI, and what gap did you see in the existing conversation around AI safety and governance?
Sandeep Shilawat: AI has been taking a shape of its own since 2019, and it really broke loose once generative AI moved into the cloud. The idea for this book had been on my mind for a while, but the Ukraine war is really where it started gaining steam, and it picked up even more momentum after Operation Epic Fury. One of the school attacks during Operation Epic Fury put a lot of wind into my sails about writing the book. Then there was an incident of an agent going rogue on Moltbook that really concerned me about the nature of AI and what philosophies we’re using to test these systems. I’d been working with a group of people from universities and industry for a long time, and I decided it was time to finish my thoughts and put them into a book. That’s the main reason I published it earlier this year.
What is the biggest misconception about red-teaming AI systems?
For years, the industry has treated testing as something you do once and then put into production, and that norm has worked well for us. Red-teaming specifically comes from the concept of wargaming — World War Two, and cyber has always used it — but it was traditionally done quarterly or annually, depending on the requirements. AI moves at a pace that makes that frequency untenable. One of the biggest things people forget about AI is that it’s live, and it’s always acting. If AI is always acting and can do things that aren’t expected, red-teaming is how you determine that — and you can’t do it once and stop. That’s one of the biggest changes I think the industry needs to make: people should be red-teaming continuously rather than as a one-time exercise.
How should companies balance the pressure to deploy AI quickly with the need to make sure these systems are both safe and reliable?
That’s a tricky question. You have to reflect on wanting to stay competitive, and for that, go-to-market speed matters — you can’t wait for your competition to get there first. That’s the market pressure companies are dealing with. But are the risks understood? Risk is often treated as an afterthought — we saw that with dot-com, with cloud, with big data, and we’re seeing it again with AI, except the scale is very different now, because everything is happening at machine speed. Companies need to think about what competitive advantage they’re actually getting from adopting AI, which will lead to a reinvention of their business processes — but also what risk they’re taking on by adopting it, and whether they understand it. Most of these companies are long-standing, innovative companies, and their biggest advantage is their market position — and that market position comes from trust. Trust is their biggest moat. They need to ask themselves: are they compromising on trust by going faster to market? That’s the trade-off, and it’s a management challenge every company executive should be thinking about on a day-to-day basis.
You have extensive experience working with federal agencies across the board — DHS and the Chief Digital and AI Office. What are some of the biggest opportunities for AI in government, and where should policymakers be most cautious?
I’ll break that into two pieces — the areas we collaborated on were mostly national security, and then there’s the public-services side. On national security, I see significant potential. One of my mentors used to say we have to do business at the speed of mission, and technology was always lagging behind those missions, which occur at speeds we never imagined. One of the scenarios we used to deal with on the JADC2 side was: what if a supersonic missile is coming from the South China Sea and will reach the US in forty-five minutes, when our decision-making processes take at least ninety days to work through? How do you scale that? That was the whole premise behind JADC2 — dealing with decision-making at machine speed using AI.
China is advancing very aggressively in several strategically important areas of AI, autonomy, robotics and military technology, so we have a real challenge on our hands. I see a significant opportunity for businesses to help the nation with national security applications of AI in a controlled fashion. Battlefields are now mostly technology-driven, and that’s turning things upside down — we’ve seen it in the Ukraine war and in Operation Epic Fury. I think the fact that Ukraine has held out against Russia this long has a lot to do with AI and drone technology.
On the public-services side: Social Security is backlogged, so is the Department of Labor, so is the Department of the Interior, and Veterans Affairs has people waiting months for treatment. AI has a tremendous application in scaling these services and helping the people stuck waiting in line. A lot of federal civilian agencies are already using AI to scale their operations.
Having worked across US and Indian technology environments, where do you see the biggest opportunities for deeper U.S.-India collaboration in AI over the next five to ten years?
I think we need to start with a basic understanding of where we are today. In the 2000s, business-process outsourcing drove the IT boom we’ve seen in India, and that was based entirely on labor-price arbitrage — a large technical workforce available at a lower cost. Then cloud came along and infrastructure got outsourced to those same companies, which consolidated some of that work. Indian industry has since pivoted, and today the majority of it runs through global capability centers, or GCCs, building capabilities used by the rest of the world, including the US. I recently wrote about GCCs, and I think they need to reinvent themselves as AI factories — that’s the fundamental shift some companies are trying to make, because AI is ubiquitous now; it isn’t going to be held back by headcount.
In fact, one of the biggest risks the Indian industry faces today is that AI could automate a substantial portion of the work historically supported by labor-arbitrage models. But that isn’t necessarily a risk — it’s a bigger opportunity for India to build AI capability that helps the rest of the world, given the scale of technology talent it has. America, on the other hand, is trying to catch up with the rest of the world, especially China, on AI capability. So I see significant room for collaboration there.
In India itself, a friend of mine is working on something related to the Kumbh Mela, giving agents to individual pilgrims that can help save human lives — that’s the kind of scale problem the U.S. and India could solve together, improving quality of life and saving lives. One more area I’d add is space technology. As long as it’s done in a controlled, trustworthy way, I think U.S. industry — companies like SpaceX — could collaborate with ISRO on AI-based space technology, which is an area growing by orders of magnitude.
Could India become a major center for AI development and deployment in your opinion, rather than primarily just a source of technology talent?
I see a lot of potential. By 2030, India will be the third-largest economy in the world, with one and a half billion people whose quality of life needs to improve — that alone creates significant organic demand for technologies like AI. The young workforce coming up, and the startup ecosystem developing in India, can propel that. That said, I think India needs to focus on building native technologies coming out of India, rather than becoming another service hub. I’d like to see India overtake other countries in the number of AI PhDs, patents filed, and original AI research and software coming out of the country — I don’t see that happening at that speed yet, though I do see a lot of potential.
One of the biggest disadvantages I see in Indian industry is AI infrastructure. GPU chips are very expensive and in short supply, and while that looks like a challenge today, I hope the Indian government and Indian industry take it upon themselves to build AI data centers using cheaper chips and different technology that will let them run models and keep innovating. Right now, it’s a significant uphill task for India to get there, but given the talent and the intent, I see tremendous potential.
If we look back at the AI industry five years from now, what is one thing you think we will realize we got wrong about AI today?
One thing I think we’re still getting wrong — and it’s unfortunate — is that people have been warning about this for a while. During the Biden administration, hundreds of people wrote letters to governments asking them to slow down AI development. For several years, leading AI researchers and technology figures have warned that AI capability may be advancing faster than our ability to govern it. I think we’re underestimating the pace and the potential of what AI can do.
I’ve been reading a book called “The Coming Wave,” and I think it significantly underestimates this wave. The precise timeline is debatable, but AI capability is advancing much faster than the governance and assurance mechanisms built to control it, and we’re outsourcing intelligence without adequate checks and balances. That’s one of the reasons the concepts in this book matter — Trustworthy AI is about alignment versus enforcement. For a long time, we’ve relied on the idea that these frontier models — open, democratized as they are — were aligned. I strongly believe alignment alone is insufficient.
I believe you need to enforce your will on these models, and if they don’t comply, they need to be shut down. Everyone’s been saying this, and yet risk teams at various frontier-model companies have been dissolved or scaled back — are we forfeiting on this front? I hope not, and that’s the thing I think we’re not prepared for. Everyone is warning us that the pace is too fast and governance is too slow. The EU has enacted an act asking companies to watermark AI content and slow down; the U.S. is trying to come up with legislation too. I think all of it is moving too slowly relative to the pace of the technology, and I hope we learn that lesson before it costs us in a major way.
As generative AI systems become increasingly autonomous, how should organizations rethink their approach to AI security and risk?
Organizations need an AI risk strategy. They need a team continuously monitoring what the AI is doing and what impact it’s having. I have a concept I call the “AI judge” — a judge-like function that continuously verifies and determines whether the AI is in control, in real time, rather than relying on humans to catch problems. And wherever they have that, they should also have a kill switch. Most organizations don’t have this today — a lot of shadow AI is spreading everywhere, with everyone using ChatGPT, other OpenAI tools, Anthropic or Gemini for their work, while enterprises often don’t have real control over it or even know what kind of data is flowing through their firewalls. That’s what they need to work on. Their overall AI strategy should include a dedicated AI risk-management section that addresses exactly this.
What was the most challenging concept in the book to translate into practical guidance?
The hardest part was probably the TIVM framework I present in the book. It comes out of the alignment problem — I don’t believe alignment works, so you have to enforce, and to enforce, you need an analytical number. To get that number, you need to know the threat, the impact, the vulnerability and the mitigation. I also introduced something called LIE — a risk score built from likelihood, impact and exploitability. The exploitability factor is the hardest of the three to explain, but together they give you a number, and once you have a number, you can have an analytical conversation instead of an anecdotal one about trust. Making the link between the LIE framework, TIVM, and actual trust in practical terms was the hardest thing to translate.
What is one misconception about trustworthy AI that you hope readers will leave the book with a different understanding of?
The one thing I hope people take away is that AI is not human — it doesn’t have consciousness. The way our kids abide by a value system is what we call alignment; that doesn’t really exist in AI. Alignment alone cannot provide the level of control required for high-consequence AI — at best, it’s probabilistic. If you leave the book understanding that alignment must be complemented by assurance and enforcement, that is the key takeaway I hope readers walk away with.
“Trustworthy AI: Red Teaming, Risk and Architecture of Secure Intelligence” is available on Amazon, in paperback ($11.99) and on Kindle ($8.99). Sandeep Shilawat can be reached through his website, www.shilawat.com.