Automating Judgement - AI's Promise and Peril in Federal Procurement
- Sarah Edwards
- Feb 19
- 6 min read
Updated: Feb 24
Artificial Intelligence (AI) is being hyped as the fourth industrial revolution. Apart from Apple, the MAG 7 is leveraging eye watering valuations to fund a relentless debt-fueled CAPEX cycle. While the Tech Bros and Star Trek fans are touting unlimited abundance where you’ll no longer need to work. The doom focused Sci-Fi crowd is pointing toward a dystopian future with mass unemployment. Predicting how new tech will impact the future is often a fool’s errand. Human systems and politics are not deterministic and amazingly complex. Truth is... we won’t really know for a while.
What do we know right now? No one seems to be making any money. Investors weary of riding a wave of vibe driven valuations are starting to ask, “where is the revenue?” GPU's with a limited lifespan being used as collateral seems risky. There are many questions around the private credit market. Examining CDS spreads may indicate holders of this debt are getting nervous and hedging against downside risk. Even some of the more famous AI researchers are making very public pivots that could spook investors. But what does all this mean for competitive public sector work? One thing you can bet on is acceleration to integrate AI into every Government workflow possible. Revenue needs to come from somewhere, fast. Lobbyists and Governments have a vested interest in accelerating integration. Like with all debt fueled manias in the last 80 years, the public always ends up holding the bag.
State and Federal Acquisition is a complex, high effort, and extremely regulated. The United States Army is trying to produce an AI approach to generate reverse RFI’s without typical solicitation/response cycles. Integrating AI into competitive source selection is another area being actively discussed within Government Agencies. The incentives are palpable even at the work layer in the organizations. The requirement owners, engineers and other professionals, get pulled from their daily tasking to evaluate your proposals. It isn’t an enjoyable duty and they would love a black box that solves the problem. It isn’t a question of if, but when we see this happen.
Today’s article is a cursory exploration of existing regulation, technical pitfalls, and potential job growth for aspiring contract law attorneys and expert witness software engineers looking for a career change. Most importantly, we will explore what this means for your business. Maybe we can find a silver lining!
The White House issued Executive Order 14275 – Restoring Common Sense to Federal Procurement in April 2025 that kicked off a FAR overhaul with debatable results relative to core intent, so far. OMB-25-21 issued a memo pushing Agencies to push ‘high-impact’ AI use cases. EO 14179 was signed days into a new administration and was essentially intended to clear out the regulatory environment. EO 14365 will go even further to clear the AI regulatory landscape banning states from issuing regulations. The Department of War issued guidance to further accelerate AI into every workflow possible. The gist is that the Trump Administration has a maximalist, all-in, view of AI. Debt bubbles and regulatory pain be damned. It’s full speed ahead!
The FAR amendments, to this point, haven’t change enough to allow for a totally AI driven source selection of your future proposals. FAR 1.102 sets the basic group rules for the entire process and requires people, not LLM’s, to make this decision. The basic tenant is maintaining public trust in the process via an accountable body. An LLM cannot be held "accountable” and has no legal person hood or professional responsibility. If AI makes a flawed recommendation that leads to an improper award, who is accountable?
FAR 15.308 places the final decision-making authority on the Source Selection Authority. If the SSA is presented with a ranked list of offerors from AI and simply signs off on the top choice, they have not exercised "independent judgment." They have deferred their judgment to the machine. This approach violates core principles clearly described in the FAR.
FAR 15.305 establishes how the Government will evaluate your proposal. This section of the FAR mandates that the agency must evaluate proposals based solely on the factors and sub-factors specified in the solicitation. The evaluation must be consistent, equitable, and documented in enough detail to support the final decision. We run into a rather important technical challenge with LLM’s: hallucinations. In other words, LLM’s will sometimes generate responses that sound plausible but are factually incorrect. Scaling was supposed to fix the hallucination issues, but it has not delivered as promised. Numerous studies show a rather wide range of hallucination probabilities. Unfortunately, during any mania the for/against divide produces highly variable data. But, the one thing we know is that hallucinations occur and will present legal challenges to the Government's approach. Prompting “Hey Claude, review this proposal and make no mistakes” won’t satisfy a highly risk averse contracting officer.
FAR 15.506 entitles the offeror’s to a debrief. It’s a little muddy regarding what this debrief will look like given the broader FAR rebirth. But, the Government will have to give you enough information to determine if a black box performed your evaluation. Outputs being unexplainable, and potentially wrong, are a huge risk to the Government. It undermines the entire concept of accountability. I haven’t found any current case law regarding a protest based on an AI evaluation. But this would be a very interesting process to watch that would undoubtedly favor your company. The first court case will set incredible precedent for how contractors compete for work in the public sector.
So, what is the likely near-term outcome? Government attorneys will almost certainly spot these massive legal risks. The initial, rational approach will be to relegate LLMs to purely administrative tasks like checking for page counts, font sizes, and basic compliance. But given the current mania and the immense pressure to show a return on AI investment, it's difficult to say how much pain will be inflicted before a rational, compliant approach is universally adopted. In my opinion, Industry is about to be the guinea pigs.
Instead of waiting to become a test case, proactive companies can adapt their proposal process now to gain a significant competitive edge. Here is how you can prepare for the AI evaluator.
A. Write for Machines and Humans
Your proposal’s structure and language will matter more than ever. The goal is to make your proposal both machine-readable and human-persuasive. Use clear headings that map directly to the RFP's section numbers (e.g., "Volume 1, Section 3.1.a: Response to Technical Subfactor 1 - System Architecture"). This allows AI to easily parse your document and align your content with the evaluation criteria.
Use simple, declarative sentences. Avoid complex, multi-clause sentences. Use clear, direct language. State your compliance and your strengths explicitly. For example, instead of "Our team is capable of...", write "Our team meets this requirement. We will provide..."
B. Front-Load Your Compliance:
Research on LLMs has shown a phenomenon known as "lost in the middle," where the model pays more attention to the beginning and end of a document or section. AI is not a human evaluator who can read back and forth to connect disparate ideas. You must assume it has a limited attention span. Therefore, state your most critical compliance points and win themes at the very beginning of each section. Master the art of technical writing for AI.
C. Ask Strategic Questions During the Q&A Period:
Use the official Q&A process to establish a written record. Most likely the Government will disclose an AI evaluation in the RFP. But, you should ask "Will the Government be using any automated tools, artificial intelligence, or machine learning algorithms to assist in the screening, scoring, or evaluation of proposals? If so, can the Government describe what safeguards will be in place to ensure the evaluation is conducted in accordance with FAR 15.305 and that human evaluators will validate all outputs?"
D. Request a Thorough Debriefing:
If you lose, the debriefing will be really important in this landscape. Thanks to the "enhanced debriefing" rules, you have the right to ask follow-up questions in writing. Even if the agency discloses its use of AI in the RFP, you are still entitled to a rational explanation of the evaluation. If the agency’s rationale seems thin, nonsensical, or they can't explain why you received a certain rating, it’s a major red flag that they are simply parroting an AI's output without understanding it. This is critical information for potential protests.
Navigating this new landscape requires more than just good proposal writing. It requires a new strategy. BWIT Solutions can help you compete in this ever evolving landscape with:
A. AI-Resilient Proposal Architectures:
We don't just write your proposals; we architect them. We structure your content to be easily parsed by AI evaluators while remaining compelling and persuasive to the human Source Selection Authority. We help you "write for the machine" without losing the narrative that makes you unique.
B. Strategic RFP Analysis and Q&A:
We analyze solicitations for AI-related risks and opportunities. We help you craft the strategic questions that protect your rights and put the agency on notice, setting the stage for a successful bid long before the proposal is even submitted.
C. Debriefing and Protest Support:
We help you dissect debriefings to identify the tell-tale signs of a flawed, AI-driven evaluation. We formulate the critical follow-up questions designed to expose a lack of rationale.
Please visit us at our Contact page if you’d like to discuss further! Perhaps over a cup of Earl Grey from the BWIT replicator? Thanks for reading!
Comments