Regulatory Oversight — AI State Regulatory Frontiers: Emerging Issues on AI, Privilege, and Work Product in Legal Practice
Host: Ashley Taylor
Guest: Gene Fishel and Daniel Waltz
Aired: 6/3/2026
Ashley Taylor (00:04):
Welcome to another episode of Regulatory Oversight, a podcast dedicated to delivering expert analysis on the latest developments shaping the regulatory landscape. I’m one of your hosts, Ashley Taylor, the co-leader of our firm’s nationally recognized State Attorneys General practice and a member of our Regulatory Investigation, Strategy, and Enforcement, or RISE, practice group. This podcast highlights insights from members of our practice group as well as guest commentary from industry leaders, regulatory specialists, and current and former government officials. Our team is committed to bringing you valuable perspective, in-depth analysis, and practical advice from some of the foremost authorities in the field today. Before we begin, I would encourage all of our listeners to visit and subscribe to our blog at RegulatoryOversight.com to stay current on the latest in regulatory news.
Ashley Taylor (00:54):
As we continue our multi-part series on artificial intelligence, I’m joined today by my colleagues Gene Fishel and Dan Waltz to discuss how a recent Southern District of New York ruling on AI-generated documents is shaping expectations around the use of AI in legal practice and what it signals for courts, lawyers, and law firms. We will also look at how these developments intersect with emerging state regulatory oversight of AI and what companies should do now to integrate AI into their operations reasonably and defensibly. Gene, a member of our RISE practice group, previously served as Senior Assistant Attorney General and Chief of the Computer Crime Section in the Virginia Attorney General’s Office and as a Special Assistant United States Attorney in the Eastern District of Virginia. He draws on more than two decades of privacy, cybersecurity, and AI enforcement experience to advise clients on compliance and regulatory investigations. Dan is also a member of our RISE practice group based in the Chicago office and a former Assistant Attorney General in the Illinois Attorney General’s Office. Dan leverages his government experience to handle complex regulatory investigations and other matters at the federal, state, and local levels, focusing on the intersection of industry and government. Gene and Dan, thank you both for joining me today to explore this ruling out of the Southern District of New York and how it is shaping the use of AI in legal practice and how it intersects with emerging state regulatory oversight of AI.
Dan Waltz (02:23):
Thanks, Ashley. It’s a pleasure to be here today.
Gene Fishel (02:24):
Thank you, Ashley. Great to be here.
Ashley Taylor (02:26):
I want to start with a basic intro. Would you all speak, maybe Gene, you could start, with how attorneys are integrating AI into their daily work?
Gene Fishel (02:37):
Sure. Well, we are definitely in an era where the legal system, including attorneys, courts, and clients, are really trying to find their way as to how to handle AI systems. We have a lot of AI platforms that are popping up, both for general public use and general AI platforms that can do anything, to more specifically tailored AI platforms for professions. And within that ecosystem of AI platforms, we have… And this is an important distinction for what we’re gonna talk about, we have publicly-available AI systems that are available to the general public, where the platforms and the companies behind those platforms can access data that’s entered into them. And we also have more proprietary or closed AI systems that law firms and companies can employ that are tailored and a little more secure in their use, where data entered may not be subject to public scrutiny. Certainly in the legal system, AI moving forward is gonna play an important role. It creates efficiencies. It’s gonna be very useful in legal practice for things like summarizing mass documents, large volumes of documents, certainly in drafting, making drafting more efficiency. But as a primer for what we’re gonna talk about today, where attorneys and sometimes clients are getting in trouble is when they rely on AI platforms for legal judgments or even legal research without verification, such as for generating case citations or statutory citations. So attorneys need to balance the efficiencies and usefulness of AI with the risks involved moving forward. And hopefully, given this recent case out of the Southern District of New York, we’re gonna highlight some of those risks that are present.
Dan Waltz (04:43):
And Gene, if I could just add on top of that, I’ve been using AI in my own practice for a variety of things; drafting, email writing, summarizing the law, just as you mentioned. One thing that I’ve seen that is particularly interesting and I think relevant to our conversation today is the pro se use of artificial intelligence tools. I’m primarily a regulatory and government investigations attorney. However, I occasionally do take on a case for a client in Chicago, for example, where there’s a pro se individual on the other side. And over the last, I would say, 12 to 16 months, there has been a remarkable increase in the pro-se cases in terms of the sophistication of the filings and their ability to articulate legal concepts. So it’s been an interesting development to see non-lawyers using artificial intelligence in ways that in some respects democratize the practice of law and I think can, in certain instances, increase access to the legal system for certain individuals. However, to your point about the judgment, the experience, anytime that I use AI, I’m using that AI through the lens of my many years of practice. I’m not gonna say exactly how many that is, but it’s enough to have some level of judgment. To your point again, Gene, where people are getting in trouble is relying too much on the artificial intelligence tool to exercise that human component that needs to be exercised in the practice of law, and that’s the judgment and decision-making that really only come with experience and knowledge.
Ashley Taylor (06:07):
Gentlemen, to frame the next segment of this discussion, perhaps it would be helpful to start with an overview of the court’s ruling in the Southern District of New York case that will, again, frame the rest of this discussion for the audience.
Gene Fishel (06:22):
Basically, what was happening in this case, and the case is United States versus Hafner. It’s out of the… It’s a federal case in the federal district court out of the Southern District of New York. An issue arose, and it was really an issue of first impression that the court ruled upon. What happened in Hafner, Hafner was the defendant in a criminal case, and he had, at some point last year in 2025, had received some grand jury subpoenas and also a target letter letting him know that he was the target of a criminal investigation. After he received these letters, but before he had been charged, so he hadn’t been indicted or arrested or anything like that, he retained counsel in anticipation of being charged criminally in federal court. What he did after that, on his own volition and not at the direction of counsel, which is very important, is he went to a publicly-available AI platform and he submitted prompts into this AI platform to basically organize information in preparation and anticipation of his upcoming defense in the criminal case. So the AI platform generated dozens of documents as a result of these prompts. What ultimately happened was federal authorities executed a search warrant at his home, seized his electronic devices and other papers, conducted a forensic analysis, and discovered 31 documents that had been produced by this AI platform. Ultimately, the defendant was charged with securities and wire fraud in federal court.
Gene Fishel (08:09):
As I mentioned, he had retained counsel, and when the time came for motions hearings, counsel, the defense counsel, asserted that the AI documents, those 31 AI-produced documents, should be protected by the attorney-client privilege and also the work-product doctrine, and thus the government shouldn’t be able to use these because of those privileges and the confidentiality. Well, as I mentioned, this was a case of first impression. The court examined the actions of the defendant and the nature of the AI program and what was produced and ultimately made an oral ruling from the bench. Ultimately, what the court ruled was these documents were not protected by the attorney-client privilege for three reasons. Number one, the defendant was not communicating with counsel when he was entering the prompts into the AI system. So, in other words, the AI system is not an attorney. The AI platform, it’s not counsel, according to the court. Number two, the defendant had no reasonable expectation of confidentiality of the information it was putting in or the information it was receiving. And what was the key point here for the court is, according to the terms of service of this AI platform, the company that ran the AI platform could use any information that a user put into the prompts to train future models on the AI platform.
Gene Fishel (09:48):
So the information put in there was not confidential, and any user should know from the terms of service in using that that there was no reasonable expectation of confidentiality. Third, the court ruled that the prompts that the defendant put into the system were not put in for the purpose of obtaining legal advice. It goes back to the fact the AI platform is not an attorney, but as the court noted, the actual terms of service concerning or attached to the AI platform use specifically stated that the AI platform is not a lawyer and cannot provide legal advice. So those three elements were present that led the court to rule that the attorney-client privilege did not attach to these documents. And as concerning the work-product doctrine assertion, the court ruled that his entering those prompts in the AI system were not at counsel’s direction. As I mentioned, he did this on his own volition. Counsel at the time wasn’t even aware that he was entering the prompts into the AI system. So, not at counsel’s direction, thus the work-product doctrine did not take and does not attach. So, important case that has in its wake been scrutinized around the country.
Ashley Taylor (11:12):
Let’s talk about another important case in this AI legal landscape: Warner v. Gilbarco, a Michigan United States District Court case. Dan, you wanna speak about that?
Dan Waltz (11:24):
So right around the same time as the case that Gene just discussed, another decision came out of the Eastern District of Michigan. And it was in a different posture and it had a different outcome, but the reasoning is very interesting and it just, it explains, I think highlights the era that we’re in now where courts and bar associations and attorneys are trying to figure out how AI interfaces with the daily practice of law. So in the case of Warner v. Gilbarco, G-I-L-B-A-R-C-O, discovery had closed and there were a number of discovery motions pending, and the magistrate judge entered an order on some of these outstanding issues. Now, this is an employment case. The plaintiff, Shayan Warner, is a pro se former employee of Gilbarco. She was representing herself. She did not have an attorney.
Dan Waltz (12:13):
At some point prior to the close of discovery, the defendants learned that she was using ChatGPT to help her draft her pleadings and discovery requests and in generally understand what was going on. And at some point prior to discovery closing, had issued a discovery request asking for her inputs and outputs for ChatGPT. It was a broad request, and there’s no evidence from the order that I can see that the defense ever built up any sort of record with respect to that specific discovery request. However, after the close of the discovery, the defendants filed a motion with the court and asked the court to overrule the plaintiff’s objections to producing the inputs and outputs of her ChatGPT logs on grounds that those were not privileged and they were not work product. And so those are the same two bases that we just heard that the court denied the request in Hepner. So the court looked at the request from the defense attorney and denied it on several grounds. So, again, this is different from Hepner because this is a civil case. The plaintiff is pro se, so she is acting as her own attorney. Ultimately, based on the procedural posture, the defendant’s request was untimely, but the court didn’t make a ruling on that basis. It addressed the merits. First, the court found that the defendant’s request for the ChatGPT logs wasn’t proper. It was not relevant. Under Rule 26, materials that are prepared by an attorney or a party in anticipation of litigation or trial are not relevant. They’re not proper subjects of discovery. And so the court said, as a preliminary matter, Rule 26 doesn’t allow this kind of discovery. But then the court moved on to address the claims that the inputs and outputs of ChatGPT were privileged and work product. And first, the court, just like in Hepner, found that there was no privilege. ChatGPT is not an attorney. Any communications with ChatGPT are not a communication between an attorney and a client, so there is no privilege. So on that basis, there was no grounds to deny the request.
Dan Waltz (14:14):
However, the court then turned to the issue of work product and concluded that, yes, even though this material is not discoverable under Rule 26, it would be improper to order the production of the pro se plaintiff’s ChatGPT inputs and outputs because that is work product. First, the court examined precedent and concluded that pro se individuals can invoke the work-product doctrine as a basis to withhold information. Then the court examined the scope of work product and waiver specifically and evaluated that work product can be waived, but waiver of work product has to be intentional and has to be to an adversary. And just like with the privilege analysis, the court stated that ChatGPT is not an adversary. It’s a tool. So there was no disclosure to an adversary. There was a reasonable expectation that those logs would be private and not discoverable. And if you think about it, that makes sense because most individuals can’t just go to ChatGPT. I, Dan Waltz, could not go to ChatGPT and ask it to give me Gene Fishel’s inputs and outputs. It’s not possible. So there is a reasonable expectation of privacy, no disclosure that would waive work product, and on that basis, the court said these materials are not properly disclosed. So there was actually a finding that work-product protections do apply to the inputs and outputs of ChatGPT.
Dan Waltz (15:34):
Now, the court must not have really liked these discovery requests because at the end of the day, the court also said that in addition to these grounds that it was not going to grant the defendant’s request and order the plaintiff to turn over those logs, it thought that this was essentially a fishing expedition by the defendant. The request was exceedingly broad, just your inputs and outputs for your ChatGPT use. I’d speculate that there’s potentially a different outcome here. I think attorneys are going to start asking for this sort of discovery regularly. So if an attorney had pursued this line of reasoning and had developed a record and perhaps had more specific requests, the ability to articulate the need a little more clearly, it’s possible that the court would have come out differently. It didn’t… In this case, there was actually a finding that work product protects the disclosure of interactions with ChatGPT. So just an interesting counterpoint with a little bit different outcome than Hepner.
Ashley Taylor (16:29):
Well, Dan and Gene, we always like to make sure that the listeners receive some practical advice from every one of our episodes. So what advice would you all give an attorney navigating these issues? And I’m sure we have a number of in-house counsel listening, so provide some comments about how they can navigate AI successfully.
Gene Fishel (16:53):
As it relates to these two cases we just cited, let’s put things in perspective. First, these two cases, to the extent they’re binding, they’re binding in these specific jurisdictions, Southern District of New York and Michigan. Also, let’s not read them as broader than they actually are because the court’s decisions here are fairly narrow. And what’s important is they’re fact-intensive inquiries. They’re really based on the specific facts, the specific AI platforms involved, and interaction between the clients or the pro se party and the AI platform. So these are not rulings saying that all AI use is gonna destroy privilege or destroy the work-product confidentiality, those sorts of things. That being said, attorneys should educate themselves on the types of AI platforms that are out there and specifically the types of AI platforms they’re using. And that includes reading the terms of service. If you’re dealing with a client that has used AI, also educating yourself on the type of platform the client is using. These cases honestly highlight the potential legal pitfalls and legal dangers of using publicly-available AI platforms. There’s a greater risk involved in destroying privilege and potentially work-product confidentiality if a client or an attorney is using something where, again, a company is using inputs and outputs to train models or other people have access to it.
Gene Fishel (18:32):
That being said, really the safest thing for attorneys is to use a closed AI system or proprietary system where access is limited to what is being put in and produced. Troutman, for example, has a proprietary system it uses that is a closed system. So attorneys need to take these considerations, public versus private, into account. Attorneys also need, if you’re gonna utilize AI, you really need internal AI use policies. Something Troutman helps other firms with is developing AI governance controls and using it. Who in your firm is touching the AI system? What kind of information could be put into the system? The people who are using the system and it involves a client matter, what is the supervisory hierarchy? If there are associates, do they have to report use to partners? Those sorts of things. So internal policies should be developed with firms. And attorneys in a particular matter need to also advise their clients of the potential dangers for using AI systems. And as far as navigating a matter, it’s also important to have AI use mentioned in client engagement agreements. How is AI gonna be used in the matter and to what extent? That goes really just to informed consent duty, where clients need to know the ethical duty for attorneys and clients need to know how AI is being used in practice. There are also other privacy considerations. Entering personal information into publicly-available AI platforms is not wise. There are trade secret, intellectual property considerations. We could spend hours talking about that, but I just raise these issues because there are many, many sensitive areas depending on what kind of law you practice, areas of law you practice that you need to consider. And so thoughtful development of internal use policies and governance controls is key for attorneys in this area.
Dan Waltz (20:43):
Gene, I agree with everything you just said. And I think it’s a really important point to emphasize that education is critically important for attorneys. This is a powerful new tool that is going to become a ubiquitous part of the practice of the law. And not only do attorneys need to know how to use it for themselves, but they need to know how to advise their clients about the use of artificial intelligence. It is going to be increasingly common that clients will have a conversation with their attorney, they’ll go home, they’ll go to their internet browser, open it up, and type their questions directly into ChatGPT. Given the risk and the unsettled nature of the attorney-client privilege and work-product protections, the client’s conduct could potentially be a litigation risk and liability. So attorneys need to counsel their clients against taking those steps and potentially creating discovery problems. In addition to just that basic human desire to ask the easiest resource for information, attorneys also need to counsel their clients about how they’re interacting with their vendors and whether where those vendors are using non-public information and how they’re using it. If you have vendors who are inputting non-public information from a company into a public GPT, there are other potential implications, potentially in discovery, for trade secrets, and other issues that could arise. And so attorneys advising clients, especially in clients in heavily regulated areas or clients who face a large amount of litigation, it’s very important that the attorney can help control the information. And one way to do that is by ensuring that the client’s use of artificial intelligence is not undermining other efforts to help that client out.
Ashley Taylor (22:21):
We’ve talked about a couple of the important cases that really seem to be shaping the contours of the law around AI. What about the other aspects of the regulatory environment, state bar opinions, et cetera? What’s that landscape look like?
Gene Fishel (22:38):
What we’re seeing is the case law that’s out there, the state bar opinions that are starting to come up. Where attorneys are starting to get in trouble is when they are using AI in a potentially unethical manner. They’re relying on an AI platform for legal judgments, or they’re relying on AI research, such as citations, without verifying the output itself, personally verifying what the AI is putting out. And so attorneys are getting in trouble by submitting made-up, completely imaginary cases and case cites and using that in support of their cases. And courts are weighing in on it, are sanctioning attorneys. State bars are weighing in on the ethical responsibilities that attorneys have. And so AI use will implicate several ethics rules, regardless of the state you’re in. Those include competence, due diligence, confidentiality, candor to a tribunal. So submitting false case cites is gonna get an attorney in trouble. There’s informed consent, as I mentioned. Clients need to know how AI is being used. There are even security considerations about entering client information into AI systems. So there are many ethical considerations that attorneys need to be aware of. And really the bottom line is whatever outputs an AI platform an attorney is using, whenever outputs are coming out of this AI platform, attorneys need to verify that that information is correct, or else there is a great risk of being sanctioned by either a court or a state bar. And we’ve seen now dozens of cases out there of sanctioning of attorneys who have just relied on, who are using AI to substitute their best judgment or research, like I mentioned. So these are the pitfalls out there. We’re gonna continue to see state bar opinions, court rules, and regulations pop up surrounding AI. Again, we’re in the nascent stages of AI regulation, but some of this case law that’s already out there is instructive.
Ashley Taylor (25:01):
Well, Gene and Dan, I want to thank you for joining me today. It’s been an interesting conversation and I appreciate hearing you all’s perspective. And I want to thank our listeners for tuning in as well. And remember to subscribe to this podcast via Apple Podcasts, Google Play, Stitcher, or whatever platform you use. And we look forward to having you join us next time.
Copyright, Troutman Pepper Locke LLP. These recorded materials are designed for educational purposes only. This podcast is not legal advice and does not create an attorney-client relationship. The views and opinions expressed in this podcast are solely those of the individual participants. Troutman does not make any representations or warranties, express or implied, regarding the contents of this podcast. Information on previous case results does not guarantee a similar future result. Users of this podcast may save and use the podcast only for personal or other non-commercial, educational purposes. No other use, including, without limitation, reproduction, retransmission or editing of this podcast may be made without the prior written permission of Troutman Pepper Locke. If you have any questions, please contact us at troutman.com.
—————————————————————————
DISCLAIMER: This transcript was generated using artificial intelligence technology and may contain inaccuracies or errors. The transcript is provided “as is,” with no warranty as to the accuracy or reliability. Please listen to the podcast for complete and accurate content. You may contact us to ask questions or to provide feedback if you believe that something is inaccurately transcribed.
