Flutterby™! : Michael Cohen on distinguishing LLMs from search

We've probably already seen the news that Donald Trump's "fixer"/lawyer Michael Cohen used fake cases created by Google Bard AI in bid to end his probation (#GiftArticle), but the filing notes:

Specifically, the citations and descriptions came from Google Bard. As a non- lawyer, I have not kept up with emerging trends (and related risks) in legal technology and did not realize that Google Bard was a generative text service that, like Chat-GPT, could show citations and descriptions that looked real but actually were not. Instead, I understood it to be a super-charged search engine and had repeatedly used it in other contexts to (successfully) find accurate information online. I did not know that Google Bard could generate non-existent cases, nor did I have access to Westlaw or other standard resources for confirming the details of cases. Instead, I trusted Mr. Schwartz and his team to vet my suggested additions before incorporating them.

I think there are a couple of interesting things here: That we as a society are needing to go back to a world of specialized search with vetted information (Westlaw), and the open web is now polluted to the point where even if we find a link, we don't know if it was LLM generated, or what the provenance is. This is just part of an SEO driven trend, but it's definitely moreso.

That the legal profession is probably going to become more valuable, because putting a reputation behind a claim means more.

And that little disclaimer that "Bard may display inaccurate info, including about people, so double-check its responses." is not enough.