NYC Chatbot Advises Restaurant Owners to Serve Cheese Bitten by Rats

Date:

Share post:


New York City leaders and small business owners are learning about the pitfalls of relying on AI the hard way. 

According to a report from the Daily Wire, the city introduced an AI-powered chatbot in October 2023 as a “one-stop shop” for assisting business owners with government operations. The objective was to answer questions posed by small business owners as they pertain to city laws and codes. 

However, the chatbot has been giving out illegal – and rather absurd – advice, including telling employers it’s acceptable to take workers’ tips and fire pregnant workers, while advising landlords to discriminate based on income. 

According to tests run by the Markup, the mistakes cover a range of topics, including housing policy, worker rights and regulations for businesses. 

Errors include telling landlords they can lock out tenants and ignore income sources like Section 8 vouchers, despite these actions being illegal in New York City. The chatbot’s inaccuracies extend to other areas, wrongly informing businesses they can operate without accepting cash and misguiding them on worker tip regulations.

Some of the AI’s responses bordered on absurdity. For example, one user testing the product was informed it was legal to serve restaurant patrons cheese that had been nibbled on by rodents. 

The chatbot, powered by Microsoft’s Azure AI, was expected to be a reliable source for navigating city regulations and avoiding penalties. However, its failure to provide correct information has raised concerns about its impact on small business compliance and legal standing. 

Granted, like other similar ai tools, New York City’s chatbot offers a disclaimer that it can “occasionally produce incorrect, harmful or biased” information. It also includes the caveat that its answers are not legal advice. Many, however, wonder if that’s enough warning to continue employing the tool.

But NYC leaders are sticking with the program, no matter how flawed it might be. 

“Anyone that knows technology knows this is how it’s done,” New York City Mayor Eric Adams said at a press conference. “Only those who are fearful sit down and say, ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”

Experts disagree, however.

“They’re rolling out software that is unproven without oversight,” Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University, told the Associated Press. “It’s clear they have no intention of doing what’s responsible.”

For the time being at least, the chatbot is still available to NYC small business owners. The question remains, though: Should they trust it? Like most other AI-based tools, it’s best to proceed with caution.




LEAVE A REPLY

Please enter your comment!
Please enter your name here

Related articles

Fashion Models and Business School Professors Agree This Is the No. 1 Way to Project Confidence 

Catwalk experience and scientific research lead to the same conclusion on how to appear ultra confident.

Winter Weather Preparation for Building Owners and Fleets

This post is part of a series sponsored by...

Google Israel R&D head Yossi Matias moving to global role

Sources inform “Globes” that Prof. Yossi Matias, who for the past eighteen years has headed Google’s research...

Please Make a Small Donation to Help More Kids Play Soccer ⋆

Help! I’m taking part in the 31st Annual Austin Marathon Gives to raise money for Soccer Assist,...