New York City Struggles to Define Which School AI Tools Children May Use

Days before the school year began, New York City announced that it would disable or discontinue the artificial intelligence components of more than 38 programs previously allowed in hundreds of public schools, citing new safety and oversight standards. The names of the flagged products were not initially released.

A list later obtained by reporters shows the restrictions reach widely used reading and math curriculums, student assessment tools, and a chatbot the Education Department itself built in partnership with Microsoft. One reading tutor used in more than 200 schools was banned outright because, officials said, it could not function without AI.

The response from companies was uneven. Several insisted their products do not use the kind of generative AI the policy prohibits and said they did not understand why they were flagged, including a translation tool meant to help students learning English. Others acknowledged prohibited features and moved quickly to switch them off.

The Education Department said the restrictions apply to student-facing generative AI features rather than entire products, and that it is working case by case to determine which tools can be reinstated. It did not explain how it decided which products met that definition. Critics cautioned against trusting either the companies or the department, and called for a more transparent vetting process. Parents were advised to contact their schools or community education councils for information about the tools in use.

Why this matters for families:

• Children encounter AI through ordinary classroom software, not only through chatbots they choose to open.
• Confusion over what counts as AI makes it harder for parents to know what their children are exposed to.
• Clear, independent vetting is needed before such tools reach young learners.

This development was reported by Route Fifty (via Chalkbeat).