The flood of AI crawlers and automated agents is forcing publishers and content owners to face a challenger: Deciding who gets access to their content, and under what terms. To help address that problem, IAB Tech Lab released new guidance today on bot and crawler management strategies, opening the document for public comment through June 26, 2026. The guidance is designed to help publishers, content owners, and ad tech companies think through how they manage non-human traffic, including AI systems scraping or accessing content. It also complements the organization’s recently released CoMP API V1, a framework intended to support communication and permissions between AI systems and publishers. “Giving content owners clear, practical guidance is key if we want adoption to move forward in a meaningful way,” Anthony Katsur, CEO of IAB Tech Lab, said in a statement.…