from the article:

Robots.txt is a line of code that publishers can put into a website that, while not legally binding in any way, is supposed to signal to scraper bots that they cannot take that website’s data.

i do understand that robots.txt is a very minor part of the article, but i think that’s a pretty rough explanation of robots.txt