What is Robots.txt File in 2020

Robots.txt is the simplest text file that is available at the root directory of the website. The robots.txt file is mainly used to instr...