Social media marketing Influence: Well-liked trends on social media can spill above into engines like google, impacting the research volume of relevant key phrases. txt file is then parsed and will instruct the robotic as to which web pages will not be to be crawled. As a search engine crawler https://www.youtube.com/watch?v=aKirkq1uHGM