Tree-based Focused Web Crawling with Reinforcement Learning

Andreas Kontogiannis, Dimitrios Kelesis, Vasilis Pollatos +2

A focused crawler aims at discovering as many web pages and web sites relevant to a target topic as possible, while avoiding irrelevant ones. Reinforcement Learning (RL) has been a…