An Approach to Reduce Web Crawler Traffic Using Asp.Net
Chetna1, Harpal Tanwar2, Navdeep Bohra3
1Chetna, M.TECH (Scholar) Jagannath University, Jaipur (Rajasthan), India
2Harpal Tanwar, Lecturer, BMCTM, Farukhnagar (MDU), India
3Navdeep Bohra, Assistant Professor, MSIT, Janakpuri, New Delhi, India
Manuscript received on July 01, 2012. | Revised Manuscript received on July 04, 2012. | Manuscript published on July 05, 2012. | PP: 464-469 | Volume-2, Issue-3, July 2012. | Retrieval Number: C0815062312 /2012©BEIESP
Open Access | Ethics and Policies | Cite
© The Authors. Published By: Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: Now days search engine transfers the web data from one place to another. They work on client server architecture where the central server manages all the information. A web crawler is a program that extracts the information over the web and sends it to the search engine for further processing. It is found that maximum traffic (approximately 40.1%) is due to the web crawler. The proposed scheme shows how web crawler can reduce the traffic using Dynamic web page and HTTP GET request using asp.net.
Keywords: Crawler, Search Engine, WWW