SOGOU-2012-CRAWL: A Crawl of Search Results in the Sogou 2012 Chinese Query Log

Whiting, S., Jose, J. M. and Alonso, O. (2016) SOGOU-2012-CRAWL: A Crawl of Search Results in the Sogou 2012 Chinese Query Log. In: 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, Pisa, Italy, 17-21 Jul 2016, pp. 709-712. ISBN 9781450340694 (doi: 10.1145/2911451.2914668)

Abstract

In 2012, Sogou, a major Chinese web search engine released a large-scale query log containing 43.5M user interactions, including submitted queries and clicked web page search results. This query log offers a deep sample of queries over a two day period from 30th December 2011 to 1st January 2012. In August 2013, we identified 1.4M predominantly Chinese language unique search result URLs that were clicked at least three times in this query log. We crawled the HTML content of these URLs to construct the supplementary SOGOU-2012-CRAWL dataset, which we release in this work. A real large-scale query log with accompanying crawl such as this offers several opportunities for reproducible information retrieval (IR) research, including query classification, intent modelling and indexing strategy. In this paper we first detail the query log and crawl dataset construction and characteristics. Following this, to demonstrate potential applications we use the crawl to indicatively analyse various time-based patterns in web content and search behaviour. In particular, we study the distribution of language-independent date expressions in the crawled web content. Based on this, we propose a simple approach for modelling the past/present/future temporal intent of queries based on the date the query was submitted by the user, and the dates appearing in the clicked search results. We observe several prominent temporal patterns which may lead to novel time-aware IR approaches.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Jose, Professor Joemon
• Authors: Whiting, S., Jose, J. M., and Alonso, O.
• College/School: College of Science and Engineering > School of Computing Science
• ISBN: 9781450340694

Related URLs

• Organisation