The 2021 edition of the SIGIR eCom Data Challenge, hosted by Coveo, ran from April 21 to June 12. Over 20 teams, coming from both industry and academia, participated in the Data Challenge and a total of 6 final design papers, where teams shared their insights and methods, were accepted. The final results of the Data Challenge were presented on July 15, 2021 during the SIGIR eCom'21 Workshop, and featured an invited talk from the NVIDIA team and a round table discussion with various participating teams.

Training data, evaluation scripts and rules can be found in the official challenge repository; relevant literature and background information about the challenge and relevant industry use cases can be found in the challenge paper pre-print.

Contacts

For questions regarding the dataset, please contact Jacopo Tagliabue.

Challenge Overview

This challenge addresses the growing need for reliable predictions within the boundaries of a shopping session, as customer intentions can be different depending on the occasion. In the context of e-commerce technology, the feedback loop determined by behavioural signals spans from hours to a few seconds and machine learning models need to adapt as fast as possible to the continuously changing nature of the customer journey.

The need for efficient procedures for personalization is even clearer if we consider the e-commerce landscape more broadly: outside of giant digital retailers, the constraints of the problem are stricter, due to smaller user bases and the realization that most users are not frequently returning customers.

We release a new session-based dataset including fine-grained browsing events (detail, add, purchase), enriched by linguistic behavior (queries made by shoppers, with items clicked and items not clicked after the query) and catalog meta-data (image, text, pricing information). On this dataset, we ask participants to showcase innovative solutions for two open problems:

1. a recommendation task, where a model is shown k events at the start of a session, and it is asked to predict future product interactions in the same session;
2. an intent prediction task, where a model is shown a session containing an add-to-cart event, and it is asked to predict whether the item will be bought before the end of the session.

Please refer to the public repository for details on rules, evaluations and everything related to the dataset.

Organizing Committee

• Jacopo Tagliabue - Coveo AI Labs
• Ciro Greco - Coveo AI Labs
• Jean-Francis Roy - Coveo
• Federico Bianchi - Postdoctoral Researcher at Università Bocconi
• Giovanni Cassani - Tillburg University
• Bingqing Yu - Coveo
• Patrick John Chia - Coveo

Acknowledgements

The organizers wish to thank Luca Bigon for his outstanding support in data collection, and Surya Kallumadi, Massimo Quadrana, Dietmar Jannach, Ajinkya Kale for precious feedback on a previous version of this paper. Finally, special thanks to Richard Tessier and Coveo's legal team for believing in this data sharing initiative.

System Description Paper

The following system description papers were accepted:

1  . Transformers with multi-modal features and post-fusion context for e-commerce session-based recommendation  [PDF]
          Gabriel Moreira, Sara Rabhi, Ronay Ak, Md Yasin Kabir and Even Oldridge.
2  . Comparison of Transformer-Based Sequential Product Recommendation Models for the Coveo Data Challenge  [PDF]
          Elisabeth Fischer, Daniel Zoller and Andreas Hotho.
3  . Utilizing Graph Neural Network to Predict Next Items in Large-sized Session-based Recommendation Industry Data  [PDF]
          Tianqi Wang, Zhongfen Deng, Houwei Chou, Lei Chen and Wei-Te Chen.
4  . Session-based Recommender System Using an Ensemble of Multiple NN Models with LSTM and Matrix Factorization  [PDF]
          Yoshihiro Sakatani.
5  . Adversarial Validation to Select Validation Data for Evaluating Performance in E‑commerce Purchase Intent Prediction  [PDF]
          Shotaro Ishihara, Shuhei Goda and Hidehisa Arai.
6  . A Session-aware DeepWalk Model for Session-based Recommendation  [PDF]
          Kaiyuan Li, Pengfei Wang and Long Xia.

Timeline  (UTC)

April 21	Data Challenge registration opens; Stage 1 opens
June 5	Data Challenge registration deadline
June 10	Stage 1 closes
June 11	Stage 2 opens; Paper submission opens
June 17	Stage 2 closes
June 29	Paper submission closes
July 7	Paper Accept/Reject
July 10	Camera ready paper submission deadline
July 15	Workshop

Leaderboards (Stage 2)

Leaderboards (Stage 1)