Image Matching Challenge 2021

The 2022 IMC challenge will take place on Kaggle.

Ground truth has been released, please only submit processed results

Local features have played a key role in a wide range of computer vision applications throughout the past 20 years. Despite the drastic advancements in many fields after the deep learning revolution, end-to-end solutions for 3D reconstruction under challenging conditions remain elusive, and solutions based on traditional, hand-crafted methods such as SIFT (1999) and RANSAC (1981) may outperform the state of the art in machine learning.

Historically, local features have been evaluated on small datasets using proxy metrics, such as patch retrieval accuracy for descriptors. This may not necessarily translate to downstream applications, misrepresenting their actual performance and hindering the development of new techniques. There is a clear need for large-scale, challenging benchmarks to both train and evaluate new strategies for image matching. Towards this end, we held in 2019 an open challenge in image matching across wide baselines. In contrast to existing benchmarks, we only consider integrated solutions and our primary metric measures the accuracy of the reconstructed poses — the final goal.

We are pleased to introduce the third edition of this challenge. It is co-located with the CVPR 2021 workshop on Image Matching: Local Features and Beyond. Winners and selected participants will be invited to give talks at the conference, as last year. Prizes are available, thanks to our sponsors.

News

September 7, 2022: Releasing ground truth for Phototourism and PragueParks test set.
April 4, 2022: The challenge has moved to Kaggle! It opens today and will run for two months, until June 2.
January 17, 2022: The workshop has been renewed for CVPR 2022! We will announce plans for the 2022 challenge soon -- stay tuned.
June 21, 2021: Announcing the results of the challenge!
June 08, 2021: The synthetic dataset image matching challenge has been announced!
May 27, 2021: A new tutorial by Dmytro on custom matcher.
May 25, 2021: A small change to the rules: you may now use descriptors of any size. Please check the submission page for details.
May 17, 2021: A new tutorial by Dmytro may help you create your submission.
May 10, 2021: The 2021 challenge has been announced!
June 12, 2020: The challenge is officially over, and we are announcing the results!
April 27, 2020: Small reorganization of the leaderboard and bugfixes.
April 17, 2020: The challenge deadline has been postponed (for details, see link).
March 18, 2020: Instructions to set up a scalable cluster to run the benchmark on Google Cloud.
March 18, 2020: We have processed the first 8 challenge submissions (in the leaderboards). CVPR is still on track, despite the pandemic. We will hold the workshop if CVPR remains on. We plan to hold the challenge whether CVPR is held or not.
March 3, 2020: Published benchmark paper on arxiv.
February 10, 2020: Introducing the 2020 challenge.

Key Dates

Challenge submission deadline: June 14, 2021
Deadline for short descriptions on submissions: June 15, 2021
Workshop date (CVPR'20): June 25, 2021

2021 Challenge Highlights

Three additional datasets: PragueParks, GoogleUrban dataset, Synthetic dataset (TBA)
Fresh new leaderboard with Phototourism/PragueParks/GoogleUrban
Additional detailed benchmarking with realistic synthetic data (TBA)

Organizers

Eduard Trulls Google

Yuhe Jin UBC

Kwang Yi UBC

Dmytro Mishkin CTU Prague

Jiri Matas CTU Prague