Double Blind FaQ

This year, ICPE is using a double-blind reviewing (DBR) model for all peer-reviewed submissions that are to be published in ICPE’s main proceedings (research track and industry track papers). Submissions for other tracks (including Artifact Evaluation) are not impacted by the information on this page.

DBR means that submissions should be anonymized, and that reviewers should not be aware of the identity or affiliation of authors. The name “double blind” refers to the reviewing process under DBR to be “blind” in both directions (not only authors do not know who their reviewers are, but also the other way around). The goal of DBR is to prevent implicit bias in the reviewing process, both positive and negative, and hence to increase fairness towards all authors. Furthermore, DBR is intended to be pragmatic - it is intended as a measure that requires little additional effort from the authors and which does not substantially restrict submissions, while still eliminating (implicit) bias and increasing fairness.

More information on DBR can be found here and here, among many other places.

How to prepare your submission for double-blind reviewing

ICPE’22 authors are required to make reasonable effort to honor the DBR process and to keep their identities hidden. Research or industry track submissions that violate the guidelines described on this page may be desk-rejected. This includes specifically:

  • Either removing author names on the title page or replacing them with a placeholder, such as “Anonymous Authors”. The title page shall also not list author job titles or affiliations. Example:

  • Removing named acknowledgements from the submission PDF, if the name or affiliation of acknowledged persons would disclose or narrow down potential authors.

  • Referring to own previous work in third person. Tools, approaches, or systems that would clearly reveal author identities to a reviewer knowledgeable in the field have to be avoided. Previous published work shall be cited and referenced in the same way as any other previous work. Bad example: “We build on our tool ICPE-Master [1].” Good example: “This research is based on the ICPE-Master tool provided by Leitner and Herbst [1]”.

  • Avoiding explicitly naming case study partners, if naming them would realistically narrow down the author’s identity to a small subset of the community (e.g., to researchers affiliated with this company). Instead, anonymous company names shall be used, for instance when describing empirical studies. Systems that are open for study to all or a large part of the community, such as open source systems, can of course be named.

  • Removing links to supplementary material that would disclose author names or affiliations, such as personal web pages, GitHub pages, Youtube channels, companion technical reports, or student thesis reports. Note that this does not mean that DBR does not allow for data to be published - data can and should be uploaded to third-party services blinded. Instructions how to upload blinded data (which can be unblinded after acceptance) can be found here.

Notably, the goal of these measures is not to make it impossible for reviewers to guess author identities. Much more, the guiding principle is to maximize the number of people who could plausibly be authors, and to avoid that reviewers automatically become aware of author identities merely by carefully reading the paper and reflecting on its content.

After paper acceptance, authors will be given ample time to update their manuscript and “unblind” it to their liking for the published (camera-ready) version (add acknowledgements, add explicit mention of company names, add more explicitly mentions of previous work, etc.), see also the ICPE Important Dates page. The information discussed here is solely for the version that will be reviewed by the conference PC.

DBR and Pre-Prints

The ICPE DBR policy does not prevent authors from submitting pre-prints, project reports, or extended technical reports. It is fine to upload submissions to systems such as arXiv prior to submission (including author names and identities). However, authors shall:

  • Use a pre-print or report title that’s sufficiently different from the submission title that merely googling for the paper title does not immediately bring up the pre-print or report. (Concerned authors please note that, e.g., on Google Scholar, similar or consecutive publications can be manually joined later, plus arXiv does allow for updates.)

  • Not reference the pre-print or report in the submission, especially in a way that would make clear that it is by the same author team. Bad example: “More information about the approach can be found in our technical report [1].”

The general philosophy, as always, is to pragmatically increase author anonymity as much as possible without restricting how authors choose to use and disseminate their research results.

DBR Submissions Based on Student Work

ICPE’22 submissions can be based on work that originally appeared in a master’s or bachelor thesis report. In this case, authors shall not cite the work in the submitted paper. Citations to the previous thesis report can and should be added after acceptance, for the camera-ready version.

Discussing ICPE’22 Submissions With Colleagues

Some authors may wonder if they need to be “extra-secretive” about work intended for DBR submission, so as not to accidentally disclose their identity to a potential reviewer. This is not the case. Authors are free to discuss and present their work prior or during DBR submission normally within reason, e.g., discuss the work with colleagues at different institutions or present the work at meetings (e.g., job talks, visits to research labs, a Dagstuhl or Shonan meeting). However, authors shall avoid dissemination that can be perceived as primarily targeting the ICPE PC specifically (e.g., sending personal emails about the submitted work to all or selected ICPE PC members to “keep them in the loop”). If the program committee chairs learn that authors share their results frivolously and with the goal to circumvent DBR, the chairs can decide to desk-reject the paper.

More Questions?

More questions shall be directed to the ICPE’22 Program Committee Chairs Philipp Leitner and Nikolas Herbst (via icpe2022@easychair.org ). We reserve the right to update this information page with more clarifications based on questions.

Disclaimer: the information on this page is largely based on similar instructions provided by ICSE’21.