Edward Lee on the toxic culture of peer review

In the last episode from my own reviewing hall of shame, I mentioned the coincidence of attending a webinar where Edward Lee talked about the ‘toxic culture of rejection’, based on a blog post he write in 2022. In this episode we hear from Edward directly and discuss the culture of rejection, in CS especially, the problems with peer reviewing, the nature of conferences, and how we might approach reviewing differently and start to change the culture around publications, acceptance rates and evaluations without losing quality standards. His bio: Edward A. Lee has been working on embedded software systems for more than 40 years. After studying and working at Yale, MIT, and Bell Labs, he landed at Berkeley, where he is now Professor of the Graduate School in EECS. His research is focused on cyber-physical systems. He leads the open-source software project Lingua Franca and previously Ptolemy II, is a coauthor of textbooks on embedded systems, signals and systems, digital communications, and philosophical and social implications of technology. His current research is focused on a polyglot coordination language for distributed real-time systems called Lingua Franca that combines features of discrete-event modeling, synchronous languages, and actors. “There's a classical view of the purpose of publication, which is essentially to add knowledge to the archive […] I feel…we should understand that the primary purpose of publication is to communicate with other humans.”“A scientific discipline progresses in a very cultural way… it's really about a human culture of developing and evolving, and, and it tends to evolve in a very chaotic way.”“Institutions should be prepared to do their own evaluation.”“The criterion should be, what is informative, interesting, and potentially valuable and useful to the community.”“Everyone involved in the [review] process knows that we're dealing with other human beings. And the phrase that I've tried to use, […]  is to pretend that this paper was written by their sister. How would that change [how you assess the paper]?”  Overview (times approximate): [FULL TRANSCRIPT for download] 0:05 Welcome to Changing Academic Life. 0:30 Intro to the episode03:03 Welcome Edward Lee starts to introduce himself 06:04 How the faculty position selection process has changed over time  - hypercompetition. 07:31 The gradual change towards hypercompetition, the randomness of the review process and the role of luck in getting papers accepted11:41 The problem of the conference peer review process in no real opportunity for dialogue compared to journal review processes12:38 This has the effect of a certain amount of randomness and conservatism. 14:44 What are conferences for? The importance of informal communication orthogonal to the  publications17:02 The obsessive focus on novelty18:25 The purpose of publication, how science progresses and the importance of dialogue and culture.22:59 The challenge of publishing multidisciplinary and systems papers26:44 Playing the game the right way31:17 The randomness of reviews and factors around this in program committees37:30 The  tensions and conflicts of  selective conferences for rankings38:07 Learning from how other communities work re conferences and journals40:21 The association of publications with funding to attend a...

There are no comments yet.

Please log in to write the first comment.