The share of a reproduction paper is based on analyses of plus in insights into current methods and problems—plus the certainty that is added is sold with validating past outcomes.

1. Relevance: Is it paper highly relevant to COLING?
2. Readability/clarity: may be the paper well-structured and well-written?
3. Data/code supply: could be the data/code (as appropriate) open to the investigation community or perhaps is here a compelling explanation offered why this isn’t feasible?
4. Analysis: In the event that paper managed to reproduce the total link between the earlier in the day work, does it plainly construct exactly exactly what must be filled in in order to do therefore? It clearly identify what information was missing/the likely causes if it wasn’t able to replicate the results of earlier work, does?
5. Generalizability: Does the paper exceed replicating the total outcomes regarding the initial to explore whether or not they could be reproduced an additional environment? Instead, in instances of non-replicability, does the paper discuss the wider implications of this outcome?
6. Informativeness: To exactly exactly exactly what level does the analysis reported when you look at the paper deepen our comprehension of the methodology utilized or perhaps the nagging issue approached? Will the given information when you look at the paper assistance practitioners due to their range of technique/resource?

Read More