The share of a reproduction paper is based on analyses of plus in insights into current methods and problems—plus the certainty that is added is sold with validating past outcomes.

1. Relevance: Is it paper highly relevant to COLING?
2. Readability/clarity: may be the paper well-structured and well-written?
3. Data/code supply: could be the data/code (as appropriate) open to the investigation community or perhaps is here a compelling explanation offered why this isn’t feasible?
4. Analysis: In the event that paper managed to reproduce the total link between the earlier in the day work, does it plainly construct exactly exactly what must be filled in in order to do therefore? It clearly identify what information was missing/the likely causes if it wasn’t able to replicate the results of earlier work, does?
5. Generalizability: Does the paper exceed replicating the total outcomes regarding the initial to explore whether or not they could be reproduced an additional environment? Instead, in instances of non-replicability, does the paper discuss the wider implications of this outcome?
6. Informativeness: To exactly exactly exactly what level does the analysis reported when you look at the paper deepen our comprehension of the methodology utilized or perhaps the nagging issue approached? Will the given information when you look at the paper assistance practitioners due to their range of technique/resource?
7. Significant contrast: along with determining the experimental outcomes being replicated, does the paper motivate why these specific email address details are a target that is important reproduction and exactly exactly what the long term implications are of these having been reproduced or been discovered to be non-reproducible?
8. General suggestion: there are lots of good submissions contending for slots at COLING 2018; essential will it be to feature this 1? Will people learn a complete great deal by scanning this paper or seeing it presented? Please be decisive—it is way better to vary from other reviewers than to grade every thing at the center.

Site paper

Documents in this track provide a brand new language resource. This might be a corpus, but additionally might be an annotation standard, device, and so forth.

1. Relevance: Is it paper highly relevant to COLING? Will the resource presented likely be of good use to the community?
2. Readability/clarity: Through the means the paper is created, is it possible to inform the way the resource had been produced, the way the quality of annotations (if any) had been examined, and exactly why the resource ought to be of great interest?
3. Originality: Does the resource fill a need within the collection that is existing of resources? Observe that originality could possibly be within the range of language/language genre or variety, within the design for the annotation scheme, into the scale of this resource, or still other parameters.
4. Site quality: what type of quality control ended up being performed? If appropriate, ended up being inter-annotator agreement calculated, if therefore, with appropriate metrics? Otherwise, the other assessment ended up being carried out, and exactly how acceptable were the outcome?
5. Site accessibility: might it be simple for researchers to down load or access that is otherwise resource to be able to put it to use in their own personal work? As to what degree can perhaps work centered on this resource be provided? answers to add: Yes, We have confirmed
6. Metadata: perform some writers explain whoever language use is captured in the resource and also to exactly just just what populations experimental results based from the resource might be generalized to? In instance of annotated resources, are the demographics associated with the annotators also characterized?
7. Significant contrast: could be the new resource situated with regards to current work with the industry, including similar resources it took motivation from or improves on? Will it be clear what exactly is unique about the resource?
8. General suggestion: there are lots of submissions that are good for slots at COLING 2018; essential could it be to feature that one? Will people discover great deal by looking over this paper or seeing it presented? Please be decisive—it is way better to vary from other reviewers rather than grade everything in the center.

Position paper

A situation paper presents a challenge to mainstream reasoning or even a futuristic vision that is new. It might start a new area or unique technology, propose changes in existing research, or provide a fresh pair of ground guidelines.

1. Relevance: Is it paper highly relevant to COLING?
2. Readability/clarity: Is it clear just exactly what the career is the fact that paper is arguing for? Would be the eliteessaywriters.com/blog/psychology-research-paper-topics review arguments because of it laid away in an understandable method?
3. Soundness: Are the arguments presented into the paper appropriate and coherent? Could be the eyesight well-defined, with success requirements? (Note: it ought to be possible to provide a top rating right here even though you don’t concur with the place taken by the authors)
4. Imagination: How unique or bold may be the place drawn in the paper? Does it express well-thought through and imaginative ground that is new?
5. Range: How much range for brand new scientific studies are opened by this paper? Just What effect could it have on current areas and concerns?
6. Significant contrast: could be the paper well-situated pertaining to past work, both place documents (taking exactly the same or opposing side on a single or comparable dilemmas) and appropriate theoretical or work that is experimental?
7. Substance: Does the paper have sufficient substance for a paper that is full-length? May be the presssing problem adequately crucial? Would be the arguments sufficiently thoughtful and diverse?
8. General suggestion: there are lots of submissions that are good for slots at COLING 2018; essential will it be to feature this 1? Please be decisive—it is way better to vary from other reviewers rather than grade every thing at the center.

A study paper provides a structured summary of the literature up to now on a topic that is specific assists the reader understand the kinds of concerns being inquired about the subject, the different approaches which have been used, the way they connect with one another, and exactly what further research areas they open. A conference-length study paper should be in regards to a adequately concentrated subject it can repeat this effectively with when you look at the web page limits.

1. Relevance: may be the paper highly relevant to COLING?
2. Readability/clarity: may be the paper generally speaking very easy to follow and well organized?
3. Organization: Does the paper arrange the relevant literature in a narrative and recognize typical strands of inquiry?
4. Scope: Does the paper determine a fairly concentrated area to review?
5. Thoroughness: because of the region identified to survey, does the paper address every one of the appropriate literary works? May be the literary works evaluated represented accurately?
6. Outlook: Does the paper recognize areas for future work and/or demonstrably mention exactly what isn’t yet managed in the literary works surveyed?
7. Context: Does the paper situate research that is current within its historic context? (We don’t expect papers first of all Pa?ini, yet during the exact same time one thing that just cites work from 2017 most likely does not capture exactly exactly how current work pertains to the larger image.)