System-Description Papers

for the WASSA-2017 Shared Task on Emotion Intensity (EmoInt)

		Part of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA-2017), which is to be held in conjunction with EMNLP-2017. The main task webpage is here.

Paper: Only, the participants who made a submission on the CodaLab website before May 22, 2017, will be given the opportunity to submit a system-description paper that describes their system, resources used, results, and analysis.  This paper will be part of the official WASSA-2017 proceedings.

Submit your system description paper here.
When entering paper details on the submission webpage, make sure to select the Type of Paper as 'Emotion Intensity Shared Task - System Description Paper' from the drop-down list.

The shared task papers will be accepted in a separate category within WASSA. (The regular WASSA papers will be in a different 'Main Workshop' category.) We will likely accept all shared task papers as long as they are written reasonably well. Your system rank and scores will not impact whether the paper is accepted or not. There will be a review process, whose primary goal is to help you improve your paper. Each team submitting to the shared task will be expected to review at least two papers by other teams. You can share the review load among your team members. The workshop paper submission deadline is June 10 June 18, 2017.

• You are not obligated to submit a system-description paper, however, we strongly encourage all participating teams to do so.
• You cannot submit essentially the same work in more than one form to WASSA-2017. For example, you cannot submit essentially the same work as a system description paper and also as a main workshop long paper. You have to choose one. 
• There is no difference in how a paper will be cited whether it is in the shared task track, or in the main workshop. Of course, only a small proportion of the papers submitted to the main workshop track will be accepted, whereas almost all papers submitted to the shared task track will be accepted.
• You are free to submit an extended version of the WASSA shared task paper to another future venue (workshop, conference, or journal). You may want to check the policies of the venue for submissions that overlap with previously published papers.

Specifications of the system-descriptin paper:

• The paper is to be four to six pages long, plus at most *four pages* for references. We encourage writing at least four pages to make sure you have included sufficient details so that others can replicate your system, and so that the paper includes sufficient experimental analysis to make the paper interesting for readers. Even though you can use up to six pages, we encourage being concise and not using pages 5 and 6 unless the work and analysis warrants it. 
• The papers are to follow the format and style files provided by EMNLP-2017. The style files to be used are the same as for EMNLP 2017, available here: http://emnlp2017.net/call-for-papers.html
• The review process is single-blind. Submissions are not anonymous. You are to enter your names and affiliations on the paper.
• Paper titles are required to follow this format: "[Insert System Name Here] at EmoInt-2017: [Insert Suitable Paper Title Text Here]"

Ans. Here are some key pointers:

• Replicability: present all details that will allow someone else to replicate your system

• Analysis: focus more on results and analysis and less on discussing rankings; report results on several variants of the system (even beyond the official submission); present sensitivity analysis of your system's parameters, network architecture, etc.; present ablation experiments showing usefulness of different features and techniques; show comparisons with baselines. You can use the gold labels that we will release later next week for the extra analysis.

• Avoid Duplication: Cite the *task* description paper that will be written by the task organizers:  

WASSA-2017 Shared Task on Emotion Intensity. Saif M. Mohammad and Felipe Bravo-Marquez. In Proceedings of the EMNLP 2017 Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media (WASSA), September 2017, Copenhagen, Denmark.
BibTex

This paper will provide details of the task, summary of data creation, competition results, and a summary of participating submissions. You can avoid repeating details of the task and data in your paper, however, briefly outlining the task and relevant aspects of the data is a good idea. We will try to make a copy of this paper available before the submission deadline.

Below is the authoritative paper on the Tweet Emotion Intensity Dataset (the dataset used in this competition):

Emotion Intensities in Tweets. Saif M. Mohammad and Felipe Bravo-Marquez. In Proceedings of the sixth joint conference on lexical and computational semantics (*Sem), August 2017, Vancouver, Canada.
BibTex


• Please do not dwell too much on rankings. Focus instead on analysis and the research questions that your system can help address.
• References: References do not count against the page limit (6 pages for system description papers). You may have at most *four pages* for references.

It may also be helpful to look at some of the papers from past SemEval competitions, e.g., from https://aclweb.org/anthology/S/S16/.

My system did not get a good rank. Should I still write a system-description paper?

Ans. We encourage all participants to submit a system description paper. The goal is to record all the approaches that were used and how effective they were. Do not dwell too much on rankings. Focus instead on analysis and the research questions that your system can help address. What has not worked is also useful information.

Designated Contact Person:

Dr. Saif M. Mohammad
Senior Research Officer at NRC (and one of the creators of the resource on this page)
saif.mohammad@nrc-cnrc.gc.ca

Terms of Use:

1. All rights for the resource(s) listed on this page are held by National Research Council Canada.

2. The resources listed here are available free for research purposes. If you make use of them, cite the paper(s) associated with the resource in your research papers and articles.

3. If interested in commercial use of any of these resources, send email to the designated contact person. A nominal one-time licensing fee may apply.

4. If referenced in news articles and online posts, then cite the resource appropriately. For example: "This application/product/tool makes use of the <resource name>, created by <author(s)> at the National Research Council Canada." If possible, hyperlink the resource name to this page.

5. If you use the resource in a product or application, then acknowledge this in the 'About' page and other relevant documentation of the application by stating the name of the resource, the authors, and NRC. For example: "This application/product/tool makes use of the <resource name>, created by <author(s)> at the National Research Council Canada." If possible, hyperlink the resource name to this page.

6. Do not redistribute the resource/data. Direct interested parties to this page. They can also email the designated contact person.

7. If you create a derivative resource from one of the resources listed on this page: 

1. Please ask users to cite the source data paper (in addition to your paper). 

2. Do not distribute the source data. See #6 above.

Examples of derivative resources include: translations into other languages, added annotations to the text instances, aggregations of multiple datasets, etc.

8. If you are interested in uploading our resource on a third-party website or to include the resource in any collection/aggregate of datasets, then:

1. Email the designated contact person to begin the process to obtain permission.

2. After obtaining permission, any curator of datasets that includes a resource listed here must take steps to ensure that users of the aggregate dataset still cite the papers associated with the individual datasets. This includes at minimum: stating this clearly in the README and providing the citing information of the source dataset.

By default, no one other than the creators of the resource have permission to upload the resource on a third-party website or to include the resource in any collection/aggregate of datasets.

9. National Research Council Canada (NRC) disclaims any responsibility for the use of the resource(s) listed on this page and does not provide technical support. However, the contact listed above will be happy to respond to queries and clarifications.

If you send us an email, we will be thrilled to know about how you have used the resource.