Description
Question 1: Multi-document Summarization (60 points) This question asks you to implement a simple, but surprisingly effective algorithm for multi-document summarization, SumBasic: Ani Nenkova and Lucy Vanderwende. The Impact of Frequency on Summarization. Microsoft Research, Redmond, Washington, Tech. Rep. MSR-TR-2005-101. 2005. http://www.cs.bgu.ac.il/~elhadad/ nlp09/sumbasic.pdf a) Use a news aggregator tool such as Google News to find four clusters of articles on the same event or topic. Each cluster should contain at least three articles, and each article should be of sufficient length to generate an interesting summary from (at least 3–4 paragraphs). You should clean the article text by removing all hyperlinks, formatting, titles and other items that are not the textual body of the articles. Use any method to do this (including by hand). You may have to deal with non-ASCII characters. You can handle them any way you like, including just replacing them by a similar-looking ASCII character. Save your input into text files called docA-B.txt, where A is an positive integer corresponding to the cluster number, and B is another positive integer corresponding to the article number within that cluster. For example doc1-2.txt is the second article in the first cluster. Put all of your documents inside a subfolder called /docs. b) Implement SumBasic, as it is described in the lecture notes, in order to generate 100-word summaries for each of your document clusters. Compare these two versions of the system: 1. orig: The original version, including the non-redundancy update of the word scores. 2. simplified: A simplified version of the system that holds the word scores constant and does not incorporate the non-redundancy update. Compare these versions against a third method, leading, which takes the leading sentences of one of the articles, up until the word length limit is reached. You may decide on how to select the article arbitrarily. You should apply the standard preprocessing steps on your input documents, including sentence segmentation, lemmatization, ignoring stopwords and case distinctions. The main method that should run 1 your code should be in a file called sumbasic.py. Your code should be run using the following command structure: python sumbasic.py * And it should print the output summary to standard output. For example, running python ./sumbasic.py simplified ./docs/doc1-*.txt > simplified-1.txt should run the simplified version of the summarizer on the first cluster, writing the output to a text file called simplified-1.txt. c) Discuss quality of each of the three methods. Does the non-redundancy update work as expected? How are the methods successful or not successful? How would you order the summary sentences with the SumBasic methods, or another extractive summarization approach? Be sure to cover all aspects of summary quality that we discussed in class. Question 2: Reading Assignment — Abstractive Summarization (40 points) Read the following paper: Trevor Cohn and Mirella Lapata. Sentence Compression Beyond Word Deletion. COLING 2008. http: //www.aclweb.org/anthology/C08-1018.pdf Write a max. one-page (c. 500 words) discussion on this paper, including the following points: 1. A brief summary of the contents of the paper, including the theoretical framework and the experiments. 2. The limitations of the approach. What do you suggest to address these limitations? 3. Relate this paper to the following concepts that we have discussed throughout the term: context-free grammar, natural language generation and extractive summarization. 4. Three questions related to the paper. These can be clarification questions, or questions about potential extensions of the paper, or its relationship to other work. What To Submit Before the deadline: You are encouraged to do a preliminary pass of the reading before November 30th, when we will discuss the paper in class. Electronically: Submit the written portions of the assignment in a single pdf file called ‘a4-written.pdf’. For the programming part of Question 1, you should submit one zip file called ‘a4-q1.zip’ with your source code, input document clusters, and output summaries. Both should be submitted to MyCourses under Assignment 4. Page 2