TU Wien:Grundlagen des Information Retrieval VU (Rauber)/Prüfung 2021-01-14

The online exam on the 14.01.2021 in English

The duration of this exam was 1 hour. It was split into 3 parts, with the first one being limited to 15 minutes. The other two were unlocked 15 minutes after the start of the exam and could have been done in any order.

Part 1 (Multiple Choice)

1. PageRank is:
   1. calculated using Random Walks following web links
   2. the most important component of modern web search engines
   3. calculated using Slater Determinants
   4. based on an algorithm developed for analysing citations of scientific articles
2. Which of these features can be extracted directly from symbolic audio representations such as MIDI:
   1. chords
   2. melodic contour
   3. pitch histograms
   4. timbre
3. Which of the following statements about Document Surrogates are true?
   1. It is considered bad practice to highlight query terms in Document Surrogates
   2. A Document Surrogate has no effect on the perceived relevance of a document
   3. A Document Surrogate contains information on why the document was retrieved
   4. Document Surrogates are no longer used in most search engines
4. In order to perform music genre classification
   1. we cannot use features such as SSDs or MFCCs
   2. we need to make sure we use a lossless audio representation as input for our analysis
   3. we may encounter music genres that are not defined and thus impossible to detect by acoustic information
   4. we need to consider features that capture timbre and rhythmic aspects of music
5. trec_eval is a utility for
   1. calculating the similarity between two files
   2. calculating evaluation metrics for information retrieval
   3. designing search user interfaces
   4. indexing a small collection of documents
6. Which of the following aspects are taken into account by the BM25 model?
   1. Term Synonyms
   2. Inverse Document Frequency
   3. Term Frequency
   4. Document Length
7. Which of these statements about audio features is true:
   1. Spectral flux measures the dynamics / changes in consecutive time frames of the spectrum
   2. The spectral roll-off measures how much energy is present in the lower frequencies
   3. Zero crossing rate is a good separator for speech vs. music
   4. The spectral centroid is computed in the time domain
8. In an index, the dictionary data structure can be used to store:
   1. document frequency
   2. BM25 values
   3. term vocabulary
   4. pointers to postings lists
9. Query Logs can be used for:
   1. query spelling correction
   2. training Learning to Rank algorithms
   3. cooling servers
   4. optimising search engine cache replacement policies
10. Which techniques can be used to pre-process documents before indexing?
    1. lemmatization
    2. trunking
    3. stop word addition
    4. tokenisation
11. Classic Boolean Retrieval supports:
    1. exact match between query and document
    2. operators such as AND, OR and NOT
    3. PageRank
    4. ranking
12. Which of the following are present in an Information Retrieval Test Collection?
    1. Queries
    2. Relevance Judgements
    3. Document Collection
    4. Search Engines

Part 2 (Evaluation Metrics)

A query result like this one was given:

Rank	Relevancy Score
1	2
2	0
3	3
4	1
5	0
6	0

The scores range from 0 for non relevant, to 3 for very relevant. For boolean metrics assume numbers 1-3 denote relevant documents, and 0 non-relevant ones. Also, there are 5 relevant documents for this query in total. Calculate the following metrics:

Precision@1, Precision@2,..., Precision@6, Recall@1, Recall@6, F-measure@6, Average Precision, R precision, Cumulative Gain, Discounted Cumulative Gain, Normalised Discounted Cumulative Gain.

Part 3 (Search Engine Design Guidelines)

There was a search engine given which the student should take a look a, and answer the following questions:

1. Name and explain two design guidelines which the website fulfills.
2. Name and explain two design guidelines which the website doesn't fulfill.
3. How could the two broken guidelines in question two be fixed?