Multilingual Metaphor Processing: Experiments with Semi-Supervised and Unsupervised Learning

Metaphor brings vividness, distinction, and clarity to our thought and communication. At the same time, it plays an important structural role in our cognition, helping us to organize and project knowledge (Lakoff and Johnson 1980; Feldman 2006) and guide our reasoning (Thibodeau and Boroditsky 2011). Metaphors arise from systematic associations between distinct, and seemingly unrelated, concepts. For instance, when we talk about “the turning wheels of a political regime,” “rebuilding the campaign machinery” or “mending foreign policy,” we view politics and political systems in terms of mechanisms—they can function, break, be mended, have wheels, and so forth. The existence of this association allows us to transfer knowledge and inferences from the domain of mechanisms to that of political systems. As a result, we reason about political systems in terms of mechanisms and discuss them using the mechanism terminology in a variety of metaphorical expressions. The view of metaphor as a mapping between two distinct domains was echoed by numerous theories in the field (Black 1962); Hesse 1966; Lakoff and Johnson 1980; Gentner 1983). The most influential of these was the Conceptual Metaphor Theory of Lakoff and Johnson (1980). Lakoff and Johnson claimed that metaphor is not merely a property of language, but rather a cognitive mechanism that structures our conceptual system in a certain way. They coined the term conceptual metaphor to describe the mapping between the target concept (e.g., politics) and the source concept (e.g., mechanism), and linguistic metaphor to describe the resulting metaphorical expressions. Other examples of common metaphorical mappings include: time is money (e.g., “That flat tire cost me an hour”); ideas are physical objects (e.g., “I can not grasp his way of thinking”); violence is fire (e.g., “violence flares amid curfew”); emotions are vehicles (e.g., “[…] she was transported with pleasure”); feelings are liquids (e.g., “[…] all of this stirred an unfathomable excitement in her”); life is a journey (e.g., “He arrived at the end of his life with very little emotional baggage”).

Manifestations of metaphor are pervasive in language and reasoning, making its computational processing an imperative task within Natural Language Processing (NLP). Explaining up to 20% of all word meanings according to corpus studies (Shutova and Teufel 2010; Steen et al. 2010), metaphor is currently a bottleneck, particularly in semantic tasks. An accurate and scalable metaphor processing system would become an important component of many practical NLP applications. These include, for instance, machine translation (MT): A large number of metaphorical expressions are culture-specific and therefore represent a considerable challenge in translation (Schäffner 2004; Zhou, Yang, and Huang 2007). Shutova, Teufel, and Korhonen (2013) conducted a study of metaphor translation in MT. Using Google Translate,¹ a state-of-the-art MT system, they found that as many as 44% of metaphorical expressions in their data set were translated incorrectly, resulting in semantically infelicitous sentences. A metaphor processing component could help to avoid such errors. Other applications of metaphor processing include, for instance, opinion mining: metaphorical expressions tend to contain a strong emotional component (e.g., compare the metaphor “Government loosened its stranglehold on business” and its literal counterpart “Government deregulated business” [Narayanan 1999]); or information retrieval: non-literal language without appropriate disambiguation may lead to false positives in information retrieval (e.g., documents describing “old school gentlemen” should not be returned for the query “school” [Korkontzelos et al. 2013]); and many others.

Because the metaphors we use are also known to be indicative of our underlying viewpoints, metaphor processing is likely to be fruitful in determining political affiliation from text or pinning down cross-cultural and cross-population differences, and thus become a useful tool in data mining. In social science, metaphor is extensively studied as a way to frame cultural and moral models, and to predict social choice (Landau, Sullivan, and Greenberg 2009; Thibodeau and Boroditsky 2011; Lakoff and Wehling 2012). Metaphor is also widely viewed as a creative tool. Its knowledge projection mechanisms help us to grasp new concepts and generate innovative ideas. This opens many avenues for the creation of computational tools that foster creativity (Veale 2011, 2014) and support assessment in education (Burstein et al. 2013).

For many years, computational work on metaphor evolved around the use of hand-coded knowledge and rules to model metaphorical associations, making the systems hard to scale. Recent years have seen a growing interest in statistical modeling of metaphor (Mason 2004; Gedigian et al. 2006; Shutova 2010; Shutova, Sun, and Korhonen 2010; Turney et al. 2011; Heintz et al. 2013; Hovy et al. 2013;Li, Zhu, and Wang 2013; Mohler et al. 2013; Shutova and Sun 2013; Strzalkowski et al. 2013; Tsvetkov, Mukomel, and Gershman 2013; Beigman Klebanov et al. 2014; Mohler et al. 2014), with many new techniques opening routes for improving system accuracy and robustness. A wide range of methods have been proposed and investigated by the community, including supervised classification (Gedigian et al. 2006; Dunn 2013a; Hovy et al. 2013; Mohler et al. 2013; Tsvetkov, Mukomel, and Gersman 2013), unsupervised learning (Heintz et al. 2013; Shutova and Sun 2013), distributional approaches (Shutova 2010; Shutova, Van de Cruys, and Korhonen 2012;Shutova 2013; Mohler et al. 2014), lexical resource-based methods (Krishnakumaran and Zhu 2007; Wilks et al. 2013), psycholinguistic features (Turney et al. 2011; Gandy et al. 2013; Neuman et al. 2013; Strzalkowski et al. 2013), and Web search using lexico-syntactic patterns (Veale and Hao 2008; Bollegala and Shutova 2013; Li, Zhu, and Wang 2013). However, even the statistical methods have been predominantly applied in limited-domain, small-scale experiments. This is mainly due to the lack of general-domain corpora annotated for metaphor that are sufficiently large for training wide-coverage supervised systems. In addition, supervised methods tend to rely on lexical resources and ontologies for feature extraction, which limits the robustness of the features themselves and makes the methods dependent on the coverage (and the availability) of these resources. This also makes these methods difficult to port to new languages, for which such lexical resources or corpora may not exist. In contrast, we experiment with minimally supervised and unsupervised learning methods that require little or no annotation; and use robust, dynamically mined lexico-syntactic features that are well suited for metaphor processing. This makes our methods scalable to new data and portable across languages, domains, and tasks, bringing metaphor processing technology a step closer to a possibility of integration with real-world NLP.

Our methods use distributional clustering techniques to investigate how metaphorical cross-domain mappings partition the semantic space in three different languages—English, Russian, and Spanish. In a distributional semantic space, each word is represented as a vector of contexts in which it occurs in a text corpus.² Because of the high frequency and systematicity with which metaphor is used in language, it is naturally and systematically reflected in the distributional space. As a result of metaphorical cross-domain mappings, the words' context vectors tend to be non-homogeneous in structure and to contain vocabulary from different domains. For instance, the context vector for the noun idea would contain a set of literally used terms (e.g., understand [an idea]) and a set of metaphorically used terms, describing ideas as physical objects (e.g., grasp [an idea], throw [an idea]), liquids (e.g., [ideas] flow), or food (e.g., digest [an idea]), and so on. Similarly, the context vector for politics would contain mechanism terms (e.g., operate or refuel [politics]), game terms (e.g., play or dominate [politics]), space terms (e.g., enter or leave [politics]), as well as the literally used terms (e.g., explain or understand [politics]), as shown in Figure 1. This demonstrates how metaphorical usages, abundant in the data, structure the distributional space. As a result, the context vectors of different concepts contain a certain degree of cross-domain overlap, thus implicitly encoding cross-domain mappings. Figure 1 shows such a term overlap in the direct object vectors for the concepts of game and politics. We exploit such composition of the context vectors to induce information about metaphorical mappings directly from the words' distributional behavior in an unsupervised or a minimally supervised way. We then use this information to identify metaphorical language. Clustering methods model modularity in the structure of the semantic space, and thus naturally provide a suitable framework to capture metaphorical information. To our knowledge, the metaphorical cross-domain structure of the distributional space has not yet been explicitly exploited in wider NLP. Instead, most NLP approaches tend to treat all types of distributional features as identical, thus possibly losing important conceptual information that is naturally encoded in the distributional semantic space.

The focus of our experiments is on the identification of metaphorical expressions in verb–subject and verb–object constructions, where the verb is used metaphorically. In the first set of experiments, we apply a flat clustering algorithm, spectral clustering (Ng et al. 2002), to learn metaphorical associations from text. The system clusters verbs and nouns to create representations of source and target domains. The verb clustering is used to harvest source domain vocabulary and the noun clustering is used to identify groups of target concepts associated with the same source. For instance, the nouns democracy and marriage are clustered together (in the target noun cluster), because both are metaphorically associated with (for example) mechanisms or games and, as such, appear with mechanism and game terms in the corpus (the source verb cluster). The obtained clusters represent source and target concepts between which metaphorical associations hold. We first experiment with the unconstrained version of spectral clustering using the method of Shutova, Sun, and Korhonen (2010), where metaphorical patterns are derived from the distributional information alone and the clustering process is fully unsupervised. We then extend this method to perform constrained clustering, where a small number of example metaphorical mappings are used to guide the learning process, with the expectation of changing the cluster structure towards capturing metaphorically associated concepts. We then analyze and compare the structure of the clusters obtained with or without the use of constraints. The learning of metaphorical associations is then boosted from a small set of example metaphorical expressions that are used to connect the verb and noun clusters. Finally, the acquired set of associations is used to identify new, unseen metaphorical expressions in a large corpus.

Although we believe that these methods would capture a substantial amount of information about metaphorical associations from distributional properties of concepts, they are still dependent on the seed expressions to identify new metaphorical language. In our second set of experiments, we investigate to what extent it is possible to acquire information about metaphor from distributional properties of concepts alone, without any need for labeled examples. For this purpose, we apply the hierarchical clustering method of Shutova and Sun (2013) to identify both metaphorical associations and metaphorical expressions in a fully unsupervised way. We use hierarchical graph factorization clustering (Yu, Yu, and Tresp 2006) of nouns to create a network (or a graph) of concepts and to quantify the strength of association between concepts in this graph. The metaphorical mappings are then identified based on the association patterns between concepts in the graph. The mappings are represented as cross-level, one-directional connections between clusters in the graph. The system then uses salient features of the metaphorically connected clusters to identify metaphorical expressions in text. Given a source domain, the method outputs a set of target concepts associated with this source, as well as the corresponding metaphorical expressions.

We then compare the ability of these methods (that require different kinds and levels of supervision) to identify metaphor. In order to investigate the scalability and adaptability of the methods, we applied them to unrestricted, general-domain text in three typologically different languages—English, Spanish, and Russian. We evaluated the performance of the systems with the aid of human judges in precision- and recall-oriented settings, achieving state-of-the-art results with little supervision. Finally, we analyze the differences in the use of metaphor across languages, as discovered by the systems, and demonstrate that statistical methods can facilitate and scale up cross-linguistic research on metaphor.

Metaphor annotation studies have typically been corpus-based and involved either continuous annotation of metaphorical language (i.e., distinguishing between literal and metaphorical uses of words in a given text), or search for instances of a specific metaphor in a corpus and an analysis thereof. The majority of corpus-linguistic studies were concerned with metaphorical expressions and mappings within a limited domain, for example, war, business, food, or plant metaphors (Santa Ana 1999; Izwaini 2003; Koller 2004; Skorczynska Sznajder and Pique-Angordans 2004; Hardie et al. 2007; Lu and Ahrens 2008; Low et al. 2010), or in a particular genre or type of discourse, such as financial (Charteris-Black and Ennis 2001; Martin 2006), political (Lu and Ahrens 2008), or educational (Cameron 2003; Beigman Klebanov and Flor 2013) discourse.

Two studies (Steen et al. 2010; Shutova and Teufel 2010) moved away from investigating particular domains to a more general study of how metaphor behaves in unrestricted continuous text. Steen and colleagues Pragglejaz Group 2007; Steen et al. 2010) proposed a metaphor identification procedure (MIP), in which every word is tagged as literal or metaphorical, based on whether it has a “more basic meaning” in other contexts than the current one. The basic meaning was defined as “more concrete; related to bodily action; more precise (as opposed to vague); historically older” and its identification was guided by dictionary definitions. The resulting VU Amsterdam Metaphor Corpus³ is a 200,000-word subset of the British National Corpus (BNC) (Burnard 2007) annotated for linguistic metaphor. The corpus has already found application in computational metaphor processing research (Dunn 2013b; Niculae and Yaneva 2013; Beigman Klebanov et al. 2014), as well as inspiring metaphor annotation efforts in other languages (Badryzlova et al. 2013). Shutova and Teufel (2010) extended MIP to the identification of conceptual metaphors along with the linguistic ones. Following MIP, the annotators were asked to identify the more basic sense of the word, and then label the context in which the word occurs in the basic sense as the source domain, and the current context as the target. Shutova and Teufel's corpus is a 13,000-word subset of the BNC sampling a range of genres, and it has served as a testbed in a number of computational experiments (Shutova 2010; Shutova, Sun, and Korhonen 2010; Bollegala and Shutova 2013).

Lönneker (2004) investigated metaphor annotation in lexical resources. The resulting Hamburg Metaphor Database contains examples of metaphorical expressions in German and French, which are mapped to senses from EuroWordNet⁴ and annotated with source-target domain mappings.

Early computational work on metaphor tended to be theory-driven and utilized hand-coded descriptions of concepts and domains to identify and interpret metaphor. The system of Fass (1991), for instance, was an implementation of the selectional preference violation view of metaphor (Wilks 1978) and detected metaphor and metonymy as a violation of a common preference of a predicate by a given argument. Another branch of approaches (Martin 1990; Narayanan 1997; Barnden and Lee 2002) implemented some aspects of the conceptual metaphor theory (Lakoff and Johnson 1980), reasoning over hand-crafted representations of source and target domains. The system of Martin (1990) explained linguistic metaphors through finding the corresponding metaphorical mapping. The systems of Narayanan (1997) and Barnden and Lee (2002) performed inferences about entities and events in the source and target domains in order to interpret a given metaphor. The reasoning processes relied on manually coded knowledge about the world and operated mainly in the source domain. The results were then projected onto the target domain using the conceptual mapping representation.

The reliance on task- and domain-specific hand-coded knowledge makes these systems difficult to scale to real-world text. Later research thus turned to general-domain lexical resources and ontologies, as well as statistical methods, in order to design more scalable solutions. Mason (2004) introduced the use of statistical techniques for metaphor processing; however, his approach had a considerable reliance on WordNet (Fellbaum 1998). His CorMet system discovered source–target domain mappings automatically, by searching for systematic variations in domain-specific verb preferences. For example, pour is a characteristic verb in both lab and finance domains. In the lab domain it has a strong preference for liquids and in the finance domain for money. From this information, Mason's system inferred the domain mapping finance–lab and the concept mapping money–liquid. The system of Krishnakumaran and Zhu (2007) used hyponymy relations in WordNet and word bigram counts to predict verbal, nominal, and adjectival metaphors. For instance, given an is-a construction (e.g., “The world is a stage”) the system verified that the two nouns were in hyponymy relation in WordNet, and if this was not the case the expression was tagged as metaphorical. Given a verb–noun or an adjective–noun pair (such as “planting ideas” or “fertile imagination”), the system computed the bigram probability of this pair (including the hyponyms/hypernyms of the noun) and if the combination was not observed in the data with sufficient frequency, it was tagged as metaphorical.

These systems have demonstrated that statistical methods, when combined with broad-coverage lexical resources, can be successfully used to model at least some aspects of metaphor, increasing the system coverage. As statistical NLP, lexical semantics, and lexical acquisition techniques developed over the years, it has become possible to build larger-scale statistical metaphor processing systems that promise a step forward both in accuracy and robustness. Numerous approaches (Li and Sporleder 2010; Shutova 2010; Turney et al. 2011; Hovy et al. 2013; Shutova and Sun 2013; Shutova, Teufel, and Korhonen 2013; Tsvetkov, Mukomel, and Gershman 2013) used machine learning and statistical techniques to address a wider range of metaphorical language in general-domain text. For instance, the method of Turney et al. (2011) classified verbs and adjectives as literal or metaphorical based on their level of concreteness or abstractness in relation to the noun they appear with. They learned concreteness rankings for words automatically (starting from a set of examples) and then searched for expressions where a concrete adjective or verb was used with an abstract noun (e.g., “dark humor” was tagged as a metaphor and dark hair was not). The method of Turney et al. (2011) has served as a foundation for the later approaches of Neuman et al. (2013) and Gandy et al. (2013), who extended it through the use of selectional preferences and the identification of source domains, respectively.

Another branch of research focused on applying statistical learning to the problem of metaphor identification (Gedigian et al. 2006; Shutova, Sun, and Korhonen 2010; Dunn 2013a; Heintz et al. 2013; Hovy et al. 2013; Mohler et al. 2013; Shutova and Sun 2013; Tsvetkov, Mukomel, and Gershman 2013; Beigman Klebanov et al. 2014). The learning techniques they have investigated include supervised classification, clustering, and Latent Dirichlet Allocation (LDA) topic modeling. We review these methods in more detail subsequently.

A number of approaches trained classifiers on manually annotated data to recognize metaphor (Gedigian et al. 2006; Dunn 2013a; Hovy et al. 2013; Mohler et al. 2013; Tsvetkov, Mukomel, and Gershman 2013; Beigman Klebanov et al. 2014). The method of Gedigian et al. (2006), for instance, discriminated between literal and metaphorical uses of the verbs of motion and cure using a maximum entropy classifier. The authors obtained their data by extracting the lexical items whose frames are related to motion and cure from FrameNet (Fillmore, Johnson, and Petruck 2003). To construct their training and test sets, they searched the PropBank Wall Street Journal corpus (Kingsbury and Palmer 2002) for sentences containing such lexical items and manually annotated them for metaphoricity. They used PropBank annotation (arguments and their semantic types) as features to train the classifier and reported an accuracy of 95.12%. This result was, however, only a little higher than the performance of the naive baseline assigning majority class to all instances (92.90%).

Dunn (2013a, 2013b) presented an ontology-based domain interaction approach that identified metaphorical expressions at the utterance level. Dunn's system first mapped the lexical items in the given utterance to concepts from SUMO ontology (Niles and Pease 2001, 2003), assuming that each lexical item was used in its default sense—that is, no sense disambiguation was performed. The system then extracted the properties of concepts from the ontology, such as their domain type (abstract, physical, social, mental) and event status (process, state, object). Those properties were then combined into feature-vector representations of the utterances. Dunn trained a logistic regression classifier using these features to perform metaphor identification, reporting an F-score of 0.58 on general-domain data.

Tsvetkov, Mukomel, and Gershman (2013) experimented with metaphor identification in English and Russian, first training a classifier on English data only, and then projecting the trained model to Russian using a dictionary. They abstracted from the words in English data to their higher-level features, such as concreteness, animateness, named-entity labels, and coarse-grained WordNet categories (corresponding to WN lexicographer files,⁵ [e.g., noun.artifact, noun.body, verb.motion, verb.cognition]). The authors used a logistic regression classifier and a combination of the listed features to annotate metaphor at the sentence level. The model was trained on the TroFi data set (Birke and Sarkar 2006) of 1,298 sentences containing literal and metaphorical uses of 25 verbs. Tsvetkov and colleagues evaluated their method on self-constructed data sets of 98 sentences for English and 140 sentences for Russian, attaining F-scores of 0.78 and 0.76, respectively. The results are encouraging and show that porting coarse-grained semantic knowledge across languages is feasible. However, it should be noted that the generalization to coarse semantic features is likely to focus on shallow properties of metaphorical language and to bypass conceptual information. Corpus-linguistic research (Charteris-Black and Ennis 2001; Kovecses 2005; Diaz-Vera and Caballero 2013) suggests that there is considerable variation in metaphorical language across cultures, which makes training only on one language and translating the model problematic for modeling conceptual structure behind metaphor.

The approach of Mohler et al. (2013) relied on the concept of semantic signature of a text, defined as a set of highly related and interlinked WordNet senses. They induced domain-sensitive semantic signatures of texts and then trained a set of classifiers to detect metaphoricity within a text by comparing its semantic signature to a set of known metaphors. The intuition behind this approach was that the texts whose semantic signature closely matched the signature of a known metaphor would be likely to contain an instance of the same conceptual metaphor. Mohler and colleagues conducted their experiments within a limited domain (the target domain of governance) and manually constructed an index of known metaphors for this domain. They then automatically created the target domain signature and a signature for each source domain among the known metaphors in the index. This was done by means of semantic expansion of domain terms using WordNet, Wikipedia links, and corpus co-occurrence statistics. Given an input text their method first identified all target domain terms using the target domain signature, then disambiguated the remaining terms using sense clustering and classified them according to their proximity to the source domains listed in the index. For the latter purpose, the authors experimented with a set of classifiers, including maximum entropy classifier, unpruned decision tree classifier, support vector machines, random forest classifier, as well as the combination thereof. They evaluated their system on a balanced data set containing 241 metaphorical and 241 literal examples, and obtained the highest F-score of 0.70 using the decision tree classifier.

Hovy et al. (2013) trained a support vector machine classifier (Cortes and Vapnik 1995) with tree kernels (Moschitti, Pighin, and Basili 2006) to capture the compositional properties of metaphorical language. Their hypothesis was that unusual semantic compositions in the data would be indicative of the use of metaphor. The system was trained on labeled examples of literal and metaphorical uses of 329 words (3,872 sentences in total), with an expectation to learn the differences in their compositional behavior in the given lexico-syntactic contexts. The choice of dependency-tree kernels helped to capture such compositional properties, according to the authors. Hovy et al. used word vectors, as well as lexical, part-of-speech tags and WordNet supersense representations of sentence trees as features. They report encouraging results, F-score = 0.75, which is an indication of the importance of syntactic information and compositionality in metaphor identification.

The key question that supervised classification poses is, what features are indicative of metaphor and how can one abstract from individual expressions to its high-level mechanisms? The described approaches experimented with a number of features, including lexical and syntactic information and higher-level features such as semantic roles, WordNet supersenses, and domain types extracted from ontologies. The results that came out of these studies suggest that in order to reliably capture the patterns of the use of metaphor in the data on a large scale, one needs to address conceptual properties of metaphor, along with the surface ones. Thus the model would need to make generalizations at the level of metaphorical mappings and coarse-grained classes of concepts, in essence representing different domains (such as politics or machines). Although our intention in this article is to model such domain structure in a minimally supervised or unsupervised way and to learn it from the data directly, the clusters produced by our models provide a representation of conceptual domains that could also be a useful feature within a supervised classification framework.

We first introduced the use of clustering techniques to learn metaphorical associations in our earlier work (Shutova, Sun, and Korhonen 2010; Shutova and Sun 2013). The metaphor identification system of Shutova, Sun, and Korhonen (2010) starts from a small seed set of metaphorical expressions, learns the analogies involved in their production, and extends the set of analogies by means of spectral clustering of verbs and nouns. Shutova, Sun, and Korhonen (2010) introduced the hypothesis of “clustering by association,” stating that in the course of distributional noun clustering, abstract concepts tend to cluster together if they are associated with the same source domain, whereas concrete concepts cluster by meaning similarity. In the course of distributional clustering, concrete concepts (e.g., water, coffee, beer, liquid) tend to be clustered together when they have similar meanings. In contrast, abstract concepts (e.g., marriage, democracy, cooperation) tend to be clustered together when they are metaphorically associated with the same source domain(s) (e.g., both marriage and democracy can be viewed as mechanisms or games). Because of this shared association structure they share common contexts in the corpus. For instance, Figure 2 shows a more concrete cluster of mechanisms and a more abstract cluster containing both marriage and democracy, along with their associated verb cluster. Such clustering patterns allow the system to discover new, previously unseen conceptual and linguistic metaphors starting from a small set of examples, or seed metaphors. For instance, having seen the seed metaphor “mend marriage” it infers that “the functioning of democracy” is also used metaphorically, since mend and function are both mechanism verbs and marriage and democracy are in the same cluster. This is how the system expands from a small set of seed metaphorical expressions to cover new concepts and new metaphors.

Shutova, Sun, and Korhonen (2010) experimented with unconstrained spectral clustering and applied their system to English data. In this article, we extend their method to perform constrained clustering, and thus investigate the effectiveness of additional supervision in the form of annotated metaphorical mappings. We then also apply the original unconstrained method and its new constrained variant to three languages—English, Spanish, and Russian—thus testing the approach in a multilingual setting.

The second set of experiments in this article are based on the method of Shutova and Sun (2013), which is inspired by the same observation about distributional clustering. Through the use of hierarchical soft clustering techniques, Shutova and Sun (2013) derive a network of concepts in which metaphorical associations are exhibited at different levels of granularity. If, in the method of Shutova, Sun, and Korhonen (2010), the source and target domain clusters were connected through the use of the seed expressions, the method of Shutova and Sun (2013) learns both the clusters and the connections between them automatically from the data, in a fully unsupervised fashion. Because one of the aims of this article is to investigate the level and type of supervision optimally required to generalize metaphorical mechanisms from text, we adapt and apply the method of Shutova and Sun (2013) to our languages of interest and compare its performance to that of the spectral clustering based methods across languages. We thus also test the method, which has been previously evaluated only on English data, in a multilingual setting.

Clustering techniques have also been previously used in metaphor processing research in a more traditional sense (i.e., to identify linguistic expressions with a similar or related meaning). Mason (2004) performed WordNet sense clustering to obtain selectional preference classes, and Mohler et al. (2013) used it to determine similarity between concepts and to link them in semantic signatures. Strzalkowski et al. (2013) and Gandy et al. (2013) clustered metaphorically used terms to form potential source domains. Birke and Sarkar (2006) clustered sentences containing metaphorical and literal uses of verbs. Their core assumption was that all instances of the verb in semantically similar sentences have the same sense, either the literal or the metaphorical one. However, the latter approaches did not investigate how metaphorical associations structure the distributional semantic space, which is what we focus on in this article.

Heintz et al. (2013) applied LDA topic modeling (Blei, Ng, and Jordan 2003) to the problem of metaphor identification in experiments with English and Spanish. Their hypothesis was that if a sentence contained both source and target domain vocabulary, it contained a metaphor. The authors focused on the target domain of governance and manually compiled a set of source concepts with which governance could be associated. They used LDA topics as proxies for source and target concepts: If vocabulary from both source and target topics was present in a sentence, this sentence was tagged as containing a metaphor. The topics were learned from Wikipedia and then aligned to source and target concepts using sets of human-created seed words. When the metaphorical sentences were retrieved, the source topics that are common in the document were excluded. This ensured that the source vocabulary was transferred from a new domain. The authors collected the data for their experiments from news Web sites and governance-related blogs in English and Spanish. They ran their system on these data, and output a ranked set of metaphorical examples. They carried out two types of evaluation: (1) top five linguistic examples for each conceptual metaphor were judged by two annotators, yielding an F-score of 0.59 for English (κ = 0.48); and (2) 250 top-ranked examples in system output were annotated for metaphoricity using Amazon Mechanical Turk, yielding a mean metaphoricity of 0.41 (standard deviation = 0.33) in English and 0.33 (standard deviation = 0.23) in Spanish.

The method of Heintz et al. (2013) relies on the ideas of the Conceptual Metaphor Theory, in that metaphorical language can be generalized using information about source and target domains. Many supervised classification approaches (e.g., Mohler et al. 2013; Tsvetkov, Mukomel, and Gershman 2013), as well as our own approach, share this intuition. However, our methods are different in their aims. If the method of Heintz et al. (2013) learned information about the internal domain structure from the data (through the use of LDA), our methods aim to learn information about cross-domain mappings, as well as the internal domain structure, from the words' distributional behavior.

In addition, in contrast to most of the systems described in this section, we experiment with minimally supervised and unsupervised techniques that require little or no annotated training data, and thus can be easily adapted to new domains and languages. Unlike most previous approaches, we also experiment with metaphor identification in a general-domain setting.

Because our approach involves distributional learning from large collections of text, the choice of an appropriate text corpus plays an important role in the experiments and the interpretation of results. We have selected comparably large, wide-coverage corpora in our three languages to train the systems. The corpora were then parsed using a dependency parser and verb–subject, verb–direct_object, and verb–indirect_object relations were extracted from the parser output. Following previous semantic noun and verb clustering experiments (Pantel and Lin 2002; Bergsma, Lin, and Goebel 2008; Sun and Korhonen 2009), we use these grammatical relations (GRs) as features for clustering. The features used for noun clustering consisted of the verb lemmas occurring in verb–subject, verb–direct_object, and verb–indirect_object relations with the nouns in our data set, indexed by relation type. The features used for verb clustering were the noun lemmas, occurring in the above GRs with the verbs in the data set, also indexed by relation type. The feature values were the relative frequencies of the features. For instance, the feature vector for democracy in English would contain the following entries: {restore-dobjn₁, establish-dobjn₂, build-dobjn₃, … , vote_in-iobjn_i, call_for-iobjn_i+1, … , survive-subjn_k, emerge-subjn_k+1, …}, where n is the frequency of the feature.

The English verb and noun data sets used for clustering contain the 2,000 most frequent verbs and the 2,000 most frequent nouns in the BNC (Burnard 2007), respectively. The BNC is balanced with respect to topic and genre, which makes it appropriate for the selection of a data set of most common source and target concepts and their linguistic realizations. The features for clustering were, however, extracted from the English Gigaword corpus (Graff et al. 2003), which is more suitable for feature extraction because of its large size. The Gigaword corpus was first parsed using the RASP parser (Briscoe, Carroll, and Watson 2006) and the verb–subject, verb–direct_object, and verb–indirect_object relations were then extracted from the GR output of the parser, from which the feature vectors were formed.

The Spanish data were extracted from the Spanish Gigaword corpus (Mendonca et al. 2011). The verb and noun data sets used for clustering consisted of the 2,000 most frequent verbs and 2,000 most frequent nouns in this corpus. The corpus was parsed using the Spanish Malt parser (Nivre et al. 2007; Ballesteros et al. 2010). verb–subject, verb–direct_object, and verb–indirect_object relations were then extracted from the output of the parser and the feature vectors were constructed for all verbs and nouns in the data set in a similar manner to the English system. For example, the feature vector for the noun democracia included the following entries: {destruir-dobj n₁, reinstaurar-dobj n₂, proteger-dobj n₃, … , elegir_a-iobj n_i, comprometer_con-iobj n_i+1, … , florecer-subj n_k, funcionar-subj n_k+1, …}.

The Russian data were extracted from the RU-WaC corpus (Sharoff 2006), a two-billion-word representative collection of text from the Russian Web. The corpus was parsed using the Malt dependency parser for Russian (Sharoff and Nivre 2011), and the verb–subject, verb–direct_object, and verb–indirect_object relations were extracted to create the feature vectors. Similarly to the English and Spanish experiments, the 2,000 most frequent verbs and 2,000 most frequent nouns, according to the RU-WaC, constituted the verb and noun data sets used for clustering.

We first experiment with a flat clustering solution, where metaphorical patterns are learned by means of hard clustering of verbs and nouns at one level of generality.⁶ This approach to metaphor identification is based on the hypothesis of clustering by association, which we first introduced in Shutova, Sun, and Korhonen (2010). Our expectation is that clustering by association would allow us to learn numerous new target domains that are associated with the same source domain from the data in a minimally supervised way. Following Shutova, Sun, and Korhonen 2010), we also use clustering techniques to collect source domain vocabulary.

We perform verb and noun clustering using the spectral clustering algorithm, which has proven to be effective in lexical acquisition tasks (Brew and Schulte im Walde 2002; Sun and Korhonen 2009) and is suitable for high-dimensional data (Chen et al. 2006). We experiment with its unconstrained and constrained versions. The unconstrained algorithm performs clustering (and thus identifies metaphorical patterns) in a fully unsupervised way, relying on the information contained in the data alone. The constrained version uses a small set of example metaphorical mappings as constraints to reinforce clustering by association. We then investigate to what extent adding metaphorical constraints affects the resulting partition of the semantic space as a whole. Further details of these two methods are provided subsequently. Once the clusters have been created in either the unconstrained or constrained setting, the identification of metaphorical expressions is boosted from a small number of linguistic examples—the seed expressions.

The seed expressions in our experiments are verb–subject and verb–direct object metaphors, in which the verb metaphorically describes the noun (e.g., “mend marriage”). Note that these are linguistic metaphors; their corresponding metaphorical mappings are not annotated. The seed expressions are then used to establish a link between the verb cluster that contains source domain vocabulary and the noun cluster that contains diverse target concepts associated with that source domain. This link then allows the system to identify a large number of new metaphorical expressions in a text corpus. In summary, the system (1) performs noun clustering in order to harvest target concepts associated with the same source domain; (2) creates a source domain verb lexicon by means of verb clustering; (3) uses seed expressions to connect source (verb) and target (noun) clusters between which metaphorical associations hold; and (4) searches the corpus for metaphorical expressions describing the target domain concepts using the verbs from the source domain lexicon.

In our case, each vertex v_i represents a word indexed by i ∈ 1,…,N. The weight between vertices v_i and v_j is denoted by w_ij ≥ 0 and represents the similarity or adjacency between v_i and v_j, taken from the adjacency matrix W. If w_ij = 0, we say vertices v_i and v_j are unconnected. Because G is taken to be undirected, W must be symmetric—this explains our use of Jensen-Shannon divergence rather than the more well-known Kullback-Leibler divergence in constructing our similarity matrix W.⁸ We denote the degree of a vertex v_i by $di:=∑j=1Nwij$⁠. The degree represents the weighted connectivity of v_i to the rest of the graph. Finally, we define the graph Laplacian of G as L : = D − W; the role of the graph Laplacian will become apparent subsequently.

Because we have dropped the constraint that h_i,k be nonzero if and only if v_i is in cluster A_k from Equation (4), then we can no longer infer the cluster memberships directly from H or T. Instead, Ng et al. (2002) approximately infer cluster memberships by clustering in the eigenspace defined by T using a clustering algorithm such as k-Means. The algorithm of Ng et al. (2002) is summarized as Algorithm 1.

Constrained clustering methods incorporate prior knowledge about which words belong in the same clusters. In our experiments, we sought methods that were well-behaved when given only positive constraints (i.e., two words belong in the same cluster) rather than both positive and negative constraints (i.e., two words do not belong in the same cluster). Because we have no hard-and-fast constraints that must be satisfied, but rather subjective information that we believe should influence the constraints, it was also important that our methods not strictly enforce constraints, but rather be capable of weighing the constraints against information available in the similarity matrix over the set of words.

In the constrained spectral clustering algorithm introduced by Ji, Xu, and Zhu (2006), constraints are introduced by a simple modification of the objective function of NCut. Suppose we have C pairs of constraints indicating that two words belong to the same cluster, and we have N words overall. For each pair c of words i and j that belong to the same cluster, we create an N-dimensional vector u_c = [u_c1,u_c2,…,u_cN]^T where u_ci = 1, u_cj = −1, and the rest of the elements are equal to zero. We then collect these vectors into the C × N constraint matrix UT = [u₁,u₂,…,u_N].

We first applied the unconstrained version of spectral clustering algorithm to our data. We experimented with different clustering granularities (producing 100, 200, 300, and 400 clusters), examined the obtained clusters, and determined that the number of clusters set to 200 is the optimal setting for both nouns and verbs in our task, across the three languages. This was done by means of qualitative analysis of the clusters as representations of source and target domains—that is, by judging how complete and homogeneous the verb clusters were as lists of potential source domain vocabulary and how many new target domains associated with the same source domain were found correctly in the noun clusters. This analysis was performed on 10 randomly selected clusters taken from different granularity settings and none of the seed expressions were used for it. Examples of clusters generated with this setting are shown in Figures 3 (nouns) and 4 (verbs) for English; Figures 5 (nouns) and 6 (verbs) for Spanish; and Figures 7 (nouns) and 8 (verbs) for Russian. The noun clusters represent target concepts associated with the same source concept.⁹ The verb clusters contain lists of source domain vocabulary.

We then experimented with adding constraints to guide the clustering process. We used two types of constraints: (1) target–source constraints (ts) directly corresponding to metaphorical mappings (e.g., marriage and mechanism); and (2) target–target constraints (tt), where two target concepts were associated with the same source domain (e.g., marriage and democracy). The constraints were generated according to the following procedure:

• 1. 

  ts constraints:

  • • 

    Select 30 target concepts.

  • • 

    For each of the target concepts select a source concept that it is associated with. This results in 30 pairs of ts constraints.

• 2. 

  tt constraints:

  • • 

    For each of the resulting 30 ts pairs of concepts, select another target concept associated with the given source.

  • • 

    Pair the two target concepts into a tt constraint.

• 3. 

  Constraints should satisfy the following criteria:

  • • 

    Constraints represent metaphorical mappings that hold in all three languages, as validated by native speakers.

  • • 

    Each concept should appear in the set of constraints only once.¹⁰

We created 30 ts and 30 tt constraint pairs following this procedure. The source and target concepts in the constraints were selected from the lists of 2,000 nouns that we clustered in the three languages. Constraints were selected and validated by the authors (who are native speakers of the respective languages) without taking the output of the unconstrained clustering step into account (i.e., prior to having seen it). The lists of constraints were first created through individual introspection, and then finalized through discussion. Tables 1, 2, and 3 show some examples of ts and tt constraints for the three languages. One pair of constraints (relationship & trade [tt] and relationship & vehicle [ts]) was excluded from the set, because relationship is usually translated into Spanish and Russian by a plural form (e.g., relaciones). We thus used 29 tt constraints and 29 ts constraints in our experiments.

Our expectation is that the tt constraints are better suited to aid metaphor discovery, as the noun clusters tend to naturally contain distinct target domains associated with the same source. The tt constraints are designed to reinforce this principle. However, introducing the ts type of constraint allows us to investigate to what extent explicitly reinforcing the source domain features in clustering allows us to harvest more target domains associated with the source.

We experimented with different constraint enforcement parameter settings (β = 0.25,1.0,4.0) in order to investigate the effect of the constraints on the overall partition of the semantic space. Our data analysis has shown that interesting effects of metaphorical constraints were most strongly manifested with β = 4.0, and we thus used this setting in our further experiments. Examples of clusters generated with the use of constraints in the three languages are shown in Figures 9, 10, and 11. Our analysis of the clusters has confirmed that the use of tt constraints resulted in clusters containing more diverse target concepts associated with the same source. Compare, for instance, the unconstrained and tt constrained clusters in Figure 9. The unconstrained cluster predominantly contains concepts related to politics, such as profession and ideology, albeit also capturing other target domains, such as religion and education. Adding the constraint marriage & politics, however, further increases the domain diversity of the cluster, adding such target concepts as life, hope, dream, and economy. The Spanish tt constrained clustering in Figure 10 shows the wider effects of constrained clustering throughout the whole noun space. Although none of the constraints is explicitly manifested in this cluster, one can see that this cluster nonetheless contains a more diverse set of target concepts associated with the same source, as compared to the original unconstrained cluster (see Figure 10). The ts constraints, as expected, highlighted the source domain features of the target word, resulting in (for example) assigning politics to the same cluster as game terms, such as round and match in English (given the ts constraint politics & game). These types of constraints are thus less likely to be suitable for metaphor identification, where purely target clusters are desired. These trends were evident across the three languages, as demonstrated by the examples in the respective figures.

Once the clusters have been obtained, we then used a set of seed metaphorical expressions to connect the source and target clusters, thus enabling the system to recognize new metaphorical expressions. The seed expressions in the three languages were extracted from naturally occurring text, manually annotated for linguistic metaphor.

English seed expressions The seed examples used in the English experiments were extracted from the metaphor corpus created by Shutova and Teufel (2010). Their corpus is a subset of the BNC covering a range of genres: fiction; news articles; essays on politics, international relations, and history; and radio broadcast (transcribed speech). As such, the corpus provides a suitable platform for testing the metaphor-processing system on real-world general-domain expressions in contemporary English. We extracted verb–subject and verb–direct object metaphorical expressions from this corpus. All phrases were included unless they fell into one of the following categories:

• • 

  Phrases where the subject or object referent is unknown (e.g., containing pronouns such as “in which they [changes] operated”) or represented by a named entity (e.g., “Then Hillary leapt into the conversation”).

• • 

  Phrases whose metaphorical meaning is realized solely in passive constructions (e.g., “sociologists have been inclined to [..]”).

• • 

  Multi-word metaphors (e.g., “go on pilgrimage with Raleigh or put out to sea with Tennyson”), because these are beyond the scope of our experiments.

The resulting data set consists of 62 phrases that are different single-word metaphors representing verb–subject and verb–direct object relations, where a verb is used metaphorically. The phrases include, for instance, “stir excitement,” “reflect enthusiasm,” “grasp theory,” “cast doubt,” “suppress memory,” “throw remark” (verb–direct object constructions); and “campaign surged,” “factor shaped […],” “tension mounted,” “ideology embraces,” “example illustrates” (subject–verb constructions). The phrases in the seed set were manually annotated for grammatical relations.

Russian and Spanish seed expressions We have collected a set of texts in Russian and Spanish, following the genre distribution of the English corpus of Shutova and Teufel (2010), insofar as possible. Native speakers of Russian and Spanish then annotated linguistic metaphors in these corpora, following the annotation procedures and guidelines of Shutova and Teufel. We then extracted the metaphorical expressions in verb–subject and verb–direct object constructions from these data, according to the same criteria used to create the English seed set. This resulted in 72 seed expressions for Spanish and 85 seed expressions for Russian. The Spanish seed set includes, for instance, the following examples: “vender influencia,” “inundar mercado,” “empapelar ciudad,” “labrarse futuro,” contagiar estado” (verb–direct object constructions); and “violencia salpicó,” “debate tropezó,” “alegría brota,” “historia gira,” “corazón saltó” (subject–verb constructions). The expressions in the seed sets were manually annotated for the corresponding grammatical relations.

Each individual seed expression implies a connection between a source domain (through the source domain verb; e.g., mend) and a target domain (through the target domain noun; e.g., marriage). The seed expressions are thus used to connect source and target clusters between which metaphorical associations hold. The system then proceeds to search the respective corpus for source and target domain terms from the connected clusters within a single grammatical relation. Specifically, the system classifies verb–direct object and verb–subject relations in the corpus as metaphorical if the lexical items in the grammatical relation appear in the linked source (verb) and target (noun) clusters. Consider the following example sentence extracted from the BNC for English.

• (1)

  Few would deny that in the nineteenth century change was greatly accelerated.

The relevant GRs identified by the parser are presented in Figure 12. The relation between the verb accelerate and its semantic object change is expressed in the passive voice and is, therefore, tagged by RASP as an ncsubj GR. Because this GR contains terminology from associated source (motion) and target (change) domains, it is marked as metaphorical and so is the term accelerate, which belongs to the source domain. The search space for metaphor identification was the BNC parsed by RASP for English; the Spanish Gigaword corpus parsed by the Spanish Malt parser for Spanish; and the RuWaC parsed by the Russian Malt parser for Russian. The search was performed similarly in the three languages: The system searched the corpus for the source and target domain vocabulary within a particular grammatical relation (verb–direct object or verb–subject). Some examples of retrieved metaphorical expressions are presented in Figures 13, 14, and 15.

We applied the unconstrained and constrained versions of our system to identify metaphor in continuous text in the three languages. Examples of full sentences containing metaphorical expressions as annotated by the unconstrained systems are shown in Figures 16, 17, and 18. We evaluated the performance of unconstrained and constrained methods in the three languages on a random sample of the extracted metaphors against human judgments.

In order to show that our metaphor identification methods generalize well over the seed set and capture diverse target domains (rather than merely synonymous ones), we compared their output with that of a baseline system built upon WordNet. In the baseline system, WordNet synsets represent source and target domains in place of automatically generated clusters. The system thus expands over the seed set by using synonyms of the metaphorical verb and the target domain noun. It then searches the corpus for phrases composed of lexical items belonging to those synsets. For example, given a seed expression “stir excitement,” the baseline finds phrases such as “arouse fervor, stimulate agitation, stir turmoil,” and so forth. The comparison against the WordNet baseline was carried out for the English systems only, because the English WordNet is considerably more comprehensive than the Spanish or the Russian one.

The quality of metaphor identification for the systems and the baseline was evaluated in terms of precision with the aid of human judges. For this purpose, we randomly sampled sentences containing metaphorical expressions as annotated by the unconstrained and constrained systems and by the baseline (for English) and asked human annotators to decide whether these were metaphorical or not.

Participants Two volunteer annotators per language participated in the experiments.¹¹ They were all native speakers of the respective languages and held at least a Bachelor's degree.

Materials We randomly sampled 100 sentences from the output of the unconstrained, tt constrained, and ts constrained systems for each language and the WordNet baseline system for English. Each sentence contained a metaphorical expression annotated by the respective system. We then also extracted 100 random sentences containing verbs in direct object and subject relations from corpora for each language. These examples were used as distractors in the experiments. The subjects were thus presented with a set of 500 sentences for English (unconstrained, tt and ts constrained, baseline, distractors) and 400 sentences for Russian and Spanish (unconstrained, tt and ts constrained, distractors). The sentences in the sets were randomized. An example of the sentence annotation format is given in Figure 19.

Task and guidelines The participants were asked to mark which of the expressions were metaphorical in their judgment. They were encouraged to rely on their own intuition of what a metaphor is in the annotation process. However, additional guidance in the form of the following definition of metaphor (Pragglejaz Group 2007) was also provided:

1. 

• For each verb establish its meaning in context and try to imagine a more basic meaning of this verb in other contexts. Basic meanings normally are: (1) more concrete; (2) related to bodily action; (3) more precise (as opposed to vague); (4) historically older.

• 2. 

  If you can establish a basic meaning that is distinct from the meaning of the verb in this context, the verb is likely to be used metaphorically.

Interannotator agreement We assessed the reliability of the annotations in terms of kappa (Siegel and Castellan 1988). The interannotator agreement was measured at κ = 0.62 (n = 2,N = 500, k = 2) in the English experiments (substantial agreement); κ = 0.58 (n = 2,N = 400, k = 2) in the Spanish experiments (moderate agreement); and κ = 0.64 (n = 2,N = 400, k = 2) in the Russian experiments (substantial agreement). The data suggest that the main source of disagreement between the annotators was the presence of highly conventional metaphors (e.g., verbs such as impose, convey, decline). According to previous studies (Gibbs 1984; Pragglejaz Group 2007; Shutova and Teufel 2010) such metaphors are deeply ingrained in our everyday use of language and thus are perceived by some annotators as literal expressions.

The system performance was then evaluated against the elicited judgments in terms of precision. The system output was compared with the judgments of each annotator individually and the average precision across annotators for a given language is reported. The results are presented in Table 4. These results demonstrate that the method is portable across languages, with the unconstrained system achieving a high precision of 0.77 in English, 0.74 in Spanish, and 0.67 in Russian. As we expected, tt constraints outperformed the ts constraints in all languages. This is likely to be the result of the explicit emphasis on the source domain features in ts-constrained clustering, which led to a number of literal expressions (containing the source domain noun) being tagged as metaphorical (e.g., approach a barrier). The effect of tt constraints is not as pronounced as we expected in English and Spanish. In Russian, however, tt constraints led to a considerable improvement of 6 percentage points in system performance, yielding the highest precision.

The constrained and unconstrained variants of our method harvested a comparable number of metaphorical expressions. Table 5 shows the number of seeds used in our experiments in each language, the number of unique metaphorical expressions identified by the unconstrained systems for these seeds, and the total number of sentences containing these expressions as retrieved in the respective corpus.¹² These statistics demonstrate that the systems expand considerably over the small seed sets they use as training data and identify a large number of new metaphorical expressions in corpora. It should be noted, however, that the output of the systems exhibits significant overlap in the constrained and unconstrained settings (e.g., 68% overlap in ts-constrained and unconstrained settings, and 73% in tt-constrained and unconstrained settings in English).

We have shown that the method leads to a considerable expansion over the seed set and operates with a high precision—that is, produces high quality annotations—in the three languages. It identifies new metaphorical expressions relying on the patterns of metaphorical use that it learns automatically through clustering. We have conducted a data analysis to compare the unconstrained and constrained variants of our method and to gain insights about the effects of metaphorical constraints. Although at first glance the performance of the systems appeared not to be strongly influenced by the use of tt constraints (except in the case of Russian), the analysis of the identified expressions revealed interesting qualitative differences. According to our qualitative analysis, the tt constrained clusters exhibited a higher diversity with respect to the target domains they contained in all languages, leading to the system capturing a higher number of new metaphorical patterns, as compared to the unconstrained clusters. As a result, it discovered a more diverse set of metaphorical expressions given the same seeds. Such examples include “mend world” (given the seed “mend marriage”); “frame rule” (given the seed “glimpse duty”); or “lodge service,” “fuel life,” “probe world,” “found science,” or “fuel economy” (given the seed “base career”). Overall, our analysis has shown that even a small number of metaphorical constraints (such as 29 in our case) has global effects throughout the cluster space, that is, influences the structure of all clusters. The fact that the tt constrained method yielded a performance similar to the unconstrained method in English and Spanish and a considerably better performance in Russian suggests that such effects are desirable for metaphor processing. Another consideration that has arisen from the analysis of the system output is that the tt clustering setting may benefit from a larger cluster size in order to incorporate both similar and diverse target concepts.

The ts constrained clusters exhibit the same trend with respect to cluster diversity. However, the explicit pairing of source and target concepts (that occasionally leads to them being assigned to the same cluster) produces a number of false positives, decreasing the system precision. For instance, in the case of the constraint difficulty & barrier, these two nouns are clustered together. As a result, given the seed “confront problem,” the system falsely tags expressions such as approach barrier or face barrier as metaphorical.

The comparison of the English system output to that of a WordNet baseline shows that the clusters in all clustering settings capture diverse concepts, rather than merely the synonymous ones, as in the case of WordNet synsets. The clusters thus provide generalizations over the source and target domains, leading to a wider coverage and acquisition of a diverse set of metaphors. The observed discrepancy in precision between the clustering methods and the baseline (i.e., as high as 37%) can be explained by the fact that a large number of metaphorical senses are included in WordNet. This means that in WordNet synsets, source domain verbs appear together with more abstract terms. For instance, the metaphorical sense of shape in the phrase “shape opinion” is part of the synset “(determine, shape, mold, influence, regulate).” This results in the low precision of the baseline system, because it tags literal expressions (e.g., influence opinion) as metaphorical, assuming that all verbs from the synset belong to the source domain.

System errors were of similar nature across the three languages and had the following key sources: (1) metaphor conventionality and (2) general polysemy. Because a number of metaphorical uses of verbs are highly conventional (such as those in “hold views, adopt traditions, tackle a problem”), such verbs tend to be clustered together with the verbs that would be literal in the same context. For instance, the verb tackle is found in a cluster with solve, resolve, handle, confront, face, and so on. This results in the system tagging resolve a problem as metaphorical if it has seen “tackle a problem” as a seed expression. However, the errors of this type do not occur nearly as frequently as in the case of the baseline.

A number of system errors were due to cases of general polysemy and homonymy of both verbs and nouns. For example, the noun passage can mean both “the act of passing from one state or place to the next” and “a section of text; particularly a section of medium length,” as defined in WordNet. Our method performs hard clustering, that is, it does not distinguish between different word senses. Hence the noun passage occurred in only one cluster, containing concepts such as thought, word, sentence, expression, reference, address, description, and so on. This cluster models the latter meaning of passage. Given the seed phrase “she blocked the thought,” the system then tags a number of false positives such as block passage, impede passage, obstruct passage, and speed passage.

Russian exhibited an interesting difference from English and Spanish in the organization of its word space. This is likely to be due to its rich derivational morphology. In other words, in Russian, more lexical items can be used to refer to the same concept than in English or Spanish, highlighting slightly different aspects of meaning. In English and Spanish, the same meaning differences tend to be expressed at the phrase level rather than at word level. For instance, the English verb to pour can be translated into Russian by at least five different verbs: lit, nalit, slit, otlit, vilit, roughly meaning to pour, to pour into, to pour out, to pour only a small amount, to pour all of the liquid out, to pour some of the liquid out, etc.¹³ As a result, some Russian words tend to naturally form highly dense clusters essentially referring to a single concept (as in case of the verbs of pouring), while at the same time sharing similar distributional features with other, related but different concepts (such as sip or spill). This property suggests that it may be necessary to cluster a larger number of Russian nouns or verbs (into the same or lower number of clusters) in order to achieve the cluster coverage and diversity comparable to the English system. With respect to our experiments, this phenomenon has led to the unconstrained clusters containing more near-synonyms (such as the many variations of pouring), and the metaphorical constraints had a stronger effect in diversifying the clusters, thus allowing us to better capture new metaphorical associations.

Although the diversity of the noun clusters is central to the acquisition of metaphorical patterns, it is also worth noting that in many cases the system benefits not only from dissimilar concepts within the noun clusters, but also from dissimilar concepts in the verb clusters. Verb clusters produced automatically relying on contextual features may contain lexical items with distinct, or even opposite meanings (e.g., throw and catch, take off and land). However, they tend to belong to the same semantic domain. It is the diversity of verb meanings within the domain cluster that allows the generalization from a limited number of seed expressions to a broader spectrum of previously unseen metaphors, non-synonymous to those in the seed set.

The fact that our approach is seed-dependent is one of its possible limitations, affecting the coverage of the system. Wide coverage is essential for the practical use of the system. In order to obtain full coverage, a large and representative seed set is necessary. Although it is difficult to capture the whole variety of metaphorical language in a limited set of examples, it is possible to compile a seed set representative of common source-target domain mappings. The learning capabilities of the system can then be used to expand from those to the whole range of conventional metaphorical mappings and expressions. In addition, because the precision of the system was measured on the data set produced by expanding individual seed expressions, we would expect the expansion of new seed expressions to yield a comparable quality of annotations. Incorporating new seed expressions is thus likely to increase the recall of the system without a considerable loss in precision. However, creating seed sets for new languages may not always be practical. We thus further experiment with fully unsupervised metaphor identification techniques.

The focus of our experiments so far has been mainly on metaphorical expressions, and metaphorical associations were modeled implicitly within the system. In addition, both the constrained and the unconstrained methods relied on a small amount of supervision in the form of seed expressions to identify new metaphorical language. In our next set of experiments, we investigate whether it is possible to learn metaphorical connections between the clusters from the data directly (without the use of metaphorical seeds for supervision) and thus to acquire a large set of explicit metaphorical associations and derive the corresponding metaphorical expressions in a fully unsupervised fashion.

This approach is theoretically grounded in cognitive science findings suggesting that abstract and concrete concepts are organized differently in the human brain (Binder et al. 2005; Crutch and Warrington 2005, 2010;Huang, Lee, and Federmeier 2010; Wiemer-Hastings and Xu 2005; Adorni and Proverbio 2012). According to Crutch and Warrington (2005), these differences emerge from their general patterns of relation with other concepts. In this section, we present a method that learns such different patterns of association of abstract and concrete concepts with other concepts automatically. Our system performs soft hierarchical clustering of nouns to create a network (or a graph) of concepts at multiple levels of generality and to determine the strength of association between the concepts in this graph. We expect that, whereas concrete concepts would tend to naturally organize into a tree-like structure (with more specific terms descending from the more general terms), abstract concepts would exhibit a more complex pattern of association. Consider the example in Figure 20. The figure schematically shows a small portion of the graph describing the concepts of mechanism (concrete), political system, and relationship (abstract) at two levels of generality. One can see from this graph that concrete concepts, such as bike or engine, tend to be strongly associated with one concept at the higher level in the hierarchy (mechanism). In contrast, abstract concepts may have multiple higher-level associates: the literal ones and the metaphorical ones. For instance, the abstract concept of democracy is literally associated with the more general concept of political system, as well as metaphorically associated with the concept of mechanism. Such multiple associations are due to the fact that political systems are metaphorically viewed as mechanisms; they can function, break, they can be oiled, and so forth. We often discuss concepts such as democracy or dictatorship using mechanism terminology, and thus a distributional learning approach would learn that they share features with political systems (from their literal uses), as well as with mechanisms (from their metaphorical uses, as shown next to the respective graph edges in the figure). Our system discovers such association patterns within the graph and uses them to identify metaphorical connections between concepts.

The graph of concepts is built using hierarchical graph factorization clustering (hgfc) (Yu, Yu, and Tresp 2006) of nouns, yielding a network of clusters with different levels of generality. The weights on the edges of the graph indicate the level of association between the clusters (concepts). The system then traverses the graph to find metaphorical associations between clusters using the weights on the edges of the graph. It then generates lists of salient features for the metaphorically connected clusters and searches the corpus for metaphorical expressions describing the target domain concepts, using the verbs from the set of salient features.

In contrast to flat clustering, which produces a partition at one level of generality, the goal of hierarchical clustering is to organize the objects into a hierarchy of clusters with different granularities. Traditional hierarchical clustering methods widely used in NLP (such as agglomerative clustering [Schulte im Walde and Brew2001; Stevenson and Joanis2003; Ferrer 2004; Devereux and Costello 2005]) take decisions about cluster membership at the level of individual clusters when these are merged. As Sun and Korhonen (2011) pointed out, such algorithms suffer from two problems—error propagation and local pairwise merging—because the clustering solution is not globally optimized. In addition, they are designed to perform hard clustering of objects at each level, by successively merging the clusters. This makes them unsuitable to model multi-way associations between concepts within the hierarchy, albeit such association patterns exist in language and reasoning (Crutch and Warrington 2005; Hill, Korhonen, and Bentz 2014). As opposed to this, hgfc allows modeling of multiple relations between concepts simultaneously via a soft clustering solution. It successively derives probabilistic bipartite graphs for every level in the hierarchy. The algorithm delays the decisions about cluster membership of individual words until the overall graph structure has been computed, which allows it to globally optimize the assignment of words to clusters. In addition, hgfc can detect the number of levels and the number of clusters at each level of the hierarchical graph automatically. This is essential for our task as these settings are difficult to pre-define for a general-purpose concept graph.

The algorithm starts from a similarity matrix that encodes similarities between the objects. Given a set of nouns, $V={vn}n=1N$⁠, we construct their similarity matrix W, using Jensen-Shannon Divergence as a similarity measure (as in the spectral clustering experiments). The matrix W in turn encodes an undirected similarity graph G, where the nouns are mapped to vertices and their similarities represent the weights w_ij on the edges between vertices i and j. Such a similarity graph is schematically shown in Figure 21(a). The clustering problem can now be formulated as partitioning of the graph G and deriving the cluster structure from it.

The number of levels (L) and the number of clusters (m_ℓ) are detected automatically, using the method of Sun and Korhonen (2011). Clustering starts with an initial setting of number of clusters (m₀) for the first level. In our experiment, we set the value of m₀ to 800. For the subsequent levels, m_ℓ is set to the number of non-empty clusters (bipartite graph nodes) on the parent level – 1. The matrix B is initialized randomly. We found that the actual initialization values have little impact on the final result. The rows in B are normalized after the initialization so the values in each row add up to one.

The whole algorithm can be summarized as follows.

The resulting graph is composed of a set of bipartite graphs defined by B_ℓ, B_ℓ−1, …, B₁. A bipartite graph has a similar structure to the one shown in Figure 20. For a given noun, we can rank the clusters at any level according to the soft assignment probability (Equation (19)). The clusters that have no member noun were hidden from the ranking because they do not explicitly represent any concept. However, these clusters are still part of the organization of the conceptual space within the model and they contribute to the probability for the clusters at upper levels (Equation (19)). We call the view of the hierarchical graph where these empty clusters are hidden an explicit graph.

Once we obtain the explicit graph of concepts, we can now identify metaphorical associations based on the weights connecting the clusters at different levels. Taking a single noun (e.g., fire) as input, the system computes the probability of its cluster membership for each cluster at each level, using the weights on the edges of the graph (Equation (19)). We expect the cluster membership probabilities to indicate the level of association of the input noun with the clusters. The system can then rank the clusters at each level based on these probabilities. We chose level 3 as the optimal level of generality for our experiments, based on our qualitative analysis of the graph.¹⁵ The system selects six top-ranked clusters from this level (we expect an average source concept to have no more than five typical target associates) and excludes the literal cluster containing the input concept (e.g., fire flame blaze). The remaining clusters represent the target concepts associated with the input source concept.

Example output for the input concepts of fire and disease in English is shown in Figure 22. One can see from the figure that each of the noun-to-cluster mappings represents a new conceptual metaphor (e.g., emotion is fire, violence is fire, crime is a disease). These mappings are exemplified in language by a number of metaphorical expressions (e.g., “His anger will burn him,” “violence flared again,” “it's time they found a cure for corruption”). Figures 23 and 24 show metaphorical associations identified by the Spanish and Russian systems for the same source concepts. As we can see from the figures, feelings tend to be associated with fire in all three languages. Unsurprisingly, however, many of the identified metaphors differ across languages. For instance, victory, success, and looks are viewed as fire in Russian, whereas immigrants and prisoners have a stronger association with fire in English and Spanish, according to the systems. All of the languages exhibit crime is a disease metaphor, with Russian and Spanish also generalizing it to violence is a disease. Interestingly, throughout our data set, Spanish data tends to exhibit more negative metaphors about corporations, as it is demonstrated by the disease example in Figure 23. Although we do not claim that this output is exhaustively representative of all conceptual metaphors present in a particular culture, we believe that these examples showcase some interesting differences in the use of metaphor across languages that can be discovered via large-scale statistical processing.

After extracting the source–target domain mappings, we now move on to the identification of the corresponding metaphorical expressions. The system does this by harvesting the salient features that lead to the input noun being strongly associated with the extracted clusters. The salient features are selected by ranking the features according to the joint probability of the feature (f) occurring both with the input source noun (w) and the target cluster (c). Under a simplified independence assumption, $p(w,c|f)=p(w|f)×p(c|f)$⁠. $p(w|f)$ and $p(c|f)$ are calculated as the ratio of the frequency of the feature f to the total frequency of the input noun and the cluster, respectively. The features ranked higher are expected to represent the source domain vocabulary that can be used to metaphorically describe the target concepts. Example features (verbs and their grammatical relations) extracted for the source domain noun fire and the violence cluster in English are shown in Figure 25.

Similarly to the spectral clustering experiments, we then search the parsed corpus for grammatical relations, in which the nouns from the target domain cluster appear with the verbs from the source domain vocabulary (e.g., “war blazed” (subj), “to fuel violence” (dobj) for the mapping violence is fire in English). The system thus annotates metaphorical expressions in text, as well as the corresponding conceptual metaphors, as shown in Figure 26. Metaphorical expressions identified by the Spanish and Russian systems are shown in Figures 27 and 28, respectively.

Because there is no large and comprehensive gold standard of metaphorical mappings available, we evaluated the quality of metaphorical mappings and metaphorical expressions identified by the system against human judgments. We conducted two types of evaluation: (1) precision-oriented, for both metaphorical mappings and metaphorical expressions; and (2) recall-oriented, for metaphorical expressions. In the first setting, the human judges were presented with a random sample of system-produced metaphorical mappings and metaphorical expressions, and asked to mark the ones they considered valid as correct. In the second setting, the human annotators were presented with a set of source domain concepts and asked to write down all target concepts they associated with a given source, thus creating a gold standard.

We compared the system performance with that of two baseline systems: an unsupervised agglomerative clustering baseline (agg) for the three languages and a supervised baseline built upon Wordnet (wn) for English.

agg: We constructed the agglomerative clustering baseline using SciPy implementation (Oliphant 2007) of Ward's linkage method (Ward 1963). The output tree was cut according to the number of levels and the number of clusters of the explicit graph detected by hgfc. The resulting tree was then converted into a graph by adding connections from each cluster to all the clusters one level above. We computed the connection weights as cluster distances measured using Jensen-Shannon Divergence between the cluster centroids. This graph was then used in place of the hgfc graph in the metaphor identification experiments.

wn: In the wn baseline, the WordNet hierarchy was used as the underlying graph of concepts to which the metaphor extraction method was applied. Given a source concept, the system extracted all its sense-1 hypernyms two levels above and subsequently all of their sister terms. The hypernyms themselves were considered to represent the literal sense of the source noun and were therefore removed. The sister terms were kept as potential target domains.

To create our data set, we extracted 10 common source concepts that map to multiple targets from the Master Metaphor List (Lakoff, Espenson, and Schwartz 1991) and linguistic analyses of metaphor (Lakoff and Johnson 1980; Shutova and Teufel 2010). These included fire, child, speed, war, disease, breakdown, construction, vehicle, system, business. We then translated them into Spanish and Russian. Each of the systems and the baselines identified 50 source–target domain mappings for the given source domains. This resulted in a set of 150 conceptual metaphors for English (hgfc,agg,wn), 100 for Spanish (hgfc,agg), and 100 for Russian (hgfc,agg). Each of these conceptual mappings represents a number of submappings since all the target concepts are clusters or synsets. These were then evaluated against human judgments in two different experimental settings.

Setting 1

Task and guidelines The judges were presented with a set of conceptual metaphors identified by the three systems, randomized. They were asked to annotate the mappings they considered valid as correct. In all our experiments, the judges were encouraged to rely on their own intuition of metaphor, but they also reviewed the metaphor annotation guidelines of Shutova and Teufel (2010) at the beginning of the experiment.

Participants Two judges per language, who were native speakers of English, Russian, and Spanish participated in this experiment. All of them held at least a bachelor's degree.

Interannotator agreement The agreement on this task was measured at κ = 0.60 (n = 2,N = 150,k = 2) for English, κ = 0.59 (n = 2,N = 100,k = 2) for Spanish, and κ = 0.55 (n = 2,N = 100,k = 2) for Russian. The main differences in the annotators' judgments stem from the fact that some metaphorical associations are less obvious and common than others, and thus need more context (or imaginative effort) to establish. Such examples where the judges disagreed included metaphorical mappings such as intensity is speed, goal is a child, collection is a system, and illness is a breakdown.

Results The system performance was then evaluated against these judgments in terms of precision (P), i.e., the proportion of the valid metaphorical mappings among those identified. We calculated system precision (in all experiments) as an average over both annotations. The results across the three languages are presented in Table 6.

Setting 2

To measure recall, R, of the systems we asked two annotators per language (native speakers with a background in metaphor, different from Setting 1) to write down up to five target concepts they strongly associated with each of the 10 source concepts. Their annotations were then aggregated into a single metaphor association gold standard, including all of the mappings listed by the annotators. The gold standard consisted of 63 mappings for English, 70 mappings for Spanish, and 68 mappings for Russian. The recall of the systems was measured against this gold standard. The results are shown in Table 6.

For each of the identified conceptual metaphors, the systems extracted a number of metaphorical expressions from the corpus. For the purposes of this evaluation, we selected the top 50 features from the ranked feature list (as described in Section 5.3) and searched the corpus for expressions where the verbs from the feature list co-occurred with the nouns from the target cluster. Figure 29 shows example sentences annotated by hgfc for English. The identification of metaphorical expressions was also evaluated against human judgments.

Materials The judges were presented with a set of randomly sampled sentences containing metaphorical expressions as annotated by the systems and by the baselines (200 each). This resulted in a data set of 600 sentences for English (hgfc, agg, wn), 400 sentences for Spanish (hgfc, agg), and 400 sentences for Russian (hgfc, agg). The order of the presented sentences was randomized.

Task and guidelines The judges were asked to mark the expressions that were metaphorical in their judgment as correct, following the same guidelines as in the spectral clustering evaluation.

Participants Two judges per language, who were native speakers of English, Russian, and Spanish, participated in this experiment. All of them held at least a bachelor's degree.

Interannotator agreement Their agreement on the task was measured at κ = 0.56(n = 2,N = 600,k = 2) for English, κ = 0.52(n = 2,N = 400,k = 2) for Spanish, and κ = 0.55(n = 2,N = 400,k = 2) for Russian.

Results The system performance was measured against these annotations in terms of an average precision across judges. The results are presented in Table 7. hgfc outperforms both agg and wn, yielding a precision of 0.65 in English, 0.54 in Spanish, and 0.59 in Russian.

As expected, hgfc outperforms both agg and wn baselines in all evaluation settings. agg has been previously shown to be less accurate than hgfc in the verb clustering task (Sun and Korhonen 2011). Our analysis of the noun clusters indicated that hgfc tends to produce more pure and complete clusters than agg. Another important reason agg fails is that it by definition organizes all concepts into a tree and optimizes its solution locally, taking into account a small number of clusters at a time. However, being able to discover connections between more distant domains and optimizing globally over all concepts is crucial for metaphor identification. This makes agg less suitable for the task, as demonstrated by our results. However, agg identified a number of interesting mappings missed by hgfc (e.g. career is a child, language is a system, corruption is a vehicle, empire is a construction), as well as a number of mappings in common with hgfc (e.g. debate is a war, destruction is a disease). The fact that both hgfc and agg identified valid metaphorical mappings across languages confirms our hypothesis that clustering techniques are well suited to detect metaphorical patterns in a distributional word space.

The wn system also identified a few interesting metaphorical mappings (e.g., cognition is fire, education is construction), but its output is largely dominated by the concepts similar to the source noun and contains some unrelated concepts. The comparison of hgfc to wn shows that hgfc identifies meaningful properties and relations of abstract concepts that cannot be captured in a tree-like classification (even an accurate, manually created one such as WordNet). The latter is more appropriate for concrete concepts, and a more flexible representation is needed to model abstract concepts. The fact that both baselines identified some valid metaphorical associations, relying on less suitable conceptual graphs, suggests that our way of traversing the graph is a viable approach to identification of metaphorical associations in principle.

hgfc identifies valid metaphorical associations for a range of source concepts. One of them (crime is a disease, or crime is a virus) happened to have been already validated in behavioral experiments with English speakers (Thibodeau and Boroditsky 2011). The most frequent type of error of hgfc across the three languages is the presence of target clusters similar or closely related to the source noun. For instance, the source noun child tends to be linked to other “human” clusters across languages—for example, the parent cluster for English; the student, resident, and worker clusters in Spanish, and the crowd, journalist, and emperor clusters in Russian. The clusters from the same domain can, however, be filtered out if their nouns frequently occur in the same documents with the source noun (in a large corpus), that is, by topical similarity. The latter is less likely to be the case for the metaphorically associated nouns. However, we leave such an experiment to future work.

The system errors in the identification of metaphorical expressions stem from multiple word senses of the salient features or the source and target sharing some physical properties (e.g., one can “die from crime” and “die from a disease,” an error that manifested itself in all three languages). Some identified expressions invoke a chain of mappings (e.g., abuse is a disease, disease is an enemy for “combat abuse”); however, such chains are not yet incorporated into the system. In some cases, the same salient feature could be used metaphorically both in the source and target domain (e.g., “to open fire” vs. “to open one's heart” in Russian). In this example the expression is correctly tagged as metaphorical, although representing a different conceptual metaphor than feeling is fire. The performance of agg in the identification of metaphorical expressions is higher than in the identification of metaphorical associations, because it outputs only few expressions for the incorrect associations. In contrast, wn tagged a large number of literal expressions due to the incorrect prior identification of the underlying associations.

The performance of the Russian and Spanish systems is slightly lower than that of the English system. This is likely to be due to errors from the data preprocessing step (i.e., parsing). The quality of parser output in English is likely to be higher than in Russian or Spanish, for which fewer parsers exist. Another important difference lies in the corpora used. Whereas the English and Spanish systems have been trained on English and Spanish Gigaword corpora (containing data extracted from news sources), the Russian system has been trained on RuWaC, which is a Web corpus containing a greater amount of noisy text (including misspellings, slang, etc.) The difference in corpora is also likely to have an impact on the mappings identified—that is, different target domains and different metaphorical mappings may be prevalent in different types of data. However, because our goal is to test the capability of clustering techniques to identify metaphorical associations and expressions in principle, the specific types of metaphors identified from different corpora (e.g., the domains covered) are less relevant.

Importantly, our results show that the method is portable across languages. This is an encouraging result, particularly because hgfc is unsupervised, making metaphor processing technology available to a large number of languages for which metaphor-annotated data sets and lexical resources do not exist.

By automatically discovering metaphors in a data-driven way, our methods allow us to investigate and compare the semantic spaces of different languages and gain insights for cross-linguistic research on metaphor. The contrastive study of differences in metaphor is important for several reasons. Understanding how metaphor varies across languages could provide clues about the roles of metaphor and cognition in structuring each other (Kövecses 2004). Contrastive differences in metaphor also have implications for second-language learning (Barcelona 2001), and thus a systematic understanding of variation of metaphor across languages would benefit educational applications. From an engineering perspective, metaphor poses a challenge for machine translation systems (Zhou, Yang, and Huang 2007; Shutova, Teufel, and Korhonen 2013), and can even be difficult for human translators (Schäffner 2004).

Although some aspects of the way that metaphor structures language may be widely shared and near-universal (Kövecses 2004), there are significant differences in how conventionalized and pervasive different metaphors are in different languages and cultures. The earliest analyses of cross-lingual metaphorical differences were essentially qualitative.¹⁶ In these studies, the authors typically produce examples of metaphors that they argue are routine and widely used in one language, but unconventionalized or unattested in another language. Languages that have been studied in such a way include Spanish (Barcelona 2001), Chinese (Yu 1998), Japanese (Matsuki 1995), and Zulu (Taylor and Mbense 1998). One drawback of these studies is that they rely on the judgment of the authors, who may not be representative of the speakers of the language at large. They also do not allow for subtler differences in metaphor use across languages to be exposed. One possibility for addressing these shortcomings involves manually searching corpora in two languages and counting all instances of a metaphorical mapping. This is the approach taken by Charteris-Black and Ennis (2001) with respect to financial metaphors in English and Spanish. They find several metaphors that are much more common in one language than in the other. However, the process of manually identifying instances is time-consuming and expensive, limiting the size of corpora and the scope of metaphors that can be analyzed in a given time frame. As a result, it can be difficult to draw broad conclusions from these studies.

Our systems present a step towards a large-scale data-driven analysis of linguistic variation in the use of metaphor. In order to investigate whether statistically learned patterns of metaphor can capture such variation, we conducted an analysis of the metaphors identified by our systems in the three languages. We ran the hgfc systems with a larger set of source domains taken from the literature on metaphor and conducted a qualitative analysis of the resulting metaphorical mappings to identify the similarities and the differences across languages. As one might expect, the majority of metaphorical mappings identified by the systems are present across languages. For instance, violence and feelings are associated with fire in all three languages, debate or argument are associated with war, crime is universally associated with disease, money with liquid, and so on. However, although the instances of a conceptual metaphor may be present in all three languages, interestingly, it is often the case that the same conceptual metaphor is lexicalized differently in different languages. For instance, although feelings are generally associated with liquids in both English and Russian, the expression “stir excitement” is English-specific and cannot be used in Russian. At the same time, the expression “mixed feelings” (another instantiation of the same conceptual metaphor) is common in both languages. Our systems allow us to trace such variation through the different metaphorical expressions that they identify for the same or similar conceptual metaphors.

Importantly, besides the linguistic variation our methods are also able to capture and generalize conceptual differences in metaphorical use in the three languages. For instance, they exposed some interesting cross-linguistic differences pertaining to the target domains of business and finance. The Spanish conceptual metaphor output manifested rather negative metaphors about business, market, and commerce: business was typically associated with bomb, fire, war, disease, and enemy. Although it is the case that business is typically discussed in terms of a war or a race in English and Russian, the other four Spanish metaphors are uncommon. Russian, in fact, has rather positive metaphors for the related concepts of money and wealth, which are strongly associated with sun, light, star, and food, possibly indicating that money is viewed primarily as a way to improve one's own life. An example of the linguistic instantiations of the Russian money is light metaphor and their corresponding word-for-word English translations is shown in Figure 30. We have validated that the word-for-word English translations of the Russian expressions in the Figure are not typically used in English by searching the BNC, where none of the expressions were found. In contrast, in English, money is frequently discussed as a weapon, that is, a means to achieve a goal or win a struggle (which is directly related to business is a war metaphor). At the same time, the English data exhibit positive metaphors for power and influence, which are viewed as light, sun, or wing. In Russian, on the contrary, power is associated with bomb and bullet, perhaps linking it to the concepts of physical strength and domination. The concepts of freedom and independence were also associated with a wing, weapon, and strength in the Russian data, however. English and Spanish data also exhibited interesting differences with respect to the topic of immigration. According to the system output, in English immigrants tend to be viewed as fire or enemies, possibly indicating danger. In Spanish, on the other hand, immigrants and, more specifically, undocumented people have a stronger association with animals, which is likely a reference to them as victims, being treated like animals.

Although these differences may be a direct result of the contemporary socio-economic context and political rhetoric, and are likely to change over time, other conceptual differences have a deeper grounding in our culture and way of life. For instance, the concept of birth tends to be strongly associated with light in Spanish and battle in Russian, each metaphor highlighting a different aspect of birth. The differences that stem from highly conventional metaphors seem to be even more deeply entrenched in the conceptual system of the speakers of a language. For instance, our analysis of system-produced data revealed systematic differences in discussing quantity and intensity in the three languages. Let us consider, for instance, the concept of heat. In English, heat intensity is typically measured on a vertical scale; for example, it is common to say “low heat” and “high heat.” In Russian, heat intensity is rather thought of in terms of strength; for example, one would say “strong heat” or “weak fire.” As opposed to this, Spanish speakers talk about heat in terms of its speed; for example, “fuego lento” (literally “slow fire”) refers to “low heat” (on the stove). This metaphor also appears to generalize to other phenomena whose level or quantity can be assessed (e.g., intelligence is also discussed in terms of speed in Spanish, height in English, and strength in Russian). Such a systematic variation provides new insights for the study of cognition of quantity, intensity, and scale. Statistical methods provide a tool to expose such variation through automatic analysis of large quantities of linguistic data.

More generally, such systematic cross-linguistic differences in the use of metaphor have significance beyond language and can be associated with contrastive behavioral patterns across the different linguistic communities (Casasanto and Boroditsky 2008; Fuhrman et al. 2011). Psychologists Thibodeau and Boroditsky (2011) investigated how the metaphors we use affect our decision-making. They presented two groups of human subjects with two different texts about crime. In the first text, crime was metaphorically portrayed as a virus and in the second as a beast. The two groups were then asked a set of questions on how to tackle crime in the city. As a result, the first group tended to opt for preventive measures in tackling crime (e.g., stronger social policies), whereas the second group converged on punishment- or restraint-oriented measures. According to the researchers, their results demonstrate that metaphors have profound influence on how we conceptualize and act with respect to societal issues. Although Thibodeau and Boroditsky's study did not investigate cross-linguistic contrasts in the use of metaphor, it still suggests that metaphor-induced differences in decision-making may manifest themselves across communities. Applying data-driven methods such as ours to investigate variation in the use of metaphor across (linguistic) communities would allow this research to be scaled-up, using statistical patterns learned from linguistic data to inform experimental psychology.

We have presented three methods for metaphor identification that acquire metaphorical patterns from distributional properties of concepts. All of the methods (unconstrained, constrained, hgfc) are based on distributional word clustering using lexico-syntactic features. The methods are minimally supervised and unsupervised and, as our experiments have shown, they can be successfully ported across languages. Despite requiring little supervision, their performance is competitive even in comparison to fully supervised systems.¹⁷ In addition, the methods identify a large number of new metaphorical expressions in corpora (e.g., given the English seed “accelerate change,” the unconstrained method identifies as many as 113 new, different metaphors in the BNC), enabling large-scale cross-linguistic analyses of metaphor.

Our experimental results have demonstrated that lexico-syntactic features are effective for clustering and metaphor identification in all three languages. However, we have also identified important differences in the structure of the semantic spaces across languages. For instance, in Russian, a morphologically rich language, the semantic space is structured differently from English or Spanish. Because of its highly productive derivational morphology, Russian exhibits a higher number of near-synonyms (often originating from the same stem) for both verbs and nouns. This has an impact on clustering, in that (1) more nouns or verbs need to be clustered in order to represent a concept with sufficient coverage and (2) the clusters need to be larger, often containing tight sub-clusters of derivational word forms. While playing a role in metaphor identification, this finding may also have implications for other multilingual NLP tasks beyond metaphor research.

Importantly, our results confirm the hypothesis that metaphor and cross-domain vocabulary projection are naturally encoded in the distributional semantic spaces in all three languages. As a result, metaphorical mappings could be learned from distributional properties of concepts using clustering techniques. The differences in performance across languages are mainly explained by the differences in the quality of the data and pre-processing tools available for them. However, both our quantitative results and the analysis of the system output confirm that all systems successfully discover metaphorical patterns from distributional information.

We have investigated different kinds of supervision: learning from a small set of metaphorical expressions, metaphorical mappings, and without supervision. Although both minimally supervised (unconstrained, constrained) and unsupervised (hgfc) methods successfully discover new metaphorical patterns from the data, our results indicate that minimally supervised methods achieve a higher precision. The use of annotated metaphorical mappings for supervision at the clustering stage does not significantly alter the performance of the system, because their patterns are already to a certain extent encoded in the data and can be learned. However, metaphorical expressions are a good starting point in learning metaphorical generalizations in conjunction with clustering techniques.

Despite its comparatively lower performance, we believe that hgfc may prove to be a practically useful tool for NLP applications. Because it does not require any metaphor annotation, it can be easily applied to a new language (including low resource languages) for which a large enough corpus and a shallow syntactic parser are available. In addition, whereas the semi-supervised constrained and unconstrained methods discover metaphorical expressions somewhat related to the seeds, the range of metaphors discovered by hgfc is unrestricted and thus considerably wider. Since the two types of methods differ in their precision vs. their coverage, one may also consider a combination of these methods when designing a metaphor processing component for a real-world application—or, depending on the needs of the application, one may choose a more suitable one.

In the future, the models need to be extended to identify not only verb–subject and verb–object metaphors, but also metaphorical expressions in other syntactic constructions (e.g., adjectival or nominal metaphors). Previous distributional clustering and lexical acquisition research has shown that it is possible to model the meanings of a range of word classes using similar techniques (Hatzivassiloglou and McKeown 1993; Boleda Torrent and Alonso i Alemany 2003; Brockmann and Lapata 2003; Zapirain, Agirre, and Màrquez 2009). We thus expect our methods to be equally applicable to metaphorical uses of other word classes and syntactic constructions. For spectral clustering systems, such an extension would require incorporation of adjectival and nominal modifier features in clustering, clustering adjectives, and adding seed expressions representing a variety of syntactic constructions. The extension of hgfc would be more straightforward, only requiring ranking additional adjectival and nominal features that the metaphorically associated clusters in the graph share.

The results of our hgfc experiments also offer support to the cognitive science findings on the differences in organization of abstract and concrete concepts in the human brain (Crutch and Warrington 2005; Wiemer-Hastings and Xu 2005; Huang, Lee, and Federmeier 2010; Adorni and Proverbio 2012). Specifically, our experiments have shown that abstract concepts exhibit both within-domain and cross-domain association patterns (i.e., the literal ones and the metaphorical ones) and that the respective patterns can be successfully learned from linguistic data via the words' distributional properties. The metaphorical patterns that the system is able to acquire (for different languages or different data sets) can in turn be used to guide further cognitive science and psychology research on metaphor and concept representation more generally. In addition, we believe that the presented techniques may have applications in NLP beyond metaphor processing and would impact a number of tasks in computational semantics that model the properties of and relations between concepts in a distributional space.

We would like to thank our anonymous reviewers for their most insightful comments. Ekaterina Shutova's research is supported by the Leverhulme Trust Early Career Fellowship.