Some objects in the visual field are more likely to attract attention because they are either intrinsically eye catching or relevant in the context of a particular task. These two factors, known as stimulus-driven and goal-directed factors, respectively, are thought to be integrated into a unique salience map, possibly located in the frontal or the parietal cortex. However, the distinct contribution of these two regions to salience representation is difficult to establish experimentally and remains debated. In an attempt to address this issue, we designed several dual tasks composed of a letter reporting task and a visual search task, allowing us to quantify the salience of each visual item by measuring its probability to be selected by attention. In Experiment 1, the salience of the visual search items depended on a combination of conspicuity and relevance factors, whereas in Experiment 2, stimulus-driven and goal-directed factors were tested separately. Then, we used transcranial magnetic stimulation to interfere transiently with the function of the right angular gyrus (ANG) or right FEFs in healthy subjects performing these dual tasks. We found that interfering with the ANG and the FEF function specifically altered the influence of salience on the letter report rate without affecting the overall letter reporting rate, suggesting that these areas are involved in salience representation. In particular, the present study suggests that ANG is involved in goal-directed salience representation, whereas FEF would rather house a global salience map integrating both goal-directed and stimulus-driven factors.