Episodic memory relies on both recollection and familiarity; why these processes are differentially engaged during retrieval remains unclear. Traditionally, recollection has been considered necessary for tasks requiring associative retrieval, whereas familiarity supports recognition of items. Recently, however, familiarity has been shown to contribute to associative recognition if stimuli are “unitized” at encoding (a single representation is created from multiple elements)—the “benefit” of unitization. Here, we ask if there is also a “cost” of unitization; are the elements of unitized representations less accessible via familiarity? We manipulated unitization during encoding and used ERPs to index familiarity and recollection at retrieval. The data revealed a selective reduction in the neural correlate of familiarity for individual words originally encoded in unitized compared with nonunitized word pairs. This finding reveals a measurable cost of unitization, suggesting that the nature of to-be-remembered stimuli is critical in determining whether familiarity contributes to episodic memory.