The influence of grammatical gender on conceptual representations of gender has proven to be a controversial topic in the linguistic relativity literature, with empirical evidence in support of the Linguistic Relativity Hypothesis being highly task and context-dependent, as well as being modulated by the type of items being investigated (animates/inanimates). In this paper, we take a megastudy approach in order to investigate differences in results based on explicit and implicit paradigms that modulate the role of language and gender in their design. We present analyses of three experiments focussing on participants (total N = 4,621) with a grammatically gendered L1 (Czech), a non-grammatically gendered L1 (English) and L1-Czech in L2-English, and on three distinct semantic categories – people, animals and inanimates (total Nitems = 1,208). Our results indicate that the most reliable effects of grammatical gender influencing conceptual gender (outside of the domain of people) are observed for items representing animals, with Czech participants showing congruency effects in both explicit and implicit paradigms, even in their L2. The evidence for effects on inanimates is substantially weaker and is highly restrained to explicit tasks. We discuss these results in relation to the Linguistic Relativity Hypothesis and highlight important methodological considerations for future research.