It is a common and widely accepted assumption that glycine and GABA are the main inhibitory transmitters in the central nervous system (CNS). But, in the past 20 years, several studies have clearly demonstrated that these amino acids can also be excitatory in the immature central nervous system. In addition, it is now established that both GABA receptors (GABARs) and glycine receptors (GlyRs) can be located extrasynaptically and can be activated by paracrine release of endogenous agonists, such as GABA, glycine, and taurine. Recently, non-synaptic release of GABA, glycine, and taurine gained further attention with increasing evidence suggesting a developmental role of these neurotransmitters in neuronal network formation before and during synaptogenesis. This review summarizes recent knowledge about the non-synaptic activation of GABA(A)Rs and GlyRs, both in developing and adult CNS. We first present studies that reveal the functional specialization of both non-synaptic GABA(A)Rs and GlyRs and we discuss the neuronal versus non-neuronal origin of the paracrine release of GABA(A)R and GlyR agonists. We then discuss the proposed non-synaptic release mechanisms and/or pathways for GABA, glycine, and taurine. Finally, we summarize recent data about the various roles of non-synaptic GABAergic and glycinergic systems during the development of neuronal networks and in the adult.