When colloidal particles are dispersed in a chiral nematic matrix, such as 90 degrees twisted nematic cell, colloidal self-assembly is substantially different compared to homogeneous nematic. The macroscopic chiral organization of the underlying nematic not only lifts the energy degeneration of recently observed “bubble-gum” colloidal dimers in non-chiral liquid crystal, but also dramatically increases the probability of their formation. We identify experimentally and numerically two basic colloidal dimer conformations in a nematic liquid crystal, that are of opposite chirality and we confirm the non-singular topological nature of their binding. Dimer-dimer interaction weakly depends on the chirality of the dimers and leads to an assembly of 2D nematic colloidal crystals of pure or "mixed" chirality.