The assembly of medium-sized silicon nanoclusters was simulated to study the starting point of the formation of cluster-derived silicon nanowires (CDSiNWs). Hydrogen-terminated clusters were found repulsing each other and inter-connecting through the hydrogen bonds, thus could not form a stable silicon nanowire (SiNW). Between the pristine silicon clusters without hydrogen saturation, the assembly takes place automatically. An orientation priority in cluster assembly is obtained, as silicon clusters Si29 are more possibly adhered along <111> direction than the other directions. Such an assembly may be the starting point of the SiNW growth along <111> direction. Moreover, it indicates the possibility of silicon tetrapods or zigzag wires formation, besides straight SiNWs.