To have words or characters represented as sparse vectors is to provide some sort of grounding for NLU system’s input. There are different ways to look at it, and there are different solutions available.
Intelligent machines, just like intelligent brains have an internal language of communication. It can be a series of 1’s and 0’s, a series of electrical spikes, an analog signal, or a symbolic signal.
If a sparse binary map represents grounding of a symbol in a 2d feature map, then we need to figure how to create the map (topological locations of features) and how to map symbols onto those features. Preferably done with model free code.