One side of the reader has the Square logo recessed (basically, three concentric squares), and the other side is just flat and featureless.

When swiping a card, the magnetic stripe must face the plain, featureless side.

In other words, if the Square logo is facing you, the magnetic stripe faces away from you.

Another way to think about it is to look at the reader from the side. See how one side of the gap is thicker? This side is thicker because it contains the magnetic stripe reading hardware, so the stripe must face that thicker side.