In an in situ powder-in-tube (PIT) fabrication of MgB2 superconducting tapes, it has been found that ball-milling treatment on a starting powder is effective for improving critical current density (Jc) of the MgB2 tapes. In the present study, microstructures/nanostructures of MgB2 tapes and the mechanism of the Jc enhancement by the ball milling treatment were investigated by means of scanning electron microscopy (SEM) and transmission electron microscopy (TEM) techniques. A mixed powder of Mg and amorphous B was ball milled for 4 h. The milled powder was packed in a pure Fe tube and then cold rolled into a tape specimen. A heat treatment was performed at 600°C for 1 h under an Ar atmosphere and subsequently furnace cooled. MgB2 tape specimens without the ball-milling treatment were also prepared for comparison. A focused ion beam (FIB)-microsampling technique was used for TEM specimen preparation. The MgB2 tape with the ball-milling treatment exhibited a dense and uniform microstructure mainly composed of nano-crystalline MgB2. On the other hand, MgB2 tapes without the ball-milling treatment showed inhomogeneous microstructures that comprise of a porous uncrystallized area and a crystallized one. It was also found that the averaged grain size of the nano-crystalline MgB2 becomes smaller by the ball-milling treatment. Thus, it can be explained that the Jc enhancement by the ball-milling treatment is owing to improvements in the grain connectivity and microstructural uniformity of the crystalline MgB2 phase in the tape.