In this paper the modal characteristics and leakage loss of hollow-core photonic bandgap fibers based on a square lattice with rounded square air holes was investigated by using a full-vector finite element method. It was found that hollow-core photonic bandgap fibers with air core arranged in square lattice can operate in the broad bandgap single mode. The leakage loss was analyzed comprehensively. Simulations show that the core diameter and the hole-to-hole spacing have a small influence on leakage loss, but for a given wavelength the desired lowest leakage loss could be obtained by tuning the value of hole-to-hole spacing, while the rounded diameter has a larger influence on leakage loss than the former factors and it has an optimal value dc/d=0.4. The number of cladding rings plays a key role in the leakage loss and the leakage loss is reduced by nearly 2 orders of magnitude per 2 additional cladding rings.