Soybean is a phosphorus-loving crop, whereas phosphorus deficiency can affect soybean growth and development finally resulting in yield loss. The grain weight per plant of soybean is a quantitative character, which is an important indicator for identifying soybean tolerance to low phosphorus. At present, most studies are still in the stage of QTL mapping, so it is particularly important to discover more regulatory genes and excellent allelic variations in order to promote the analysis of the regulatory mechanism of soybean tolerance to low phosphorus and its breeding utilization. In this study, 395 soybean germplasm resources were used to conduct genome wide association scanning (GWAS) and preliminarily predict candidate genes by using grain weight per plant under low phosphorus and relative conditions as identification indicators for low phosphorus tolerance traits, combined with high-density SNP markers. The results showed that the tolerance to low phosphorus was significantly different among different soybean germplasms. The mean values of grain weight per plant under normal phosphorus, low phosphorus and relative conditions were 4.08 g, 1.15 g and 0.34, and their coefficients of variation were higher than 50%. Through cluster analysis, 4 landraces such as ZDD00219 and ZDD00163 and 16 cultivars such as ZDD00383 and ZDD23714 were screened as phosphorus efficient soybean germplasms. GWAS identified 32 SNP loci significantly associated with low phosphorus tolerance of soybean, including 23 SNP loci associated with single plant grain weight under low phosphorus conditions and nine SNP loci associated with relative single plant grain weight. Two candidate genes were proposed by LD block analysis, including a WRKY DNA binding domain gene that was detected with specific expression in roots, and a pyridoxal phosphate phosphatase gene that was detected with specific expression in root hairs.