In order to further improve the accuracy of the tag number estimation algorithms in dense tags environment, based on the analysis and comparison of the traditional bit estimation algorithms, a novel optimized bit estimation algorithm is proposed. Firstly, based on the binomial distribution theory, the observation of non-selected bits is used to calculate the idle bit rate. Then, a tag number estimation model in dense tags environment is established by determining the threshold of idle bit rate. Finally, the mathematical expression of the relationship between the estimated tag number and the slot consumption is derived. The simulation results show that the proposed algorithm has better estimation accuracy than the traditional bit estimation algorithms and is stable under different numbers of tags.