I know that p-value is used to determine the significance of results after a hypothesis test in statistics. I am confused about how to use it as a threshold in selecting the best features for my model?

The idea is that you perform a hypothesis test (t, z, anything else works too as long as it makes sense here) to test whether a given feature has a relationship with the target (alternate hypothesis ) or not (null hypothesis). You do this test for all variables one by one and look at the p-value (significance level) to determine which variable is most expendable (lowest relation to the target).

You can look at this post from the forum for an algorithm using this approach.

Also look at this post to see how this might be a bad feature selection criteria.