Legend
Best result
Good result
Bad sign

Password Polices Tested

Policy implementations
The password polices that implemented as distinct software products.

Formal polices
The password polices that exist as a formal description of what "good" password should be look like.

Input Data

Applying Polices to Passwords

There are two group of passing passwords experiments:

  • Pass passwords from real leaks through polices.
    More passwords is allowed by policy is better. But because usually big part real passwords are weak it should not be more than necessary.
  • Pass presumably strong passwords through polices.
    In this case, the metric is plain: more is better.

Apply polices to real passwords

passwords 14char passwdqc pwquality o365 swisscom complexify zxcvbn
1000000yandex2014 0.034469 0.021339 0.048604 0.030262 0.259723 0.033666 0.147317
rockyou-all 0.022204 0.016047 0.03188 0.023172 0.24867 0.020302 0.054312
psy-real.png

Apply polices to strong passwords

passwords 14char passwdqc pwquality o365 swisscom complexify zxcvbn
CMIYC2010-uncracked 0.002987 0.598367 0.06133 0.641378 0.894066 0.032457 0.424333
phrases-rand39 0.9851 0.9998 0.998 0.3304 1.0 0.9994 0.9976
random10 0.0 1.0 0.2503 0.7102 1.0 0.0 0.9999
random20 1.0 0.9728 0.9966 0.0 1.0 1.0 1.0
psy-strong.png

Guessing Passwords Passed Polices

Total Guesses

The following table accumulates total guesses per password dump, policy. The numbers are relation between total number of guessed accounts and number of accounts. I use term accounts because password dump contain duplicates, i.e. accounts that have the same passwords.

The NOMETER represents guessing attacks against passwords dumps without any password policy applied.

Total guesses.
Less is better.

passwords NOMETER 14char passwdqc pwquality o365 swisscom complexify zxcvbn
1000000yandex2014 0.275119 0.000646 1.4e-05 0.000414 0.001312 0.025865 0.000216 0.000346
rockyou-all 0.479516 0.000166 3e-05 0.000293 0.002978 0.080141 0.000152 0.000595

Average total guesses per policy.

avg-pass-guess-w-nometer.png

Average total guesses per policy without NOMETER.

avg-pass-guess.png

Average total guesses per policy without NOMETER and Swisscom policy.

avg-pass-guess-wo-swisscom.png

Total guesses per policy.
In relation between guesses and size of password dump.

total-guesses.png

Effectiveness of Guesses

The next data seems to me much more interesting. It represents effectiveness of dictionary attacks against different polices.

Most effective attack per policy.
The relation between size of dictionary and number of guessed accounts.
Lower is better.

NOMETER 14char complexify o365 passwdqc pwquality swisscom zxcvbn
1241.283982 0.003362 0.0012 0.10247 2.7e-05 0.000279 24.616187 0.009882

Average effectiveness of attack per policy.
Lower is better.

NOMETER 14char complexify o365 passwdqc pwquality swisscom zxcvbn
Dicts. < 106 158.82237965 0.0002697 0.0001198 0.01298045 7e-07 0.0 2.29603645 0.00172205
Dicts. > 106 0.2442868125 3.1375e-05 2.19375e-05 0.001080125 2.6875e-06 3.98125e-05 0.02769325 9.325e-05

Average effectiveness of attack per policy.
Logarithmic scale. Lower is better.

avg-session.png

Summary

NOMETER 14char complexify o365 passwdqc pwquality swisscom zxcvbn
Acceptance of real passwords Don't use it! Best Good
Acceptance of strong passwords Don't use it! Good Good Best Best
Total guesses Don't use it! Warn Best Warn
Protection from online attacks Don't use it! Warn Best Best Warn