Tighten a bit for ssh, differentiate:
/etc/pf.conf
table <bruteforce> persist block quick from <bruteforce> # tighter for ssh pass quick proto { tcp, udp } from any to any port ssh \ flags S/SA keep state \ (max-src-conn 15, max-src-conn-rate 5/3, \ overload <bruteforce> flush global) pass inet proto tcp from any to $int_if:network port $tcp_services \ flags S/SA keep state \ (max-src-conn 100, max-src-conn-rate 15/5, \ overload <bruteforce> flush global)