class: center, middle # Recent and not so recent changes in OpenBSD that make life better ## (and may turn up elsewhere too) #### A Sunday Lunch at [SEMIBUG](https://semibug.org/) #### August 22, 2021 ### Peter N. M. Hansteen This presentation: [home.nuug.no/~peter/openbsd_moments/](https://home.nuug.no/~peter/openbsd_moments/) or [full text](https://bsdly.blogspot.com/2021/08/recent-and-not-so-recent-changes-in.html) --- # Recent and not so recent changes in OpenBSD that make life better The *"functional, free, secure by default"* [OpenBSD](https://www.openbsd.org/) has been an important part of open source for 25+ years, and has been fairly central in my last couple of decades. This is about moments when OpenBSD helped make life better. I will assume you know about the code audit, emphasis on security and quality and so forth. For more on OpenBSD, my occasionally updated [OpenBSD and you](https://home.nuug.no/~peter/openbsd_and_you/) presentation might be good for background. --- # So who am I and what can I offer? I am Peter N. M. Hansteen <peter@bsdly.net> In information technology since the late 1980s, OpenBSD user since OpenBSD 2.5 (1999) Security Engineer going on Cloud Expert (;D) -- Unix/Linux sysadmin, networker Wrote [The Book of PF](https://www.nostarch.com/pf3), and [OpenBSD](https://www.openbsd.org/) has frequently made my life better. This presentation is about those moments. From The Other West Coast (Bergen, Norway) --- # The installer was always good, got better I came to [OpenBSD](https://www.openbsd.org/) from mainly Linux and FreeBSD 20+ years back. Came for the security focus, ordered the 2.5 CD set and the [wireframe daemon shirt](https://www.bsdly.net/~peter/blogpix/tshirt-2.jpg). Installed at first, *experimentally*, on even-then-crappy hardware (80386/33MHz, 8MB RAM, 100MB IDE disk). *The thing worked!* *Everything made sense!* *Everything had a (readable) man page!* I later heard about "no commit without documentation" rule + anonymous CVS -- *follow commits in real time!* --- # The installer got better After all those years, the installer is still stubbornly *text-only*, portably *the same across [hardware platforms](http://www.openbsd.org/plat.html)*, and has spawned * repeatable, scriptable ([autoinstall(8)](https://man.openbsd.org/autoinstall) from [OpenBSD 5.5](https://www.openbsd.org/55.html)) * [sysupgrade(8)](https://man.openbsd.org/sysupgrade) from [OpenBSD 6.6](https://www.openbsd.org/66.html)) for (almost) hands-off upgrades snapshot to snapshot or one release to the next and still incremental improvements (of course) in almost every release. --- # Something for the laptop: hardware support I got myself a new laptop late May 2021, and went from **"Oh sh*t, the SSD isn't recognized"** followed by <img src="20210528_085741_cropped.jpg" alt="OpenBSD 6.9-current panic message" style="width:292px;height:120px;"><p> To **full support of everything** -- Intel Core 11th gen evo -- within days, and will be in [OpenBSD 7.0](https://www.openbsd.org/70.html) I *did* run kernels from jsg@'s drm510 branch for a couple of weeks, then back to regular [sysupgrade](https://man.openbsd.org/sysupgrade)s. The SSD thing was down to a BIOS option -- **pseudo-RAID on a single device does not make sense** The gory details are at [bsdly.blogspot.com](https://bsdly.blogspot.com/2021/07/the-impending-doom-of-your-operating.html) or the .no IT mag [digi.no](https://bsdly.blogspot.com/2021/07/the-impending-doom-of-your-operating.html) (in *Norwegian*) --- # Living the life dynamic Laptop network config is usually dynamic. OpenBSD 7.0 will ship with [dhcpleased(8)](https://man.openbsd.org/dhcpleased) [enabled by default](https://undeadly.org/cgi?action=article;sid=20210722072359). This completes a five year process of rebuilding dynamic configuration with several unix programs that *do one thing well*: * [slaacd(8)](https://man.openbsd.org/slaacd) stateless IPv6 address autoconfiguration, ([OpenBSD 6.2](https://www.openbsd.org/62.html)) * [rad(8)](https://man.openbsd.org/rad) IPv6 router advertisement daemon ([OpenBSD 6.4](https://www.openbsd.org/64.html)) * [unwind(8)](https://man.openbsd.org/unwind) validating DNS resolver; learns resolvers to query from DHCP + other sources ([OpenBSD 6.5](https://www.openbsd.org/65.html)) * [resolvd(8)](https://man.openbsd.org/resolvd) manages + edits [/etc/resolv.conf](https://man.openbsd.org/resolv.conf), [dhcpleased(8)](https://man.openbsd.org/dhcpleased) is the DHCP client, feeds into the config ([OpenBSD 6.9](https://www.openbsd.org/69.html)) Have your laptop *join* networks withing range -- --- # Living the life dynamic put likely networks in */etc/hostname.if* (e.g. hostname.iwx0) ``` join adipose wpakey thedoctorknows join tardis wpakey biggerontheinside join cybermen wpakey nowedont inet autoconf inet6 autoconf ``` and your laptop life is dynamically better. --- # The thing that lured me in In 2001, I was longing for something saner than Linux' *iptables*, still only experimenting with OpenBSD and IPF. Then the IPF license was not *actually* free, needed to be replaced. With [OpenBSD 3.0](https://www.openbsd.org/30.html). PF was born, a '**working prototype**' that performed better than IPF. That had me dive in, starting the process that had me produce [The Book of PF](https://nostarch.com/pf3) and numerous [blog posts](https://bsdly.blogspot.com/). More about that later. The IPF incident lead to a *license audit*, summed up by **deraadt@** on [misc@](https://marc.info/?l=openbsd-misc&m=104570938124454&w=2): Lots of problems fixed, including no license or no copyright. The relevant developers were tracked down, a frequent response was ### "Say what? Are people still using this?" --- # SSH, open and better PF was written from scratch to replace non-free code. But it was not the first *nonlibreectomy* performed by the OpenBSD project. The by the late 1990s, the original SSH was turning commercial and proprietary. OpenBSD developers dug out the last free version, re-implemented more modern features and got rid of bugs. [OpenSSH](https://www.openssh.com/) debuted in [OpenBSD 2.6](https://www.openbsd.org/26.html). It soon grew a *-portable* variant which has been widely ported. In recent years, OpenSSH has a steady high-ninteties percent market share. With time, *telnetd* became irrelevant and was removed in [OpenBSD 3.8](https://www.openbsd.org/38.html). OpenSSH was the first daemon to become *privilege separated* in an overhaul up to [OpenBSD 3.2](https://www.openbsd.org/32.html). --- # And yes, that packet filter I must confess: - PF has been an important part of my life since the early noughties - I have contributed to the (popular, wrong) perception that OpenBSD is mainly a firewall OS But lots of *useful* and *fun* tools and features sprung from PF, some were ported or imitated elsewhere, some remain OpenBSD only. My favorites follow. --- # Beating up spammers with OpenBSD spamd(8) since OpenBSD 3.3 I was already a mail server admin back when I found OpenBSD and PF. When [OpenBSD 3.3](https://www.openbsd.org/33.html) dropped with [spamd(8)](https://man.openbsd.org/spamd) the spam deferral daemon, featuring blocklists and stuttering (1 byte per second), I loved the idea. Just minor surgery to [pf.conf](https://man.openbsd.org/pf.conf) and spamd config, and the FreeBSD mail server with spamassassin and clamav grew noticeably quieter. --- # Going grey, then trapping The next big thing in [spamd(8)](https://man.openbsd.org/spamd) was a greylisting mode. The mail server's fans grew even quieter. The came **greytrapping**: any host that tries to deliver to *known bad*, non existent addresses in our domains are added to the blocklist, get the 10 bytes per sec treatment for 24 hours. I do my own trapping: * harvest spamtraps from bounce entries in mail server logs * by July 2007, I started to [publish](https://bsdly.blogspot.com/2007/07/hey-spammer-heres-list-for-you.html) both the *trapped* IP addresses and the *spamtraps*. *Trapped* IP addresses typically in the 3000 to 5000 range, exported once per hour, has been past the 20' mark Bizarrely, number of *spamtraps* is now over 270', still growing. It's absurd but fun, see [Badness, Enumerated by Robots](https://bsdly.blogspot.com/2018/08/badness-enumerated-by-robots.html) for details. --- # The brutes, the password gropers and the state tracking options If you run any internet facing service with a login option, your logs will contain noise from *password guessing*, aka *password gropers*. OpenBSD 3.6-current introduced *state tracking options*: act on state data, set and enforce limits by actions, such as ``` table <bruteforce> persist block quick from <bruteforce> pass inet proto tcp from any to $localnet port $tcp_services \ flags S/SA keep state \ (max-src-conn 100, max-src-conn-rate 15/5, \ overload <bruteforce> flush global) ``` => more than 100 connections *or* more than 15 new over 5 seconds -> added to the table, blocked and connections cut. --- # State tracking, generally do expire Useful for other services too. A few examples for inspiration can be found in the article [Forcing the password gropers through a smaller hole with OpenBSD's PF queues](https://bsdly.blogspot.com/2017/04/forcing-password-gropers-through.html). **Do** remember to *expire* entries after a while -- I now favor a few weeks over the classical 24 hours. Like *greytrapping*, this lets you build a configuration that *adapts* to network conditions and *learns* from the traffic it sees. The *buzzwordability* potential is enormous, I'm puzzled as to why none of the other **big names** imitated this and marketed to the max. --- # NAT's guts ripped out [OpenBSD 4.7](https://www.openbsd.org/47.html) contained a totally rewritten NAT -- IPv4 network address translation -- system. The result of a several thousand line diff that *shrank* the code and made it *faster*. Previous versions had separate **nat on $interface** or **rdr on $interface** rules. Now we have **nat-to**, **rdr-to** as *options* on **match** or **pass** rules. Rulesets with NAT logic could be a lot more flexible overnight. Also prompted the re-write of *The Book of PF* to its *second edition*. --- # We went to modern queueing Traffic shaping in OpenBSD used to be *ALTQ*, code labeled experimental for 15 years, several different algorithms. *ALTQ* was rolled into PF on OpenBSD, but the syntax was '*inelegant at best*'- Replaced with another not-small diff in [OpenBSD 5.5](https://www.openbsd.org/55.html), simpler scheme with priorities or variations on one algorithm, *Hierarchical Fair Service Curve* (HFSC) and a saner syntax: ``` queue rootq on $ext_if bandwidth 20M queue main parent rootq bandwidth 20479K min 1M max 20479K qlimit 100 queue qdef parent main bandwidth 9600K min 6000K max 18M default queue qweb parent main bandwidth 9600K min 6000K max 18M queue qpri parent main bandwidth 700K min 100K max 1200K queue qdns parent main bandwidth 200K min 12K burst 600K for 3000ms queue spamd parent rootq bandwidth 1K min 0K max 1K qlimit 300 ``` Queue assignment with *match* or *pass* rules. (Yes, I *do* punish spammers extra here). Triggered the third edition of [The Book of PF](https://nostarch.com/pf3), with new and old plus conversion tips. *ALTQ* lives on in FreeBSD and NetBSD, removed from OpenBSD in [OpenBSD 5.6](https://www.openbsd.org/56.html). --- # pflow(4) offers network insights lite If you run a network, you will at times need to look into what the traffic there looks like. If nothing else to see which endpoints are involved, the amount of data transferred, which protocols and so forth. For some purposes just the metadata is all you need, and that's where *NetFlow* or *IPFIX* comes in. In [OpenBSD 4.5](https://www.openbsd.org/45.html), the tools for a Netflow *sensor* were introduced with the [pflow(4)](https://man.openbsd.org/pflow) virtual network device and the **pflow** state tracking option. Set up the interface, add the pflow option to your rules, and collect with tools such as [nfsen](http://nfsen.sourceforge.net/) or [flow-tools](https://code.google.com/archive/p/flow-tools/) in packages. My story of a noisy network and how pflow helped is [Yes, You Too Can Be An Evil Network Overlord - On The Cheap With OpenBSD, pflow And nfsen](https://bsdly.blogspot.com/2014/02/yes-you-too-can-be-evil-network.html). --- # LibreSSL, the great deobfuscation The [Heartbleed](https://en.wikipedia.org/wiki/Heartbleed) bug was **not** the reason why [LibreSSL](https://www.libressl.org) now exists. OpenBSD developers had felt physical agony at reading the OpenSSL code for years. The pain levels reached a critical threshold ("Why, oh *why* is sh*t code like that in our tree?") A few core OpenBSD hackers decided a few weeks before Heartbleed to fork the OpenSSL code and [*flense*](https://en.wikipedia.org/wiki/Flensing) the code of bugs and misfeatures. At first, unfixed bugs from **openssl.org**'s request tracker. In parallell, **jsing@** reformatted the code to less eye-stinging form so the thing was approaching readable. --- # LibreSSL, the great deobfuscation With readable code, the digging turned up * Code was never deleted, no matter how obsolete or irrelevant * bypassed malloc, wrote their own instead, never did free memory but reused it LIFO, and stuck private data in logs * written in "OpenSSL C", according to **beck@** a dialect of "worst common denominator". Details to be found in Bob Beck's BSDCan talk on the first 30 days of LibreSSL (the start of code flensing) The default TLS library since [OpenBSD 5.6](https://www.openbsd.org/56.html). Offers a **-portable** variant for porting. In my experience, *"Just Works™"*, likely a healthier alternative than its predecessor. --- # This was my list of life improving OpenBSD events - I'd love to hear yours As I warned earlier, this has been about *my* personal list of OpenBSD events that I remember fondly. I am sure *your* list is at least a little different. I am sure there are things from the [innovations](https://www.openbsd.org/innovations.html) page that I have simply forgotten about. I would love to hear about your favorite [OpenBSD](https://www.openbsd.org/) moments. --- # More items for your OpenBSD reading [www.openbsd.org](https://www.openbsd.org/) The official OpenBSD website – to donate: [www.openbsd.org/donations.html](https://www.openbsd.org/donations.html) and please do donate, corporates may prefer [www.openbsdfoundation.org](https://www.openbsdfoundation.org/) - a Canadian non-profit [undeadly.org](https://undeadly.org/) - The OpenBSD Journal news site [bsdly.blogspot.com](https://bsdly.blogspot.com/) - My rant^H^H^H^Hblog posts [flak.tedunangst.com/](https://flak.tedunangst.com/) tedu@ on developments Michael W Lucas: [Absolute OpenBSD, 2nd ed.](https://www.nostarch.com/openbsd2e) Peter N. M. Hansteen: [The Book of PF, 3rd ed.](https://www.nostarch.com/pf3) Henning Brauer: [OpenBSD sucks (… least)](https://quigon.bsws.de/papers/2015/eurobsdcon/) This presentation: [home.nuug.no/~peter/openbsd_moments/](https://home.nuug.no/~peter/openbsd_moments/) [Recent and not so recent changes in OpenBSD that make life better (and may turn up elsewhere too)](https://bsdly.blogspot.com/2021/08/recent-and-not-so-recent-changes-in.html) - full text article