In information technology since the late 1980, OpenBSD user since OpenBSD 2.5 (1999)
Security Engineer going on Cloud Expert (;D) -- Unix/Linux sysadmin, networker
Wrote [The Book of PF](https://www.nostarch.com/pf3), and [OpenBSD](https://www.openbsd.org/) has frequently made my life better. This presentation is about those moments.
From The Other West Coast (Bergen, Norway)
# OpenBSD: How it all started
OpenBSD's origin story is really the history of the Internet itself.
OpenBSD is Descended from 'BSD Unix' - Berkeley Software Distribution
Tthe *Berkeley Software Distribution* aka *BSD* lived at the CSRG, UC Berkeley from 1974 (also Ken Thompson's 1975 sabbatical) -> active development until c 1992
Became over time the complete OS *BSD Unix*, and was the home of the TCP/IP reference implementation
CSRG's DARPA research funding ended c 1992, initial 386BSD (Jolitz et al 1991-1992) spawned [FreeBSD](https://www.freebsd.org/) (BSD on PC architecture) and [NetBSD](https://www.netbsd.org/) (BSD on anything), [BSDI](https://en.wikipedia.org/wiki/Berkeley_Software_Design) (commercial - 1-800-ITS-UNIX generated USL vs BSDI lawsuit, settled 1994)
[OpenBSD](https://www.openbsd.org/) forked from NetBSD 1995 (tree created Oct 18, 1995)
The first project to provide [anonymous, public CVS](http://www.openbsd.org/anoncvs.html) - *See code change in real time!*
# You are already an OpenBSD user!
You may not know it, but you are. OpenBSD code is in
* macOS, iOS
* Cisco networking products (and likely others)
* All Linux distributions, all Unixes
... and Microsoft put [OpenSSH](https://www.openssh.com/) in base on recent Windows versions
# OpenBSD: Code audit and security evolution
Early and continuing emphasis on security - code audit started 1995 and still ongoing:
* Assume hostile environment
* Look for unsafe behaviors
* Find one bug, fix similar bugs everywhere in the tree
* *(repeat ...)*
*"Where it is possible to spot damage, fail hard"*
Lead to exploit mitigation techniques (W^X, privsep, ASLR, see eg [www.openbsd.org/papers/ven05-deraadt/index.html](https://www.openbsd.org/papers/ven05-deraadt/index.html) and the 10 years later [www.openbsd.org/papers/ru13-deraadt](https://www.openbsd.org/papers/ru13-deraadt))
First free OS with *strong crypto* in base, illegal to re-export from the US early on(!)
# Why use OpenBSD? Proactive security
*All of these have been [enabled by default](http://www.openbsd.org/security.html#default) for 10+ years:*
* *Address space randomization* (aka ASLR) no fixed jump targets or gaps
* *W^X* memory can be writable XOR executable
* *Guard pages* 'fence-like' unreadable, unwritable page after [malloc()](http://man.openbsd.org/malloc)ed chunks, detect overruns
* *Privilege separation* daemons run bulk of their code as different non-privileged users (most in chroot without shell) - [sshd](https://man.openbsd.org/sshd) was the first, the rest followed
* *Privilege revocation* privsep'd daemons drop privilege as soon as possible
# Why use OpenBSD? Proactive security (cont)
**Exploit mitigation (cont)**
* *chroot* jail -- daemons run in restricted environment ($HOME /var/empty, no shell)
* *ProPolice* random stack gap inserted, fixed returns fail
Newer developments include:
* OpenBSD 5.9 introduced *[pledge(2)](https://man.openbsd.org/pledge)* syscall to restrict program behavior to predeclared profile
* OpenBSD 6.4 introduced *[unveil(2)](https://man.openbsd.org/unveil)* syscall to restrict file system access to predeclared profile
* OpenBSD 6.2 introduced *KARL* (kernel address randomized link) - kernel relinked with randomized layout for each boot, see the [undeadly.org article](http://undeadly.org/cgi?action=article&sid=20170613041706) or the [tech@ message](https://marc.info/?l=openbsd-tech&m=149732026405941) (kernel object files grew base by ~300MB)
# OpenBSD: Usable, Portable and secure
*Secure, correct code* that runs on 14 hardware [platforms](https://www.openbsd.org/plat.html)
*Usable system* - sane defaults, complete and readable documentation ([man pages](https://man.openbsd.org/) -- commit new features w/o matching man page not allowed -- plus [FAQ](https://www.openbsd.org/faq/) and other guides)
*Free to use* - short (2 paragraph) BSD license preferred (see [www.openbsd.org/policy.html](https://www.openbsd.org/policy.html)), also see [www.openbsd.org/goals.html](https://www.openbsd.org/goals.html)
By a small team: Approx 100 active developers (historically '~356 hackers'), see [marc.info/?l=openbsd-misc&m=144515087006177&w=2](http://marc.info/?l=openbsd-misc&m=144515087006177&w=2)
Coordinated by a Canadian in Canada, US export restrictions do not apply.
*Release every six months* (May and November), most recently [OpenBSD 6.9](https://www.openbsd.org/69.html) May 1, 2021 (50th release, 25 years), with [OpenBSD 7.0](https://www.openbsd.org/69.html) slated for November 1, 2021
Emphasis on security and usability (yes, devs run it on their laptops).
# Why use OpenBSD?
It's a UNIX. A real UNIX (no **systemd**)
Uncluttered base system - full install w/X, compilers etc is approx 962MB total on amd64
No services enabled (listening) by default. Installer asks if you want to run [sshd(8)](https://man.openbsd.org/sshd)
Full featured system (c, c++ (clang or gcc), perl and tools in base)
Supports modern to fairly ancient hardware across [14 platforms](https://www.openbsd.org/plat.html) (alpha, amd64, arm64, armv7, hppa, i386, landisk, loongson, luna88k, macppc, octeon, powerpc64, riscv64, sparc64)
# OpenBSD: Ported software goes under /usr/local
Ported software in package system (*11310* packages prebuilt on amd64 for [OpenBSD 6.9](https://www.openbsd.org/69.html)), mips64 had 8182 and mips64el is still building.
Clear separation of base system and packages. Use of packages encouraged, but if you want to build from checked out **ports** source (make install), you can.
See [You've Installed It. Now What? Packages!](https://bsdly.blogspot.com/2013/04/youve-installed-it-now-what-packages.html).
I will assume you know about the code audit, emphasis on security and quality and so forth.
For more on OpenBSD, my occasionally updated [OpenBSD and you](https://home.nuug.no/~peter/openbsd_and_you/) presentation might be good for background.
# The installer was always good, got better
I came to [OpenBSD](https://www.openbsd.org/) from mainly Linux and FreeBSD 20+ years back.
Came for the security focus, ordered the 2.5 CD set and the [wireframe daemon shirt](https://www.bsdly.net/~peter/blogpix/tshirt-2.jpg).
Installed at first, *experimentally*, on even-then-crappy hardware (80386/33MHz, 8MB RAM, 100MB IDE disk).
*The thing worked!*
*Everything made sense!*
*Everything had a (readable) man page!*
I later heard about "no commit without documentation" rule + anonymous CVS -- *follow commits in real time!*
# The installer got better
After all those years, the installer is still stubbornly *text-only*, portably *the same across [hardware platforms](http://www.openbsd.org/plat.html)*, and has spawned
* repeatable, scriptable ([autoinstall(8)](https://man.openbsd.org/autoinstall) from [OpenBSD 5.5](https://www.openbsd.org/55.html))
* [sysupgrade(8)](https://man.openbsd.org/sysupgrade) from [OpenBSD 6.6](https://www.openbsd.org/66.html)) for (almost) hands-off upgrades snapshot to snapshot or one release to the next
and still incremental improvements (of course) in almost every release.
# Something for the laptop: hardware support
I got myself a new laptop late May 2021, and went from **"Oh sh*t, the SSD isn't recognized"** followed by
To **full support of everything** -- Intel Core 11th gen evo -- within days, and will be in [OpenBSD 7.0](https://www.openbsd.org/70.html)
I *did* run kernels from jsg@'s drm510 branch for a couple of weeks, then back to regular [sysupgrade](https://man.openbsd.org/sysupgrade)s.
The SSD thing was down to a BIOS option -- **pseudo-RAID on a single device does not make sense**
The gory details are at [bsdly.blogspot.com](https://bsdly.blogspot.com/2021/07/the-impending-doom-of-your-operating.html) or the .no IT mag [digi.no](https://bsdly.blogspot.com/2021/07/the-impending-doom-of-your-operating.html) (in *Norwegian*)
# Living the life dynamic
Laptop network config is usually dynamic. OpenBSD 7.0 will ship with [dhcpleased(8)](https://man.openbsd.org/dhcpleased) [enabled by default](https://undeadly.org/cgi?action=article;sid=20210722072359).
This completes a five year process of rebuilding dynamic configuration with several unix programs that *do one thing well*:
* [slaacd(8)](https://man.openbsd.org/slaacd) stateless IPv6 address autoconfiguration, ([OpenBSD 6.2](https://www.openbsd.org/62.html))
* [rad(8)](https://man.openbsd.org/rad) IPv6 router advertisement daemon ([OpenBSD 6.4](https://www.openbsd.org/64.html))
* [unwind(8)](https://man.openbsd.org/unwind) validating DNS resolver; learns resolvers to query from DHCP + other sources ([OpenBSD 6.5](https://www.openbsd.org/65.html))
* [resolvd(8)](https://man.openbsd.org/resolvd) manages + edits [/etc/resolv.conf](https://man.openbsd.org/resolv.conf), [dhcpleased(8)](https://man.openbsd.org/dhcpleased) is the DHCP client, feeds into the config ([OpenBSD 6.9](https://www.openbsd.org/69.html))
Have your laptop *join* networks withing range --
# Living the life dynamic
put likely networks in */etc/hostname.if* (e.g. hostname.iwx0)
join adipose wpakey thedoctorknows
join tardis wpakey biggerontheinside
join cybermen wpakey nowedont
and your laptop life is dynamically better.
# The thing that lured me in
In 2001, I was longing for something saner than Linux' *iptables*, still only experimenting with OpenBSD and IPF.
Then the IPF license was not *actually* free, needed to be replaced.
With [OpenBSD 3.0](https://www.openbsd.org/30.html). PF was born, a '**working prototype**' that performed better than IPF.
That had me dive in, starting the process that had me produce [The Book of PF](https://nostarch.com/pf3) and numerous [blog posts](https://bsdly.blogspot.com/). More about that later.
The IPF incident lead to a *license audit*, summed up by **deraadt@** on [misc@](https://marc.info/?l=openbsd-misc&m=104570938124454&w=2): Lots of problems fixed, including no license or no copyright.
The relevant developers were tracked down, a frequent response was
### "Say what? Are people still using this?"
# SSH, open and better
PF was written from scratch to replace non-free code. But it was not the first *nonlibreectomy* performed by the OpenBSD project.
The by the late 1990s, the original SSH was turning commercial and proprietary.
OpenBSD developers dug out the last free version, re-implemented more modern features and got rid of bugs.
[OpenSSH](https://www.openssh.com/) debuted in [OpenBSD 2.6](https://www.openbsd.org/26.html). It soon grew a *-portable* variant which has been widely ported.
In recent years, OpenSSH has a steady high-ninteties percent market share.
With time, *telnetd* became irrelevant and was removed in [OpenBSD 3.8](https://www.openbsd.org/38.html).
OpenSSH was the first daemon to become *privilege separated* in an overhaul up to [OpenBSD 3.2](https://www.openbsd.org/32.html).
# And yes, that packet filter
I must confess:
- PF has been an important part of my life since the early noughties
- I have contributed to the (popular, wrong) perception that OpenBSD is mainly a firewall OS
But lots of *useful* and *fun* tools and features sprung from PF, some were ported or imitated elsewhere, some remain OpenBSD only.
My favorites follow.
# Beating up spammers with OpenBSD spamd(8) since OpenBSD 3.3
I was already a mail server admin back when I found OpenBSD and PF.
When [OpenBSD 3.3](https://www.openbsd.org/33.html) dropped with [spamd(8)](https://man.openbsd.org/spamd) the spam deferral daemon, featuring blocklists and stuttering (1 byte per second), I loved the idea.
Just minor surgery to [pf.conf](https://man.openbsd.org/pf.conf) and spamd config, and the FreeBSD mail server with spamassassin and clamav grew noticeably quieter.
# Going grey, then trapping
The next big thing in [spamd(8)](https://man.openbsd.org/spamd) was a greylisting mode. The mail server's fans grew even quieter.
The came **greytrapping**: any host that tries to deliver to *known bad*, non existent addresses in our domains are added to the blocklist, get the 10 bytes per sec treatment for 24 hours.
I do my own trapping:
* harvest spamtraps from bounce entries in mail server logs
* by July 2007, I started to [publish](https://bsdly.blogspot.com/2007/07/hey-spammer-heres-list-for-you.html) both the *trapped* IP addresses and the *spamtraps*.
*Trapped* IP addresses typically in the 3000 to 5000 range, exported once per hour, has been past the 20' mark
Bizarrely, number of *spamtraps* is now over 270', still growing. It's absurd but fun, see [Badness, Enumerated by Robots](https://bsdly.blogspot.com/2018/08/badness-enumerated-by-robots.html) for details.
# The brutes, the password gropers and the state tracking options
If you run any internet facing service with a login option, your logs will contain noise from *password guessing*, aka *password gropers*.
OpenBSD 3.6-current introduced *state tracking options*: act on state data, set and enforce limits by actions, such as
table <bruteforce> persist
block quick from <bruteforce>
pass inet proto tcp from any to $localnet port $tcp_services \
flags S/SA keep state \
(max-src-conn 100, max-src-conn-rate 15/5, \
overload <bruteforce> flush global)
=> more than 100 connections *or* more than 15 new over 5 seconds -> added to the table, blocked and connections cut.
# State tracking, generally do expire
Useful for other services too. A few examples for inspiration can be found in the article [Forcing the password gropers through a smaller hole with OpenBSD's PF queues](https://bsdly.blogspot.com/2017/04/forcing-password-gropers-through.html).
**Do** remember to *expire* entries after a while -- I now favor a few weeks over the classical 24 hours.
Like *greytrapping*, this lets you build a configuration that *adapts* to network conditions and *learns* from the traffic it sees.
The *buzzwordability* potential is enormous, I'm puzzled as to why none of the other **big names** imitated this and marketed to the max.
# NAT's guts ripped out
[OpenBSD 4.7](https://www.openbsd.org/47.html) contained a totally rewritten NAT -- IPv4 network address translation -- system. The result of a several thousand line diff that *shrank* the code and made it *faster*.
Previous versions had separate **nat on $interface** or **rdr on $interface** rules.
Now we have **nat-to**, **rdr-to** as *options* on **match** or **pass** rules.
Rulesets with NAT logic could be a lot more flexible overnight.
Also prompted the re-write of *The Book of PF* to its *second edition*.
# We went to modern queueing
Traffic shaping in OpenBSD used to be *ALTQ*, code labeled experimental for 15 years, several different algorithms. *ALTQ* was rolled into PF on OpenBSD, but the syntax was '*inelegant at best*'-
Replaced with another not-small diff in [OpenBSD 5.5](https://www.openbsd.org/55.html), simpler scheme with priorities or variations on one algorithm, *Hierarchical Fair Service Curve* (HFSC) and a saner syntax:
queue rootq on $ext_if bandwidth 20M
queue main parent rootq bandwidth 20479K min 1M max 20479K qlimit 100
queue qdef parent main bandwidth 9600K min 6000K max 18M default
queue qweb parent main bandwidth 9600K min 6000K max 18M
queue qpri parent main bandwidth 700K min 100K max 1200K
queue qdns parent main bandwidth 200K min 12K burst 600K for 3000ms
queue spamd parent rootq bandwidth 1K min 0K max 1K qlimit 300
Queue assignment with *match* or *pass* rules. (Yes, I *do* punish spammers extra here).
Triggered the third edition of [The Book of PF](https://nostarch.com/pf3), with new and old plus conversion tips.
*ALTQ* lives on in FreeBSD and NetBSD, removed from OpenBSD in [OpenBSD 5.6](https://www.openbsd.org/56.html).
# pflow(4) offers network insights lite
If you run a network, you will at times need to look into what the traffic there looks like.
If nothing else to see which endpoints are involved, the amount of data transferred, which protocols and so forth.
For some purposes just the metadata is all you need, and that's where *NetFlow* or *IPFIX* comes in.
In [OpenBSD 4.5](https://www.openbsd.org/45.html), the tools for a Netflow *sensor* were introduced with the [pflow(4)](https://man.openbsd.org/pflow) virtual network device and the **pflow** state tracking option.
Set up the interface, add the pflow option to your rules, and collect with tools such as [nfsen](http://nfsen.sourceforge.net/) or [flow-tools](https://code.google.com/archive/p/flow-tools/) in packages.
My story of a noisy network and how pflow helped is [Yes, You Too Can Be An Evil Network Overlord - On The Cheap With OpenBSD, pflow And nfsen](https://bsdly.blogspot.com/2014/02/yes-you-too-can-be-evil-network.html).
# LibreSSL, the great deobfuscation
The [Heartbleed](https://en.wikipedia.org/wiki/Heartbleed) bug was **not** the reason why [LibreSSL](https://www.libressl.org) now exists.
OpenBSD developers had felt physical agony at reading the OpenSSL code for years.
The pain levels reached a critical threshold ("Why, oh *why* is sh*t code like that in our tree?")
A few core OpenBSD hackers decided a few weeks before Heartbleed to fork the OpenSSL code and [*flense*](https://en.wikipedia.org/wiki/Flensing) the code of bugs and misfeatures.
At first, unfixed bugs from **openssl.org**'s request tracker.
In parallell, **jsing@** reformatted the code to less eye-stinging form so the thing was approaching readable.
# LibreSSL, the great deobfuscation
With readable code, the digging turned up
* Code was never deleted, no matter how obsolete or irrelevant
* bypassed malloc, wrote their own instead, never did free memory but reused it LIFO, and stuck private data in logs
* written in "OpenSSL C", according to **beck@** a dialect of "worst common denominator".
Details to be found in Bob Beck's BSDCan talk on the first 30 days of LibreSSL (the start of code flensing)
The default TLS library since [OpenBSD 5.6](https://www.openbsd.org/56.html). Offers a **-portable** variant for porting.
In my experience, *"Just Works™"*, likely a healthier alternative than its predecessor.
# This was my list of life improving OpenBSD events - I'd love to hear yours
As I warned earlier, this has been about *my* personal list of OpenBSD events that I remember fondly.
I am sure *your* list is at least a little different.
I am sure there are things from the [innovations](https://www.openbsd.org/innovations.html) page that I have simply forgotten about.
I would love to hear about your favorite [OpenBSD](https://www.openbsd.org/) moments.
# More items for your OpenBSD reading
[www.openbsd.org](https://www.openbsd.org/) The official OpenBSD website – to donate: [www.openbsd.org/donations.html](https://www.openbsd.org/donations.html) and please do donate, corporates may prefer [www.openbsdfoundation.org](https://www.openbsdfoundation.org/) - a Canadian non-profit
[undeadly.org](https://undeadly.org/) - The OpenBSD Journal news site
[bsdly.blogspot.com](https://bsdly.blogspot.com/) - My rant^H^H^H^Hblog posts
[flak.tedunangst.com/](https://flak.tedunangst.com/) tedu@ on developments
Michael W Lucas: [Absolute OpenBSD, 2nd ed.](https://www.nostarch.com/openbsd2e)
Peter N. M. Hansteen: [The Book of PF, 3rd ed.](https://www.nostarch.com/pf3)
Henning Brauer: [OpenBSD sucks (… least)](https://quigon.bsws.de/papers/2015/eurobsdcon/)
This presentation: [home.nuug.no/~peter/openbsd__moments_nuug/](https://home.nuug.no/~peter/openbsd__moments_nuug/)
[Recent and not so recent changes in OpenBSD that make life better (and may turn up elsewhere too)](https://bsdly.blogspot.com/2021/08/recent-and-not-so-recent-changes-in.html) - full text article
## Meld deg inn i NUUG [her](https://www.nuug.no/innmelding.shtml)