Puppet 4 on Debian 9 Stretch with nginx (with Puppet 3 agents if needed)
As part of upgrading my machines from Jessie to Stretch, I finally had to pick up Puppet 4. My hosts running testing were trying to do so for a while already, but since Puppet requires the master to be newer than the agents, I've always just had a pinning rule in place to stick all machines to Puppet 3.x.
It's been quite the operation and I'm not done yet, but let me write down some of my findings for others to maybe use. As always, there are many different ways to achieve this goal, there are existing docs, but they're all outdated in one way or another. (As surely this one will be in a year.)
It's been quite the operation and I'm not done yet, but let me write down some of my findings for others to maybe use. As always, there are many different ways to achieve this goal, there are existing docs, but they're all outdated in one way or another. (As surely this one will be in a year.)
During the upgrade, one of the first issues I ran into was that Puppet 3 agents are simply not compatible with Puppet 4 masters. When you try to have them connect, you'll get a poorly formatted error message that looks more or less like:
(And another fun note: Puppet Master inside Passenger, too, is deprecated in v5, which kind of summarises my whole experience with Puppet TBH.)
Anyway, for getting Puppet+passenger to work, there's the puppet-master-passenger package. Which I didn't want to use though, because I've worked reasonably hard about a year ago to migrate away from Apache.
Screw backward compatibility, just give me Puppet+nginx+Passenger!
So, how to do this the, seemingly to me, right way? I've looked at using the Passenger packets from Debian, but there seems to be no nginx module. I tried out the existing passenger package which offered to recompile nginx for me, at which point I most definitely lost interest. :-/ But Phusion's official repository has a proper libnginx-mod-http-passenger package! I'm not a big fan of third-party repos, but when I only need it on my master and not on the entire fleet, I'll survive.
Here's most of my work, described in Puppet DSL (mind you, not directly reusable for others). tl;dr on it: Adds the repo, pinned to be used only for packages with "passenger" somewhere in the name, installs the packages from it that I need, plus nginx (that one from Debian proper) with only the Puppet master service enabled using a snippet like this.
Only thing left to do is the file used as Passenger glue, and this is where the existing written instructions fall short, and where the Debian package wins: The default file is pretty bare whereas the Debian one contains a few tricks to rewrite Puppet 3 agent requests into something your Puppet 4 master will understand!
So what I did: I've fetched the puppet-master-passenger package, but instead of installing it (which pulls in Apache, etc.), I've taken out only the /usr/share/puppet/rack/puppet-master portion, and put that into /etc/puppet/rack. (This is where apt install -d and dpkg --extract come in handy.)
One last thing here that is mentioned but not explained in other docs, yet very important: Ensure config.ru is owned by GID+UID puppet. As it turns out, that's the only thing instructing the Passenger framework what user to run the scripts as. I suppose implicit setuid behaviour like this is what the Ruby on Rails crowd considers elegant... :-)
Anyway, that's all I needed to do to get the Puppet 3 agents to talk to the new master again. However, there are other known incompatibilities, expect syntax errors or unwanted changes if you're not careful! Besides what's listed on that page, I ran into the issue of File objects that used to get created owned as root by default, now inherit the numeric UID+GID from the Puppet master. This was easily fixed with the following snippet in the site manifest (which is also where I learned one can get global defaults for object types this way!):
Warning: Find /production/node/.... resulted in 404 with the message: {"message":"Not Found: Error: Invalid URL - Puppet expects requests that conform to the /puppet and /puppet-ca APIs.\n\nNote that Puppet 3 agents aren't compatible with this version; if you're running Puppet 3, you must either upgrade your agents to match the server or point them to a server running Puppet 3.\n\nMaster Info:\n Puppet version: 4.8.2\n Supported /puppet API versions: v3\n Supported /puppet-ca API versions: v1","issue_kind":"HANDLER_NOT_FOUND"}As helpfully pointed out by someone on this ticket, this is actually just a generic 404 message that kind of assumes you're the wrong Puppet version talking to this master. Now the Debian NEWS file tells you that for Puppet 3 agents to work, you need to switch from the old stand-alone webrick master to Puppet master running under Passenger (kinda Ruby's equivalent of Python's WSGI?) One important note here is: You won't find this information in official Puppet docs, because this compatibility is actually a Debian-specific patch!(And another fun note: Puppet Master inside Passenger, too, is deprecated in v5, which kind of summarises my whole experience with Puppet TBH.)
Anyway, for getting Puppet+passenger to work, there's the puppet-master-passenger package. Which I didn't want to use though, because I've worked reasonably hard about a year ago to migrate away from Apache.
Screw backward compatibility, just give me Puppet+nginx+Passenger!
So, how to do this the, seemingly to me, right way? I've looked at using the Passenger packets from Debian, but there seems to be no nginx module. I tried out the existing passenger package which offered to recompile nginx for me, at which point I most definitely lost interest. :-/ But Phusion's official repository has a proper libnginx-mod-http-passenger package! I'm not a big fan of third-party repos, but when I only need it on my master and not on the entire fleet, I'll survive.
Here's most of my work, described in Puppet DSL (mind you, not directly reusable for others). tl;dr on it: Adds the repo, pinned to be used only for packages with "passenger" somewhere in the name, installs the packages from it that I need, plus nginx (that one from Debian proper) with only the Puppet master service enabled using a snippet like this.
Only thing left to do is the file used as Passenger glue, and this is where the existing written instructions fall short, and where the Debian package wins: The default file is pretty bare whereas the Debian one contains a few tricks to rewrite Puppet 3 agent requests into something your Puppet 4 master will understand!
So what I did: I've fetched the puppet-master-passenger package, but instead of installing it (which pulls in Apache, etc.), I've taken out only the /usr/share/puppet/rack/puppet-master portion, and put that into /etc/puppet/rack. (This is where apt install -d and dpkg --extract come in handy.)
One last thing here that is mentioned but not explained in other docs, yet very important: Ensure config.ru is owned by GID+UID puppet. As it turns out, that's the only thing instructing the Passenger framework what user to run the scripts as. I suppose implicit setuid behaviour like this is what the Ruby on Rails crowd considers elegant... :-)
Anyway, that's all I needed to do to get the Puppet 3 agents to talk to the new master again. However, there are other known incompatibilities, expect syntax errors or unwanted changes if you're not careful! Besides what's listed on that page, I ran into the issue of File objects that used to get created owned as root by default, now inherit the numeric UID+GID from the Puppet master. This was easily fixed with the following snippet in the site manifest (which is also where I learned one can get global defaults for object types this way!):
# Sets the default for Puppet 3 agents. Should be safe to remove when
# they're gone.
# https://docs.puppet.com/puppet/3.8/deprecated_resource.html#default-copying-of-source-permissions
File {
source_permissions => ignore,
}
That's pretty much it, I think. Many thanks to Apollon Oikonomopoulos in this thread for reminding me to use the right config.ru file. To those who dislike third party repositories, Georg Faerber's response on using uwsgi may also be interesting!
Trackbacks
No Trackbacks
Comments
Display comments as Linear | Threaded