This morning I found one of our Puppet installations in a state of total fail. Specifically:
May 25 16:22:47 www-app01 puppet-agent[7779]: Could not send report: Error 400 on SERVER: no 'environments' in {:rootenvironment=>#<Puppet::Node::Environment:0x7fb712b7e780 @manifest="/", @modulepath=[], @name=:"root", @watching=true, @configversion=nil>, :currentenvironment=>#<Puppet::Node::Environment:0x7fb712b7e780 @manifest="/", @modulepath=[], @name=:"root", @watching=true, @configversion=nil>} at top of [[0, nil, nil]]
A few things stood out immediately. How did a working Puppet installation go completely belly up after running without problems for so long? Apparently Puppet updated itself on May 23rd from 3.6.0 to 3.6.1 – even without any configuration in a module/class instructing it to do so (note to self, look into this later).
After banging my head against Google search for a while I noticed a new directory $configdir/environments. According to the Puppet documentation you can enable environments in one of two ways, but you still needed to make configuration changes. As an experiment I renamed the "environments" directory and Puppet started processing again.
This is just another in a long, long series of issues with Puppet that are driving me away from the platform. I get the move fast and break things approach to development – and I want you to do so – just don't completely break things on minor releases.
