Deploying Puppet Open Source

In this guide we will go over best practices to deploy Puppet Open Source using the recommended workflow (r10k), PuppetDB, and the foreman. You can deploy Puppet server on any of their supported *nix distributions. In this tutorial we will assume it to be on CentOS 7 as this seems to have the best support. Continue reading Deploying Puppet Open Source

Hobby vs responsibility

One of my hobbies is video gaming. In my teen years, I would spend on average of 60-80 hours a week playing either Counter Strike, Team Fortress 2, or any other Valve software title. It was not until I went to college and got a job when my average went to 30-40 hours a week. After being married and having children, my average is now down to 8-12 hours a week. As my responsibilities increased, my hobby time decreased. I would consider myself addicted to video games (i.e. I cannot stop playing them); however, I do not feel like I am missing out from partaking in my hobby. Here is what I have done to lower my addiction to video games:

  1. I stopped playing multiplayer FPS
  2. I only buy video games that run on Linux
  3. I play during specific time frames only

My current favorite game is Factorio – you crash landed on a distant planet and have to build a rocket to escape.

Repercussions from a 1.1 Tbsp DDoS

In case you missed it, the largest recorded Direct Denial of Service (DDoS) occurred. While under DDoS, a victim’s server (or servers) is under high load and cannot complete all requests that are requested by it. Basically, a DDoS victim is someone the attacker wants silenced on the internet. In order to send a DDoS of that magnitude, the attacker has to have control over many computers – a botnet. It is believed that this attack originated from over 150,000 computers in the IoT category (smart TVs, refrigerators, thermostats, etc.). Due to their poor default security, the IoT devices are easy targets for hackers who intend on adding them to their botnets. A recent article on Ars Technica points out the current issues with IoT and Linux kernel security, but with most articles of this nature, provides no clear cut solution to the problem we are experiencing. Below are my thoughts to this current situation and how it may be resolved.

We need a governing body to issue a seal of approval for IoT and anything that is compiled with the Linux kernel. Then we, as consumers, must use, buy, and encourage others to buy from the companies that have this seal. The governing body should ensure each company seeking the seal comply with the following criteria:

  1. Every new device created and sent to market has a minimum of 5 years worth of bi-monthly security patches and updates since the day of release to the public.
  2. In the event the company goes bankrupt, dissolves, or cannot support any older product they have released in the past 5 years, the company must provide schematics, instructions, or software that open source enthusiasts can recreate, patch, or upgrade the legacy product.
  3. No known vulnerability must be willingly left unpatched.
  4. When a CVE is identified on a company’s product, a test case must be created and run on that code base for every future release.
  5. A notification service must be in place when new updates are released and must be available in RSS or email form.
  6. Automatic updates should occur over HTTPS
  7. Backdoors, admin terminals, etc. should require a physical connector be applied on the device in order to grant access.

    For a potential company to get this approval, it may seem like an arduous task to get all the controls in place; however, by applying DevOps methodologies, these tasks can be a simple feat. This would require the governing body to not only enforce the list, but also have the training available to comply to this list. For this reason, I suggest the Linux Foundation to become this governing body and issue out seals of approval.

    First puppet module published

    I completed my first public module for puppet and submitted it to the puppet forge. It seems too simple to compile into a build and submit it to the forge; however, I made it public for these reasons:

    1. I needed experience with puppet code testing. This helped me at the most basic level.
    2. I felt like someone else could benefit from the code – even if it is one person.
    3. I wanted to do it.

    Still, the code seems too juvenile to be submitted to the forge. All it does is take the hostname of a Digital Ocean droplet and submit its IP address as a new DNS record inside of Digital Ocean DNS. The code is located here.

    I almost want to follow up with this and develop my duplicity module into reusable code for the community.

    Signs you are doing IT wrong

    1. You still use FTP
    2. You use SFTP
    3. You have a single server hosting 1 website, MySQL, and PHP. It has 4+ GB of RAM and you only have ~2,000 visitors a day.
    4. You login via root
    5. You don’t use version control
    6. You use a control panel for servers which you have SSH access.
    7. It takes you over an hour to migrate 1 website
    8. Your DNS TTL records are over 10 minutes
    9. Your SQL server is not accessible over SSL/TLS
    10. You use mod_php instead of reverse proxying to php-fpm
    11. You develop for the web on Windows
    12. You chmod 777
    13. You use modules/plugins that require chmod 777
    14. You have no backups
    15. You host multiple websites on one server (internal-only websites excluded)
    16. You SSH with passwords
    17. You reuse passwords
    18. You don’t read books
    19. You don’t attend conferences
    20. You attend more than 6 conferences a year
    21. You use skype for communication
    22. You make a separate mobile site
    23. You add more RAM to fix your memory leaks

    Iced coffee is the best

    I am not a very big fan of hot drinks, but I enjoy drinking a cup/glass/thermos/pot/gallon of coffee. I especially drink it more when my taste buds dance around and say, “Wow! That was some good, quality coffee!” A few weeks ago I set out to find a better way to make my favorite drink – iced coffee. In my opinion, the best method of procuring coffee is in whole bean form. I tend to buy a brand that is roasted in my region – supporting the local economy – that also tastes good. I store the whole bean bag in my freezer and the grounded bean in a small coffee can in my refrigerator.

    At first, I tried pouring hot coffee over Frozen coffee cubes, then added my refrigerated creamer. This lasted for a few weeks, but I couldn’t notice a huge difference in taste between water iced cubes and coffee iced cubes.

    Secondly, I tried cold brewing coffee – placing ground coffee beans in cold water into the refrigerator overnight. This only resulted in weak, flavorless coffee.

    Next, I tried hot brewing coffee, pouring it into a container, and letting it sit in the refrigerator overnight. This seems to be the best option so far. I still get to keep my 1.5 tbsp ratio for coffee beans and resulting liquid. The iced cubes do not melt when the coffee is poured over them. I think I will stick to this option for now.

    My IoT device history

    The internet of things (IoT) is getting pretty saturated with devices most of which are either smart watches or activity trackers. Smart watches do not appeal to me as I have a very nasty habit of destroying the clock face of my watches. Last November, I was able to get a Vivosmart from Garmin for $60 plus tax and shipping. It was great – did step tracking, allowed for notifications, allowed me to dismiss calls and see texts. My brother-in-law also received one as a gift a few months later (he preferred it over the Fitbit which did less and cost more).

    A month ago we both noticed the pixels disappearing on the Garmin Vivosmart display. I was able to submit a warranty request through their website quite painlessly, and the offered to upgrade me to a Garmin Vivosmart HR! Of course I took the upgrade offer and paid the shipping for the old device to be sent back. My brother-in-law had a complete different experience. The website at first said his Garmin Vivosmart was out of warranty (even though it was newer than mine) then it eventually – a day later – said it was in warranty. He was given the option to replace the Garmin Vivosmart with a non-HR model, but they gave him a shipping label. It is quite odd that we both had different experiences within a few days of submitting our warranty requests.

    I recently received the newer model after waiting the RMA process and I am quite impressed. At first, the font was too skinny and hard to read, but all I had to do was upgrade to the latest firmware and it was fixed. One neat feature that was added (besides the obvious HR function) was the ability to see the weather – up to a 4 day forecast. Two new goal trackers were also added – a stair counter and strenuous activity counter. The plain Garmin Vivosmart’s battery lasted over a week. I haven’t depleted the new model all the way, but I am assuming it will last 4-5 days depending on my use.

    Provisioning VMs with cloud init

    One of the easiest ways to deploy a virtual machine in oVirt is first to install the OS then turn it into a template. This will allow you to copy that template to deploy new instances. One mundane task after a new template is copied to a new instance is logging in, changing the IP, setting the hostname, setting up Puppet, running puppet, etc. cloud-init is the tool designed to fix that mundane task process by allowing those steps to be automated. oVirt/RHEV (as well as OpenStack, AWS, and others) allow you to pass in user data which is then supplied to cloud-init after the template is copied over and turned on. This allows for scripting on the new VM – easing deployment.

    For my environment, I wanted a CentOS 7 template. To have that, I must first install CentOS on a new VM and seal it (Windows calls this Sysprep). Before I seal it, I must install cloud-init and any other tools I might use for deployment – such as puppet. Here are the steps to obtain just that:

    Continue reading Provisioning VMs with cloud init

    Securing PWM

    In last week’s post we set up PWM insecurely. In this post, we are going to secure it down and install mysql to store the reset questions. This guide assumes you have this CentOS 7 server publicly accessible with ports 80 and 443 available to the entire world. First, we will need to install mysql, set up a database, and add a user to that database. To do that, we need to edit our manifest.pp and append the following:

     class { '::mysql::server':
         root_password           => 'My4cc0unt$$password!',
         remove_default_accounts => true,
         package_name            => 'mariadb-server',
         package_ensure          => 'installed',
         service_name            => 'mariadb',
     mysql::db { 'pwm':
         user     => 'pwm',
         password => 'pwm_passworD2!', # Can't do a password hash here :(
     class { 'mysql::bindings':
         java_enable => true,
    file { '/opt/tomcat8/pwm/lib/mysql-connector-java.jar':
         ensure  => link,
         target  => '/usr/share/java/mysql-connector-java.jar',
         require => Class['mysql::bindings']

    We will also need to install additional modules: Continue reading Securing PWM

    Password management portal for end users

    We in IT have heard it often, the #1 request coming into help desk ticket systems is password resets, account lockouts, and the like. PWM is a password reset web application written in Java for use with LDAP directories. You can configure it to work with Active Directory, OpenLDAP, FreeIPA, and others. There are already a handful of good tutorials on how to set up PWM (I think of this one in particular); however, I want to demonstrate the puppet apply command in this tutorial.


    This guide assumes you have an Active Directory server with TLS set up (to change passwords) which is beyond the scope of this post. It also assumes you have a CentOS 7 instance which can communicate to the Active Directory server. It also assumes this is in an environment without a puppet master/server. The end manifest can be uploaded to a master and used that way.

    Continue reading Password management portal for end users