We’re without Al again this episode, but we carry on regardless!

Stu talks about Puppet which is a configuration management system, comparable to Ansible, Salt Stack or Chef.

Like Chef and Salt, Puppet is predominently agent based, where the agent is installed on the endpoint, and it calls out to a central server, every X period of time (Jerry mentions 30 minutes at one point in the show, while Stu says 15 minutes) to get the state the device should be in, and it then tries to remediate all those items which are not compliant with the state.

Puppet is more like Chef than Ansible or Salt in that it uses a Ruby “Domain Specific Language” (or DSL) to define the configuration of the node, rather than YAML.

We then get into a more general conversation about configuration management software, including talking about how Salt Stack allows you to create entire tasks and variables using jinja2 templates, and Jon mentions he did something like this with Ansible variables. Jon mentions seeing a video from an early PuppetConf where a member of the board (he thinks the CTO) decided to learn Puppet by wiping and reinstalling his machine every day using Puppet. Sadly, he can’t find this video now, and would appreciate listeners pointing him to that video, if they can find it!

Jon talks about Architecture Decision Records (or “All” Decision Records) writing bash scripts, and using BATS to perform unit testing of bash files. He also mentions that it’s possible to “mock” specific commands in BATS.

Lastly, Stu proposes we talk at about using Cloud Native services in AWS, Azure, etc. versus using Infrastructure as a Service. A series of specific services on AWS and Azure are mentioned. We talk about how vendor-lock-in can occur and some of the things you can do to help prevent that. Jon mentions the books “The Phoenix Project” and “The Unicorn Project” by Gene Kim which discuss the idea of “Core” services (which make money for the company or project) and “Context” services (which don’t, and can be outsourced.) We also talk about the issues involved in not transforming your services when you “Lift and Shift” services into a cloud service.

We’re a member of the Other Side Podcast Network. The lovely Dave Lee does our Audio Production.

We want to remind our listeners that we have a Telegram channel and email address if you want to contact the hosts. We also have Patreon, if you’re interested in supporting the show. Details can all be found on our Contact Us page.

First episode of 2021! We’re in lockdown number 3 in England!

Jon admits to writing a private-only diary using WordPress (he doesn’t mention he also has a separate photo diary). Jerry mentions that another of his friends also has recently started a diary using WordPress, and suggests that maybe this is a new trend.

Jon is also Internet Famous due to a post he made on StatusNet in 2009 (mirrored to twitter) that got captured in the screenshot of a StatusNet client and posted to Wikipedia.

Jon wrote a post on his blog talking about how he got into his career. He would encourage anyone else to write something similar, particularly if they’ve taken an unusual route into their career!

Al asks the team what he should learn about. He talks about the tooling they’re using – Bamboo, Azure DevOps, Terraform, Ansible. We talk a bit about what Bamboo is, what a code pipeline entails, and how they’ve used it. Jon mentions that Lorna Jane Mitchell talks about moving from Travis to Github Actions on her Twitch Stream. We then drill into using Terraform modules.

Jon mentions about “Architecture Decision Records” and cites files in the gov.uk public repo as an example of this. It’s similar in principle to IETF RFCs. He found it via the Last Week in AWS newsletter issue 195 (which at the time of writing was only available to subscribers). He mentions the tooling (“adr-tools“) which you can use to write these records.

Al then asks where we find time to learn. We all talk about what we do, some at more length than others.

Al talks about being OK about being alone. He mentions about his life coach, the “Alonement” podcast, and the talk he gave at OggCamp about staying positive on a digital world.

Jon then reminds our listeners to check in with family, friends, colleagues and neighbours to make sure they’re OK.

In this bumper feedback episode we talk about line endings in files, OpenStack, secrets management, and protecting your network.

• Iain asks:

  Hello all,

  I hear a lot about Openstack but whenever I try to find out stuff about it, I get vague buzzword-laden and vague comments from “evangelists”. Could any of you guys explain to an IT-literate but not a sysadmin end user, what the hell Openstack is?

  Stu explains what OpenStack is built for, where it’s often deployed these days, and some of the issues he’s seen with it.

  Jon talks about some of the components inside OpenStack, and how OpenStack upgrades can have issues.

  Jon and Stu talk about companies who were selling OpenStack distributions, and comparisons to Kubernetes.

  Jerry mentions that many of the problems OpenStack was created to solve are now mostly solved by Kubernetes. He also mentions that we discussed Kubernetes in Episode 51.

  Jon mentions Eucalyptus, nominally as an alternative to using AWS S3 or OpenStack Swift (the object storage module), but also mentions it could be used to virtualize some of the other services provided by AWS.

• Al asked about “Dark Matter Engineering” which he’s heard about on Coder Radio. We presume it’s code that isn’t released into the public, or never gets any traction. We also discuss Linode and compare it to Digital Ocean as a result of the adverts run on Coder Radio.

• Jay provides some feedback:

  Hi, in your last podcast someone mentioned having an issue with VSCode in windows always saying that files were all edited.

  What’s probably going on is a wrong setting for the core.autocrlf setting.

  You can fix it by opening powershell in windows and running

  git config --global core.autocrlf input

  https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration#_formatting_and_whitespace

  There are 3 settings, but I always recommend the ‘input’ one, as it converts everything to LF endings on commit, and checks out without modification.

  Also, you may be interested in a recent networking video series I made: https://jaytuckey.name/2020/10/18/how-websites-load-a-deep-dive-into-the-ip-network-stack-and-how-it-is-used-to-connect-to-a-site/

  Jon talks about how he’s got Microsoft’s Windows Subsystem for Linux (WSL) setup and how he organizes his “My Documents” directory structure. He mentions “Symbolic Links” to easily find Windows directories in his WSL environment.

  We also talk about Line Endings (CR – Carriage Return, LF – Line Feed, CR+LF). Jon incorrectly recalls CR as ASCII character 10 (it should be 13) and LF as ASCII character 13 (it should be 10).

• WIE E asks for help with Secrets Management in a Continuous Integration (CI)/Continuous Delivery (CD)/Continuous Deployment (CD) environment.

  Stu talks about Gitlab, HashiCorp Vault, and AWS IAM roles, which Jerry extends to include Azure System Assigned Identities. Jerry mentions that you can use your provisioning system to create a per-system key during a build, which never commit to your version control system.

  Jon mentions about protecting CI/CD/CD systems and references the exploit of a CI/CD system on the Matrix.org project.

• Yannick asks:

  VPN: always on or not?

  How to protect the target network – i.e. does my machine becomes the weakest link in the network and what can/should I do to protect the network ?

  Jon talks about his views on always-on Client-to-Server Virtual Private Network (VPN) connections.

  Al mentions that he thought the question was talking about Site-to-Site VPNs, and Jon suggests that VPNs typically now auto-establish themselves when traffic is initiated from the “Encryption Domain” on one side of the network to the “Encryption Domain” on the other side of the network. Jon refers to IPsec Phase 1 and Phase 2 which are two stages of a VPN tunnel, dealing with the initial connection between the “left” and “right” sides of a VPN tunnel, and the connections between two encryption domains (subnets or hosts at either end of the tunnel). Jon also mentions about various encryption algorithms like DES, Triple DES, AES, and hashing algorithms like SHA1.

  Jerry quotes “Clarke’s Third Law“: Any sufficiently advanced technology is indistinguishable from magic.

  Jon mentions about the Diffie Hellman Key Exchange video, and then talks about browse-down management environments and references the National Cyber Security Centre (NCSC) End User Device security guidance for hardening machines. He also talks about segregating network segments for protecting trusted and untrusted networks, and then goes into “Zero Trust” networks, mentions “CASB“. Jon and Stu both talk about broadcast domains in a network, and how you can work around that.

  Jerry mentions about Bastion Hosts, and Jon explains why they’re not really a good idea.

  Jon butchers talking about User Behaviour Analytics (UBA) systems. He also mentions about a protocol break.

Wrap up

We’re a member of the Other Side Podcast Network. The lovely Dave Lee does our Audio Production.

We want to remind our listeners that we have a Telegram channel and email address if you want to contact the hosts. We also have Patreon, if you’re interested in supporting the show. Details can all be found on our Contact Us page.

The whole crew is back together for the first time in a while, talking about: Git commit hooks, Windows as a development environment, cloud network firewalls, and Azure DevOps.

What have we been up to?

• Jerry started a new job, and he’s re-started using Windows 10 as part of his job after a few years of using OSX and Linux. He’s using Windows Subsystem for Linux (WSL). We talk a bit about the difference between WSL 1 and WSL 2, and comment about how WSL 2 uses Hyper-V and what that means compared to using VirtualBox or VMWare for virtual machines. We compare WSL 1 to a “reverse” WINE.
• Stu mentions that you can’t run some network controls (like traceroute) with WSL 1 because some of the kernel calls are not available.
• Jon notes that he used the terraform for Windows binary in WSL by mistake and couldn’t authenticate to AWS because he’d installed the AWS CLI for Linux. Installing Terraform for Linux and also putting the AWS configuration files into both the Linux expected path (~/.aws/config) and the Windows expected path (%UserProfile%\.aws) worked around this issue!
• Jon then mentions using /etc/wsl.conf to configure mounting the Windows drives into WSL, and notes that you can configure it to permit POSIX style file permissions with this sample:

[automount]
options = "metadata"

• Jon also creates a symbolic link between /mnt/c/Users/Jon/Documents and ~/Documents to “easily” get into the Windows paths that are backed up in Windows.
• Jerry notes that he’s using the Windows version of Virtual Desktops. He’s also using the Microsoft Terminal application.
• Al mentions that if you navigate to \\wsl$ in Windows Explorer, you can access the Linux file system from Windows Subsystem for Linux. Stu mentions he has this open in the left hand pane in Explorer all the time!
• Al said he’s using Visual Studio Code (VSCode), and uses that to open Windows Subsystem for Linux, and he also mentioned that if you type in code in any path in WSL (or Command Prompt, for that matter [ed.]) it will open that folder in Visual Studio Code.
• Jerry notes that he’s just moved to using VSCode, but has installed the Vim extension. Jon asks whether he’s installed the “Butterflies” extension, referring to an XKCD comic.
• We talk briefly about using Git in VSCode, versus using it from the command line. Jon mentions a specific bug he has. Jon talks about the differences in line endings between Linux and Windows systems.
• Al talks about using Azure DevOps with it’s pipelines. We talk about it’s history, and compare it to other products. Al mentions using Azure DevOps to trigger Terraform using PowerShell. Al also mentions using AWX (the open source upstream version of Ansible Tower), and having an agent for Azure Devops running on his AWX service.
• Jon mentions the DevOps.fm podcast in the context of Azure DevOps. Stu asks about running PowerShell on Linux. Jerry mentions a Binary Times podcast episode where they interview the person, “dementor”/”the mentor” who runs the Powershell On Linux, Al mentions the Makers Corner podcast which also interviewed the same person.
• Jon is writing Terraform to deliver 3rd party security appliances in AWS and Azure. He notes that most of the AWS appliances use a Transit Gateway to set this up. Jerry and Stu mention how they use Terraform Modules. Jerry mentions automating Jenkins with Terraform. Stu and Jon talk about using count and for each statements. Jon also mentions about defining which “providers” to use in the Terraform files. He also notes that you can get into a dependency loop if you have several modules with different provider files. Talking of Providers, Jon mentions using the “null” provider, but doesn’t explain what he uses it for.
• Jon talks about Git hooks, and Jerry talks about a python project called pre-commit which can help to automate some of these pre-commit hooks, like calling a linter or a unit testing system (like ShellCheck) before the commit completes. Stu mentions using the GitLab Continious Integration (CI) system instead of using Pre-Commit hooks. Jon suggests when it might be preferable to use Pre-Commit hooks instead.
• Stu mentions about SourceHut, which is an alternative to GitHub which uses email for patch sharing.
• Al talks about using the Azure Firewall product, and Jon and Al drills down into how Azure Networking works. Jon then explains how High Availability events occur in AWS and Azure with 3rd party firewalls. Jon also mentions AWS Firewall Manager. Jon also mentions that Terraform and Ansible have code to write and amend AWS and Azure Firewall rules.

Feedback

• We received feedback (although the source has now been lost) about how we pronounced “Oriented” as “Orientated”. These are both valid words in UK English and are interchangable in UK English.

Wrap up

We’re a member of the Other Side Podcast Network. The lovely Dave Lee does our Audio Production.

We want to remind our listeners that we have a Telegram channel and email address if you want to contact the hosts. We also have Patreon, if you’re interested in supporting the show. Details can all be found on our Contact Us page.

No Jerry this time, but we do have Al back!

What have we been up to?

• Jon mentions lug.org.uk and how he’s redeveloping it.
• Stu mentions working with Gitlab Pipelines, compares it to Jenkins, and Al compares it to Azure DevOps (which is the modern, cloud hosted, version of Team Foundation Service, abbreviated to TFS).
• Al mentions that he got Terraform working in Azure DevOps, and using a remote state file.
• Jon mentions that we’ve had feedback that mentions that we’re not explaining all the terms we use.
• Al talks about how he’s looking at executing Ansible playbooks when Windows machines are added to Active Directory. Jon mentions that a Dynamic Inventory module for Ansible is basically just a python script that returns a JSON value, so you could have a python script contact Active Directory and ask for a list of machines in an OU. Jon also mentions about using Azure’s Virtual Machine Inventory to get a list of machines, or alternatively, using Terraform to trigger an action in AWX (Ansible Tower‘s upstream open source project) to provision that machine.

Feedback

• Wayne (from the Binary Times podcast) contacted us to suggest that we’re not being very good at explaining what terms mean. We try to clear some of the terms up that we use!

Techniques for Rebuilding a machine using Post-Provisioning Tools (like Ansible)

• Jon suggests a process of using Git on /etc/ and /home/<user>/ with Vagrant to test each stage of the build, and to see what files are changed by each action performed. Once you’ve got your build instructions based on that, you can use something like Ansible, Salt, Puppet or Chef to apply pos-install statements.
• Stu mentions using Chocolatey for installing packages in Windows. Jon mentions that using BoxStarter works well for automating Chocolately installs. He mentions using boxstarter paths which are currently not documented – https://boxstarter.org/package/<yourpackage> and https://boxstarter.org/package/url/?some_path_to_a_boxstarter_set_of_instructions.
• Jon mentions the Ubuntu Server Vagrant Box file, and the Desktopify script written by Martin Wimpress. He also talks about provisioning Windows machines where Terraform renames machines and adds them to the Active Directory Domain.

Object Orientated Programming

• Al wanted to know more about what Object Orientated Programming (OOP) is, as he’s started looking at ASP.NET and has previously only known Classic ASP. Jon talks about it from his previous PHP experience and perspective. Jon talks about when he used OOP in a previous role to perform network device backups. Jon mentions he’d used OOP in CCHits.net and the now defunct CampFireManager.

Wrap up

We’re a member of the Other Side Podcast Network. The lovely Dave Lee does our Audio Production.

We want to remind our listeners that we have a Telegram channel and email address if you want to contact the hosts. We also have Patreon, if you’re interested in supporting the show. Details can all be found on our Contact Us page.

• Al couldn’t make it for this recording
• Jon broke his QNAP NAS with Debian. He doesn’t go into any details, and will leave it to another show (as long as he can remember what he did until then)!
• Stu has been blogging, now on Consul, Saltstack and Prometheus on LOTS of different platforms.
• Jerry started a new job. The CTO at his new firm is possibly a listener! His NAS has failed, and he’s building a ZFS mirror to move the data to. He used Syncthing to move the data off to another drive, and is using Backblaze to run a backup.
• We discuss possible other uses of Syncthing, in particular, one YouTube creator talks about how he uses SyncThing for his video editing workflow (part 2).
• Jon also mentions that he’s been watching some of Martin Wimpress’ YouTube channel, and in particular, the series where he created the Desktopify script for turning a Raspberry Pi Ubuntu Server image into a Desktop Flavour.
• We talk about “Hashistack” (referring to the collection of tools released by Hashicorp, which are “Terraform“, “Packer“, “Consul“, “Vagrant“, “Vault” and “Nomad“). All the hosts provide summaries of how each of these tools work (except Nomad) and why you might use them.
  • In the exit for the show, Jon throws in a note about Martin Wimpress‘ alternative to Vagrant using Bash .
• We discuss using Git.
  • Jon talks about DangItGit (and a slightly more rudely named version of that site), and mentions a comic on XKCD about Git.
  • Then he mentions some of the things about git which may give it a bit of a bad name, like SubModules.
  • He also mentions that you can use “hooks” which are scripts that run before or after certain actions (like a commit or a push), and Stu talks about how he’s used that in the past.
  • Stu also talks about some of the tools in Github and Gitlab which are similar to hooks, that run when Github or Gitlab actions occur (like a pull/merge request being raised, or an issue being opened).
  • Stu also talks about CI/CD pipelines and Gitlab runners.
  • Stu and Jon talk about Pull Requests (Github terminology) and Merge Requests (Gitlab terminology). We all talk about Issues and Wikis within Github and Gitlab.
  • Jon talks about what a Fork is.
  • Stu reminds us that Git is not the only version control system, and that Subversion (SVN) is also out there. Jerry mentions CVS. Jon mentions Mercurial (HG) and we talk about where version control systems have come from.
  • We diverge into how Blockchain is similar to Git… and why it isn’t the same.

We’re a member of the Other Side Podcast Network. The lovely Dave Lee does our Audio Production.

We want to remind our listeners that we have a Telegram channel and email address if you want to contact the hosts. We also have Patreon, if you’re interested in supporting the show. Details can all be found on our Contact Us page.