Author: admin
How to call a json rest API using Ansible
So a very useful thing to understand is rest api’s and how to call them as a lot of organisations have these and want to integrate them into automation, a popular method is the http method
They are very simple calls { GET, POST, PUT, DELETE, PATCH }
For the sake of this post. Im going to use commvault public api’s https://api.commvault.com/
You will need to two things.
- The api endpoint which is usually an http url
Example:
- The raw json body of the of the api
Example:
{
"csFailoverConfigInfo": {
"configStatus": 0,
"isAutomaticFailoverEnabled": false
}
}
Now keep in mind if you are using an api that requires a login. In order for it to work, you will need to store the auth token to pass later to the last task later for the api call to work as intended. You can look at one of my other posts under vmware, where i used a http login to handle the tasks later, as a reference.
You can call these preliminary task as includes to store the token.
It will look something like this before it gets to the api task. You can also just do it all one on book if you wanted to. But for the purposes of this post. Im just giving ya highlevel.
- name: Login task include_role: name: commvault_login tasks_from: login.yml - name: Setfact for authtoke set_fact: authtoken: "{{ login_authtoken }}" delegate_to: localhost
Now in order for you to pass json api to ansible. You will need to convert the json raw body into yaml format. You can use visual studio code plugins or a site like https://json2yaml.com/
So if we are to use the above raw json example it would look like this
csFailoverConfigInfo:
configStatus: 0
isAutomaticFailoverEnabled: false
So now we want to pass this information to the task in the form of a variable. A really cool thing with ansible and this type of action. Is you can create a variable name and simply pass the new yaml converted body right below the varible. You can pass this as extra-vars or create a group variable with the same name and use that.
For those you who use tower passing them as extra-vars to test something can be a pain, since it doesn’t allow you to change the passed vars and rerun the previous run just used, you have to start all over. So I prefer the command line way as its easier to be agile
disable_api_body:
csFailoverConfigInfo:
configStatus: 0
isAutomaticFailoverEnabled: false
So now we ansible to use the rest api with ansible. You create a task that after the login is run and the token is stored inside as a fact. It run the following task, in our case this call will be a POST. It will post the headers to the url which will disabled commvault live_sync which is essentially commvault failover redundancy for the backup server itself.
- name: Disable Commvault livesync uri: url: http://{{ commvault_primary }}/webconsole/api/v2/CommServ/Failover method: POST body_format: json body: "{{ disable_api_body }}" return_content: true headers: Accept: application/json Content-Type: application/json Authtoken: "{{ login_authtoken }}" status_code: 200 validate_certs: false register: disable_livesync retries: "4" delay: "10" delegate_to: localhost - debug: var: disable_livesync
When you run the book and your have an active failover setup correctly with commvault. In the command center under the control panel you should see livesync. If you click on this you should see either it is checked or unchecked.
How to Deploy LVM’s with Ansible
Provisioning-LVM-Filesystems:
This role is designed to use ansible-merge-vars module. An Ansible plugin to merge all variables in context with a certain suffix (lists or dicts only) and create a new variable that contains the result of this merge. This is an Ansible action plugin, which is basically an Ansible module that runs on the machine running Ansible rather than on the host that Ansible is provisioning.
Benefits: Configuring disks into LVM
Note: This post assumes you have already ansible installed and running.
Install ansible-merge-vars module:
1. root@KVM-test-box:~# pip install ansible_merge_vars
Requirement already satisfied: ansible_merge_vars in
/usr/local/lib/python3.8/dist-packages (5.0.0)
By default, Ansible will look for action plugins in an action_plugins folder adjacent to the running playbook. For more information on this, or to change the location where ansible looks for action plugin.
from ansible_merge_vars import ActionModule
Role Setup:
Once the plugin has been setup, you now you will want to setup a role.
Now we will create a task that will merge variable names associated with a list and then itemise the list for variables we will pass to provision the filesystem via the inventory/host_var or group_var
– name: Merge VG variables
merge_vars:
suffix_to_merge: vgs__to_merge
merged_var_name: merged_vgs
expected_type: ‘list’
– name: Merge LV variables
merge_vars:
suffix_to_merge: lvs__to_merge
merged_var_name: merged_lvs
expected_type: ‘list’
– name: Merge FS variables
merge_vars:
suffix_to_merge: fs__to_merge
merged_var_name: merged_fs
expected_type: ‘list’
– name: Merge MOUNT variables
merge_vars:
suffix_to_merge: mnt__to_merge
merged_var_name: merged_mnt
expected_type: ‘list’
– name: Create VGs
lvg:
vg: “{{ item.vg }}”
pvs: “{{ item.pvs }}”
with_items: “{{ merged_vgs }}”
– name: Create LVs
lvol:
vg: “{{ item.vg }}”
lv: “{{ item.lv }}”
size: “{{ item.size }}”
pvs: “{{ item.pvs | default(omit) }}”
shrink: no
with_items: “{{ merged_lvs }}”
– name: Create FSs
filesystem:
dev: “{{ item.dev }}”
fstype: “{{ item.fstype }}”
with_items: “{{ merged_fs }}”
– name: Mount FSs
mount:
path: “{{ item.path }}”
src: “{{ item.src }}”
state: mounted
fstype: “{{ item.fstype }}”
opts: “{{ item.opts | default(‘defaults’) }}”
dump: “{{ item.dump | default(‘1’) }}”
passno: “{{ item.passno | default(‘2’) }}”
with_items: “{{ merged_mnt }}”
Note: Now this currently task has no safe guards for /dev/sda or checks to ensure the disk is wiped properly in order for the disks to be added to the volume group. I have created such safe guards for others. But for the purposes of this blog post this is basics. If you want to my help you can contact me via email or the ticketing system.
Now what we are going to do is define our inventory file with what file lvm we want to crave out.
Setup inventory:
1.Go inside your inventory/host_var or group_var file and create a file for testserver1
- .nano inventory/host_var/testserver1
2. save the file.
Definitions of the variables above:
vgs__to_merge: This section is the creation volume/physical groups
– vg: vg_vmguest (this is the volume group name)
pvs: /dev/sdb (this is the physical assigned to the above volume group
– vg: vg_sl_storage (This the second volume name)
pvs: /dev/sdc (This is the second physical disk assigned to the above
volume
*You can add as many as you like*
lvs__to_merge: This section is the logical Volume creations
– vg: vg_vmguest (this is the volume group created)
lv: lv_vg_vmguest (this is the logical volume that is attached to above vg
size: 100%FREE (this says please use the whole disk)
shrink: no (this is needed to so the disk space is used correctly)
– vg: vg_sl_storage (this is the second volume created)
lv: lv_vg_sl_storage (this is the second lvm created attached to above vg)
size: 100%FREE (this is use the whole disk)
shrink: no (this is needed so the disk space is properly used)
fs__to_merge: This section formats the lvm
– dev: /dev/vg_vmguest/lv_vg_vmguest (lvm name)
fstype: ext4 (file system you want to format with)
– dev: /dev/vg_sl_storage/lv_vg_sl_storage (2nd lvm name)
fstype: ext4 (file system you want to format with)
mnt__to_merge: This section will create the path,mount, and add to fstab
– path: /vmguests (path you want created for mount)
src: /dev/vg_vmguest/lv_vg_vmguest (lvm you want to mount)
fstype: ext4 (this is for fstab adding)
– path: /sl_storage (this is second path to create)
src: /dev/vg_sl_storage/lv_vg_sl_storage (second lvm you want to mount)
fstype: ext4 (to add to fstab)
Running your playbook:
cd ansible/
Example: of justdofs.yml
– hosts: all
gather_facts: yes
any_errors_fatal: true
roles:
– role: provision-fs
Command:
ansible/$ ansible-playbook -i inventory/hosts justdofs.yml -u root -k –limit=’testservernick1′
Example of successful play:
ntailor@test-box:~/ansible/computelab$ ansible-playbook –i inventory/hosts justdofs.yml -u root -k –limit=’testservernick1‘
SSH password:
PLAY [all] *******************************************************************************************************************************************************************************************************
TASK [provision-fs : Merge VG variables] *************************************************************************************************************************************************************************
ok: [testservernick1]
TASK [provision-fs : Merge LV variables] *************************************************************************************************************************************************************************
ok: [testservernick1]
TASK [provision-fs : Merge FS variables] *************************************************************************************************************************************************************************
ok: [testservernick1]
TASK [provision-fs : Merge MOUNT variables] **********************************************************************************************************************************************************************
ok: [testservernick1]
TASK [provision-fs : Create VGs] *********************************************************************************************************************************************************************************
ok: [testservernick1] => (item={‘vg’: ‘vg_vmguest‘, ‘pvs‘: ‘/dev/sdb‘})
ok: [testservernick1] => (item={‘vg’: ‘vg_sl_storage‘, ‘pvs‘: ‘/dev/sdc‘})
TASK [provision-fs : Create LVs] *********************************************************************************************************************************************************************************
ok: [testservernick1] => (item={‘vg’: ‘vg_vmguest‘, ‘lv’: ‘lv_vg_vmguest‘, ‘size’: ‘100%FREE’, ‘shrink’: False})
ok: [testservernick1] => (item={‘vg’: ‘vg_sl_storage‘, ‘lv’: ‘lv_vg_sl_storage‘, ‘size’: ‘100%FREE’, ‘shrink’: False})
TASK [provision-fs : Create FSs] *********************************************************************************************************************************************************************************
ok: [testservernick1] => (item={‘dev’: ‘/dev/vg_vmguest/lv_vg_vmguest‘, ‘fstype‘: ‘ext4’})
ok: [testservernick1] => (item={‘dev’: ‘/dev/vg_sl_storage/lv_vg_sl_storage‘, ‘fstype‘: ‘ext4’})
TASK [provision-fs : Mount FSs] **********************************************************************************************************************************************************************************
ok: [testservernick1] => (item={‘path’: ‘/vmguests‘, ‘src‘: ‘/dev/vg_vmguest/lv_vg_vmguest‘, ‘fstype‘: ‘ext4’})
ok: [testservernick1] => (item={‘path’: ‘/sl_storage‘, ‘src‘: ‘/dev/vg_sl_storage/lv_vg_sl_storage‘, ‘fstype‘: ‘ext4’})
PLAY RECAP *******************************************************************************************************************************************************************************************************
testservernick1 : ok=8 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
How to add VM-Tags and Custom attributes with Anisble(VMware)
So whether your using cloud or doing in house deploys. Tagging is a used a lot especially in cloud type environments. Which we will cover in later posts.
So a common reason to tag your vm is wanting to have the creation date and the type of server or environment its using, handy for backup solutions & other auditables.
Pre-requisites: Assumed.
Step by Step:
1.Create a roles directory inside /etc/ansible/roles
b.mkdir -p /etc/ansible/roles/ custom-tags-attributes-vmware/defaults
2.Now you want to create a task for the snapshots.
d.Create a file called main.yml
Note: Okay for the custom attributes to work you need to get the MOID, UUID & Folder of the vm by using the vm name. So we need to gather facts about the vm and the set those facts as variables that we can pass to the next tasks.
– name: get list of facts
vmware_guest_facts:
hostname: “{{ vcenter_host }}”
username: “{{ vcenter_username }}”
password: “{{ vcenter_password }}”
name: “{{ inventory_hostname }}”
datacenter: “{{ vcenter_dc }}”
validate_certs: False
delegate_to: localhost
ignore_errors: true
register: vm_facts
Note: So when we use the vmware_guest_facts module to gather the facts about the vm by register the facts to a variable “vm_facts”, which when you run the playbook with –vvvv will spit out the facst you can pass as indicated below.
“hw_folder“: “/SysUnix/Testing“,
“hw_guest_full_name“: “Red Hat Enterprise Linux 7 (64-bit)”,
“hw_guest_ha_state“: true,
“hw_guest_id“: “rhel7_64Guest”,
“hw_interfaces“: [
“eth0”
],
“hw_is_template“: false,
“hw_memtotal_mb“: 2048,
“hw_name“: “v-sits-test4”,
“hw_power_status“: “poweredOn“,
“hw_processor_count“: 2,
“hw_product_uuid“: “4226d4e1-6be8-9447-5ced-b037075e2ffd”,
“hw_version“: “vmx-11”,
“instance_uuid“: “50263518-c95b-c3be-5c77-4e1ea69ec295”,
“ipv4”: “192.168.1.29“,
“ipv6”: null,
“module_hw“: true,
“moid“: “vm-296678”,
“snapshots”: [],
“vimref“: “vim.VirtualMachine:vm-296678”,
“vnc“: {}
Note: Now that we have the UUID, MOID, & Folder. We now want to create static variables for UUID, MOID, & folder by using the previous variable we registered as “vm_facts”, and we want to set them as static variables by setting them as facts we can past to the tasks after. As indicated below. Again setting facts is the same as defining variables in bash. Just ansible way to do it.
– set_fact:
vm_uuid: “{{ vm_facts.instance.instance_uuid }}”
– set_fact:
moid: “{{ vm_facts.instance.moid }}”
– set_fact:
vm_folder: “{{ vm_facts.instance.hw_folder }}”
Note: Now want to use the ansible server date and pass that as a variable so you don’t have to input the date as manual value each time you deploy a new host. So we want to grab the date and setup a static fact and then pass it as its own variable. Like we did above…
– name: Get Date
shell: date +%Y-%m-%d
register: date
delegate_to: localhost
Note: We use shell module to get the date in the format we want, then register the result as the variable {{ date }}. We then set a static fact of the result and create another variable called date with using the result from the above.
– set_fact:
date: “{{ date.stdout }}”
Note: We now want to pass all the fact to the “vmware_guest_custom_attributes” module UUID, MOID, DATE, and Folder indicated as below.
– name: Add multiple virtual machine custom attributes
vmware_guest_custom_attributes:
hostname: “{{ vcenter_host }}”
username: “{{ vcenter_username }}”
password: “{{ vcenter_password }}”
name: “{{ inventory_hostname }}”
datacenter: “{{ vcenter_dc }}”
folder: “{{ vm_folder }}”
uuid: “{{ vm_facts.instance.instance_uuid }}”
moid: “{{ vm_facts.instance.moid }}”
state: present
validate_certs: False
use_instance_uuid: True
attributes:
– name: Creation Date
value: “{{ date }}”
# – name: MyAttribute2 – Note: You can add additional attributes if you wish
# value: test2 – Note: You can add additional attributes if you wish
delegate_to: localhost
register: attributes
Note: Okay so depending on which module you use, some require you to gather facts about the vmware categories. This is just incase you need the cateogory_id which is usually derived from using a REST API in json or other method. The “community.vmware.vmware_category_facts”will be able to pull the info and then you can pass it as a variable or a static value. IF YOU NEED, however this is NOT needed. If you use the “vmware_tag_manager” module”
– name: Gather facts about tag categories
community.vmware.vmware_category_facts:
hostname: “{{ vcenter_host }}”
username: “{{ vcenter_username }}”
password: “{{ vcenter_password }}”
validate_certs: no
delegate_to: localhost
register: all_tag_category_facts
ok: [v-sits-test4] => {
“changed”: false,
“invocation”: {
“module_args“: {
“hostname”: “vmware.nicktailor.com“,
“password”: “VALUE_SPECIFIED_IN_NO_LOG_PARAMETER”,
“port”: 443,
“protocol”: “https”,
“username”: “admin“,
“validate_certs“: false
}
},
“tag_category_facts“: [
{
“category_associable_types“: [
“VirtualMachine“
],
“category_cardinality“: “SINGLE”,
“category_description“: “VM Type – Clone, Decomm, Dev, Prod, SRM, SLM, Template or Test”,
“category_id“: “urn:vmomi:InventoryServiceCategory:f1024eb4-d7d4-49fe-9725-4dcba39fbe3b:GLOBAL”,
“category_name“: “VMType“,
“category_used_by“: []
},
{
“category_associable_types“: [
“VirtualMachine“
],
“category_cardinality“: “SINGLE”,
“category_description“: “Team or Department”,
“category_id“: “urn:vmomi:InventoryServiceCategory:888a0877-5335-4477-9347-6d8de5b3e60e:GLOBAL”,
“category_name“: “Team/Dept”,
“category_used_by“: []
}
]
}
Note: Now that we want to create the tag for the vm using the ““vmware_tag_manager” module. The only variable that is passed from outside the defaults at the inventory_hostname level “host_var/nicktest1” is the {{ vm_tag }} Make sure you have this defined for the role to work properly.
– name: Add tags to a virtual machine
vmware_tag_manager:
hostname: ‘{{ vcenter_host }}’
username: ‘{{ vcenter_username }}’
password: ‘{{ vcenter_password }}’
validate_certs: no
tag_names:
– “{{ vm_tag }}” – passed at the host_var/nicktest1
object_name: “{{ inventory_hostname }}”
object_type: VirtualMachine
state: present
delegate_to: localhost
Note: You will likely have a group_var from you vmdeploy role that you can use for here.
vcenter_username: admin
vcenter_password: should be vault encrypted variable
vcenter_host: vmware.nicktailor.com
vcenter_dc: London
/etc/ansible/inventory/TEST/hosts
Run your playbook: from /etc/ansible
Playbook log:
[root@ansible-server]# ansible-playbook –i inventory/TEST/hosts justcustomattrib.yml –ask-vault-pass –limit=’v-sits-test4′
Vault password:
PLAY [all] **********************************************************************************************************************************
TASK [custom-tags-attributes-vmware : get list of facts] ************************************************************************************
[DEPRECATION WARNING]: The ‘vmware_guest_facts‘ module has been renamed to ‘vmware_guest_info‘. This feature will be removed in version
2.13. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ok: [v-sits-test4]
TASK [custom-tags-attributes-vmware : set_fact] *********************************************************************************************
ok: [v-sits-test4]
TASK [custom-tags-attributes-vmware : set_fact] *********************************************************************************************
ok: [v-sits-test4]
TASK [custom-tags-attributes-vmware : set_fact] *********************************************************************************************
ok: [v-sits-test4]
TASK [custom-tags-attributes-vmware : Get Date] *********************************************************************************************
changed: [v-sits-test4]
TASK [custom-tags-attributes-vmware : set_fact] *********************************************************************************************
ok: [v-sits-test4]
TASK [custom-tags-attributes-vmware : Add multiple virtual machine custom attributes] *******************************************************
ok: [v-sits-test4]
TASK [custom-tags-attributes-vmware : Gather facts about tag categories] ********************************************************************
ok: [v-sits-test4]
TASK [custom-tags-attributes-vmware : Add tags to a virtual machine] ************************************************************************
ok: [v-sits-test4]
PLAY RECAP **********************************************************************************************************************************
v-sits-test4 : ok=9 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
How to remove snapshots with Ansible(VMware)
Okay, so lots of folks ask me about this, and there are a number of ways you can do this.
But if you’re using vmware and redhat satellite for central patch management for your redhat environment.
Then depending on how you patch your systems. If you snapshot every group prior to patching. Then this post will be perfect for you.
Process:
Pre-requisites: Assumed.
Step by Step:
1.Create a roles directory inside /etc/ansible/roles
b.mkdir -p /etc/ansible/roles/remove-snapshot/defaults
2.Now you want to create a task for the snapshots.
d.Create a file called main.yml
– name: Login into vCenter and get cookies
delegate_to: localhost
uri:
url: https://{{ vcenter_host }}/rest/com/vmware/cis/session
force_basic_auth: yes
validate_certs: no
method: POST
user: ‘{{ vcenter_username }}’
password: ‘{{ vcenter_password }}’
register: login
Note: Okay so what we want to do is find the virtual machine in vsphere by name and then grab its folder value and pass it as a variable so you don’t need to define it statically in your host_var. The main reason is, say you deployed a vm and months later moved it to another folder, your code will likely have the origin folder which would be annoying, and the ansible documentation doesn’t really cover this approach, you basically figure it out as you do it. So im going to save you all time. Here is how you do it. The below will gather vm_facts based on the inventory_hostname
– name: Find Guest’s Folder using name
vmware_guest_find:
hostname: “{{ vcenter_host }}”
username: “{{ vcenter_username }}”
password: “{{ vcenter_password }}”
validate_certs: no
name: “{{ inventory_hostname }}”
delegate_to: localhost
ignore_errors: true
register: vm_facts
Note: It will than gather those facts and find the folder value. You then register the facts to a variable “vm_facts” Now it will spit out what it finds when you do –vvvv when you do your play. From there you can see the folder setting. You now want to set that that folder setting as its own variable that you can pass to another task as indicated below.
ok: [ansible-server] => {
“changed”: false,
“folders”: [
“/SysUnix/Teststuff“
],
“invocation”: {
“module_args“: {
“datacenter“: null,
“hostname”: “vmware.nicktailor.com“,
“name”: ” ansible-server“,
“password”: “VALUE_SPECIFIED_IN_NO_LOG_PARAMETER”,
“port”: 443,
“proxy_host“: null,
“proxy_port“: null,
“use_instance_uuid“: false,
“username”: “svc_ans“,
“uuid“: null,
“validate_certs“: false
}
}
}
– name: “vm_folder – setting folder value”
set_fact:
folder : “{{ vm_facts.folders }}”
Note: So you can see that the facts has a sub fact called “folders”. We want to pass that by setting that value as its own variable by making it a fact. Ansible way to set variables is setting facts. So we make that value above into a variable “vm_facts.folders” and then pass that into the next task where it asks for folders. This will get around the having to provide the exact folder the vm_resides to create snapshotting for an array of hosts. This section is basically identical to create except the “state” is set to absent
– name: Remove Snapshot
vmware_guest_snapshot:
hostname: “{{ vcenter_host }}”
username: “{{ vcenter_username }}”
password: “{{ vcenter_password }}”
datacenter: “{{ vcenter_dc }}”
validate_certs: no
name: “{{ inventory_hostname }}”
state: absent
snapshot_name: “Ansible Managed Snapshot”
folder: “‘{{ vm_facts.folders }}'”
description: “This snapshot is created by Ansible Playbook”
delegate_to: localhost
Note: You will likely have a group_var from you vmdeploy role that you can use for here.
vcenter_username: admin
vcenter_password: should be vault encrypted variable
vcenter_host: vmware.nicktailor.com
vcenter_dc: London
/etc/ansible/inventory/TEST/hosts
Run your playbook: from /etc/ansible
Playbook log:
[root@ansible-server]# ansible-playbook –i inventory/TEST/hosts justremovevmsnap.yml –ask-vault-pass –limit=’nicktest1‘
Vault password:
PLAY [all] **********************************************************************************************************************************
TASK [remove_snapshot : Login into vCenter and get cookies] *********************************************************************************
ok: [nicktest1]
TASK [remove_snapshot : Find Guest’s Folder using name] *************************************************************************************
ok: [nicktest1]
TASK [remove_snapshot : vm_folder – setting folder value] ***********************************************************************************
ok: [nicktest1]
TASK [remove_snapshot : remove Snapshot] ****************************************************************************************************
ok: [nicktest1]
PLAY RECAP **********************************************************************************************************************************
nicktest1 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Note: This uses the snapshot name to remove as the one you used to create. If another snapshot exists with a different name it will ignore it entirely. If you pass snapshot variable with another name and run the book again, it will remove another snapshot. The same applies to removing snapshots, it will remove based on the name.
How to create snapshots with Ansible (VMware)
Okay, so lots of folks ask me about this, and there are a number of ways you can do this.
But if you’re using vmware and redhat satellite for central patch management for your redhat environment.
Then depending on how you patch your systems. If you snapshot every group prior to patching. Then this post will be perfect for you.
Patching Processes:
– (these are not in this post) part 2
Note: There is a step to change the content view of all your hosts and cleaning your yum repos on all the hosts. I have not written a post on this step yet, but you should obviously automate that. I will eventually get around to including it, when I have some time.
Pre-requisites: Assumed.
Step by Step:
b.mkdir -p /etc/ansible/roles/create-snapshot/defaults
2.Now you want to create a task for the snapshots.
d.Create a file called main.yml
3.Add the following code and save the file
– name: Login into vCenter and get cookies
delegate_to: localhost
uri:
url: https://{{ vcenter_host }}/rest/com/vmware/cis/session
force_basic_auth: yes
validate_certs: no
method: POST
user: ‘{{ vcenter_username }}’
password: ‘{{ vcenter_password }}’
register: login
Note: Okay so what we want to do is find the virtual machine in vsphere by name and then grab its folder value and pass it as a variable so you don’t need to define it statically in your host_var. The main reason is, say you deployed a vm and months later moved it to another folder, your code will likely have the origin folder which would be annoying, and the ansible documentation doesn’t really cover this approach, you basically figure it out as you do it. So im going to save you all time. Here is how you do it. The below will gather vm_facts based on the inventory_hostname
– name: Find Guest’s Folder using name
vmware_guest_find:
hostname: “{{ vcenter_host }}”
username: “{{ vcenter_username }}”
password: “{{ vcenter_password }}”
validate_certs: no
name: “{{ inventory_hostname }}”
delegate_to: localhost
ignore_errors: true
register: vm_facts
Note: It will than gather those facts and find the folder value. You then register the facts to a variable “vm_facts” Now it will spit out what it finds when you do –vvvv when you do your play. From there you can see the folder setting. You now want to set that that folder setting as its own variable that you can pass to another task as indicated below.
ok: [ansible-server] => {
“changed”: false,
“folders”: [
“/SysUnix/Teststuff“
],
“invocation”: {
“module_args“: {
“datacenter“: null,
“hostname”: “vmware.nicktailor.com“,
“name”: “ ansible-server“,
“password”: “VALUE_SPECIFIED_IN_NO_LOG_PARAMETER”,
“port”: 443,
“proxy_host“: null,
“proxy_port“: null,
“use_instance_uuid“: false,
“username”: “admin“,
“uuid“: null,
“validate_certs“: false
}
}
}
– name: “vm_folder – setting folder value”
set_fact:
folder : “{{ vm_facts.folders }}”
Note: So you can see that the facts has a sub fact called “folders”. We want to pass that by setting that value as its own variable by making it a fact. Ansible way to set variables is setting facts. So we make that value above into a variable “vm_facts.folders” and then pass that into the next task where it asks for folders. This will get around the having to provide the exact folder the vm_resides to create snapshotting for an array of hosts.
– name: Create Snapshot
vmware_guest_snapshot:
hostname: “{{ vcenter_host }}”
username: “{{ vcenter_username }}”
password: “{{ vcenter_password }}”
datacenter: “{{ vcenter_dc }}”
validate_certs: no
name: “{{ inventory_hostname }}”
state: present
snapshot_name: “Ansible Managed Snapshot”
folder: “‘{{ vm_facts.folders }}'”
description: “This snapshot is created by Ansible Playbook”
delegate_to: localhost
Note: You will likely have a group_var from you vmdeploy role that you can use for here.
vcenter_username: admin
vcenter_password: should be vault encrypted variable
vcenter_host: vmware.nicktailor.com
vcenter_dc: London
/etc/ansible/inventory/TEST/hosts
Run your playbook: from /etc/ansible
Playbook log:
[root@ansible–server]# ansible-playbook –i inventory/TEST/hosts justcreatevmsnap.yml –ask-vault-pass –limit=’nicktest1‘
Vault password:
PLAY [all] **********************************************************************************************************************************
TASK [create_snapshot : Login into vCenter and get cookies] *********************************************************************************
ok: [nicktest1]
TASK [create_snapshot : Find Guest’s Folder using name] *************************************************************************************
ok: [nicktest1]
TASK [create_snapshot : vm_folder – setting folder value] ***********************************************************************************
ok: [nicktest1]
TASK [create_snapshot : Create Snapshot] ****************************************************************************************************
ok: [nicktest1]
PLAY RECAP **********************************************************************************************************************************
nicktest1 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Note: This uses the snapshot name to create. If another snapshot exists with a different name it will ignore it entirely. If you pass snapshot variable with another name and run the book again, it will create another snapshot. The same applies to removing snapshots, it will remove based on the name. We will cover that in my post to remove snapshots.
How to Create a New Host in Foreman with Ansible
Okay…this one was quite difficult to find online. Seems like the ansible documentation for the foreman module was seriously lacking or not kept up to date by anyone. I searched for awhile to see if anyone had an actual working model of it.
Not even in the ansible chat rooms did anyone know….which was weak.
So I spent some time getting this to work smoothly, and you will probably not find anywhere else on the web on how to do this. If you do show me….so I can kick myself.
Lets get dangerous then. 😊
Step by step:
Foreman – Already setup and your “computer resource” is hooked in (VMware)
Note: The compute profile(vmware) when hooked in, will also trigger a new vm creation in vsphere prepped to do DHCP. You can combine variables from vmware_guest module and this module as they require similar variables to be passed. To setup a one stop shop to deploy in foreman and vmware with just using ansible. Iv done this already…..
Special notes: The foreman I had setup did not have organisation or location configured. This caused the module to not function properly and I had to contact one of the developers who helped me patch the code so I didn’t require them to be configured or defined. Which I will show you all how to do.
Ansible – Assuming you have it setup and working with python 2.7 not sure this module will work with python 3. Havent tried that yet…..
Module – TheForeman Collection
Note: You can find the locations of these certs on foreman server. You will to copy them over to ansible for the callback to work properly. However, it is not needed to complete the host creation, you likely just see an error at the end of the play.
[callback_foreman]
url = ‘http://foreman-1.tdr.corp-apps.com’
ssl_cert = /etc/foreman-proxy/ssl-cert.pem
ssl_key = /etc/foreman-proxy/ssl-pvt.pem
verify_certs = /etc/foreman-proxy/ssl-ca
Okay once installed you. If you look at the ansible documentation on how to manage hosts using this module…from redhat.
It utterly useless…and will not work if you try to use the examples below.
https://people.redhat.com/evgeni/fam-antsibull/plugins/host_module.html
– name: “Create a host”
host:
username: “admin”
password: “changeme“
server_url: “https://foreman.example.com”
name: “new_host“
hostgroup: my_hostgroup
state: present
The fix was to avoid trying to touch a specific resource that is only available when you have Org/Loc enabled.
diff –git plugins/module_utils/foreman_helper.py plugins/module_utils/foreman_helper.py
index 432c76df..c9a3abda 100644
— plugins/module_utils/foreman_helper.py
+++ plugins/module_utils/foreman_helper.py
@@ -396,8 +396,9 @@ class ForemanAnsibleModule(AnsibleModule):
_host_update = next(x for x in _host_methods if x[‘name’] == ‘update’)
for param in [‘location_id‘, ‘organization_id‘]:
– _host_update_taxonomy_param = next(x for x in _host_update[‘params’] if x[‘name’] == param)
– _host_update[‘params’].remove(_host_update_taxonomy_param)
+ _host_update_taxonomy_param = next((x for x in _host_update[‘params’] if x[‘name’] == param), None)
+ if _host_update_taxonomy_param is not None:
+ _host_update[‘params’].remove(_host_update_taxonomy_param)
@_check_patch_needed(fixed_version=’2.0.0′)
def _patch_templates_resource_name(self):
Trick: with ansible you can write some of the code and run the playbook and if there are missing variables it will tell you what they are.
fatal: [testnick1]: FAILED! => {
“changed”: false,
“invocation”: {
“module_args“: {
“activation_keys“: null,
“architecture”: null,
“build”: null,
“comment”: null,
“compute_attributes“: null,
“compute_profile“: null,
“compute_resource“: null,
“config_groups“: null,
“content_source“: null,
“content_view“: null,
“domain”: null,
“enabled”: null,
“environment”: null,
“hostgroup“: “my_hostgroup“,
“image”: null,
“interfaces_attributes“: null,
“ip“: null,
“kickstart_repository“: null,
“lifecycle_environment“: null,
“location”: null,
“mac”: null,
“managed”: null,
“medium”: null,
“name”: “testnick1”,
“openscap_proxy“: null,
“operatingsystem“: null,
“organization”: null,
“owner”: null,
“owner_group“: null,
“parameters”: null,
“password”: “VALUE_SPECIFIED_IN_NO_LOG_PARAMETER”,
“provision_method“: null,
“ptable“: null,
“puppet_ca_proxy“: null,
“puppet_proxy“: null,
“puppetclasses“: null,
“pxe_loader“: null,
“realm”: null,
“root_pass“: null,
“server_url“: “http://foreman-1.nictailor.com/”,
“state”: “present”,
“subnet”: null,
“subnet6”: null,
“username”: “ntailor“,
“validate_certs“: true
}
},
“msg“: “The hostname must be FQDN”
}
PLAY RECAP ************************************************************************************************************************************************************************
testnick1 : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
Create a Host: This code is what you need for this module to work.
– name: “Create a host”
theforeman.foreman.host:
username: “{{ foreman_user }}”
password: “{{ vcenter_password }}”
server_url: “{{ server_url }}”
name: “{{ inventory_hostname }}”
hostgroup: “{{ host_group }}”
managed: no
build: no
compute_profile: “{{ compute_profile }}”
compute_resource: “{{ computer_resource }}”
compute_attributes:
cpus: “{{ vm_cpu_count }}”
memory_mb: “{{ vm_memory }}”
interfaces_attributes:
– type: “interface”
primary: true
compute_attributes:
name: nic1
network: “{{ vm_vlan_name }}”
interface: “{{ vm_interface }}”
subnet: “{{ vm_subnet }}”
ip: “{{ vm_ip }}”
domain: “{{ domain }}”
provision: yes
operatingsystem: “{{ operating_system }}”
medium: “{{ medium }}”
architecture: x86_64
pxe_loader: PXELinux BIOS
puppet_ca_proxy: “{{ puppet_ca_proxy }}”
puppet_proxy: “{{ puppet_proxy }}”
root_pass: “{{ root_pass }}”
environment: tdr
# ptable: Centos – LVM – / , swap
ptable: “{{ ptable }}”
# owner: unix
state: present
validate_certs: false
delegate_to: localhost
– name: “Switch host on”
theforeman.foreman.host_power:
username: “{{ foreman_user }}”
password: “{{ foreman_password }}”
server_url: “{{ server_url }}”
hostname: “{{ inventory_hostname }}”
state: on
validate_certs: false
delegate_to: localhost
Note: You can find all these variables inside foreman GUI with a bit of digging.
foreman_user: Reptilianfilth
foreman_password: { generally want a ansible vault password }
compute_profile: vmware
computer_resource: vcenter.nic.internal
domain: nic.internal
medium: 7.8-CentOS
puppet_ca_proxy: puppet-2.nic.internal
puppet_proxy: puppet-2.nic.internal
#VM creation variables
vm_network: niccorp-192.168.65_corp
vm_interface: VMXNET3
vm_subnet: 192.168.65.0
vm_ip: 192.168.65.103
domain: nic.internal
managed: no
host_group: Base-Server/Centos-7.8.2003
operating_system: Centos 7.8.2003
ptable: Centos – LVM – / , swap
root_pass: changemetwiceaday
medium: 7.8-CentOS
Special Note: Now if you wanted to have it so you can use foreman module or vmware_guest module combining the variables names between the modules.
You can do as below. You will need to ensure the variables match but it works. You can get around having to rely on DHCP with this.
#VM creation variables foreman and vmware together
vm_vlan_name: nic_192.168.44_db_stor2
vm_datastore: esx_nicrcorp
vm_dvswitch: VDS-nic-Corporate
vm_interface: VMXNET3
vm_subnet: 192.1268.44.0
vm_ip: 192.168.44.14
vm_netmask: 255.255.255.0
vm_gateway: 192.168.44.254
vm_dns_servers: [192.168.1.1]
vm_dns_suffix: nic.internal
vm_cpu_count: 4
vm_memory: 16384
vm_state: poweredon
vm_connected: true
domain: tdr.internal
managed: no
host_group: Base-Server/Centos-7.8.2003
operating_system: Centos 7.8.2003
ptable: Centos – LVM – / , swap
root_pass: changemetwiceaday
medium: 7.8-CentOS
Before you to start one last thing. If you remember in the defaults we outlined
compute_profile: vmware
(this is the foreman profile it will use, so whatever defaults you have set for network and disksize here is what will be used to trigger foreman to create a host in vcenter, so it good to go check this in foreman first.)
Run playbook: from /etc/ansible
[root@nick ansible]# ansible-playbook –i inventory/TDR/hosts foremancreatehost.yml –ask-vault-pass –limit ‘testnick3.tdr.internal’
Vault password:
PLAY [all] **********************************************************************************************************************************************
TASK [ansible-provision-foreman : Create a host] ********************************************************************************************************
changed: [testnick3.tdr.internal]
PLAY RECAP *******************************************************************************************************************************************************************************
testnick3.tdr.internal : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
403 Client Error: Forbidden for url: http://foreman-1.nic.corp.com/api/v2/reports (if you see this, just ignore it) Its just callback report.
HOW TO CHECK CPU, MEMORY, & DISKS THRESHHOLDS on an ARRAY of HOSTS.
So I was tinkering around as usual. I thought this will come in handy for other engineers
If you a large cluster of servers that can suddenly over night loose all its MEM,CPU,DISK due to the nature of your businesses. Its difficult to monitor that from a GUI and on an array of hosts more often than not.
Cloud Scenario……
Say you find a node that is dying because too many clients are using resources and you need migrate instances off to another node, only you don’t know which nodes have the needed resources without having to go look at all the nodes individually.
This tends be every engineers pain point. So I decide to come up with quick easy solution for emergency situations, where you don’t have time to sifting through alert systems that only show you data on a per host basis, that tend to load very slowly.
This bash script will check the CPU, MEM, DISK MOUNTS (including NFS) and tell which ones are okay and which ones are
CPU – calculated by the = 100MaxThrottle – Cpu-idle = CPU-usage
note: it also creates a log /opt/cpu.log on each host
MEM – calculate by Total Mem / Used Memory * 100 = Percentage of Used Memory
note: it also creates a log /opt/mem.log on each host
Disk – Any mount that reaches the warn threshold… COMPLAIN
Now, itemised the bash script so you can just comment out item you don’t want to use at the bottom of the script if you wanted to say just check CPU/MEM
#Written By Nick Tailor
#!/bin/bash
now=`date -u -d”+8 hour” +’%Y-%m-%d %H:%M:%S’`
#cpu use threshold
cpu_warn=’75’
#disk use threshold
disk_warn=’80’
#—cpu
item_cpu () {
cpu_idle=`top -b -n 1 | grep Cpu | awk ‘{print $8}’|cut -f 1 -d “.”`
cpu_use=`expr 100 – $cpu_idle`
echo “now current cpu utilization rate of $cpu_use $(hostname) as on $(date)” >> /opt/cpu.log
if [ $cpu_use -gt $cpu_warn ]
then
echo “cpu warning!!! $cpu_use Currently HIGH $(hostname)”
else
echo “cpu ok!!! $cpu_use% use Currently LOW $(hostname)”
fi
}
#—mem
item_mem () {
#MB units
LOAD=’80.00′
mem_free_read=`free -h | grep “Mem” | awk ‘{print $4+$6}’`
MEM_LOAD=`free -t | awk ‘FNR == 2 {printf(“%.2f%”), $3/$2*100}’`
echo “Now the current memory space remaining ${mem_free_read} GB $(hostname) as on $(date)” >> /opt/mem.log
if [[ $MEM_LOAD > $LOAD ]]
then
echo “$MEM_LOAD not good!! MEM USEAGE is HIGH – Free-MEM-${mem_free_read}GB $(hostname)”
else
echo “$MEM_LOAD ok!! MEM USAGE is beLOW 80% – Free-MEM-${mem_free_read}GB $(hostname)”
fi
}
#—disk
item_disk () {
df -H | grep -vE ‘^Filesystem|tmpfs|cdrom’ | awk ‘{ print $5 ” ” $1 }’ | while read output;
do
echo $output
usep=$(echo $output | awk ‘{ print $1}’ | cut -d’%’ -f1 )
partition=$(echo $output | awk ‘{ print $2 }’ )
if [ $usep -ge $disk_warn ]; then
echo “AHH SHIT!, MOVE SOME VOLUMES IDIOT…. \”$partition ($usep%)\” on $(hostname) as on $(date)”
fi
done
}
item_cpu
item_mem
#item_disk – This is so you can comment out whole sections of the script without having to do the whole section by individual lines.
Now the cool part.
Now if you have a centrally managed jump host that allows you to get out from your estate. Ideally you would want to setup ssh keys on the hosts and ensure you have sudo permissions on the those hosts.
We want to loop this script through an array of hosts and have it run and then report back all the findings in once place. This is extremely handy if your in resource crunch.
This assumes you have SSH KEYS SETUP & SUDO for your user setup.
Create the script
Next
Server1
Server2
Server3
Server4
Run your forloop with ssh keys and sudo already setup.
Logfile – cpumem.status.DEV – will be the log file that has all the info
Output:
cpu ok!!! 3% use Currently dev1.nicktailor.com
17.07% ok!! MEM USAGE is beLOW 80% – Free-MEM-312.7GB dev1.nicktailor.com
5% /dev/mapper/VolGroup00-root
3% /dev/sda2
5% /dev/sda1
1% /dev/mapper/VolGroup00-var_log
72% 192.168.1.101:/data_1
28% 192.168.1.102:/data_2
80% 192.168.1.103:/data_3
AHH SHIT!, MOVE SOME VOLUMES IDIOT…. “192.168.1.104:/data4 (80%)” on dev1.nicktailor.com as on Fri Apr 30 11:55:16 EDT 2021
Okay so now I’m gonna show you a dirty way to do it, because im just dirty. So say your in horrible place that doesn’t use keys, because they’re waiting to be hacked by password. 😛
DIRTY WAY – So this assumes you have sudo permissions on the hosts.
Note: I do not recommend doing this way if you are a newb. Doing it this way will basically log your password in the bash history and if you don’t know how to clean up after yourself, well………………….you’re going to get owned.
I’m only showing you this because some cyber security “folks” believe that not using keys is easier to deal with in some parallel realities iv visited… You can do the exact same thing above, without keys. But leave massive trail behind you. Hence why you should use secure keys with passwords.
Not Recommended for Newbies:
Forloop AND passing your ssh password inside it.
Log file – cpumem.status.DEV – will be the log file that has all the info
Output:
cpu ok!!! 3% use Currently dev1.nicktailor.com
17.07% ok!! MEM USAGE is beLOW 80% – Free-MEM-312.7GB dev1.nicktailor.com
5% /dev/mapper/VolGroup00-root
3% /dev/sda2
5% /dev/sda1
1% /dev/mapper/VolGroup00-var_log
72% 192.168.1.101:/data_1
28% 192.168.1.102:/data_2
80% 192.168.1.103:/data_3
AHH SHIT!, MOVE SOME VOLUMES IDIOT…. “192.168.1.104:/data4 (80%)” on dev1.nicktailor.com as on Fri Apr 30 11:55:16 EDT 2021
How to deploy Open-AKC(Authorized Key Chain)
Acting as a centralised trust management platform:
By allowing the “authorized_keys” mechanism on the hosts to be completely disabled, OpenAKC
permits SSH trust across an entire estate to be managed (with rich control and monitoring features)
centrally by “systems administration” or “information security” staff. This means that users, or
application developers etc. cannot add or remove trust relationships, effectively enforcing any
whitelist or approval process you might want to establish for the creation of trust relationships
within an estate.
A practical ‘Jump Host’:
Having worked in many Linux environments, the author of the software has seen a number of ‘jump
host’ solutions using dubious mechanisms such as “sudo” and others used to map users from AD or
LDAP directories to SSH trust relationships using shared private keys etc. and these solutions are
almost universally highly insecure.
OpenAKC implements a new approach and acts as an “drop in” upgrade for a legacy solution by simply migrating users to personal rather than sharedkeys with a “self service” key management mechanism. Requiring personal keys have pass phrases ensures increased security, and avoids ad-hoc automation from user accounts. The system provides rich control and monitoring features, while users can use familiar tools with little to no disruption to their workflow.
The problems everyone thinks about, but never finds a good solution.
The problem with joining every server to the domain:
◦ As soon as someone sudo’s to root. There is zero control on limiting what this root user can do. Which is a huge problem when getting compromised. Every key stroke is not logged and if you have multiple people sudo as root at the same time the logs can get blurry.
Well guess what….this is exactly what im talking about today. This architecture does take a few steps and time to understand its inner workings but security wise trumps anything out there currently being used by most folks.
This is a overview of a simplified architecture. This can be scaled out to new or legacy environment using many distros without interruption.
Combined jump host/security server architecture: This approach is generally used when it’s a small group managing things:
Benefits: The advantage here is your security server and your jumphost are on the same server. If you’re admin team is also managing the security then this approach is ideal because when diagnosing and editing role rules everything is in one place. Another key benefit… There are only couple client packages that are deployed on client machines upon deployment the client is already brought into the trust, so no additional work is required to bring it in.
Cons: If your jumphost goes down, so does your security server. Now this can be scaled out so there are two servers if one is unavailable it tries the second. But if you have one this could be a single point of failure. Keep in mind this is a VM and would takes seconds to bring back up, so durability is very good from a infrastructure stand point.
Or the alternative:
A Segregated jumphost/security server architecture: This approach is generally used when you have large groups and large infrastructure managing things.
Benefits: Now the advantage here is if you have a multiple groups and need to tighten the security. You can have a your security server and your jump hosts segregated. These two machines are the only two machines that talk you AD. You have roles setup on the security server. If your user is listed in the role on the security server and your apart the AD group that is allowed to login via ssh.
The security server will allow you to login as root via registered ssh keys. You can also strip away “roots” abilities, and every keystroke is logged per user on the security server. A key benefit here is that because the client machines are not joined to the domain. A hacker would never be able to determine information that leads to the heart of your network such as AD, the groups setup, and any of its users. The other benefit is the jump hosts are easily deployed as needed and ready to go if needed. Since the security stuff is not local, the jump host is also secured.
Cons: If your security servers are not functioning and you have disabled user/pass. You will be locked out unless you single user. However this is easily fixed by setting up a strong root/pass that can bypass all security and only give this to one person, such as your manager or someone in secureops. Which would be only used ever used in an extreme dire situation. I have yet to see this ever required once this is implemented correctly.
Special Features:
How to deploy Segregated jumphost/security server architecture:
Okay so the first thing we want to do is setup our security server host to Active Directory. The reason for this is we want to be able to setup centralised user management coupled with using authorised keys to jump to client machines.
Centos 7
Note: make sure you disable firewalld and selinux on your machine. Just a extra layer that not needed and only serves to tie your own hands behind your own back.
Install the following packages will be be using sssd/kerebros
[root@securityack1 ~]# realm discover AD.NICKTAILOR.COM
ad.nicktailor.com
type: kerberos
realm-name: AD.NICKTAILOR.COM
domain-name: ad.nicktailor.com
configured: kerberos–member
server-software: active-directory
client-software: sssd
required-package: oddjob
required-package: oddjob-mkhomedir
required-package: sssd
required-package: adcli
required-package: samba-common-tools
login-formats: %U
login-policy: allow-realm-logins
it will ask you for the AD admin password, enter it and it should go to the next prompt. This step will create the all the necessary files /etc/sssd/sssd.conf /etc/krb5.conf /etc/smb.conf file you need.
yum install openakc-server
Example:
[nicktailor@security1 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/nicktailor/.ssh/id_rsa):
/home/tailorn/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/nictailor/.ssh/id_rsa.
Your public key has been saved in /home/nicktailor/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:udhNKEp0txzfup7IxhUwNA+VSviWP1mu/aKPA5vZb3w tailorn@jumphost1.nicktailor.com
The key’s randomart image is:
+—[RSA 2048]—-+
| o+… |
| . ++. |
| . .oo=. |
| . . o=*… |
| . ..S.o=. |
| . . +.+=.. |
| . ..oBo= |
| .*.++= E |
| .o.**o+. |
+—-[SHA256]—–+
[nicktailor@jumphost1 ~]$ openakc register
OpenAKC Copyright (C) 2019-2020 A. James Lewis. Version is 1.0.0~alpha18-1.el7.
This program comes with ABSOLUTELY NO WARRANTY; see “license” option.
This is free software, and you are welcome to redistribute it
under certain conditions; See LICENSE file for further details.
Passphrase is requested to ensure you own this key.
Enter passphrase:
Escalating to perform API call
Connected to OpenAKC server. Sending key registration request
OK: Request processed
Here you will create a role block that will allow said user to jump to any client so long as your key is registered the openakc security server.
RULE=2020/01/13 19:17,2030/01/13 20:17,user,nicktailor
DAY=any
TIM=any
SHELL=/bin/bash
CMD=any
SCP=s,^/,/data/,g
CAP=cap_linux_immutable
REC=yes
FROM=any
RULE=2020/01/13 19:17,2030/01/13 20:17,group,linuxusers
DAY=any
TIM=any
SHELL=/bin/bash
CMD=any
SCP=s,^/,/data/,g
CAP=cap_linux_immutable
REC=yes
FROM=any
Note: The CAP section allows you to disable root abilities there is a long list of things. This particular one. Disables root from being able to edit any file with an immutable flag. So all your root users can not change these types of file eve as root.
You starting to see why this is how you do shit? Think for yourselves not because someone else said that best practice. Jeez…
Note: this is just a development setup I am doing for you. It can be scaled with two security servers using gluster of nfs to share the files system. This server will be what all the clients check with before allowing anyone to enter. They will need to first exist in AD and then their root key will need to be registered here, and then they will need to be allowed in the appropriate role and sssd and ssh groups before they can get in. This also eliminate the need to join any clients to the domain and protects against any hacker from being able to query the domain controller for user groups and if users exist. The would of also needed to get on to the jump host, have your key and also know your root passphrase. Unlikely going to happen
Okay now were going to setup the jumphost jump1.nickatilor.com
Note: You can have as many jumphosts as you like. This should be the only entry point to your servers. The jump host also talks to the security server. So even if you tried to go directly the server. You couldn’t since its only allowed from here. This is how you set shit up so you don’t end up like solarwinds and so many other idiots who follow rules old and out of date rule blindly.
Note: You also add this server to the domain. So please follow the joining of the domain setups from above and then carry on from here.
APIS=”nickack1.nicktailor.com“
PORT=”889″
ssh-copy-id –i ~/.ssh/id_rsa.pub root@192.168.1.200
[nicktailor@jumphost1 keys]$ openakc ping
OpenAKC Copyright (C) 2019-2020 A. James Lewis. Version is 1.0.0~alpha18-1.el7.
This program comes with ABSOLUTELY NO WARRANTY; see “license” option.
This is free software, and you are welcome to redistribute it
under certain conditions; See LICENSE file for further details.
Connected to OpenAKC server. Sending Test Run Ping Message
Test Run Response – OK: Pong! – from server – securityakc1.nicktailor.com
If you see that above that is a good sign
Now were going to add a client machine so you jump to it. This is the easiest part. So say you have a bunch of legacy systems and you want to centralise login without joining to the domain, you want everyone who using root to be tracked by logging every keystroke and logging what incident they logged in for to use root.
curl https://netlore.github.io/OpenAKC/repos/openakc-el7.repo | sudo tee
/etc/yum.repos.d/openakc.repo
vi /etc/openakc/openakc.conf
APIS=”192.168.1.200“
ENABLED=”yes”
PORT=”889″
CACHETIME=”60″
DEBUG=”no”
PERMITROOT=”yes”
AUDIT=”yes”
QUIZ=”no”
HIDE=”restrict”
FAKESUDO=”yes”
Note: Quiz if you set it to yes. When you log into root it will ask for a service now ticket number and description which will get logged on the security server.
That’s it!….
Now if you go back the jump host and try to log in
[@jumphost1 ~]$ ssh root@192.168.1.38
Enter passphrase for key ‘/home/nicktailor/.ssh/id_rsa‘:
OpenAKC (v1.0.0~alpha18-1.el7) – Interactive Session Initialized
[root@nickclient1 ~]#
This is session is now being logged and and you can not see if the user belongs to the domain
[root@nickclient1 ~]# id nickatilor
id: nicktailor: no such user
This is how you setup security like pro. Hope you enjoyed this.
Hope you enjoyed this.
Special thank you the author of the project “James Lewis” I enjoyed learning this and am a big fan of the innovation behind the open-akc project.
How to add new users:
That’s it your done. Now the new user is able to login into the estate via ssh from the jump host.
How to add a custom tomcat installation to SystemD with ansible.
Okay so say you have a custom install of tomcat and java, which is what a lot of people do because java update and tomcat updates can bring things down. So things need to be tested before updates and standard patch cycles can end up affecting the environment.
But you want to handle the startup and stopping via systemd to be able to get status outputs and let system handle the service on reboots. This is how to do it slick.
Ansible Setup:
Role:
Setup the new role:
Main.yml
===========================================
Note: this will install the redhat tomcat version of tomcat. Do not worry we are not going to be using this tomcat. This is just so redhat automatically setups all the needed services and locations. We will then update the SystemD config for tomcat to use the custom version.
– name: Install the latest version of tomcat
package:
name: tomcat
state: latest
Note: This symlink is important as tomcat default install by redhat is inside /opt/tomcat. Update the src to the custom location of your tomcat
– name: Create symbolic link for “tomcat” in /opt
file:
src: /custom/install/tomcat
path: /opt/tomcat
force: yes
state: link
Note: This will enable tomcat to start up on reboot
– name: Enable tomcat service on startup
shell: systemctl enable tomcat
Note: This is the tomcat systemd service file that systemd uses for the default install. We are going to empty.
– name: Null tomcat.service file
shell: “>/etc/systemd/system/tomcat.service“
Note: We are now going to add our custom block for tomcat into the tomcat.service file we just emptied above using the blockinfle module. This means that this whole section will also be managed by ansible as well. Make sure you adjust the java_home if your java isn’t location inside tomcat. Along with the user,group,umask for to your custom tomcat.
– name: Edit tomcat.service for systemd
blockinfile:
dest: /etc/systemd/system/tomcat.service
insertafter:
block: |
[Unit]
Description=Apache Tomcat Web Application Container
After=syslog.target network.target
[Service]
Type=forking
Environment=JAVA_HOME=/opt/tomcat
Environment=CATALINA_PID=/opt/tomcat/temp/tomcat.pid
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/tomcat
Environment=’CATALINA_OPTS=-Xms512M -Xmx1024M -server –XX:+UseParallelGC‘
Environment=’JAVA_OPTS=-Djava.awt.headless=true –Djava.security.egd=file:/dev/./urandom‘
ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/bin/kill -15 $MAINPID
User=tomcat
Group=tomcat
UMask=
RestartSec=10
Restart=always
[Install]
WantedBy=multi-user.target
Note: This will then reload the custom tomcat via systemd
– name: Start tomcat service with Systemd
systemd:
name: tomcat
daemon_reload: yes
Note: This will then check to see if the new tomcat is service running and out to the ansible playbook log.
– name: get service facts
service_facts:
– name: Check to see if tomcat is running
debug:
var: ansible_facts.services[“tomcat.service“]
Ansibe playbook log:
[root@nickansible]# ansible-playbook –i inventory/DEV/hosts justtomcatrole.yml –limit ‘nicktestvm‘ -k
SSH password:
PLAY [all] ************************************************************************************************************************************************************************************************
TASK [AddTomCatSystemD : Create symbolic link for “tomcat” in /opt] ***************************************************************************************************************************************
changed: nicktestvm]
TASK [AddTomCatSystemD : Enable tomcat service on startup] ************************************************************************************************************************************************
changed: nicktestvm]
TASK [AddTomCatSystemD : Null tomcat.service file] ********************************************************************************************************************************************************
changed: nicktestvm]
TASK [AddTomCatSystemD : Edit tomcat.service for systemd] *************************************************************************************************************************************************
changed: nicktestvm]
TASK [AddTomCatSystemD : Start tomcat service with Systemd] ***********************************************************************************************************************************************
ok: nicktestvm]
TASK [AddTomCatSystemD : get service facts] ***************************************************************************************************************************************************************
ok: nicktestvm]
TASK [AddTomCatSystemD : Check to see if tomcat is running] ***********************************************************************************************************************************************
ok: nicktestvm] => {
“ansible_facts.services[\”tomcat.service\”]”: {
“name”: “tomcat.service“,
“source”: “systemd“,
“state”: “running”,
“status”: “enabled”
}
}
PLAY RECAP ************************************************************************************************************************************************************************************************
nicktestvm : ok=7 changed=4 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
==========================
[root@nicktestvm ~]# cat /etc/systemd/system/tomcat.service
# BEGIN ANSIBLE MANAGED BLOCK
[Unit]
Description=Apache Tomcat Web Application Container
After=syslog.target network.target
[Service]
Type=forking
Environment=JAVA_HOME=/opt/tomcat
Environment=CATALINA_PID=/opt/tomcat/temp/tomcat.pid
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/tomcat
Environment=’CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC’
Environment=’JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom’
ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/bin/kill -15 $MAINPID
User=tomcat
Group=tomcat
UMask=0028
RestartSec=10
Restart=always
[Install]
WantedBy=multi-user.target
# END ANSIBLE MANAGED BLOCK
SystemD Status:
root@nicktestvm ~]# systemctl status tomcat
● tomcat.service – Apache Tomcat Web Application Container
Loaded: loaded (/etc/systemd/system/tomcat.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2020-12-24 05:11:21 GMT; 21h ago
Process: 6333 ExecStop=/bin/kill -15 $MAINPID (code=exited, status=0/SUCCESS)
Process: 6353 ExecStart=/opt/tomcat/bin/startup.sh (code=exited, status=0/SUCCESS)
Main PID: 6363 (java)
CGroup: /system.slice/tomcat.service
└─6363 /usr/local/java/java -Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -server -Xms1…
Dec 24 05:11:21 nicktestvm systemd[1]: Starting Apache Tomcat Web Application Container…
Dec 24 05:11:21 nicktestvm startup.sh[6353]: Existing PID file found during start.
Dec 24 05:11:21 nicktestvm startup.sh[6353]: Removing/clearing stale PID file.
Dec 24 05:11:21 nicktestvm systemd[1]: Started Apache Tomcat Web Application Container.
How to deploy an EC2 instance with Terraform
Okay so terraform is like ansible but for the cloud. Its probably a bit cooler in what it can do than ansible….but im still still picking it up myself. But ultimately the same thing. If you know ansible you can pick up terraform without too much difficulty.
After a couple of cool writes you should be good to go.
Note: Some interesting things to note. If you create a EC2 with terraform and go and delete it manually after. It appears that terraform does not know the states. If you ask terraform to show you the state. As far as it knows the EC2 is all good. Kinda stupid in my opinion…..This is where ansible take the cake, since ansible will actually go and check to see if that VM/EC2 is actually there. But anyway. Its about learning new ways to do things.
Terrform Install:
• Unzip and install terraform
unzip ./terraform_0.14.3_linux_amd64.zip -d /usr/local/bin/
Example–Log:
[root@nick ~]# unzip ./terraform_0.14.3_linux_amd64.zip -d /usr/local/bin/
Archive: ./terraform_0.14.3_linux_amd64.zip
replace /usr/local/bin/terraform? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
inflating: /usr/local/bin/terraform
[root@nick ~]# terraform -v
Terraform v0.14.3
Note: This is to set the environment variable path for terraform if its not set already.
Setup Terraform to communicate with AWS:
• You need to add your aws accesskeyid and secretaccesskey
Note: You can find these inside your aws console under “My Security Credentials” on the top right corner of AWS console.
Then Click Access keys (access key ID and secret access key)◦
Should be listed here, if not create one and download the keys.
◦ echo “export AWS_ACCESS_KEY_ID=accesskeyid” >> /usr/local/bin/profile_terraform.sh
vi provider.tf
#—Content of provider.tf
provider “aws” {
profile = “default”
region = “us-east-1”
}
- Save file
[root@nick]# cat provider.tf
provider “aws” {
profile = “default”
region = “us-east-1”
}
vi create_ec2.tf
#—Content of create_ec2.tf
resource “aws_instance” “nicktest-1” {
ami = “ami-0fc61db8544a617ed”
instance_type = “t2.micro”
}
- Save file
root@nick bin]# terraform init
Initializing the backend…
Initializing provider plugins…
– Finding latest version of hashicorp/aws…
– Installing hashicorp/aws v3.22.0…
– Installed hashicorp/aws v3.22.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run “terraform init” in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running “terraform plan” to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
[root@nick bin]#
[root@nick bin]# terraform plan
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_instance.nicktest-1 will be created
+ resource “aws_instance” “nicktest-1” {
+ ami = “ami-0fc61db8544a617ed”
+ arn = (known after apply)
+ associate_public_ip_address = (known after apply)
+ availability_zone = (known after apply)
+ cpu_core_count = (known after apply)
+ cpu_threads_per_core = (known after apply)
+ get_password_data = false
+ host_id = (known after apply)
+ id = (known after apply)
+ instance_state = (known after apply)
+ instance_type = “t2.micro“
+ ipv6_address_count = (known after apply)
+ ipv6_addresses = (known after apply)
+ key_name = (known after apply)
+ outpost_arn = (known after apply)
+ password_data = (known after apply)
+ placement_group = (known after apply)
+ primary_network_interface_id = (known after apply)
+ private_dns = (known after apply)
+ private_ip = (known after apply)
+ public_dns = (known after apply)
+ public_ip = (known after apply)
+ secondary_private_ips = (known after apply)
+ security_groups = (known after apply)
+ source_dest_check = true
+ subnet_id = (known after apply)
+ tenancy = (known after apply)
+ volume_tags = (known after apply)
+ vpc_security_group_ids = (known after apply)
+ ebs_block_device {
+ delete_on_termination = (known after apply)
+ device_name = (known after apply)
+ encrypted = (known after apply)
+ iops = (known after apply)
+ kms_key_id = (known after apply)
+ snapshot_id = (known after apply)
+ throughput = (known after apply)
+ volume_id = (known after apply)
+ volume_size = (known after apply)
+ volume_type = (known after apply)
}
+ enclave_options {
+ enabled = (known after apply)
}
+ ephemeral_block_device {
+ device_name = (known after apply)
+ no_device = (known after apply)
+ virtual_name = (known after apply)
}
+ metadata_options {
+ http_endpoint = (known after apply)
+ http_put_response_hop_limit = (known after apply)
+ http_tokens = (known after apply)
}
+ network_interface {
+ delete_on_termination = (known after apply)
+ device_index = (known after apply)
+ network_interface_id = (known after apply)
}
+ root_block_device {
+ delete_on_termination = (known after apply)
+ device_name = (known after apply)
+ encrypted = (known after apply)
+ iops = (known after apply)
+ kms_key_id = (known after apply)
+ throughput = (known after apply)
+ volume_id = (known after apply)
+ volume_size = (known after apply)
+ volume_type = (known after apply)
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
————————————————————————
Note: You didn’t specify an “-out” parameter to save this plan, so Terraform
can’t guarantee that exactly these actions will be performed if
“terraform apply” is subsequently run.
[root@nick bin]# terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_instance.nicktest-1 will be created
+ resource “aws_instance” “nicktest-1” {
+ ami = “ami-0fc61db8544a617ed”
+ arn = (known after apply)
+ associate_public_ip_address = (known after apply)
+ availability_zone = (known after apply)
+ cpu_core_count = (known after apply)
+ cpu_threads_per_core = (known after apply)
+ get_password_data = false
+ host_id = (known after apply)
+ id = (known after apply)
+ instance_state = (known after apply)
+ instance_type = “t2.micro“
+ ipv6_address_count = (known after apply)
+ ipv6_addresses = (known after apply)
+ key_name = (known after apply)
+ outpost_arn = (known after apply)
+ password_data = (known after apply)
+ placement_group = (known after apply)
+ primary_network_interface_id = (known after apply)
+ private_dns = (known after apply)
+ private_ip = (known after apply)
+ public_dns = (known after apply)
+ public_ip = (known after apply)
+ secondary_private_ips = (known after apply)
+ security_groups = (known after apply)
+ source_dest_check = true
+ subnet_id = (known after apply)
+ tenancy = (known after apply)
+ volume_tags = (known after apply)
+ vpc_security_group_ids = (known after apply)
+ ebs_block_device {
+ delete_on_termination = (known after apply)
+ device_name = (known after apply)
+ encrypted = (known after apply)
+ iops = (known after apply)
+ kms_key_id = (known after apply)
+ snapshot_id = (known after apply)
+ throughput = (known after apply)
+ volume_id = (known after apply)
+ volume_size = (known after apply)
+ volume_type = (known after apply)
}
+ enclave_options {
+ enabled = (known after apply)
}
+ ephemeral_block_device {
+ device_name = (known after apply)
+ no_device = (known after apply)
+ virtual_name = (known after apply)
}
+ metadata_options {
+ http_endpoint = (known after apply)
+ http_put_response_hop_limit = (known after apply)
+ http_tokens = (known after apply)
}
+ network_interface {
+ delete_on_termination = (known after apply)
+ device_index = (known after apply)
+ network_interface_id = (known after apply)
}
+ root_block_device {
+ delete_on_termination = (known after apply)
+ device_name = (known after apply)
+ encrypted = (known after apply)
+ iops = (known after apply)
+ kms_key_id = (known after apply)
+ throughput = (known after apply)
+ volume_id = (known after apply)
+ volume_size = (known after apply)
+ volume_type = (known after apply)
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only ‘yes’ will be accepted to approve.
Enter a value: yes
aws_instance.nicktest-1: Creating…
aws_instance.nicktest-1: Still creating… [10s elapsed]
aws_instance.nicktest-1: Still creating… [20s elapsed]
aws_instance.nicktest-1: Creation complete after 27s [id=i-07883b59bcc922e51]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
[root@nick bin]# terraform show
# aws_instance.nicktest-1:
resource “aws_instance” “nicktest-1” {
ami = “ami-0fc61ef7a2b8544a617ed”
arn = “arn:aws:ec2:us-east-1:047906495434:instance/i-07883b59bcc922e51″
associate_public_ip_address = true
availability_zone = “us-east-1e”
cpu_core_count = 1
cpu_threads_per_core = 1
disable_api_termination = false
ebs_optimized = false
get_password_data = false
hibernation = false
id = “i-07833343459bcc922e51″
instance_state = “running”
instance_type = “t2.micro“
ipv6_address_count = 0
ipv6_addresses = []
monitoring = false
primary_network_interface_id = “eni-08aa9d6ec05f22570”
private_dns = “ip-172-31-62-225.ec2.internal”
private_ip = “172.31.62.187“
public_dns = “ec2-3-85-136-118.compute-1.amazonaws.com”
public_ip = “3.85.136.265“
secondary_private_ips = []
security_groups = [
“default”,
]
source_dest_check = true
subnet_id = “subnet-fce7f5c2”
tenancy = “default”
volume_tags = {}
vpc_security_group_ids = [
“sg-2838b90d”,
]
credit_specification {
cpu_credits = “standard”
}
enclave_options {
enabled = false
}
metadata_options {
http_endpoint = “enabled”
http_put_response_hop_limit = 1
http_tokens = “optional”
}
root_block_device {
delete_on_termination = true
device_name = “/dev/xvda“
encrypted = false
iops = 100
throughput = 0
volume_id = “vol-0d0cfc51b13e65986”
volume_size = 8
volume_type = “gp2”
}
}
[root@nick bin]#