Pablo Iranzo Gómez's blog

Filtering email with imapfilter

Since some time ago, email filter management was not scaling for me as I was using server-side filtering, I had to deal with the web-based interface which was missing some elements like drag&drop reordering of rules, cloning, etc.

As I was already using offlineimap to sync from the remote mailserver to my system into a maildir folder, I had almost all the elements I needed.

After searching for several options imapfilter seemed to be a perfect fit, so I started with a small set of rules and start integration with my email process.

On my first attempts, I setup a pre-sync hook on offlineimap by using as well as the postsync hook I already had:

presynchook  = time imapfilter
postsynchook = ~/.mutt/postsync-offlineimap.sh

Initial attempts were not good at all, applying filters on the remote imapserver was very time consuming and my actual 1 minute delay after finishing one check was becoming a real 10-15 minute interval between checks because of the imapfiltering and this was not scaling as I was putting new rules.

After some tries, and as I already had all the email synced offline, moved filtering to be locally instead of server-side, but as imapfilter requires an imap server, I tricked dovecot into using the local folder to be offered via imap:

protocols = imap
mail_location = maildir:~/.maildir/FOLDER/:INBOX=~/.maildir/FOLDER/.INBOX/
auth_debug_passwords=yes

This also required to change my foldernames to use "." in front of them, so I needed to change mutt configuration too for this:

set mask=".*"

and my mailfoders script:

set mbox_type=Maildir
set folder="~/.maildir/FOLDER"
set spoolfile="~/.maildir/FOLDER/.INBOX"

#mailboxes `echo -n "+ "; find ~/.cache/notmuch/mutt/results ~/.maildir/FOLDER -type d -not -name 'cur' -not -name 'new' -not -name 'tmp' -not -name '.notmuch' -not -name 'xapian' -not -name 'FOLDER' -printf "+'%f' "`

mailboxes `find ~/.maildir/FOLDER -type d -name cur -printf '%h '|tr " " "\n"|grep -v "^/home/iranzo/.maildir/FOLDER$"|sort|xargs echo`
#Store reply on current folder
folder-hook . 'set record="^"'

After this, I could start using imapfilter and start working on my set of rules... but first problem appeared, apparently I started having some duplicated email as I was cancelling and rerunning the script while debugging so a new tool was also introduced to 'dedup' my imap folder named IMAPdedup with a small script:

#!/bin/bash
(
for folder in $(python ~/.bin/imapdedup.py -s localhost  -u iranzo    -w '$PASSWORD'  -m -c -v  -l)
do
    python ~/.bin/imapdedup.py -s localhost  -u iranzo    -w '$PASSWORD'  -m -c  "$folder"

done
) 2>&1|grep "will be marked as deleted"

This script was taking care of listing all email foders on 'localhost' with my username and password (can be scripted or use external tools to gather it) and dedup email after each sync (in my postsync-offlinemap.sh as well as lbdq script for fetchning new addresses, notmuch and running imapfilter after syncing (to cath the limited filtering I do sever-side)

I still do some server-side filtering (4 rules), to get on a "Pending sort" folder all email which is either:

• New support cases remain at INBOX
• All emails from case updates, bugzilla, etc to _pending
• All emails containing 'list' or 'bounces' in from to _pending
• All emails not containing me directly on CC or To, to _pending

This more or less ensures a clean INBOX with most important things still there, and easier rule handling for email sorting.

So, after some tests, this is at the moment a simplified version of my filtering file:

---------------
--  Options  --
---------------

options.timeout = 30
options.subscribe = true
options.create = false

function offlineimap (key)
    local status
    local value
    status, value = pipe_from('grep -A2 ACCOUNT ~/.offlineimaprc | grep -v ^#|grep '.. key ..'|cut -d= -f2')C
        value = string.gsub(value, ' ', '')
        value = string.gsub(value, '\n', '')
        return value
end

----------------
--  Accounts  --
----------------

-- Connects to "imap1.mail.server", as user "user1" with "secret1" as
-- password.
EXAMPLE = IMAP {
    server = 'localhost',
    username = 'iranzo',
    password = '$PASSWORD',
    port = 143
}
-- My email
myuser = 'ranzo'

function mine(messages)
    email=messages:contain_cc(myuser)+messages:contain_to(myuser)+messages:contain_from(myuser)
    return email
end

function filter(messages,email,destination)
    messages:contain_from(email):move_messages(destination)
    messages:contain_to(email):move_messages(destination)
    messages:contain_cc(email):move_messages(destination)
    messages:contain_field('sender', email):move_messages(destination)
end

function deleteold(messages,days)
    todelete=messages:is_older(days)-mine(messages)
    todelete:move_messages(EXAMPLE['Trash'])
end


-- Define the msgs we're going to work on

-- Move sent messages to INBOX to later sorting
sent = EXAMPLE.Sent:select_all()
sent:move_messages(EXAMPLE['INBOX'])

inbox = EXAMPLE['INBOX']:select_all()
pending = EXAMPLE['INBOX/_pending']:select_all()
todos = pending + inbox

-- Mark as read messages sent from my user
todos:contain_from(myuser):is_recent():mark_seen()

-- Delete google calendar forwards
todos:contain_to('piranzo@gapps.example.com'):delete_messages()

-- Move all spam messages to Junk folder
spam = todos:contain_field('X-Spam-Score','*****')
spam:move_messages(EXAMPLE['Junk'])

-- Move Jive notifications
filter(todos,'jive-notify@example.com',EXAMPLE['INBOX/EXAMPLE/Customers/_jive'])

-- Filter EXAMPLEN
filter(todos,'dev-null@rhn.example.com',EXAMPLE['Trash'])

-- Filter PNT
filter(todos:contain_subject('[PNT] '),'noreply@example.com',EXAMPLE['Trash'])

-- Filter CPG (Customer Private Group)
filter(todos:contain_subject('Red Hat - Group '),'noreply@example.com',EXAMPLE['INBOX/EXAMPLE/Customers/Other/CPG'])

-- Remove month start reminders
todos:contain_subject('mailing list memberships reminder'):delete_messages()

-- Delete messages about New accounts created (RHN)
usercreated=todos:contain_subject('New Red Hat user account created')*todos:contain_from('noreply@example.com')
usercreated:delete_messages()

-- Search messages from CPG's
cpg = EXAMPLE['INBOX/EXAMPLE/Customers/Other/CPG']:select_all()
cpg:contain_subject('Cust1'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust1/CPG'])
cpg:contain_subject('Cust2'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust2/CPG'])
cpg:contain_subject('Cust3'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust3/CPG'])
cpg:contain_subject('Cust4'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust4/CPG'])

-- Move bugzilla messages
filter(todos:contain_subject('] New:'),'bugzilla@example.com',EXAMPLE['INBOX/EXAMPLE/Customers/_bugzilla/new'])
filter(todos,'bugzilla@example.com',EXAMPLE['INBOX/EXAMPLE/Customers/_bugzilla'])

-- Move all support messages to Other for later processing
filter(todos:contain_subject('(NEW) ('),'support@example.com',EXAMPLE['INBOX/EXAMPLE/Customers/_new'])
filter(todos:contain_subject('Case '),'support@example.com',EXAMPLE['INBOX/EXAMPLE/Customers/Other/cases'])

EXAMPLE['INBOX/EXAMPLE/Customers/_new']:is_seen():move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Other/cases'])

support = EXAMPLE['INBOX/EXAMPLE/Customers/Other/cases']:select_all()
-- Restart the search only for messages in Other to also process if we have new rules

support:contain_subject('is about to breach its SLA'):delete_messages()
support:contain_subject('has breached its SLA'):delete_messages()
support:contain_subject(' has had no activity in '):delete_messages()

-- Here the process is customer after customer and mark as read messages from non-prio customers
support:contain_body('Cust1'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust1/cases'])
support:contain_body('Cust2'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust2/cases'])
support:contain_body('Cust3'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust3/cases'])
support:contain_body('Cust4'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust4/cases'])

-- For customer swith common matching names, use header field
support:contain_field('X-SFDC-X-Account-Number', 'XXXX'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust5/cases'])
support:contain_body('Customer         : COMMONNAME'):move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust6/cases'])

-- Non prio customers (mark updates as read)
cust7 = support:contain_body('WATCHINGCUST') + support:contain_body('Cust7')
cust7:mark_seen()
cust7:move_messages(EXAMPLE['INBOX/EXAMPLE/Customers/Cust7/cases'])

-- Filter other messages by domain
filter(todos,'todos.es', EXAMPLE['INBOX/EXAMPLE/Customers/Cust8'])

-- Process all remaining messages in INBOX + all read messages in pending-sort for mailing lists and move to lists folder
filter(todos,'list', EXAMPLE['INBOX/Lists'])
filter(todos,'bounces',EXAMPLE['INBOX/Lists'])

-- Add EXAMPLE lists, inbox and _pending and Fedora default bin for reprocessing in case a new list has been added
lists = todos + EXAMPLE['INBOX/Lists']:select_all() + EXAMPLE['INBOX/Lists/Fedora']:select_all()

-- Mailing lists

-- EXAMPLE
filter(lists,'outages-list',EXAMPLE['INBOX/Lists/EXAMPLE/general/outage'])
filter(lists,'announce-list',EXAMPLE['INBOX/Lists/EXAMPLE/general/announce'])

-- Fedora
filter(lists,'kickstart-list',EXAMPLE['INBOX/Lists/Fedora/kickstart'])
filter(lists,'ambassadors@lists.fedoraproject.org',EXAMPLE['INBOX/Lists/Fedora/Ambassador'])
filter(lists,'infrastructure@lists.fedoraproject.org',EXAMPLE['INBOX/Lists/Fedora/infra'])
filter(lists,'announce@lists.fedoraproject.org',EXAMPLE['INBOX/Lists/Fedora/announce'])
filter(lists,'lists.fedoraproject.org',EXAMPLE['INBOX/Lists/Fedora'])

-- OSP
filter(lists,'openstack@lists.openstack.org',EXAMPLE['INBOX/Lists/OpenStack'])
filter(lists,'openstack-es@lists.openstack.org',EXAMPLE['INBOX/Lists/OpenStack/es'])

-- Filter my messages not filtered back to INBOX
mios=pending:contain_from(myuser)
mios:move_messages(EXAMPLE['INBOX'])

-- move messages we're in BCC to INBOX for manual sorting
hidden = pending - mine(pending)
hidden:move_messages(EXAMPLE['INBOX'])

-- Start processing of messages older than:
maxage=60

-- Delete old messages from mailing lists
deleteold(EXAMPLE['INBOX/Lists/EXAMPLE/general/media'],maxage)
deleteold(EXAMPLE['INBOX/Lists/EXAMPLE/general/outage'],maxage)

-- delete old cases
maxage=180

-- for each in $(cat .imapfilter/config.lua|grep -i cases|tr " ,()" "\n"|grep cases|sort|uniq|grep -v ":" );do echo "deleteold($each,maxage)";done
deleteold(EXAMPLE['INBOX/EXAMPLE/Customers/Cust1/cases'],maxage)
deleteold(EXAMPLE['INBOX/EXAMPLE/Customers/Cust2/cases'],maxage)
deleteold(EXAMPLE['INBOX/EXAMPLE/Customers/Cust3/cases'],maxage)
deleteold(EXAMPLE['INBOX/EXAMPLE/Customers/Other/cases'],maxage)

deleteold(EXAMPLE['INBOX/EXAMPLE/Customers/_bugzilla'],maxage)

-- Empty trash every 7 days
maxage=7
deleteold(EXAMPLE['Trash'],maxage)

As this is applied filtering twice, offlineimap might be uploading part of your changes already, making it faster to next syncs, and suffle some of your emails while it runs.

The point of adding the already filtered set to be filtered again (CPG, cases, etc) is that if a new customer is consiredered to be filter on a folder of its own, the messages will be picked up and moved accordingly automatically ;-)

Hope it helps, and happy filtering!

RHEV-M with nested VM for OSP

Since some time ago, I've been mostly dealing with OpenStack, requiring different releases to test for different tests, etc.

Virtualization, as provided by KVM requires some CPU flags to get accelerated operations, vmx and svm depending on your processor architecture, but, of course, this is only provided on bare-metal.

In order to get more flexibility at the expense of performance, nestedvt allows to expose those flags to the VM's running at the hypervisor so you can run another level of VM's inside those VM's (this starts to sound like the movie Inception).

The problem, so far is that this required changes on the kernel and drivers to make it work, and was lacking lot of stability, so this is something NOT SUPPORTED FOR PRODUCTION USE but which makes perfect sense for demo environments, labs, etc, allowing you to maximize the use of your hardware for better flexibility but at the cost of performance.

As I was using RHEV for managing my home-lab I hit the first issue, my hypervisors (HP Proliant G7 N54L) where using RHEL-6 as operating system, and the support for nested was not very good, but luckily, RHEV-M 3.5 includes support for hypervisors running on RHEL-7, enabling to use latest features included in kernel, networking stack, etc.

First step, was to redeploy the servers, wasn't that hard, but required some extra steps as I had another unsupported approach (servers were sharing local storage over NFS for providing Storage Domains to environment, HIGHLY UNSUPPORTED), so I moved them from NFS to iSCSI provided by an external server and with the help of the kickstart I use for other systems, I started the process.

Once the two servers were migrated, the last one, finished moving VM's from NFS to iSCSI and needed to be put on maintenance and enable the other two (as a safety measure, RHEL-6 and RHEL-7 hosts cannot coexist on the same cluster in RHEV).

From here, just needed to enable NestedVT on the environment.

NestedVT 'just' requires to expose the svm or vmx flag to the VM running directly from the bare-metal host, and we need to do that for every VM we start. On normal system with libvirt, we can just edit the XML for the VM definition and define the CPU like this:

<cpu mode='custom' match='exact'>
    <model fallback='allow'>Opteron_G3</model>
    <feature policy='require' name='svm'/>
</cpu>

For RHEV, however, we don't have an XML we can edit, as it is created dynamically with the contents of the database for the VM (disks, NICS, name, etc), but we've the VDSM-Hooks mechanism for doing this.

Hooks in vdsm are a powerful and dangerous tool, as they can modify in-flight the XML used to create the VM, and allow lot of features to be implemented.

In the past, for example, those hooks could be used to provide DirectLUN support to RHEV, or fixed BIOS Serial Number for VM's where the product was still lacking the official feature, and in this case, we'll use them to provide the CPU flags we need.

As you can imagine, this is something that has lot of interested people behind, and we can find upstream a repository with VDSM-Hooks.

In this case, the one that we're needing is 'nestedvt', so we can proceed to install it on our hosts like:

wget http://mirrors.ibiblio.org/ovirt/pub/ovirt-3.4/rpm/el7/noarch/vdsm-hook-nestedvt-4.14.17-0.el7.noarch.rpm
rpm -Uvh vdsm-hook-nestedvt-4.14.17-0.el7.noarch.rpm

You'll need to put a host in maintenance and activate for VDSM to refresh the hooks installed and start new VM so we have the hook injecting the XML.

After it boots, egrep 'svm|vmx' /proc/pcuinfo should show the flags there.

But wait...

RHEV also includes a security feature that makes it impossible for a VM to spy on the communications meant to other VM's that makes it impossible to simulate other MAC's within it, and this is performed via libvirt filters on the interfaces.

To come to our rescue, another hook comes to play in, this time macspoof which allows to disable this security measure for a VM so it can execute virtualization within.

First, let's repeat the procedure and install the hook on all of our hypervisors:

wget http://mirrors.ibiblio.org/ovirt/pub/ovirt-3.4/rpm/el7/noarch/vdsm-hook-macspoof-4.14.17-0.el7.noarch.rpm
rpm -Uvh vdsm-hook-macspoof-4.14.17-0.el7.noarch.rpm

This will enable the hook in the system, but we also need to make the RHEV-M Engine aware of it, so we need to define a new Custom Property for VM's:

engine-config -s "UserDefinedVMProperties=macspoof=(true|false)"

This will ask us for the compatibility version (we'll choose 3.5) and enable a new true/false property for VM's that require this security measure lifted. We're doing of course this approach instead of disabling it for everyone to limit it's use to just the VM's needing it, not losing all the benefits on security provided.

As a side note, macspoof plugin is available in official repositories for RHEL7 hypervisor, so you can use this instead of oVirt's repo one.

Now when we create a new VM, for example to use with OpenStack, we can go to custom properties for this vm, select 'macspoof' and set a value of 'true' and once the VM is started will be able to see the processor extensions for virtualization and at the same time, the VM's created within, will be able to communicate with the outside world.

Enjoy!

Intel AMT on Linux for remote control/fencing

Hi,

Some time ago, and after discussing with a colleague, I had a look on Intel's AMT, and this week I demoed it for another colleague as a cheap-replacement for having power fencing capabilities on commodity hardware.

AMT provides a server-like Out of band management like iLO, iDrac, RSB etc and it's included in i3 with vPro processors/chipsets of some equipment.

I did the test on a Lenovo X200/201 system I had as old laptop.

The steps used for configuring it, require to:

• first enable the support in the BIOS, usually named 'Intel AMT' or 'Intel Active Management Technology'.
• After this step it was possible to use the command to enter the special AMT firmware Intel(R) Management Engine which on this laptop is enabled with CTRL-P.
• If this is the first time you enable it, you'll require to change the default admin password to something secure, usually mixed upper-lower case, symbol and numbers.
  • For this example we'll be using Qwer123$ as password.
• Explore the settings, enable it and validate network settings.
  • I've enabled DHCP on both LAN and Wireless for IPv4 and IPv6, and enabled KVM redirection
• Once finished, save changes and exit from firmware screen and let the system boot.

From another host, you can perform the remaining configuration steps, from now on, the 'target' system will be intercepting packets sent to specific port via the network cards and redirect to AMT firmware instead of going to target host. This is something important to note, the packets are only intercepting when coming from OUTSIDE the host so we'll use a second computer to access it.

You can use a browser pointing to target system's IP at port 16992, for example: http://target:16992

From that web interface and once logging with admin and the password set Qwer123$ we can continue doing some configuration, like the power states to control (for example, this laptop could be remotely powered when it was with the charger connected even if laptop was powered off).

Now, for doing the 'command-line' part, we will need to install one package on our system and rum some scripts.

# First we'll install amtterm wsmancli

dnf -y install amtterm wsmancli

# This will provide the two commands we'll later use, wsman for configuration and amttool for power control

# We need to define the host to use and password as well as the password we'll use for console redirection (via VNC)

AMT_PASSWORD='Qwer123$'
AMT_HOST=target
VNC_PASSWORD='Qwer123$'

# we can define those vars (specially AMT_PASSWORD) in our .profile or .bash_profile in order to avoid typing them everytime

# set the vnc password (must be 8 characters MAX)
wsman put http://intel.com/wbem/wscim/1/ips-schema/1/IPS_KVMRedirectionSettingData -h ${AMT_HOST} -P 16992 -u admin -p ${AMT_PASSWORD} -k RFBPassword=${VNC_PASSWORD}

# enable KVM redirection to port 5900 (this will also intercept 5900 port for console redirection, so make it sure you'll not need it later)
wsman put http://intel.com/wbem/wscim/1/ips-schema/1/IPS_KVMRedirectionSettingData -h ${AMT_HOST} -P 16992 -u admin -p ${AMT_PASSWORD} -k Is5900PortEnabled=true

# disable opt-in policy (do not ask user for console access)
wsman put http://intel.com/wbem/wscim/1/ips-schema/1/IPS_KVMRedirectionSettingData -h ${AMT_HOST} -P 16992 -u admin -p ${AMT_PASSWORD} -k OptInPolicy=false

# disable session timeout (do not timeout sessions)
wsman put http://intel.com/wbem/wscim/1/ips-schema/1/IPS_KVMRedirectionSettingData -h ${AMT_HOST} -P 16992 -u admin -p ${AMT_PASSWORD} -k SessionTimeout=0

# enable KVM (enable keyboard/video/monitor redirection)
wsman invoke -a RequestStateChange http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_KVMRedirectionSAP -h ${AMT_HOST} -P 16992 -u admin -p ${AMT_PASSWORD} -k RequestedState=2

# OPTIONAL: view settings (validate all the settings)
wsman get http://intel.com/wbem/wscim/1/ips-schema/1/IPS_KVMRedirectionSettingData -h ${AMT_HOST} -P 16992 -u admin -p ${AMT_PASSWORD}

After this step, we should be able to use vinagre target to access the KVM redirection and remotely control our system.

For example, to control power of host you can use:

# Check host status:
amttool $AMT_HOST info

# Power up a powered-off host:
amttool $AMT_HOST powerup

# Power down a powered-on host:
amttool $AMT_HOST powerdown

Check man amttool for other commands like reset, powercycle.

IMPORTANT: note that some power state changes can only be performed based on previous status, you can check with info the available ones and current status of system.

As a bonus, there's a RFE¹ for requesting this tool to be incorporated as power fencing mechanism in fence-agents once 'amtterm' is included in RHEL, in the meantime it's already available in Fedora, and when it comes to RHEL, hopefully could also be used as fence agent for Clusters and RHEV.

Enjoy!

Install RHEL7/Centos/Fedora on a software raid device

Table of contents

1. Why or why not use a RAID via software
   1. Pros
   2. Cons
2. Performing the setup
3. Creating the raid devices with Multiple Devices mdadm

Installing Linux on a RAID has lot of advantages, from using RAID1 to enjoy protection against drive failures or RAID0 to combine the size of several drives to create bigger space for files with all the smaller disks we have.

There are several RAID level definitions and may have different uses depending on our needs and hardware availability.

For this, I focused on using raid1 for the system disks (for greater redundancy/protection against failures) and raid0 (for combining several disks to make bigger space available for non important data)..

Why or why not use a RAID via software

Pros

• There's no propietary data on the disks that could require this specific controller in case the hardware fails.
• Can be performed on any system, disk combination, etc

Cons

• The use of dedicated HW RAID cards allows to offload the CPU intensive tasks for raid calculation, etc to the dedicated processor, freeing internal CPU for system/user usage.
• Dedicated cards may have fancier features that require no support from the operating system as are all implemented by the card itself and presented to the OS as a standard drive.

Performing the setup

As I was installing on a HP Microserver G8 recently, I had to first disable the advanced mode for the included controller, so it behaved like a standard SATA one, once done, I was able to boot from my OS image (in this case EL7 iso).

Once the ISO is booted in rescue mode, I could switch to the second console with ALT-F2 so I could start executing commands on the shell.

First step is to setup partitioning, in this case I did two partitions, first one for holding /boot and the second one for setting up the LVM physical volume where the other Logical Volumes will be defined later.

I've elected this setup over others because mdadm allows transparent support for booting (grub supports booting form it) and easy to manage setup.

For partitions, remember to allocate at least 500mb for /boot and as much as needed for your SO, for example, if only base OS is expected to have RAID protection, having a 20Gb partition will be enough, leaving the remaining disk to be used for a RAID0 device for allocating non-critical files.

For both partitions, set type with fdisk to fd: Linux RAID autodetect, and setup the two drives we'll use for initial setup using the same values, for example:

fdisk /dev/sda
n # for new partition
p # for primary
<ENTER> # for first sector
+500M # for size
t # for type
fd # for Linux RAID autodetect
n # new partition
p # primary
<ENTER>
+20G #for size
t #for type
2 # for select 2nd partition
fd # for Linux RAID autodetect
# n for new partition
p # for primary
<ENTER> # for first sector
<ENTER> # for remaining disk
t # for type
3 # for third partition
fd # for Linux RAID Autodetect
w # for Writing changes

And repeat that for /dev/sdb

At this point, we'll have both sda and sdb with the same partitions defined: sd{a,b}1 with 500Mb for /boot and sd{a,b}2 with 20Gb for LVM and the remaining disk for RAID0 LVM.

Now, it's time to create the raid device on top, for simplicity, I tend to use md0 for /boot, so let's start with it.

Creating the raid devices with Multiple Devices mdadm

Let's create the raid devices for each system, starting with /boot:

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sda2 /dev/sdb2
mdadm --create /dev/md2 --level=0 --raid-devices=2 /dev/sda3 /dev/sdb3

Now, check the status of the raid device creation by issuing:

cat /proc/mdstat

Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 sda1[0] sdb1[1]
      534760 blocks level 1, 64k chunk, algorithm 2 [2/2] [UU]
            [==>..................]  recovery = 12.6% (37043392/292945152) finish=127.5min speed=33440K/sec
md1 : active raid1 sda2[0] sdb2[1]
      20534760 blocks level 1, 64k chunk, algorithm 2 [2/2] [UU]
            [=====>...............]  recovery = 25.9% (37043392/692945152) finish=627.5min speed=13440K/sec
...

When it finishes, all the devices will appear as synced, and we can start the installation of the operating system.

What I did, after this point, is to reboot the install media, so I could use anaconda installer to select manually the filesystems, creating /boot on /dev/md0, then the Physical Volume on /dev/md1 for the operating system.

Select the manual partitioning during the installation to define above devices as their intended usage, and once it has been installed, create the additional Physical volume on /dev/md2 and define the intended mountpoints, etc.

Enjoy!

Customize RHEL/CentOS installation media (EL4/EL5+)

Table of contents

1. Introduction
2. Preparing everything
3. DVD background image at boot prompt
4. Including updates
5. Removing unused packages
6. Adding extra software
7. Recreating metadata
   1. EL4
   2. EL5
8. Finishing

Introduction

A standard install media, (let's talk about a DVD for easier start) has several files/folders at his root, but most important are:

• isolinux (where the loader lives)
• images (for extra files for installer to load)
• Packages for installation (RedHat/ for EL4, Server/Client for EL5)

Usually, a distribution has for it's main binaries more than 2 Gb of data, that enables one target to act as a multifunction server/workstation, but that you will not usually load on the same system. Furthermore, since the DVD creation, there have been so many updates/patches that make your installation a 'outdated' install that you'll need to upgrade to have recent patches.

Wouldn't it be better to have one install media suited for your target systems with all available updates applied?

Preparing everything

First, we'll need to copy all of our DVD media to a folder in our harddrive, including those hidden files on DVD root (the ones telling installer which cd-sets are included and some other info).

Let's asume that we'll work on /home/user/DVD/

After we've copied everything from our install media, we'll start customizing :)

DVD background image at boot prompt

We can customize DVD background image and even keyboard layout by tweaking "isolinux/isolinux.cfg" with all required fields (Check Syslinux Documentation to check proper syntax)

On Kickstart: instalaciones automatizadas para anaconda (spanish) you can also check how to create a kickstart, so you can embed it on this DVD and configure isolinux.cfg to automatic provision a system

Including updates

The easiest way would be to install a system with all required package set from original DVD media, and then connect that system to an update server to fetch but not install them.

• EL4: up2date -du # yum upgrade —downloadonly
• EL5: yum upgrade —downloadonly

After you download every single update, you'll need to copy them to a folder like /home/user/DVD/updates/.

Well, now let's start the funny work:

For each package in updates/, you'll need to remove old version from original folder (remember: Client/ Server/ or RedHat/RPMS ), and place in that folder the updated one...

After some minutes, you'll have all updates in place... and you can remove the DVD/updates/ folder as it will be empty after placing each updated RPM in the folder where the previous versions was.

Removing unused packages

Well, after having everthing in place, we'll start removing unused files. Usually, we could check every package install status on 'test' system by checking rpm, but that's going to be a way looooong task, so we can 'automate' it a bit by doing:

• If you have ssh password less connection between your systems (BUILD and TARGET):

On BUILD system:

#!bash
for package in *.rpm
do
    NAME=`rpm -q —queryformat '%NAME' $package` ssh TARGET "rpm -q $NAME >/dev/null 2>&1 || echo rm $package" |tee things-to-do
done

• If you don't have ssh password-less setup (using private/public key auth or kerberos), you can do something similar this way:

On BUILD system:

#!bash
for package in *.rpm
do
    NAME=`rpm -q —queryformat '%NAME' $package` echo "$package:$NAME" > packages-on-DVD
done

Then copy that file on your TARGET system and running:

On TARGET system:

#!bash
for package in `cat packages-on-DVD`
do
    QUERY=`echo $package|cut -d ":" -f 2` FILE=`echo $package|cut -d ":" -f 1` rpm -q —queryformat '%NAME' $QUERY >/dev/null 2>&1 || echo rm $FILE|tee things-to-do
done

After you finish, you'll have a file named things-to-do, in which you'll see commands like 'rm packagename-version.rpm'

If you're confident about it's contents, you can run sh things-to-do and have all 'not installed on TARGET' packages removed from your DVD folder.

Adding extra software

In the same way we added updates, we can also add new software to be deployed along base system like monitoring utilities, custom software, HW drivers, etc, just add packages to desired folders before going throught next steps.

Recreating metadata

After all our adds and removals, we need to tell installer that we changed packages, and update it's dependencies, install order, etc.

EL4

This one is trickier, but it is still possible in a not so hard way, first of all, we need to update some metadata files (hdlist) and Package order for installation, it can be dificult if we add extra packages, as we'll have special care:

Gererate first version of hdlists:

#!bash
export PYTHONPATH=/usr/lib/anaconda-runtime:/usr/lib/anaconda
/usr/lib/anaconda-runtime/genhdlist —withnumbers /home/user/DVD/
/usr/lib/anaconda-runtime/pkgorder /home/user/DVD/ i386 |tee /home/user/order.txt

Review order.txt to check all packages added by hand to check correct or include missing packages and then continue with next commands:

#!bash
export PYTHONPATH=/usr/lib/anaconda-runtime:/usr/lib/anaconda
/usr/lib/anaconda-runtime/genhdlist —withnumbers /home/user/DVD/ —fileorder /home/user/order.txt

EL5

Using createrepo we'll recreate metadata, but we've to keep care and use comps.xml to provide 'group' information to installer, so we'll need to run:

#!bash
createrepo -g /home/DVD/Server/repodata/groupsfile.xml /home/DVD/Server/

Finishing

At this step you'll have a DVD structure on your hard drive, and just need to get an ISO to burn and test:

#!bash
mkisofs -v -r -N -L -d -D -J -V NAME -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -x lost+found -m .svn -o MyCustomISO.iso /home/user/DVD/

Now, it's time to burn MyCustomISO.iso and give it a try ;-)

PD: While testing is just better to keep using rewritable media until you want to get a 'release'