How to setup same DKIM settings for multiple (virtualmin) servers

You already have one server with a dkim key pair and use virtualmin “DomainKeys Identified Mail”:

  1. Edit the dkim option on the source server and add the new domain (I personally use sub domains for all my servers) so for example if your initial mail server is s1.domain.com and your new server (the one you will copy the dkim key on) is s2.domain.com your “Domains to sign for” section must contain:

s1.domain.com

s1

s2.domain.com

s2

  1. Save this on the source server. The key will be updated….
  2. The private key on my ubuntu server is filed under: /etc/dkim.key
  3. If your target server is the same [OS], first of all proceed to enable DKIM on the virtualmin “DomainKeys Identified Mail”, you may want to use the same details than the source server in the “Domains to sign for” section & “Save”, this will create the key pairs.
  4. I personally then disabled dkim on the TARGET server before doing the next step, but it might not be required(!?).
  5. When complete, edit the /etc/dkim.key on the TARGET server with vi or otherwise replace the private key with the one from the SOURCE server (you should make a backup of the file first, always do a backup!).
  6. Go back to the virtualmin “DomainKeys Identified Mail” in the TARGET server page and enable the dkim outgoing email but with the option “Force generation of new private key?” to “NO” and “Save”

The private key will then be read from the /etc/dkim.key and used to generate the exact same public and DKIM DNS records for domains as the source server together with all the required settings to make it work.

You’re all set and the DNS can be edited if the DKIM DNS records for domains has changed, mine does not seem to have.

Synchronisation between two Virtualmin servers

When, like me, you get paranoid to losing your data or web server functionality…

I have created a little perl script to allow the synchronization of MySQL databases and /home between my Webservers.

The master server (where the script runs from) is the main production server, the salve server is on standby just in case.

I can appreciate that the passwords are clearly inserted in the script and it is a security issue. Nonetheless, I am the only admin and no other users have ftp or other privileges on both servers. I am sure that there is a way to prevent this but I am happy with the current situation and, obviously I am also making a regular backup of the master server on an external backup provider (I use rsync.net).

#!/usr/bin/perl
# Performs a synchronisation of home folder and dumps sql databases 
# from one Virtual server to another using rsync and secure shell 
# 
# Written by G.Serex Sharpnet UK (c) 03.12.2020 

# Var definitions 

############### SQL Config ############# 
# SQL root username 
$username = "root"; 
# Local SQL root password 
$password = "localmysqlpassword"; 
# Remote SQL root password 
$rpassword = "remotemysqlpassword"; 
# The dumped files path . (absolute path + trailing / please) 
$dumped_dbs_path = "/root/mysql/"; 
# The dumped file name 
$dumped_db = "dump.sql"; 
# Name of the database to exclude from the dump (here the mysql and sys are obviously dedicated to each server, so don't dump them!) 
$exclude_database = "mysql,sys,information_schema,performance_schema"; 

################ SSH Config ################# 

# The remote host name 
$remotehost = "ipaddress"; 

#The ssh username 
$sshusername = "root"; 

#The ssh port 
$sshport = "xx"; 

#____ E N D _ V A R _ D E F S. ________________ 

# First check and optimise the lot.

# A little house keeping 
system("/usr/bin/mysqlcheck --optimize --all-databases --auto-repair -u $username -p$password"); 

# Dump the dbs 
system("/usr/bin/mysqlpump -u $username -p$password --exclude-databases=$exclude_database --add-drop-table --result-file=$dumped_dbs_path$dumped_db"); 

# Transfer them abroad 
system("/usr/bin/rsync -avz -e 'ssh -p $sshport' $dumped_dbs_path $sshusername\@$remotehost:$dumped_dbs_path"); 

# Restore the dump abroad 

system("/usr/bin/ssh -p $sshport $sshusername\@$remotehost 'mysql -u root -p$rpassword < $dumped_dbs_path$dumped_db'"); 

# rsync the home directory 

system("/usr/bin/rsync -avz --delete -e 'ssh -p $sshport' /home/ $sshusername\@$remotehost:/home"); 
exit;

CSF/LFD Sasl Auth Failure

My Daily Logwatch reports a large amount of SMTP Authentication errors like the following:

--------------------- sasl auth daemon Begin ------------------------

 **Unmatched Entries** 

:auth failure: [user=smr@clarky.net] [service=smtp] [realm=clarky.net] [mech=pam] [reason=PAM auth error] 
: auth failure: [user=megaplan@clarky.net] [service=smtp] [realm=clarky.net] [mech=pam] [reason=PAM auth error] 
: auth failure: [user=pdf@clarky.net] [service=smtp] [realm=clarky.net] [mech=pam] [reason=PAM auth error] 
: auth failure: [user=development@clarky.net] [service=smtp] [realm=clarky.net] [mech=pam] [reason=PAM auth error]

This can cause a large amount of lines reported on the email report.

To counter this, here are the steps on an Ubuntu system.

1.Edit /etc/csf/csf.conf to find CUSTOM1_LOG and edit the content to:

CUSTOM1_LOG= "/var/log/mail.log"

2. Add the regex to catch the failed attempts against SASL by adding the following regular expression in /usr/local/csf/bin/regex.custom.pm:

 if (($lgfile eq $config{
CUSTOM1_LOG}) and ($line =~ /^\S+\s+\d+\s+\S+ \S+ postfix\/smtpd\[\d+\]: warning:.*\[(\d+\.\d+\.\d+\.\d+)\]: SASL [A-Z]*? authentication failed/)) {
    return ("Failed SASL login from",$1,"mysaslmatch","3","25","3600");
 }

3. restart CSF and LFD:

#csf -r
#service lfd restart
#systemctl restart lfd

After that, be prepared to get a lot of emails about the blocked IPs for the set time (default is 3600 seconds or 1 hour). They eventually taper out with the most offending IP’s getting permanently blocked 🙂

WordPress twenty seventeen different header images in different pages

spam blockers

Came across the challenge of having to customise the WordPress Twentyseventeen theme to cater for header images that match content of specific pages, let’s say, for example that one has a WP site with portraits of different animals in various pages of the site, whilst twenty seventeen theme allows you to upload a serie of header images that can be randomly rotated on the pages as the are loaded by the end user it does not specifically allow to have the same system dedicated to each pages. The following example remedies this.

After digging a bit and following some other posts on the web, here are the steps required to enable this feat.

Note that you must be able to edit the PHP files and thus be proficient in editing PHP and be familiar with WP inner workings…

First edit the wp-content/themes/twentyseventeen/header.php

(make a backup first!):

...

<div id="page" class="site">
 <a class="skip-link screen-reader-text" href="#content"><?php _e( 'Skip to content', 'twentyseventeen' ); ?></a>

<header id="masthead" class="site-header" role="banner">

<?php get_template_part( 'template-parts/header/header', 'image' ); ?>

 <?php if ( has_nav_menu( 'top' ) ) : ?>
 <div class="navigation-top">
 <div class="wrap">
 <?php get_template_part( 'template-parts/navigation/navigation', 'top' ); ?>
 </div><!-- .wrap -->
 </div><!-- .navigation-top -->
 <?php endif; ?>
</header><!-- #masthead -->

...

Remove or comment out the orange line and replace with the following code.

...

<div id="page" class="site">
 <a class="skip-link screen-reader-text" href="#content"><?php _e( 'Skip to content', 'twentyseventeen' ); ?></a>

<header id="masthead" class="site-header" role="banner">

<?php
// start simply copy header-image.php to header-dog.php and mod the file accordingly
if(is_page("dog-paintings-and-dog-portraits")) {

get_template_part( 'template-parts/header/header', 'dog' );

}
else{

get_template_part( 'template-parts/header/header', 'image' );

}
// end mod
 ?>

<?php if ( has_nav_menu( 'top' ) ) : ?>
 <div class="navigation-top">
 <div class="wrap">
 <?php get_template_part( 'template-parts/navigation/navigation', 'top' ); ?>
 </div><!-- .wrap -->
 </div><!-- .navigation-top -->
 <?php endif; ?>

...

in the example above the page is called

dog-paintings-and-dog-portraits

The header file called is called header-dog.php under the wp-content/themes/twentyseventeen/template-parts/header/ folder (which is a copy of the header-image.php and edited to deal with the other images)

if there are more than one page, simply add more conditionals ifs for each individual pages and copy header-image.php to match like above.

More to come as I write the code…

Plesk 11 to VM – Part 3: Customisation and migration

Intro

In this series of Best Practice articles I talk about migrating websites and emails from a Plesk 11 run server to a Virtualmin run server. It describes in details the steps involved, including the eventual pitfalls and other thing I came across whilst doing it. Note that this documentation is addressed to people with medium expertise in linux/system administration.

Useful customisation

Most of VM defaul settings are fine, however in order to make it a little easier to migrate websites, the following mods are useful.

Server Template – Apache directives

As Plesk websites are hosted under $Documentroot/httpdocs this can be set by default before creating website by editing the Apache website default config:

VM Menu > Server Templates > Default Settings > Edit template section: apache website

Change: DocumentRoot ${HOME}/public_html
To: DocumentRoot ${HOME}/httpdocs

Change: <Directory ${HOME}/public_html>
To: <Directory ${HOME}/httpdocs>

Under “Users’ website subdirectory to create“.

Change: Default (public_html)
To: [httpdocs]

Account Plans

One may customise them to match previous Plesk hosting “plans”, however this is not mandatory.

Virtual website creation – Admin username

The only requirement in this process is to create the Administration username as a “Custom username” giving it the domain.tld name (i.e. mydomain.com). This way the websites paths will end up being: /home/domain.tld, which makes it easier to do a path change from /var/www/vhosts to /home wherever there are scripts configurations containing full paths.

Perl

I am a perl script writer thus I use perl extensively, over the years I have created scripts that require a few additional modules. I shall not go into details with them (yet), however there’s an important bit I must mention, some perl scripts I use have a different #!. Some point to the sometimes non-existent /usr/local/bin/perl, so in order to avoid these script throwing an error I simply link it to the whereis perl (usally /usr/bin/perl or /bin/perl):

ln -s /usr/bin/perl /usr/local/bin/perl

Website content transfer

To transfer website content from Plesk to VM it is a matter of rsyncing each website with the rsync command,  to minimise the load, the following commands are run at nice 20 from the source server, note that the target server must have the virtual server already configured prior to doing this, otherwise an error will be thrown:

rsync -avz --rsh='ssh -p{port}' /var/qmail/mailnames/{domain}/{user}/  root@{server}:/home/{domain}/homes/{user}/

Where:

  • port is the ssh port (if different than 22, which is recommended for security)
  • domain is the domain to be transferred
  • target is is the target server IP or domain

Depending on the size of the website this might take some time. Note that I am not really needing to transfer statistical information over. most if not all the required files are under httpdocs and cgi-bin in my case.

After the transfer, on the target server, ownership of the directories must be set:

chown -R {username}:{username} /home/{domain}/httpdocs /home/{domain}/cgi-bin

Where:

  • username is the owner of the httpd and cgi-bin content (in my case I created a username matching the domain, making it easier to find the correct directory under /home when working over the CLI.
  • domain is the domain

Email content transfer

The operation of transferring emails to the target server are pretty much the same as website transfer, with a couple of exceptions.

nice -20 rsync -avz --rsh='ssh -p{port}' /var/qmail/mailnames/{domain} root@{target}:/home/{domain}/home/

Where:

  • port is the ssh port (if different than 22, which is recommended for security)
  • domain is the domain to be transferred
  • target is is the target server IP or domain

After the transfer, on the target server, ownership of the directories must be set:

IMPORTANT: note user is user.domain (not just domain)

chown -R {user}.{domain}:{domain} /home/{domain}/homes/{user}/

Where:

  • user is the name of the user
  • domain is the name of the domain

IMAP changes (At client level, here using Thunderbird)

Server name: no change
Username: replace @ with .
Connection security: from SSL/TLS to STARTTLS
Authentication method: from encrypted password to normal password

SMTP changes (At client level, here using Thunderbird)

Server name: no change
Port from 25 to 587
Connection security: from SSL/TLS to STARTTLS
Authentication method: from encrypted password to normal password
Username: replace @ with .

Database content transfer

Source server

mysqlcheck --optimize {database} --auto-repair  -uadmin -p`cat /etc/psa/.psa.shadow`
mysqldump -uadmin -p`cat /etc/psa/.psa.shadow` --flush-logs --opt {database} > {database}.sql

Where:

  • database is the database name

Target server (after rsyncing all the dbs dumps on it)

VM Menu > Edit Databases > (manage current database usually called {domain}_{tld}) or create a new one if required.
click Manage …
Click Execute SQL
Run SQL from file Tab
From local file (browse to file)
Click Execute

Special characters

Sites  with accents (like French’s)

The charset must be edited in the following Module:

Select Domain > Menu > Services > Configure Website > Languages >  Character set for documents     (x) iso-8859-1

Save, apply changes.

languages

Plesk 11 to VM – Part 1: Why move?

Intro

In this series of Best Practice articles I talk about migrating websites and emails from a Plesk 11 run server to a Virtualmin run server. It describes in details the steps involved, including the eventual pitfalls and other thing I came across whilst doing it. Note that this documentation is addressed to people with medium expertise in linux/system administration.

Fed up with Plesk?Fed up with Plesk?

Since  2003 I have been using two brands of Dedicated Servers Control Panels: Ensim (RIP) and Plesk. So it’s died 🙁 oh well, Ensim was an awful CP, never liked it, I liked Plesk (from 8 to 11) a bit more and it has been my selected Control Panel for many years. However the time has come to make way for a relatively new comer: Virtualmin or VM for short, a very mature and simpler CP indeed with a vibrant Community.

Deciding to select another Control Panel is a big decision, especially when your bottom line is at stake, so after having extensively tested and used VM for the last few months I finally decided to make the move, VM is slick, simple and easy to use, it’s like Ronseal, it does what it says on the tin.

Continue reading Plesk 11 to VM – Part 1: Why move?

Plesk 11 to VM – Part 2: Bare server set up

Intro

In this series of best practice articles I talk about migrating websites and emails from a Plesk 11 managed server to a Virtualmin managed server. It describes in some details the steps involved in doing it remedy the pitfalls and other things I came across whilst doing it. Note that this documentation is address to people with medium expertise in linux/system administration.

Hardware

A fresh install of a super fast multi-core web server with sufficient RAM and [Hardware] RAID 1 partitioned as follow: Continue reading Plesk 11 to VM – Part 2: Bare server set up

Reverting to passworded SSH (root) logins

My normal ssh logins are with public keys authentication (aka passwordless), however, under certain circumstances I’d like to revert back to good old passwords logins. Like when I want to transfer a Plesk server across another server using the Plesk Migration & Transfer manager.

After scouring the Net for some inspiration, which failed, and some tinkering, I finally found a neat solution to switch between the two (passwordless and normal authentication logins) quickly, the process is relatively simple* but frankly puzzling IMHO. I shall explain why: Continue reading Reverting to passworded SSH (root) logins

Yum clamav upgrade broken on virtualmin VPS

I got a vps with virtualmin installed, I recently upgraded clam antivirus using webmin/virtualmin, this resulted in the clamd failing to start.

After some research it turns out that I am running clamd with user clam instead of clamav. Yet yum upgrades clam with the user clamav and other directories in the configuration;

In short here is what I experienced after the upgrade (these must be changed back into the /etc/clamd.conf file): Continue reading Yum clamav upgrade broken on virtualmin VPS