Plesk 11 to VM – Part 3: Customisation and migration

Intro

In this series of Best Practice articles I talk about migrating websites and emails from a Plesk 11 run server to a Virtualmin run server. It describes in details the steps involved, including the eventual pitfalls and other thing I came across whilst doing it. Note that this documentation is addressed to people with medium expertise in linux/system administration.

Useful customisation

Most of VM defaul settings are fine, however in order to make it a little easier to migrate websites, the following mods are useful.

Server Template – Apache directives

As Plesk websites are hosted under $Documentroot/httpdocs this can be set by default before creating website by editing the Apache website default config:

VM Menu > Server Templates > Default Settings > Edit template section: apache website

Change: DocumentRoot ${HOME}/public_html
To: DocumentRoot ${HOME}/httpdocs

Change: <Directory ${HOME}/public_html>
To: <Directory ${HOME}/httpdocs>

Under “Users’ website subdirectory to create“.

Change: Default (public_html)
To: [httpdocs]

Account Plans

One may customise them to match previous Plesk hosting “plans”, however this is not mandatory.

Virtual website creation – Admin username

The only requirement in this process is to create the Administration username as a “Custom username” giving it the domain.tld name (i.e. mydomain.com). This way the websites paths will end up being: /home/domain.tld, which makes it easier to do a path change from /var/www/vhosts to /home wherever there are scripts configurations containing full paths.

Perl

I am a perl script writer thus I use perl extensively, over the years I have created scripts that require a few additional modules. I shall not go into details with them (yet), however there’s an important bit I must mention, some perl scripts I use have a different #!. Some point to the sometimes non-existent /usr/local/bin/perl, so in order to avoid these script throwing an error I simply link it to the whereis perl (usally /usr/bin/perl or /bin/perl):

ln -s /usr/bin/perl /usr/local/bin/perl

Website content transfer

To transfer website content from Plesk to VM it is a matter of rsyncing each website with the rsync command,  to minimise the load, the following commands are run at nice 20 from the source server, note that the target server must have the virtual server already configured prior to doing this, otherwise an error will be thrown:

rsync -avz --rsh='ssh -p{port}' /var/qmail/mailnames/{domain}/{user}/  root@{server}:/home/{domain}/homes/{user}/

Where:

  • port is the ssh port (if different than 22, which is recommended for security)
  • domain is the domain to be transferred
  • target is is the target server IP or domain

Depending on the size of the website this might take some time. Note that I am not really needing to transfer statistical information over. most if not all the required files are under httpdocs and cgi-bin in my case.

After the transfer, on the target server, ownership of the directories must be set:

chown -R {username}:{username} /home/{domain}/httpdocs /home/{domain}/cgi-bin

Where:

  • username is the owner of the httpd and cgi-bin content (in my case I created a username matching the domain, making it easier to find the correct directory under /home when working over the CLI.
  • domain is the domain

Email content transfer

The operation of transferring emails to the target server are pretty much the same as website transfer, with a couple of exceptions.

nice -20 rsync -avz --rsh='ssh -p{port}' /var/qmail/mailnames/{domain} root@{target}:/home/{domain}/home/

Where:

  • port is the ssh port (if different than 22, which is recommended for security)
  • domain is the domain to be transferred
  • target is is the target server IP or domain

After the transfer, on the target server, ownership of the directories must be set:

IMPORTANT: note user is user.domain (not just domain)

chown -R {user}.{domain}:{domain} /home/{domain}/homes/{user}/

Where:

  • user is the name of the user
  • domain is the name of the domain

IMAP changes (At client level, here using Thunderbird)

Server name: no change
Username: replace @ with .
Connection security: from SSL/TLS to STARTTLS
Authentication method: from encrypted password to normal password

SMTP changes (At client level, here using Thunderbird)

Server name: no change
Port from 25 to 587
Connection security: from SSL/TLS to STARTTLS
Authentication method: from encrypted password to normal password
Username: replace @ with .

Database content transfer

Source server

mysqlcheck --optimize {database} --auto-repair  -uadmin -p`cat /etc/psa/.psa.shadow`
mysqldump -uadmin -p`cat /etc/psa/.psa.shadow` --flush-logs --opt {database} > {database}.sql

Where:

  • database is the database name

Target server (after rsyncing all the dbs dumps on it)

VM Menu > Edit Databases > (manage current database usually called {domain}_{tld}) or create a new one if required.
click Manage …
Click Execute SQL
Run SQL from file Tab
From local file (browse to file)
Click Execute

Special characters

Sites  with accents (like French’s)

The charset must be edited in the following Module:

Select Domain > Menu > Services > Configure Website > Languages >  Character set for documents     (x) iso-8859-1

Save, apply changes.

languages

Plesk 11 to VM – Part 1: Why move?

Intro

In this series of Best Practice articles I talk about migrating websites and emails from a Plesk 11 run server to a Virtualmin run server. It describes in details the steps involved, including the eventual pitfalls and other thing I came across whilst doing it. Note that this documentation is addressed to people with medium expertise in linux/system administration.

Fed up with Plesk?Fed up with Plesk?

Since  2003 I have been using two brands of Dedicated Servers Control Panels: Ensim (RIP) and Plesk. So it’s died 🙁 oh well, Ensim was an awful CP, never liked it, I liked Plesk (from 8 to 11) a bit more and it has been my selected Control Panel for many years. However the time has come to make way for a relatively new comer: Virtualmin or VM for short, a very mature and simpler CP indeed with a vibrant Community.

Deciding to select another Control Panel is a big decision, especially when your bottom line is at stake, so after having extensively tested and used VM for the last few months I finally decided to make the move, VM is slick, simple and easy to use, it’s like Ronseal, it does what it says on the tin.

Continue reading Plesk 11 to VM – Part 1: Why move?

Plesk 11 to VM – Part 2: Bare server set up

Intro

In this series of best practice articles I talk about migrating websites and emails from a Plesk 11 managed server to a Virtualmin managed server. It describes in some details the steps involved in doing it remedy the pitfalls and other things I came across whilst doing it. Note that this documentation is address to people with medium expertise in linux/system administration.

Hardware

A fresh install of a super fast multi-core web server with sufficient RAM and [Hardware] RAID 1 partitioned as follow: Continue reading Plesk 11 to VM – Part 2: Bare server set up

Reverting to passworded SSH (root) logins

My normal ssh logins are with public keys authentication (aka passwordless), however, under certain circumstances I’d like to revert back to good old passwords logins. Like when I want to transfer a Plesk server across another server using the Plesk Migration & Transfer manager.

After scouring the Net for some inspiration, which failed, and some tinkering, I finally found a neat solution to switch between the two (passwordless and normal authentication logins) quickly, the process is relatively simple* but frankly puzzling IMHO. I shall explain why: Continue reading Reverting to passworded SSH (root) logins

Yum clamav upgrade broken on virtualmin VPS

I got a vps with virtualmin installed, I recently upgraded clam antivirus using webmin/virtualmin, this resulted in the clamd failing to start.

After some research it turns out that I am running clamd with user clam instead of clamav. Yet yum upgrades clam with the user clamav and other directories in the configuration;

In short here is what I experienced after the upgrade (these must be changed back into the /etc/clamd.conf file): Continue reading Yum clamav upgrade broken on virtualmin VPS

Primary Default Website in Virtualmin (Apache)

When no virtual websites are in existence, The Apache web server will point to /var/www/html and /var/www/cgi-bin in a base install of Virtualmin (the default). However when a Virtual website is created, this changes to the first available virtual site, so for example, if the hostname is some.domain.com and no virtual sites are configured, typing this URL in a browser will serve the content in /var/www/html. If a virtual site called virtualsite.com is created, the same request will serve the content of /home/virtualsite/public_html.

By design this is normal

Whilst this is the way Apache is designed and there’s nothing wrong with that, it becomes a problem when some applications are only available outside the virtual server realm (i.e. not suexec’ed). For example the server has Nagios or Munin installed and you want to be able to access it even after a virtual server has been created. The solution is to manually create the first virtual server and call it the same as the server hostname (some.domain.com).

Continue reading Primary Default Website in Virtualmin (Apache)

SSL smtp on Virtualmin

By default, virtual servers aren’t setting up SMTP to use SSL. Following this information regarding SSL SMTP I am just summarizing:

 netstat -an | grep :465

Returns nothing, so.

vi /etc/postfix/master.cf

find the lines:

#smtps inet n - n - - smtpd
# -o smtpd_tls_wrappermode=yes
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_client_restrictions=permit_sasl_authenticated,reject
# -o milter_macro_daemon_name=ORIGINATING

And uncomment them:

Continue reading SSL smtp on Virtualmin

index.shtml and #include virtual in Virtualmin

For some reason the default virtualmin install does not have index.shtml in the Directoryindex directive. To enable it, one must edit the directive:

Virtualmin > Services > Configure Website > Edit Directives 

Find the following line:

DirectoryIndex index.html index.htm index.php index.php4 index.php5

Add index.shml at the end  of the line:

DirectoryIndex index.html index.htm index.php index.php4 index.php5 index.shtml

Click “Save” then “Apply Changes” (on the top right hand side of the “Virtual Server Options” page). Checking your virtual website will show the index.shtml page.

To make it a default configuration

In order to enable  this for all further virtual websites:

Continue reading index.shtml and #include virtual in Virtualmin

Virtualmin Virtual Servers (GPL)

Following my last post I now found out that Webmin does not allow multi virtual servers :-(, BUT there is a module called Virtualmin Virtual Servers (GPL) which is what I need.

To save myself hassle, I installed a compatible OS (CentOS 6 64bit) and simply installed virtualmin from the install.sh command after downloading it:

cd /root
wget http://software.virtualmin.com/gpl/scripts/install.sh
sh ./install.sh

After this my next  move was to install csf lfd from configserver.com:
(there is now a new URI for the archive which I changed on 2017-02-08)

wget http://www.configserver.com/free/csf.tgz
wget https://download.configserver.com/csf.tgz
tar zxvf csf.tgz
cd csf
sh ./install.sh

Once installed integrate it in Webmin and you’re good to go.

– Install the csf webmin module in:
Webmin > Webmin Configuration > Webmin Modules > From local file > /etc/csf/csfwebmin.tgz > Install Module

Email access problems

All worked well from the word go except SMTP/POP3 (Dovecot) server; I was getting Failed to connect to localhost:143 : Connection refused  (as well as port 993,995,110 and 25) when trying to connect either via my email client or usermin.

System Information was reporting that Dovecot IMAP / POP3 Server was offline, and trying to start Dovecot failed:

Starting dovecot: Error: socket() failed: Address family not supported by protocol
Error: service(pop3-login): listen(::, 110) failed: Address family not supported by protocol
Error: socket() failed: Address family not supported by protocol
Error: service(pop3-login): listen(::, 995) failed: Address family not supported by protocol
Error: socket() failed: Address family not supported by protocol
Error: service(imap-login): listen(::, 143) failed: Address family not supported by protocol
Error: socket() failed: Address family not supported by protocol
Error: service(imap-login): listen(::, 993) failed: Address family not supported by protocol
Fatal: Failed to start listeners

After editing the /etc/dovecot/dovecot.conf file and commenting out/insert: listen = *  Here is how my edited file looks like:

# A comma separated list of IPs or hosts where to listen in for connections. 
# "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces.
# If you want to specify non-default ports or anything more complex,
# edit conf.d/master.conf.
#listen = *, ::
listen = *

Now, why would Dovecot server come with this line disabled is baffling, anyway, after this, Dovecot starts fine allowing usermin and email clients to connect without problems:

dovecot

 

Server heartbeat

This little perl program allows you to check the availability of an IP address via a cron command that outputs the result in a text file.

#!/usr/bin/perl
# This script pings IP addresses
#
# In a live application, read host list
# from a config file
@hosts = ("192.168.1.1","192.168.1.19");
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
$year += 1900;
$mon += 1;
$min = sprintf("%02d", $min);
$sec = sprintf("%02d", $sec);
$mon = sprintf("%02d", $mon);
$mday = sprintf("%02d", $mday);
$year = sprintf("%02d", $year % 100);

@live = ();
foreach $h (@hosts) {
        $r = `ping -c2 $h`;
        if ($r =~ /2 re/) {
                push @live,$h;
                }
        }
$alive = "@live";
print ("You have $alive on $mday $mon $year @ $hour:$min:$sec\n");

This script can then be used in a cron like:

*/5 * * * * /path/to/file/pingtest.pl >> /some/path/pingtest.txt

This command will write the result on a new line in the file called pingtest.txt every 5 minutes. Which will look like that:

You have 192.168.1.1 192.168.1.19 on 06 01 13 @ 20:05:01
You have 192.168.1.1 192.168.1.19 on 06 01 13 @ 20:10:01
You have 192.168.1.1 192.168.1.19 on 06 01 13 @ 20:15:01