map freely

Ho provato a buttarmi (finalmente dico io) su Slony. Si tratta di un sistema per replicare uno o più database da un server postgresql su un altro o piu’ server (anche remoti).

Ho fatto un piccolo test (per ora) cercando di replicare il db spaziale (con estensione postgis) della carta tecnica regionale numerica su un altro server.

Su entrambi i server gira Debian stable (ad oggi “squeeze”) ed hanno rispettivamente gli IP: 192.168.1.71 e 100.0.4.231.

Creiamo il database (vuoto) sul server slave con il comando:

$ su

# su postgres

# psql

=# CREATE DATABASE ctr_new_slave OWNER sit template=template_gis;

e lo strutturiamo come “ctr_new” (db master):

$ pg_dump -s -U postgres -h localhost ctr_new | psql -U postgres -h 100.0.4.71 ctr_new_slave

Quest’ultimo comando è stato impartito dal server master.

Installare slony via apt-get:

# apt-get install slony1-2-bin postgresql-8.4-slony1-2

Il database (già presente) deve avere abilitato il linguaggio procedurale plpgsql. Essendo un db creato mediante un “template_gis” che già incorpora questo linguaggio non ho dovuto fare niente. In caso di nuovo db ricordarsi invece di abilitarlo con:

(come utente postgres) # createlang plpgsql nome_db

Una prerogativa per il funzionamento di slony è che ogni tabella del db abbia una chiave primaria (notare che possono essere replicate anche solo alcune tabelle specificandole nel file di configurazione che vedremo dopo).

Configurazione degli script altperl

Ci sono diversi modi per configurare e manovrare slony: in questo esempio useremo gli script altperl (si tratta di script in perl appunto).

ci spostiamo della directory “/etc/slony/” e copiamo al suo interno il file di configurazione di esempio che si trova in “/usr/share/doc/slony1-2-bin/examples/slon_tools.conf-sample.gz” (dopo averlo decompresso e averne fatto una copia chiamata slon_tools.conf)

(come root):

# cd /etc/slony

# cp /usr/share/doc/slony1-2-bin/examples/slon_tools.conf .

modifichiamo il file con nano:

# nano slon_tools.conf

riporto il file:

======================================================

# $Id: slon_tools.conf-sample,v 1.8.2.4 2009-08-17 22:21:38 devrim Exp $
# Author: Christopher Browne
# Copyright 2004-2009 Afilias Canada
# Revised extensively by Steve Simms

# Keeping the following three lines for backwards compatibility in
# case this gets incorporated into a 1.0.6 release.
#
# TODO: The scripts should check for an environment variable
# containing the location of a configuration file.  That would
# simplify this configuration file and allow Slony-I tools to still work
# in situations where it doesn’t exist.
#
if ($ENV{“SLONYNODES”}) {
    require $ENV{“SLONYNODES”};
} else {

    # The name of the replication cluster.  This will be used to
    # create a schema named _$CLUSTER_NAME in the database which will
    # contain Slony-related data.
    $CLUSTER_NAME = ‘replication’;

    # The directory where Slony should record log messages.  This
    # directory will need to be writable by the user that invokes
    # Slony.
    $LOGDIR = ‘/var/log/slony1′;

    # SYNC check interval (slon -s option)
    # $SYNC_CHECK_INTERVAL = 1000;

    # Which node is the default master for all sets?
    $MASTERNODE = 1;

    # Which debugging level to use?  [0-4]
    $DEBUGLEVEL = 2;

    # Include add_node lines for each node in the cluster.  Be sure to
    # use host names that will resolve properly on all nodes
    # (i.e. only use ‘localhost’ if all nodes are on the same host).
    # Also, note that the user must be a superuser account.

    add_node(node     => 1,
         host     => ’192.168.1.71′,
         dbname   => ‘ctr_new’,
         port     => 5432,
         user     => ‘postgres’,
             password => ”);

    add_node(node     => 2,
         host     => ’100.0.4.71′,
         dbname   => ‘ctr_new_slave’,
         port     => 5432,
         user     => ‘postgres’,
             password => ”);

}

# The $SLONY_SETS variable contains information about all of the sets
# in your cluster.

$SLONY_SETS = {

    # A unique name for the set
    “set1″ => {

    # The set_id, also unique
    ”set_id” => 1,

    # Uncomment the following line to change the origin
    # (a.k.a. master) for the set.  The default is $MASTERNODE.
    #
    # “origin” => 1,

    # If this is set to 1, table and sequence names will be folded to lower-case
    # to match the way that PostgreSQL handles unquoted names.
    # For example, CREATE TABLE ACCOUNT(…) actually turns into CREATE TABLE account(…);
    # unless you put quotes around the table name
    # Slony always quotes object names, so you may get a mis-match between the table-name
    # as PostgreSQL understands it, and as Slony represents it.
    # default value is 0
    #
    # foldCase => 0,

    # The first ID to use for tables and sequences that are added
    # to the replication cluster.  This must be unique across the
    # cluster.
    #
    # TODO: This should be determined automatically, which can be
    # done fairly easily in most cases using psql.  create_set
    # should derive it, and give an option to override it with a
    # specific value.
    ”table_id”    => 1,
    ”sequence_id” => 1,

    # This array contains a list of tables that already have
    # primary keys.
        “pkeyedtables” => [
               'public.aree_ter',
               'public.astefer',
               'public.asteflu',
               'public.astevia',
               'public.cam_fabb',
               'public.curve',
               'public.discont',
               'public.ele_div',
               'public.err_l',
               'public.fabbr_l',
               'public.fabbric',
               'public.ferrovie',
               'public.geometry_columns',
               'public.idrogr_a',
               'public.idrograf',
               'public.infras_a',
               'public.infrastr',
               'public.inquadra',
               'public.lim_amm',
               'public.limamm_a',
               'public.nodofer',
               'public.nodoflu',
               'public.nodovia',
               'public.opere',
               'public.punti_q',
               'public.sim_disc',
               'public.sim_elem',
               'public.sim_fabb',
               'public.sim_idro',
               'public.sim_infr',
               'public.sim_oper',
               'public.sim_vege',
               'public.sim_viab',
               'public.spatial_ref_sys',
               'public.testi',
               'public.veget_a',
               'public.vegetaz',
               'public.viab_a',
               'public.viabilit',
               ],

    # For tables that have unique not null keys, but no primary
    # key, enter their names and indexes here.
    #”keyedtables” => {
    #    ‘table3′ => ‘index_on_table3′,
    #    ‘table4′ => ‘index_on_table4′,
    #},

    # If a table does not have a suitable key or set of keys that
    # can act as a primary key, Slony can add one.
    #
    # Note: The Slony development team does not recomment this
    # approach — you should create your own primary keys instead.
    # “serialtables” => ["table5"],

    # Sequences that need to be replicated should be entered here.
    #”sequences” => ['sequence1',
    #        'sequence2',
    #        ],
    },

};

# Keeping the following three lines for backwards compatibility in
# case this gets incorporated into a 1.0.6 release.
#
# TODO: The scripts should check for an environment variable
# containing the location of a configuration file.  That would
# simplify this configuration file and allow Slony tools to still work
# in situations where it doesn’t exist.
#
if ($ENV{“SLONYSET”}) {
    require $ENV{“SLONYSET”};
}

# Please do not add or change anything below this point.
1;

======================================================

Ho modificato il file di esempio togliendo le parti relative a “keyedtables” e “serialtables” e inserendo i nomi delle tabelle (con relativo schema) da replicare. Inoltre ho lasciato inalterate le impostazioni per “$CLUSTER_NAME = ‘replication’;” e “$LOGDIR = ‘/var/log/slony1′;”. La prima identificherà con un prefisso un nuovo schema nei due db e l’altra indica dove salvare i file di log.

Configurazione di slonik

Una volta editato il file di configurazione possiamo usalro per generare gli script da passare al modulo “slonik” (siamo sempre all’interno della  cirectory “/etc/slony/”).

# slonik_init_cluster > initcluster

# slonik_create set 1 > createset

# slonik_subscribe_set 1 2 > subscribeset

In questo modo vengono creati 3 file ognuno dei quali contiene il codice per impostare un cluster e farlo partire.

Accendere i motori

Ora possiamo fare partire il tutto:

# slonik < initcluster

che dovrebbe produrre un output simile a questo:

<stdin>:10: Set up replication nodes
<stdin>:13: Next: configure paths for each node/origin
<stdin>:16: Replication nodes prepared
<stdin>:17: Please start a slon replication daemon for each node

Può succedere che compaiano dei messaggi di errore (nel mio caso avevo scritto alcuni nomi di tabelle sbagliate nel file slon_tools.conf). In questo caso correggere il file suddetto, rimuovere i tre script creati con “# rm initcluster createset subscribset“, poi entrare dei due db “ctr_new” e “ctr_new_slave” e rimuovere lo schema “_replication” con “=# DROP SCHEDA -replication CASCADE;“. Infine ripartire con la creazione degli script di configurazione.

Avviamo slon sul server 1:

# slon_start 1

Invoke slon for node 1 – /usr/bin/slon -p /var/run/slony1/node1.pid -s 1000 -d2 replication ‘host=192.168.1.71 dbname=ctr_new user=postgres port=5432′ >>/var/log/slony1/node1-ctr_new.log 2>&1 </dev/null &
Slon successfully started for cluster replication, node node1
PID [5815]
Start the watchdog process as well…

e avviamo slon sul server 2:

# slon_start 2
Invoke slon for node 2 – /usr/bin/slon -p /var/run/slony1/node2.pid -s 1000 -d2 replication ‘host=100.0.4.71 dbname=ctr_new_slave user=postgres port=5432′ >>/var/log/slony1/node2-ctr_new_slave.log 2>&1 </dev/null &
Slon successfully started for cluster replication, node node2
PID [5879]
Start the watchdog process as well…

Ora creiamo cluster e subscribe

# slony < createset

<stdin>:16: Subscription set 1 created
<stdin>:17: Adding tables to the subscription set
<stdin>:21: Add primary keyed table public.aree_ter
<stdin>:25: Add primary keyed table public.astefer
<stdin>:29: Add primary keyed table public.asteflu
<stdin>:33: Add primary keyed table public.astevia
<stdin>:37: Add primary keyed table public.cam_fabb
<stdin>:41: Add primary keyed table public.curve
<stdin>:45: Add primary keyed table public.discont
<stdin>:49: Add primary keyed table public.ele_div
<stdin>:53: Add primary keyed table public.err_l
<stdin>:57: Add primary keyed table public.fabbr_l
<stdin>:61: Add primary keyed table public.fabbric
<stdin>:65: Add primary keyed table public.ferrovie
<stdin>:69: Add primary keyed table public.geometry_columns
<stdin>:73: Add primary keyed table public.idrogr_a
<stdin>:77: Add primary keyed table public.idrograf
<stdin>:81: Add primary keyed table public.infras_a
<stdin>:85: Add primary keyed table public.infrastr
<stdin>:89: Add primary keyed table public.inquadra
<stdin>:93: Add primary keyed table public.lim_amm
<stdin>:97: Add primary keyed table public.limamm_a
<stdin>:101: Add primary keyed table public.nodofer
<stdin>:105: Add primary keyed table public.nodoflu
<stdin>:109: Add primary keyed table public.nodovia
<stdin>:113: Add primary keyed table public.opere
<stdin>:117: Add primary keyed table public.punti_q
<stdin>:121: Add primary keyed table public.sim_disc
<stdin>:125: Add primary keyed table public.sim_elem
<stdin>:129: Add primary keyed table public.sim_fabb
<stdin>:133: Add primary keyed table public.sim_idro
<stdin>:137: Add primary keyed table public.sim_infr
<stdin>:141: Add primary keyed table public.sim_oper
<stdin>:145: Add primary keyed table public.sim_vege
<stdin>:149: Add primary keyed table public.sim_viab
<stdin>:153: Add primary keyed table public.spatial_ref_sys
<stdin>:157: Add primary keyed table public.testi
<stdin>:161: Add primary keyed table public.veget_a
<stdin>:165: Add primary keyed table public.vegetaz
<stdin>:169: Add primary keyed table public.viab_a
<stdin>:173: Add primary keyed table public.viabilit
<stdin>:176: Adding sequences to the subscription set
<stdin>:177: All tables added

Ed infine:

# slony < subscribeset

<stdin>:4: NOTICE:  subscribe set: omit_copy=f
<stdin>:4: NOTICE:  subscribe set: omit_copy=f
CONTEXT:  SQL statement “SELECT  “_replication”.subscribeSet_int( $1 ,  $2 ,  $3 ,  $4 ,  $5 )”
funzione PL/pgSQL “subscribeset” linea 68 a PERFORM
<stdin>:10: Subscribed nodes to set 1

Tutto è partito: se andiamo a vedere i file di log in “/var/log/slony/” vediamo cio’ che accade.

Per fare un test ho provato a caricare la stessa tabella (spaziale) in QGIS (la stessa tabella del db master e dello slave). Modificando (aggiunto un elemento geometrico) la tabella e facendo un refresh si notano quasi istantaneamenete i cambiamenti replicati anche nel db slave!

Ogni commento è benvenuto.