288 |
clusters. For instance, using the I<heartbeat> system |
clusters. For instance, using the I<heartbeat> system |
289 |
(http://linux-ha.org/) to watch the health of nodes and in the |
(http://linux-ha.org/) to watch the health of nodes and in the |
290 |
case of one machine failure start services on a secondary node. |
case of one machine failure start services on a secondary node. |
291 |
|
See section bellow for more informations. |
292 |
|
|
|
Appropriate scripts that can call monit to start/stop specific |
|
|
services are needed on both nodes - typical usage: |
|
293 |
|
|
294 |
FILE DESCRIPTION |
|
295 |
----------------------------------- |
=head2 Monit with Heartbeat |
296 |
/etc/inittab starts monit |
|
297 |
/etc/rcS.d/S41heartbeat execute "monit start heartbeat" |
The first thing you have to do is install and configure |
298 |
/etc/init.d/monit-node1 execute "monit -g node1 start" |
I<heartbeat> (http://www.linux-ha.org/downloads) . |
299 |
/etc/init.d/monit-node2 execute "monit -g node2 start" |
The Getting Started Guide is very usefull for this task |
300 |
|
(http://www.linux-ha.org/download/GettingStarted.html). |
301 |
This way hearbeat can easily control the cluster state and if one |
|
302 |
node fails, hearbeat will start monit-xxxxx on the running node |
B<Starting up a Node> |
303 |
and monit is instructed to start the services of the failing node |
|
304 |
and monitor them... |
This is the normal start sequence for a cluster-node. |
305 |
|
With this sequence, there should be no error-case, which is not |
306 |
|
handled either by heartbeat or monit. For example, if monit |
307 |
|
dies, initd restart it. If heatbeat dies, monit restart it. If |
308 |
|
the node dies, heartbeat on the other node dedect it and restart |
309 |
|
the services there. |
310 |
|
|
311 |
|
1) initd starts monit with group local |
312 |
|
2) monit starts heartbeat in local group |
313 |
|
3) heartbeat requests monit to start the node group |
314 |
|
4) monit starts the node group |
315 |
|
|
316 |
|
B<Monit F</etc/monitrc>> |
317 |
|
|
318 |
|
This sample describe a cluster with 2 nodes. |
319 |
|
Services running on Node 1 are in group I<node1>, Node 2 services |
320 |
|
are in I<node2>. |
321 |
|
|
322 |
|
The local group entries are mode I<active>, the node group |
323 |
|
entries are mode I<manual> and controlled by heartbeat |
324 |
|
|
325 |
|
# |
326 |
|
# local services on every host |
327 |
|
# |
328 |
|
# |
329 |
|
check heartbeat with pidfile /var/run/heartbeat.pid |
330 |
|
start program = "/etc/init.d/heartbeat start" |
331 |
|
stop program = "/etc/init.d/heartbeat start" |
332 |
|
mode active |
333 |
|
alert foo@bar |
334 |
|
group local |
335 |
|
# |
336 |
|
# |
337 |
|
check postfix with pidfile /var/spool/postfix/pid/master.pid |
338 |
|
start program = "/etc/init.d/postfix start" |
339 |
|
stop program = "/etc/init.d/postfix stop" |
340 |
|
mode active |
341 |
|
alert foo@bar |
342 |
|
group local |
343 |
|
# |
344 |
|
# node1 services |
345 |
|
# |
346 |
|
check apache with pidfile /var/apache/logs/httpd.pid |
347 |
|
start program = "/etc/init.d/apache start" |
348 |
|
stop program = "/etc/init.d/apache stop" |
349 |
|
depends named |
350 |
|
alert foo@bar |
351 |
|
mode manual |
352 |
|
group node1 |
353 |
|
# |
354 |
|
# |
355 |
|
check named with pidfile /var/tmp/named.pid |
356 |
|
start program = "/etc/init.d/named start" |
357 |
|
stop program = "/etc/init.d/named stop" |
358 |
|
alert foo@bar |
359 |
|
mode manual |
360 |
|
group node1 |
361 |
|
# |
362 |
|
# node2 services |
363 |
|
# |
364 |
|
check named-slave with pidfile /var/tmp/named-slave.pid |
365 |
|
start program = "/etc/init.d/named-slave start" |
366 |
|
stop program = "/etc/init.d/named-slave stop" |
367 |
|
mode manual |
368 |
|
alert foo@bar |
369 |
|
group node2 |
370 |
|
# |
371 |
|
# |
372 |
|
check squid with pidfile /var/squid/logs/squid.pid |
373 |
|
start program = "/etc/init.d/squid start" |
374 |
|
stop program = "/etc/init.d/squid stop" |
375 |
|
depends named-slave |
376 |
|
alert foo@bar |
377 |
|
mode manual |
378 |
|
group node2 |
379 |
|
|
380 |
|
B<initd F</etc/inittab>> |
381 |
|
|
382 |
|
Monit is started on both nodes with initd. You have to add a |
383 |
|
entry in F</etc/inittab> starting monit with the local group, |
384 |
|
where heartbeat is member of. |
385 |
|
|
386 |
|
#/etc/inittab |
387 |
|
mo:2345:respawn:/usr/local/bin/monit -i -d 10 -c /etc/monitrc -g local |
388 |
|
|
389 |
|
B<heartbeat F</etc/ha.d/haresources>> |
390 |
|
|
391 |
|
When heartbeat starts, heartbeat lookup for the node entry and |
392 |
|
start the script F</etc/init.d/monit-node1> or |
393 |
|
F</etc/init.d/monit-node2>. The script calls monit |
394 |
|
to start the node specific group. |
395 |
|
|
396 |
|
# /etc/ha.d/haresources |
397 |
|
node1 IPaddr::172.16.100.1 monit-node1 |
398 |
|
node2 IPaddr::172.16.100.2 monit-node2 |
399 |
|
|
400 |
|
|
401 |
|
B<F</etc/init.d/monit-node1>> |
402 |
|
|
403 |
|
#!/bin/bash |
404 |
|
# |
405 |
|
# sample script for starting/stopping all services for node1 |
406 |
|
# |
407 |
|
prog="/usr/local/bin/monit -g node1" |
408 |
|
start() |
409 |
|
{ |
410 |
|
echo -n $"Starting $prog:" |
411 |
|
$prog start |
412 |
|
echo |
413 |
|
} |
414 |
|
|
415 |
|
stop() |
416 |
|
{ |
417 |
|
echo -n $"Stopping $prog:" |
418 |
|
$prog stop> |
419 |
|
echo |
420 |
|
} |
421 |
|
|
422 |
|
case "$1" in |
423 |
|
start) |
424 |
|
start;; |
425 |
|
stop) |
426 |
|
stop;; |
427 |
|
*) |
428 |
|
echo $"Usage: $0 {start|stop}" |
429 |
|
RETVAL=1 |
430 |
|
esac |
431 |
|
exit $RETVAL |
432 |
|
|
433 |
|
|
434 |
=head1 ALERT MESSAGES |
=head1 ALERT MESSAGES |