Article ID: 118479, created on Nov 6, 2013, last review on Jun 17, 2016

  • Applies to:
  • Virtuozzo 6.0

Symptoms

After installing Parallels Cloud Storage updates you can see the following error on connecting to the cluster:

Unable connect to cluster, timeout (15 sec) expired

The following messages can be found in /var/log/pstorage/$CLUSTER/mds-1/mgs.log.gz:

05-11-13 04:16:53.648 BUG at cs_wd.c:1885/wd_replay_chunks_balanced()
05-11-13 04:16:53.648 pstorage version: 6.0.4-33 (Debug)
05-11-13 04:16:53.648 ---------- [18 stack frames] ----------
05-11-13 04:16:53.648 /usr/lib64/libpcs_io.so(+0x1dd17) [0x7fd8a3647d17]
05-11-13 04:16:53.648 /usr/lib64/libpcs_io.so(show_trace+0xb5) [0x7fd8a3647e55]
05-11-13 04:16:53.648 /usr/lib64/libpcs_io.so(pcs_err+0x35) [0x7fd8a3647415]
05-11-13 04:16:53.648 /usr/lib64/libpcs_io.so(+0x1d438) [0x7fd8a3647438]
05-11-13 04:16:53.648 /usr/bin/mdsd(wd_replay_chunks_balanced+0x2cc) [0x45f94c]
05-11-13 04:16:53.648 /usr/bin/mdsd() [0x46d8f4]
05-11-13 04:16:53.648 /usr/bin/mdsd(paxos_next_round+0x8c) [0x47090c]
05-11-13 04:16:53.648 /usr/bin/mdsd() [0x4732c2]
05-11-13 04:16:53.648 /usr/bin/mdsd() [0x473537]
05-11-13 04:16:53.648 /usr/bin/mdsd(learner_rcv_learn+0x127) [0x473737]
05-11-13 04:16:53.648 /usr/bin/mdsd() [0x470cef]
05-11-13 04:16:53.648 /usr/bin/mdsd() [0x474477]
05-11-13 04:16:53.648 /usr/lib64/libpcs_io.so(+0x21c6d) [0x7fd8a364bc6d]
05-11-13 04:16:53.648 /usr/lib64/libpcs_io.so(+0xd6c6) [0x7fd8a36376c6]
05-11-13 04:16:53.648 /usr/lib64/libpcs_io.so(+0xbfc9) [0x7fd8a3635fc9]
05-11-13 04:16:53.648 /usr/lib64/libpcs_io.so(+0xc352) [0x7fd8a3636352]
05-11-13 04:16:53.648 /lib64/libpthread.so.0() [0x3b79a07851]
05-11-13 04:16:53.648 /lib64/libc.so.6(clone+0x6d) [0x3b792e892d]
05-11-13 04:16:53.660 pcs_log_terminate
05-11-13 04:17:23.712 pcs_set_logrotate_filenum: 10
05-11-13 04:17:23.712 pcs_set_logrotate_size: 104857600
05-11-13 04:17:23.714 pcs_cfg_def_long: int.slow.commits = 0
05-11-13 04:17:23.714 create_cfg_block_from_data: 0x1cfa110
05-11-13 04:17:23.714 starting MDS#12 (v.12) ...

And in /var/log/pstorage/$CLUSTER/mds-1/fatal.log:

 MDS#12 reports hard error (134 / SIGABRT)
 MDS#12 reports hard error (134 / SIGABRT)
 MDS#12 reports hard error (134 / SIGABRT)

Cause

The problem is recognized as a product bug PSBM-23601. The issue is fixed since PCS 6.0 update 6, which means that it will not happen again after the update is installed.

Resolution

To resolve the problem, it is necessary to recreate the MDS servers that crashed with the update.

If the amount of crashed MDS servers exceeds the quorum, thus making the cluster unmanageable, contact Parallels Support for resolution.

Search Words

reports hard error (134 / SIGABRT)

PSBM-23601

wd_replay_chunks_balanced

BUG at cs_wd.c:1885/wd_replay_chunks_balanced()'

Unable connect to cluster

cloud storage unreachable

timeout (15 sec) expired

c62e8726973f80975db0531f1ed5c6a2 2897d76d56d2010f4e3a28f864d69223 0dd5b9380c7d4884d77587f3eb0fa8ef

Email subscription for changes to this article
Save as PDF