Article ID: 118712, created on Nov 18, 2013, last review on Jun 17, 2016

  • Applies to:
  • Virtuozzo 6.0

Symptoms

The pstorage mount point exists, but any attempt to access it hangs.

Commands like df or ls /pstorage/<clustername> hang completely, and it is required to reset the terminal to get it working again.

Parallels Cloud Storage got unmounted with the following messages in dmesg:

    [1173444.675118] fuse_aio_complete_req: request (rw=WRITE fh=0x7f52cc01bfa0 pos=163258368 size=32768) completed with err=-103
    [1173444.675125] kaio_rw_aio_complete: kaio failed with err=-103 (rw=WRITE; state=1/0x0; clu=7; iblk=155; aux=-1)
    [1173444.675128]  bio=ffff881001dbb680: bi_sector=15760 bi_size=4096
    [1173444.675155] Buffer I/O error on device ploop45253p1, logical block 1714
    [1173444.675158] lost page write due to I/O error on ploop45253p1

The following errors can be found in /var/log/pstorage/<clustername>/pstorage-mount.log.gz:

    29-10-13 03:42:33.382 BUG at fuse_cache_tbl.c:203/fuse_cache_tbl_invalidate()
    29-10-13 03:42:33.382 pstorage version: 6.0.4-36 (Debug)
    29-10-13 03:42:33.382 ---------- [16 stack frames] ----------
    29-10-13 03:42:33.382 /usr/lib64/libpcs_io.so(+0x1df17) [0x7f52d7e41f17]
    29-10-13 03:42:33.382 /usr/lib64/libpcs_io.so(show_trace+0xb5) [0x7f52d7e42055]
    29-10-13 03:42:33.382 /usr/lib64/libpcs_io.so(pcs_err+0x35) [0x7f52d7e41615]
    29-10-13 03:42:33.382 /usr/lib64/libpcs_io.so(+0x1d638) [0x7f52d7e41638]
    29-10-13 03:42:33.382 /usr/bin/pstorage-mount(fuse_cache_tbl_invalidate+0x83) [0x418ae3]
    29-10-13 03:42:33.382 /usr/bin/pstorage-mount(fuse_read_cache_drop_blk+0x5e) [0x41af3e]
    29-10-13 03:42:33.382 /usr/bin/pstorage-mount(fuse_read_cache_drop_range+0xa8) [0x41b038]
    29-10-13 03:42:33.382 /usr/bin/pstorage-mount() [0x40ecdc]
    29-10-13 03:42:33.382 /usr/bin/pstorage-mount() [0x41fca6]
    29-10-13 03:42:33.382 /usr/bin/pstorage-mount() [0x42b6b6]
    29-10-13 03:42:33.382 /usr/lib64/libpcs_io.so(rpc_work_input+0x351) [0x7f52d7e376d1]
    29-10-13 03:42:33.382 /usr/lib64/libpcs_io.so(+0xd896) [0x7f52d7e31896]
    29-10-13 03:42:33.382 /usr/lib64/libpcs_io.so(+0xc129) [0x7f52d7e30129]
    29-10-13 03:42:33.382 /usr/lib64/libpcs_io.so(+0xc4b2) [0x7f52d7e304b2]
    29-10-13 03:42:33.382 /lib64/libpthread.so.0(+0x7851) [0x7f52d7676851]
    29-10-13 03:42:33.382 /lib64/libc.so.6(clone+0x6d) [0x7f52d797492d]
    29-10-13 03:42:33.383 pcs_log_terminate

Cause

The problem is recognized as a product bug with internal ID PSBM-23229.

Resolution

The issue has been fixed since PCS 6.0 Update 5.

To make the pstorage mount point available again, follow the instruction of this article: What to do if pstorage-mount process crashed?

Search Words

Pstorage got unmounted

BUG at fuse_cache_tbl.c:203/fuse_cache_tbl_invalidate()

PSBM-23229

Storage mount crashed

pstorage node crash

c62e8726973f80975db0531f1ed5c6a2 2897d76d56d2010f4e3a28f864d69223 0dd5b9380c7d4884d77587f3eb0fa8ef

Email subscription for changes to this article
Save as PDF