Monday, December 28, 2015

Commvault Simpana; Linux Informix Db Jobs failing.

So for the past month I've had a reoccurring issue with one Linux client and backing up the Informix db.

        Error Code: [82:129]
Description: The job is pending because: The remote end has closed network connection unexpectedly
Source: mi-2k2-cv, Process: CVD


Here's some pertinent info from the client logs IFXXBSA.log and bar_act.log:

IFXXBSA.log
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject:  Query.CopyType:3|LGName:|CopyGpName:|ResourceType:ND
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject:  Query.ObjectType=4|ObjectStatus=2
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject:  Query.Description=
7940 b740d940 12/18 16:46:28 527474 ReadFromOnconfig() -  nitems : 2
7940 b740d940 12/18 16:46:28 527474 ReadFromOnconfig() -  Parameter SERVERNUM= 0
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject: Call GetBackupInfoByJob to set/update the ArchFileMap cache
7940 b740d940 12/18 16:46:28 527474 ::GetSubclientDir() - The subclient directory is [/opt/simpana/iDataAgent/jobResults/2/5789].
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject: GetBackupInfoByJob did not succeed.. ignore and continue
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject: /ifmx_online/datadbs01/0 . Try ArchiveManager. size=1|PID=7940
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject: Unable to find object key=/ifmx_online/datadbs01/0 in the map.
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject: Check map(for debugging): key=/ifmx_online/rootdbs/0|val=/ifmx_online/rootdbs/0.1450475164.7940
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject: /ifmx_online/datadbs01/0 not found in CV database
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject: Trying to find /ifmx_online/datadbs01/0 in ixbar..|obj_field2=datadbs01|startrefversion=0
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject: /ifmx_online/datadbs01/0 not found in /opt/IBM/informix/etc/ixbar.0.
7940 b740d940 12/18 16:46:28 527474 BSAQueryObject: Exit, status:26
7940 b740d940 12/18 16:46:28 527474 BSAEndTxn: Enter, vote:2
7940 b740d940 12/18 16:46:28 527474 BSAEndTxn: Error: Transfer Incomplete
7940 b740d940 12/18 16:46:28 527474 BSAEndTxn: Exit, status:3
7940 b740d940 12/18 16:46:28 527474 BSATerminate: Enter - PID:7933, TID:7940, xbsaParams.pid:7933
7940 b740d940 12/18 16:46:28 527474 IfxBackupRestoreBase::ExitHere() - Job[0] thread[7940]: m_totalFilesTrans=0 m_totalFilesTransferred=0
7940 b740d940 12/18 16:46:28 527474 IfxBackupRestoreBase::Finalize() - Job[0] thread[7940]: Exit.
7940 b740d940 12/18 16:46:28 527474 IfxAgentBase::Finalize() - delete m_hostName
7940 b740d940 12/18 16:46:28 527474 BSATerminate: Exit, status=0, exstatus=3



bar_act.log

 2015-12-18 16:04:29 6482  6476 onbar_d complete, returning -1 (0xffffffff)
 2015-12-18 16:24:38 7113  7106 onbar_d -b -L 0
 2015-12-18 16:24:38 7113  7106 Working with cvsm as generic storage manager.
 2015-12-18 16:24:38 7113  7106 Archive started on rootdbs, datadbs01, llogdbs01 (Requested Level 0).
 2015-12-18 16:24:38 7113  7106 (-43078) Open or close failed on file '/opt/IBM/informix/etc/ixbar.0', errno = 2 .
No such file or directory
 2015-12-18 16:24:38 7113  7106 Begin level 0 backup rootdbs.
 2015-12-18 16:24:38 7113  7106 Starting Filter /bin/gzip.
 2015-12-18 16:24:38 7116  7113 Successfully connected to Storage Manager.
 2015-12-18 16:24:58 7116  7113 The child process for the backup and restore filter is terminating
with exit code 0.
 2015-12-18 16:24:58 7113  7106 Successfully connected to Storage Manager.
 2015-12-18 16:25:01 7113  7106 Completed level 0 backup rootdbs (Storage Manager copy ID: 7113 1450473878).
 2015-12-18 16:25:01 7113  7106 Begin level 0 backup datadbs01.
 2015-12-18 16:25:01 7113  7106 Starting Filter /bin/gzip.
 2015-12-18 16:25:01 7139  7113 ASSERT: file bar_unix.c line 1448 - contact product support
 2015-12-18 16:25:01 7139  7113 See also: /storage/informixtmp//core
 2015-12-18 16:25:01 7113  7106 (-43082) Writing to backup and restore filter failed with error 136103679.
 2015-12-18 16:25:12 7113  7106 Begin backup of critical file '/opt/IBM/informix/etc/ixbar.0'.
 2015-12-18 16:25:12 7113  7106 (-43078) Open or close failed on file '/opt/IBM/informix/etc/ixbar.0', errno = 2 .
No such file or directory
 2015-12-18 16:25:12 7113  7106 onbar_d complete, returning -1 (0xffffffff)



The resolution for this particular issue was removing a pair of functions from an onconfig file on the client, specifically:

BACKUP_FILTER /bin/gzip
RESTORE_FILTER /bin/gunzip


Restarted Informix via onmode -ky then oninit -v then re-tried the incremental backup job and EUREKA! much success.

No comments:

Post a Comment