Probleem met Synology DS215J en ESXi

[Madhouse1979]

Ik heb een probleem met waarschijnlijk mijn DS215J en ik hoop dat jullie me hierbij kunnen helpen.

Ik zal eerst even de situatie schetsen: Ik heb een Intel NUC als ESXi server en daarnaast een DS213 en een DS215J. De DS215J gebruik ik als iSCSI target voor ESXi en om de ongeveer 72u is het ISCSI target offline (zie screenshot) en zijn mijn VM's niet meer bereikbaar. Ik heb de DS215J bewust gekocht hiervoor omdat ik mijn DS213 (had ik eerst als iSCSI) voor andere doeleinden gebruik. Ik heb als test op mijn DS213 een iSCSI target aangemaakt en die blijft gewoon bereikbaar. Het lijkt dus echt aan de DS215J te liggen.

Als ik op 'Rescan All..' klik in ESXi dan komt het iSCSI target weer online en zijn al mijn VM's weer bereikbaar.

Wie heeft er een idee wat er gebeurt en wat ik er aan zou kunnen doen?

[hans_lenze]

Heb je het probleem ook als je de 215j als NFS datastore gebruikt?

[Trommelrem]

Wat zegt de Event Log?

[Madhouse1979]

NFS heb ik nog niet geprobeerd, dat zou ik kunnen doen. Nadeel is wel dat het weer 72u duurt voordat ik het weet.

Event log zegt helemaal niets helaas

[Trommelrem]

Dat geloof ik niet. Als de verbinding verbreekt dan moet er op z'n minst daarvan een melding komen. Al hoeft het niet direct een foutmelding te zijn.

[Madhouse1979]

Sorry voor mijn late antwoord. In ESX staat het volgend in de log wat de term 'iSCSI' bevat. Dit is van afgelopen woensdag en dit zou de tijd moeten zijn dat het weer is gebeurt.

2015-09-30T15:28:57.626Z cpu2:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:00.626Z cpu2:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:03.627Z cpu2:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:06.626Z cpu2:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:09.626Z cpu2:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:12.625Z cpu2:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:15.626Z cpu2:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:18.625Z cpu2:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:21.626Z cpu1:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:24.626Z cpu0:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:27.627Z cpu2:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:30.627Z cpu1:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:33.627Z cpu0:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:36.626Z cpu2:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:39.626Z cpu2:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:42.626Z cpu0:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:45.627Z cpu2:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:29:48.627Z cpu2:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:34:48.626Z cpu2:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:39:48.626Z cpu1:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:44:48.626Z cpu2:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:49:48.625Z cpu1:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:54:48.626Z cpu0:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T15:59:48.626Z cpu3:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:04:48.626Z cpu2:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:09:48.625Z cpu1:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:14:48.625Z cpu3:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:19:48.626Z cpu3:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:24:48.626Z cpu1:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:29:48.627Z cpu0:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:34:48.626Z cpu1:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:39:48.626Z cpu1:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:44:48.625Z cpu3:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:49:48.626Z cpu3:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:54:48.627Z cpu0:32800)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T16:59:48.625Z cpu0:32801)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:
2015-09-30T17:04:10.518Z cpu3:33208)iscsi_vmk: iscsivmk_ConnNetRegister: socket 0x4303ef60a020 network resource pool netsched.pools.persist.iscsi associated
2015-09-30T17:04:10.518Z cpu3:33208)iscsi_vmk: iscsivmk_ConnNetRegister: socket 0x4303ef60a020 network tracker id 16768 tracker.iSCSI.192.168.1.21 associated
2015-09-30T17:04:10.771Z cpu3:33208)WARNING: iscsi_vmk: iscsivmk_StartConnection: vmhba36:CH:0 T:0 CN:0: iSCSI connection is being marked "ONLINE"
2015-09-30T17:04:10.771Z cpu3:33208)WARNING: iscsi_vmk: iscsivmk_StartConnection: Sess [ISID: 00023d000002 TARGET: iqn.2000-01.com.synology:synsrv02.target-1.9af4fb5d7b TPGT: 0 TSIH: 0]
2015-09-30T17:04:10.771Z cpu3:33208)WARNING: iscsi_vmk: iscsivmk_StartConnection: Conn [CID: 0 L: 192.168.1.19:36514 R: 192.168.1.21:3260]
2015-09-30T17:04:10.898Z cpu2:32802)HBX: 276: 'synsrv02iSCSI': HB at offset 4120576 - Reclaimed heartbeat [Timeout]:
2015-09-30T17:26:59.604Z cpu2:33097)Loading module iscsi_trans ...
2015-09-30T17:26:59.604Z cpu2:33097)Elf: 1865: module iscsi_trans has license VMware
2015-09-30T17:26:59.605Z cpu2:33097)XXX iscsi_transport_init: Built @ 17:53:36 May 4 2015
2015-09-30T17:26:59.605Z cpu2:33097)Mod: 4942: Initialization of iscsi_trans succeeded with module ID 19.
2015-09-30T17:26:59.605Z cpu2:33097)iscsi_trans loaded successfully.
2015-09-30T17:26:59.622Z cpu3:33099)Loading module iscsi_trans_compat_shim ...
2015-09-30T17:26:59.622Z cpu3:33099)Elf: 1865: module iscsi_trans_compat_shim has license VMware
2015-09-30T17:26:59.623Z cpu3:33099)Mod: 4942: Initialization of iscsi_trans_compat_shim succeeded with module ID 20.
2015-09-30T17:26:59.623Z cpu3:33099)iscsi_trans_compat_shim loaded successfully.
2015-09-30T17:26:59.640Z cpu0:33100)Loading module iscsi_trans_incompat_shim ...
2015-09-30T17:26:59.641Z cpu0:33100)Elf: 1865: module iscsi_trans_incompat_shim has license VMware
2015-09-30T17:26:59.641Z cpu0:33100)Mod: 4942: Initialization of iscsi_trans_incompat_shim succeeded with module ID 21.
2015-09-30T17:26:59.641Z cpu0:33100)iscsi_trans_incompat_shim loaded successfully.
2015-09-30T17:27:04.439Z cpu1:33205)Loading module iscsi_vmk ...
2015-09-30T17:27:04.439Z cpu1:33205)Elf: 1865: module iscsi_vmk has license VMware
2015-09-30T17:27:04.443Z cpu1:33205)WARNING: iscsi_vmk: iscsivmk_ModuleInit:532: Offloading digest calculation using CRC32 instruction set
2015-09-30T17:27:04.444Z cpu1:33205)iscsi_vmk: iscsivmk_RegisterSCSIAdapter:1606: iscsi_vmk module registered iSCSI adapter vmhba36 with the mid-layer
2015-09-30T17:27:04.444Z cpu1:33205)Mod: 4942: Initialization of iscsi_vmk succeeded with module ID 69.
2015-09-30T17:27:04.444Z cpu1:33205)iscsi_vmk loaded successfully.
2015-09-30T17:27:05.930Z cpu0:33206)iscsi_vmk: iscsivmk_ConnNetRegister: socket 0x430326486a10 network resource pool netsched.pools.persist.iscsi associated
2015-09-30T17:27:05.930Z cpu0:33206)iscsi_vmk: iscsivmk_ConnNetRegister: socket 0x430326486a10 network tracker id 16768 tracker.iSCSI.192.168.1.21 associated
2015-09-30T17:27:06.183Z cpu0:33206)WARNING: iscsi_vmk: iscsivmk_StartConnection: vmhba36:CH:0 T:0 CN:0: iSCSI connection is being marked "ONLINE"
2015-09-30T17:27:06.183Z cpu0:33206)WARNING: iscsi_vmk: iscsivmk_StartConnection: Sess [ISID: 00023d000002 TARGET: iqn.2000-01.com.synology:synsrv02.target-1.9af4fb5d7b TPGT: 0 TSIH: 0]
2015-09-30T17:27:06.183Z cpu0:33206)WARNING: iscsi_vmk: iscsivmk_StartConnection: Conn [CID: 0 L: 192.168.1.19:52305 R: 192.168.1.21:3260]
2015-09-30T17:27:06.681Z cpu1:33048)ScsiScan: 1173: Path 'vmhba36:C0:T0:L0': Vendor: 'SYNOLOGY' Model: 'iSCSI Storage' Rev: '3.1'
2015-09-30T17:27:29.416Z cpu1:33398)Loading module vmkapi_v2_1_0_0_iscsi_shim ...
2015-09-30T17:27:29.416Z cpu1:33398)Elf: 1865: module vmkapi_v2_1_0_0_iscsi_shim has license VMware
2015-09-30T17:27:29.417Z cpu1:33398)VmkapiV2_1_0_0IscsiShim: init_module:90: Shim for iscsi vmkapi 2.1.0-0 to vmkapi 2.3.0-0 is active.
2015-09-30T17:27:29.417Z cpu1:33398)Mod: 4942: Initialization of vmkapi_v2_1_0_0_iscsi_shim succeeded with module ID 101.
2015-09-30T17:27:29.417Z cpu1:33398)vmkapi_v2_1_0_0_iscsi_shim loaded successfully.
2015-09-30T17:27:29.429Z cpu0:33399)Loading module vmkapi_v2_0_0_0_iscsi_shim ...
2015-09-30T17:27:29.430Z cpu0:33399)Elf: 1865: module vmkapi_v2_0_0_0_iscsi_shim has license VMware
2015-09-30T17:27:29.430Z cpu0:33399)VmkapiV2_0_0_0IscsiShim: init_module:90: Shim for iscsi vmkapi 2.0.0-0 to vmkapi 2.3.0-0 is active.
2015-09-30T17:27:29.430Z cpu0:33399)Mod: 4942: Initialization of vmkapi_v2_0_0_0_iscsi_shim succeeded with module ID 102.
2015-09-30T17:27:29.430Z cpu0:33399)vmkapi_v2_0_0_0_iscsi_shim loaded successfully.
2015-09-30T17:27:29.443Z cpu0:33400)Loading module vmkapi_v2_2_0_0_iscsi_shim ...
2015-09-30T17:27:29.443Z cpu0:33400)Elf: 1865: module vmkapi_v2_2_0_0_iscsi_shim has license VMware
2015-09-30T17:27:29.443Z cpu0:33400)Vmkapiv2_2_0_0IscsiShim: init_module:90: Shim for iscsi vmkapi 2.2.0-0 to vmkapi 2.3.0-0 is active.
2015-09-30T17:27:29.443Z cpu0:33400)Mod: 4942: Initialization of vmkapi_v2_2_0_0_iscsi_shim succeeded with module ID 103.
2015-09-30T17:27:29.443Z cpu0:33400)vmkapi_v2_2_0_0_iscsi_shim loaded successfully.

[Madhouse1979]

hans_lenze schreef op donderdag 01 oktober 2015 @ 08:22:
Heb je het probleem ook als je de 215j als NFS datastore gebruikt?

Getest en de NFS share blijft actief alleen de iSCSI niet.

[hans_lenze]

Madhouse1979 schreef op maandag 05 oktober 2015 @ 07:48:
[...]

Getest en de NFS share blijft actief alleen de iSCSI niet.

Dan kun je netwerk en iets van de hardware al afstrepen. Blijft over software en services.

Waarom crashed iSCSI wel en NFS niet?
Aan welke kant (Synology of host) crashed er iets?

Tijd om alle logs te verzamelen en door te spitten.

[Madhouse1979]

Ik ga aan de slag met de logs, kijken wat ik daar kan vinden.

[hans_lenze]

Op basis van "2015-09-30T15:29:33.627Z cpu0:32802)WARNING: HBX: 3278: 'synsrv02iSCSI': HB at offset 4120576 - Reclaiming timed out HB failed: No connection:" gok ik dat de iSCSI daemon onderuit gaat op de Synology. Ik zou daar beginnen met zoeken.

Edit: De service wordt iscsitrg genoemd, dat helpt met greppen van de logjes

[ Voor 21% gewijzigd door hans_lenze op 05-10-2015 10:56 ]

[Madhouse1979]

Thanks voor je hulp! Ik ga morgen even verder zoeken..