Can’t create elements on newly added agent to a cluster.

Solved765 viewsclustering DMS clustering
4
0 Comments

Hello All,

I have recently added an agent to a cluster. The DMA I added can see the rest of the cluster, the files are syncing fine. However when opening up the Protocols & Templates page locally I don't see anything.

When trying to create an element I get the pop up message:
"Unable to save the element on the DMA"

Looking in the SLDatabaseConnection logs I get the error:

2023/01/17 11:47:48.377|SLDBConnection|CassandraConnection::Connect|ERR|0|175|Cassandra.NoHostAvailableException: All hosts tried for query failed (tried [::1]:9042: SocketException 'No connection could be made because the target machine actively refused it'; 127.0.0.1:9042: AuthenticationException 'Provided username root and/or password are incorrect')
at Cassandra.ControlConnection.<Connect>d__31.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Cassandra.ControlConnection.<Init>d__30.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Cassandra.Tasks.TaskHelper.<WaitToCompleteAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Cassandra.Cluster.<Init>d__36.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Cassandra.Cluster.<ConnectAsync>d__44`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Cassandra.Cluster.<ConnectAsync>d__43.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Cassandra.Tasks.TaskHelper.WaitToComplete(Task task, Int32 timeout)
at Cassandra.Cluster.ConnectAndCreateDefaultKeyspaceIfNotExists(Dictionary`2 replication, Boolean durableWrites)
at SLCassandraClassLibrary.DBGateway.Cassandra.StorageManagers.SingleNode.CassandraConnection.Connect()

And in the SLNet logs I get:

023-01-17 11:50:33.684|158|Facade.HandleMessage|Incoming (Local SLNet): Skyline.DataMiner.Net.Messages.SLDataGateway.GetDataFromConfigFileRequest
2023-01-17 11:50:33.684|158|Cache|Exception updating event caches with event "Alarm: 7393711 (View Group: Configuration) - 117606/GCSS-NM02/Alarm Group at 01/17/2023 11:50:32". Reason: System.Runtime.Remoting.RemotingException: Failed to connect to an IPC Port: The system cannot find the file specified.

Server stack trace:
at System.Runtime.Remoting.Channels.Ipc.IpcPort.Connect(String portName, Boolean secure, TokenImpersonationLevel impersonationLevel, Int32 timeout)
at System.Runtime.Remoting.Channels.Ipc.ConnectionCache.GetConnection(String portName, Boolean secure, TokenImpersonationLevel level, Int32 timeout)
at System.Runtime.Remoting.Channels.Ipc.IpcClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Skyline.DataMiner.Net.Messages.SLDataGateway.ISLDataGateway.ExecuteRequest(BaseRequest request)
at Skyline.DataMiner.Net.Facade.HandleClientRequestMessage(IConnectionInfo connInfo, ClientRequestMessage oneMsg, Boolean canQueue)
at Skyline.DataMiner.Net.Facade.HandleMessageInternal(IConnectionInfo connInfo, DMSMessage oneMsg, Int32 groupID, Int32 groupTotal)
at Skyline.DataMiner.Net.Facade.HandleMessage(IConnectionInfo connInfo, DMSMessage oneMsg, Int32 groupID, Int32 groupTotal)
at Skyline.DataMiner.Net.CacheManager.CacheLocalDatabaseInfo()
at Skyline.DataMiner.Net.CacheManager.get__cachedLocalDatabaseInfo()
at Skyline.DataMiner.Net.CacheManager.get_HasCassandraDatabase()
at Skyline.DataMiner.Net.CacheManager.CacheGeneralInfo()
at Skyline.DataMiner.Net.AlarmEventCache.GetEnsuredGeneralInfoEventMessage()
at Skyline.DataMiner.Net.AlarmEventCache.AddAlarmToTree(ArrayList tree, AlarmEventMessage alarm)
at Skyline.DataMiner.Net.AlarmEventCache.AddOneAlarm(AlarmEventMessage alarmMessage)
at Skyline.DataMiner.Net.AlarmEventCache.AddEvent(ElementBaseEventMessage message)
at Skyline.DataMiner.Net.EventCache.Cache(ElementBaseEventMessage message)

Below snippet are the errors we are getting in the alarms console.

Could someone guide me as to how to resolve this issue and if the two messages are related?

Thanks,

Ryan

ryan yahia Selected answer as best 18th April 2023

1 Answer

5
12.27K 2 Comments

Hi Ryan,

Based on the info available, your DataMiner agent is unable to communicate with the database, which results in the problems you are experiencing.

Do you know what type of database architecture is expected for your DataMiner system? (e.g. MySQL or Cassandra, Locally hosted casandra database or remote hosted cassandra cluster, ...)?

ryan yahia Selected answer as best 18th April 2023
ryan yahia commented

Hello Ive,

This is a Cassandra database which is hosted locally on the machine.

Thanks,
Ryan
Ive Herreman [SLC] [DevOps Catalyst] commented

Hi Ryan,

Can you check through the windows task manager on the server if the Cassandra service is running (on the services tabpage)?
(also make sure it’s not restarting in a loop)

If the service is not running, we’ll need some more info from the debug log file available in c:Program FilesCassandralogsdebug.log

IF the service is running, you can try to validate the status of the Cassandra service by running the “nodetool status” command in the cmd prompt on the server. (note: first cd to c:program filescassandrabin)