Hi All,
I am attempting to determine the expected payload for a SetDataMinerInfoMessage message, the current NotifyType I am interested in replicating is NT_FILL_ARRAY_WITH_COLUMN.
This call is intended to be called with the NotifyProtocol method within a protocol, but this option is not available for Automation Scripts. As I am unable to find an equivalent alternative for automation scripts the next best thing would be to use the underlying SetDataMinerInfoMessage call instead.
The trouble with SetDataMinerInfoMessage is that it contains many fields: What, IInfo1, IInfo2, bInfo1, bInfo2, StrInfo1, StrInfo2, Uia1, Uia2, Puia1, Puia2, Sa1, Sa2, Psa1, Psa2, UiaNotifier, ElementID, Var1 and Var2. Depending on the call you make the fields that are required vary massively with little consistency between each call.
I have used this call many times and managed to work out the expected payload by following a cube session in the client test tool. This works well for calls made by the client, but it doesn't appear to work for QActions. If I follow SLManagedScripting, I can see other SLNet messages such as GetPartialTableMessage, but not any of the SetDataMinerInfoMessage.
This leads me back to my goal of working out NT_FILL_ARRAY_WITH_COLUMN which can only natively invoke in a QAction. So my question is, is there a way to sniff out SetDataMinerInfoMessage calls?
Alternatively, I would settle for documentation breaking down each NotifyType with the expected payload for its corresponding SetDataMinerInfoMessage. But I imagine this is not likely something you would want to provide to customers as it's not intended to be called directly.
Any help will be greatly appreciated.
Thanks
With guidance from Floris's reply, I have managed to achieve setting a column from an Automation Script. Below are some methods to hopefully help someone in the future.
Basic Method
SetColumn(engine, 127001, 1234, 1000, 1005, new[] { "1", "2" }, new object[] { "Hello", "World" });
public void SetColumn(Engine engine, int dataMinerId, int elementId, int tablePid, int columnPid, string[] keys, object[] values)
{
if (keys == null || keys.Length == 0)
{
throw new ArgumentException("Cannot be null or empty.", "keys");
}if (values == null || values.Length == 0)
{
throw new ArgumentException("Cannot be null or empty.", "values");
}var primaryKeys = keys.Cast<object>().ToArray();
var ids = new object[] { dataMinerId, elementId, tablePid, columnPid };
var tableData = new object[] { primaryKeys, values };try
{
var message = new SetDataMinerInfoMessage()
{
Var1 = ids,
Var2 = tableData,
What = (int)NotifyType.NT_FILL_ARRAY_WITH_COLUMN_ONLY_UPDATES,
};engine.SendSLNetSingleResponseMessage(message);
}
catch (DataMinerCOMException e)
{
if (e.ErrorCode == -2147220718 || e.ErrorCode == -2147220916)
{
// 0x80040312, Unknown destination DataMiner specified.
// 0x8004024C, SL_NO_SUCH_ELEMENT, "The element is unknown."
throw new ArgumentNullException(FormattableString.Invariant($"Element with DMA ID '{dataMinerId}' and element ID '{elementId}' was not found."), e);
}
else if (e.ErrorCode == -2147220959)
{
// 0x80040221, SL_INVALID_DATA, "Invalid data".
throw new ArgumentNullException(FormattableString.Invariant($"Invalid data - element: '{dataMinerId}/{elementId}', table ID: '{tablePid}', column ID: '{columnPid}', data: {JsonConvert.SerializeObject(tableData)}"), e);
}
else
{
throw;
}
}
}
IDms Extension Method
var dms = engine.GetDms();
var element = dms.GetElement("Test Protocol");
var table = element.GetTable(1000);
var column = table.GetColumn<string>(1005);
column.SetValues(new[] { "1", "2" }, new object[] { "Hello", "World" });
public static class IDmsExtensions
{
public static void SetValues(this IDmsColumn column, string[] keys, object[] values)
{
if (keys == null || keys.Length == 0)
{
throw new ArgumentException("Cannot be null or empty.", "keys");
}if (values == null || values.Length == 0)
{
throw new ArgumentException("Cannot be null or empty.", "values");
}var table = column.Table;
var element = table.Element;var primaryKeys = keys.Cast<object>().ToArray();
var ids = new object[] { element.AgentId, element.Id, table.Id, column.Id };
var tableData = new object[] { primaryKeys, values };try
{
var message = new SetDataMinerInfoMessage()
{
Var1 = ids,
Var2 = tableData,
What = (int)NotifyType.NT_FILL_ARRAY_WITH_COLUMN_ONLY_UPDATES,
};element.Host.Dms.Communication.SendSingleResponseMessage(message);
}
catch (DataMinerCOMException e)
{
if (e.ErrorCode == -2147220718 || e.ErrorCode == -2147220916)
{
// 0x80040312, Unknown destination DataMiner specified.
// 0x8004024C, SL_NO_SUCH_ELEMENT, "The element is unknown."
throw new ElementNotFoundException(element.DmsElementId, e);
}
else if (e.ErrorCode == -2147220959)
{
// 0x80040221, SL_INVALID_DATA, "Invalid data".
var message = FormattableString.Invariant($"Invalid data - element: '{element.DmsElementId.Value}', table ID: '{table.Id}', column ID: '{column.Id}', data: {JsonConvert.SerializeObject(tableData)}");
throw new IncorrectDataException(message);
}
else
{
throw;
}
}
}
}
Hi Aston,
The NT_* notifies are an internal messaging system that actually have two separate handlers. One is SLDataMiner, and the other is SLProtocol.
The problem is that each process implements the calls that were designed for it, or those that have to be shared between the two as they were developed.
As such, not every NotifyType is supported by both processes and will only work when sent to the intended process. When using QActions, protocol.NotifyProtocol() will send it to SLProtocol, while the SetDataMinerInfoMessage request will send it to SLDataMiner. Some of the messages intended for SLProtocol will then be forwarded to the SLProtocol by the agent hosting the element.
Our documentation explains the notifies that are safe to be called by user code: NT Notify Types | DataMiner Docs.
I've checked the SLDataMiner code, and it looks like NT_FILL_ARRAY_WITH_COLUMN_ONLY_UPDATES (336) | DataMiner Docs is supported, while NT_FILL_ARRAY_WITH_COLUMN (220) | DataMiner Docs is not.
So using this notify, you may have some success in setting rows directly.
Now, the reason that SetDataMinerInfoMessage is complex is because it is a generic wrapper for all internal notify types, which do not have a fixed structure aside from having 2 input arguments and an output argument. As such, the message facilitates different typed input values for both arguments, that can be serialized. It can't accept objects like protocol.NotifyProtocol does, because the data needs to be serialized for the wire first. So your the input argument used, is the one that matches the notify type. It is possible that not every notify is compatible with SetDataMinerInfoMessage, because they were not designed to be used this way.
Hi,
We do not recommend the use of SLNet calls in scenarios like this as these are internal calls that are not officially supported and we cannot guarantee that they will still work in the future.
We recommend instead always using the correct UI or automation options provided in DataMiner Automation, QAction, or through our web API.
In the specific case of the SetDataMinerInfoMessage, it is a call that can perform multiple distinct operations which use all those fields you listed.
Those fields are not very easy to document for such a reason and for your specific use case, I would advise either using a combination of IDmsTable.GetPrimaryKeys with IDmsTable.GetColumn and IDmsColumn.SetValue to perform the sets in a loop or to serialize the data and send it to the element as a single set and then have the element itself do the NotifyProtocol call from a QAction.
The advantage of the first option is that the connector does not need to be modified at the cost of some performance due to the need for more calls.
Thanks João,
I totally understand that there is a level of risk associated with using internal calls. I am going to be a bit cheeky and mention that DmsTable already uses a bunch of SetDataMinerInfoMessage calls for AddRow, RowExists, SetRow and DeletRow.
There is always going to be a risk no matter what calls are made. For example, if a breaking change is made in the DmsTable library any protocol/automation script made by a customer would then have to be updated to fix any issues.
As a customer, we know about this first hand when we updated SRM and it broke dozens of automation scripts. We can accept the risk if it is the better solution at the time the decision was made. (Sorry, this is not a dig at you, just seen this argument mentioned multiple times).
Regarding your proposed solution, this is 100% how I started to approach it, but it dawned on me that this is not going to be good for large tables. If you have 10,000 rows, updating each cell in a column, that’s 10,000 requests to SLNet, the DataBase, any clients connected to the element, and anything subscribed to the table such as services.
I would rather take the slightly risky option of using the corresponding SetDataMinerInfoMessage for setting an entire column currently available for Protocols.
Thanks, Floris!
Thank you for the insight, I wasn’t aware of there being two handlers at play, I was on the assumption that there was just one.
SetDataMinerInfoMessage will likely still remain a mystery when it comes to expected fields and what options are available with each handler.
You are indeed correct about NT_FILL_ARRAY_WITH_COLUMN_ONLY_UPDATES working with automation scripts while NT_FILL_ARRAY_WITH_COLUMN does not. Based on this I have managed to create a method to set a column from an automation script. I have created an answer on this thread for those who would like to replicate it.
I guess my initial question regarding sniffing out all SetDataMinerInfoMessage calls using the client test tool, will likely come down to the handler. If it was handled by SLProtocol it will likely not be shown whereas if it was SLDataMiner there is a chance to see it via a follow.