Teaser : kbmMW’s new object notation

kbmMW now supports XML, JSON, BSON, Messagepack and YAML natively, and includes a new common Object Notation format for easy conversion.

JSON and XML has been supported natively by kbmMW for years. Later kbmMW also supported marshalling of native Delphi objects to and from JSON and XML.

In addition to it being a very nice way to stream Delphi objects, the feature also allowed for a way to convert XML to JSON and visa versa.

However, the upcoming version of kbmMW now includes a new object notation framework, that makes it easy to convert from one notation to another, without going through a native Delphi object.

The new version of kbmMW not only supports JSON and XML, but now also YAML (which is a more human readable format) and the compact binary Messagepack and BSON formats, natively, both for creating your own object notation trees in code, and for marshalling Delphi objects to and from.

With the many new object notation formats supported by kbmMW, it also made sense to find a way to make it easy to convert between them, without having to marshal to and from native Delphi objects.

For that reason the kbmMW Object Notation format was invented. The object notation format is based on a number of classes:

  • TkbmMWONCustomObject (from which the classes listed below descends)
  • TkbmMWONObject (for key/value type data)
  • TkbmMWONArray (for array type data)
  • TkbmMWONNative (for string, integer, floats, date/time, binary values, custom native values and more)

In addition a number of abstract stream classes was developed, which purpose is to handle converting the object notation trees to and from various string or binary formats, and a set of abstract classes that handles marshalling object notation trees to and from native Delphi objects.

JSON, YAML, BSON and Messagepack object notation classes directly descends from the Object Notation classes.

For JSON:

  • TkbmMWJSONObject
  • TkbmMWJSONArray
  • TkbmMWJSONNative
For YAML:

  • TkbmMWYAMLObject
  • TkbmMWYAMLArray
  • TkbmMWYAMLNative
For BSON:

  • TkbmMWBSONObject
  • TkbmMWBSONArray
  • TkbmMWBSONNative
For Messagepack:

  • TkbmMWMessagepackObject
  • TkbmMWMessagepackArray
  • TkbmMWMessagepackNative

Since XML is a more rich notation format, it was decided to leave its SAX based parser (which the XML DOM class use) and instead add support for converting the XML DOM tree to and from the Object Notation format. This way its easy to convert between different notation formats, without sacrificing the more advanced features of our native XML parser.

The above specialized object notation classes, are in reality very very slim.
Thus the underlying TkbmMWON…. classes are sufficient for conversion. So even if one build a JSON object notation tree in code, it can still be understood by the Messagepack formatter code etc.

Generally it’s recommended using the non specialized classes instead of the specialized ones for simplicity reasons.

So lets try to build a simple object notation tree:

var
 father,child:TkbmMWONObject;
 children:TkbmMWONArray;
begin
 father:=TkbmMWONObject.Create;
 father.AsString['name']:='Joe Simpson';
 father.AsInt32['age']:=42;
 children:=TkbmMWONArray.Create;
 child:=TkbmMWONObject.Create;
 child.AsString['name']:='Joe Simpson Jr';
 child.AsInt32['age']:=12;
 children.Add(child);
 father.AsArray['children']:=children;
 ...

This basically makes a father object with name and age, and a list of children (containing only one child with name and age). The object notation tree should now look like this:

2017-02-01-01_09_51-xbox

So lets convert this to JSON:

var
 json:TkbmMWJSONStreamer;
 s:string;
begin
 json:=TkbmMWJSONStreamer.Create;
 try
  // If set to true, an exception will be raised in case a type is not supported
  // by the specific streamformat. An event also exists which one can hook
  // into, to get information about what type fails etc.
  json.ExceptOnUnknownType:=true;
  s:=json.SaveToUTF16String(father);

  // s now contains the serialized object in JSON format.
 finally
  json.Free;
 end;
end;

Resulting in s containing:

{"name":"Joe Simpson","age":42,"children":[{"name":"Joe Simpson Jr","age":12}]}

or to YAML:

var
 yaml:TkbmMWYAMLStreamer;
 s:string;
begin
 yaml:=TkbmMWYAMLStreamer.Create;
 try
  // If set to true, an exception will be raised in case a type is not supported
  // by the specific streamformat. An event also exists which one can hook
  // into, to get information about what type fails etc.
  yaml.ExceptOnUnknownType:=true;
  s:=yaml.SaveToUTF16String(father); 

  // s now contains the serialized object in YAML format.
 finally
  yaml.Free;
 end;
end;

Resulting in s containing (with linefeeds):

name: Joe Simpson
age: 42
children:
  -
    name: Joe Simpson Jr
    age: 12

BSON and Messagepack follows the same pattern as JSON and YAML.

And for XML:

var
 xml:TkbmMWDOMXML;
 s:string;
begin
 xml:=TkbmMWDOMXML.Create;
 try
  xml.LoadFromObjectNotation(father);
  xml.Top.Name:='father';
  xml.AutoIndent:=true;
  xml.AutoLineFeed:=true;
  xml.Typed:=false;
  s:=xml.SaveToString;

  // s now contains the serialized object in XML format.
 finally
  xml.Free;
 end;
end;

Resulting in s containing (with linefeeds):

<father>
 <name>Joe Simpson</name>
 <age>42</age>
 <children i="0">
  <name>Joe Simpson Jr</name>
  <age>12</age>
 </children>
</father>

In BSON, the output would be a binary value (a stream or a byte array). A Base64 encoded version of the above object notation tree would be:

XQAAAAJuYW1lAAwAAABKb2UgU2ltcHNvbgAQYWdlACoAAAAEY2hpbGRyZW4ALwAAAAMwACcAAAACbmFtZQAPAAAASm9lIFNpbXBzb24gSnIAEGFnZQAMAAAAAAAA

and a Base64 encoded Messagepack stream would look like:

g6RuYW1lq0pvZSBTaW1wc29uo2FnZSqoY2hpbGRyZW6RgqRuYW1lrkpvZSBTaW1wc29uIEpyo2FnZQw=

The Object Notation classes also supports letting the developer indicate if a property key/value pair of type TkbmMWONNative should be streamed as an attribute or an element in XML.

Further the TkbmMWONNative class supports custom values, like the BSON Decimal128 value. As no other streamers understand that value type, using custom types do limit lossless conversion between different object notation formats.

Delphi native objects can be marshalled to and from the Object Notation format, and thus also to and from the above supported specialized object notation formats formats.

Teaser : Easy REST with kbmMW and Delphi

kbmMW has for many years supported REST, but it did require some manual work in marshalling data in and out. Further the URL’s allowed for calling kbmMW’s REST interface had to be formatted fairly strictly.

Using the new kbmMW smart service which is being developed, you can extremely easily publish functions to be consumed by the public via REST via any URL you like.

A kbmMW smart service which have some functions available for calling from smart clients (a new kbmMW client type) and REST clients, could look like this:

type

 // This service is known to smart clients as SMARTDEMO
 // Methods registered for REST access will, if they
 // provide a relative path, be sub pathed under /myserver
 [kbmMW_Service('SMARTDEMO')]
 [kbmMW_Rest('path:/myserver')]
 TkbmMWCustomService2 = class(TkbmMWCustomSmartService)
 public

 // This method can be called both from REST and
 // from smart clients.
 // Its called from REST clients like this:
 // http://../helloworld
 [kbmMW_Method('HelloWorld')]
 [kbmMW_Rest('method:get, path: "/helloworld"')]
 function HelloWorld:string;

 // This method can be called both from REST and
 // from smart clients.
 // Its called from REST clients like this:
 // http://../myserver/myechostring/somestring
 // or
 // http://../myserver/echostring/somestring
 // somestring (can be anything) will be echoed back.
 // The argument AString will automatically receive
 // the value provided as somestring.
 [kbmMW_Method('EchoString')]
 [kbmMW_Rest(
 'method:get, '+
 'path: [ "echostring/{AString}","myechostring/{AString}" ]')]
 function EchoString(
  [kbmMW_Rest('value: "{AString}"')] const AString:string):string;

// This method is only available from REST calls, not
 // from smart clients. (kbmMW_Method attribute is missing)
 // It can be called from a browser like this:
 // http://.../someabspath/addnumbers?arg1=10&arg2=20
 // The functions arguments are automatically populated
 // with the arguments.
 [kbmMW_Rest('method:get, path: "/someabspath/addnumbers"')]
 function AddNumbers(
  [kbmMW_Rest('value: "$arg1", required: true')] const AValue1:integer;
  [kbmMW_Rest('value: "$arg2", required: true')] const AValue2:integer;
  [kbmMW_Arg(mwatRemoteLocation)] const ARemoteLocation:string):integer;

// This method gets and owns a TPerson instamce, which it must
 // itself free. The method is not available for REST calls.
 [kbmMW_Method]
 function StorePerson(
  [kbmMW_Arg(mwatOwnedValue)] const APerson:TPerson):integer;

 // This method is only available from smart clients.
 // It receives a TPerson instamce which is owned by the
 // caller. When called from a smart client, the kbmMW framework
 // is the owner and will automatically free it.
 [kbmMW_Method]
 function StorePerson2(const APerson:TPerson):integer;

 // This method is only available from smart clients.
 // It receives a IPerson2 interface.
 [kbmMW_Method]
 function StorePerson3(const APerson:IPerson2):integer;

 // This method can be called both from REST and
 // from smart clients.
 // The method returns a TPerson instamce which is
 // automatically freed by the kbmMW framework after marshalling.
 // (Its default to free a returned object)
 [kbmMW_Rest('method:get, path: "getperson/{id}", freeResult:true')]
 [kbmMW_Method(true)]
 function GetPerson([kbmMW_Rest('value:{id}')]id:string):TPerson;
end;

// Functions published by the service.
//------------------------------------

function TkbmMWCustomService2.HelloWorld:string;
begin
 Result:='Hello world';
end;

function TkbmMWCustomService2.EchoString(const AString:string):string;
begin
 Result:=AString;
end;

function TkbmMWCustomService2.ReverseString(
 const AString:string;
 const AClientIdentity:TkbmMWClientIdentity):string;
begin
 Result:=StrUtils.ReverseString(AString);
end;

function TkbmMWCustomService2.AddNumbers(
 const AValue1,AValue2:integer;
 const ARemoteLocation:string):integer;
begin
 Result:=AValue1+AValue2;

// In ARemoveLocation the reported remote location
 // for the client is found.
end;

function TkbmMWCustomService2.StorePerson(
 const APerson:TPerson):integer;
begin
 // The TPerson instance is owned by this function,
 // and must be manually freed.
 APerson.Free;
 Result:=0;
end;

function TkbmMWCustomService2.StorePerson2(
 const APerson:TPerson):integer;
begin
 // The TPerson instance is automatically freed
 // upon exit of this function.
 Result:=0;
end;

function TkbmMWCustomService2.StorePerson3(
 const APerson:IPerson2):integer;
begin
 // The lifetime of IPerson2 depends on reference counting.
 Result:=0;
end;

function TkbmMWCustomService2.GetPerson(id:string):TPerson;
begin
 Result:=TPerson.Create;
 Result.Name:='Sofie Mogensen';
 Result.Age:=87;
end;

initialization
 TkbmMWRTTI.EnableRTTI(TkbmMWCustomService2);

Examples on REST calls:

GET http://localhost/myserver/getperson/1

{“person”:{“Name”:”Sofie Mogensen”,”Address”:””,”Age”:87}}

GET http://localhost/someabspath/addnumbers?arg1=10&arg2=20

30

Its dead easy to create the server. Just have a central form/datamodule with a kbmMWServer instance, a transport with the new REST streamformat specified, and configured to listen on port 80, and add all the smart service units you want (like the above). Then only the following 2 lines are needed:

kbmMWServer1.AutoRegisterServices;
kbmMWServer1.Active:=true;

and you are running your REST and smart client server!

As usual all this can be combined with traditional services and clients. While REST calls results in objects automatically being streamed and unstreamed as JSON, smart clients will automatically stream and unstream object and interface instances in messagepack format, which is now also supported.